pi-gauntlet 8.0.2 → 8.1.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (29) hide show
  1. package/CHANGELOG.md +9 -0
  2. package/README.md +7 -5
  3. package/agents/loop-implementer.md +33 -9
  4. package/bin/gauntlet-performance.mjs +10 -1
  5. package/bin/gauntlet-performance.test.mjs +85 -5
  6. package/bin/gauntlet-telemetry-seal.mjs +1 -1
  7. package/extensions/lib/plan-check.test.ts +4 -2
  8. package/extensions/lib/plan-check.ts +1 -1
  9. package/extensions/lib/telemetry-collect.test.ts +13 -6
  10. package/extensions/lib/telemetry-collect.ts +4 -3
  11. package/extensions/lib/telemetry-record.test.ts +14 -0
  12. package/extensions/lib/telemetry-record.ts +2 -2
  13. package/extensions/phase-tracker.test.ts +45 -0
  14. package/extensions/plan-tracker.test.ts +100 -1
  15. package/extensions/plan-tracker.ts +99 -3
  16. package/extensions/telemetry.test.ts +103 -0
  17. package/extensions/telemetry.ts +19 -6
  18. package/package.json +2 -2
  19. package/skills/brainstorming/reference/amendment-surface.md +1 -1
  20. package/skills/finishing-a-development-branch/SKILL.md +5 -5
  21. package/skills/finishing-a-development-branch/reference/legs.md +4 -4
  22. package/skills/finishing-a-development-branch/reference/verification-set.md +34 -0
  23. package/skills/gauntlet-handoff/SKILL.md +1 -1
  24. package/skills/gauntlet-resume/SKILL.md +4 -5
  25. package/skills/gauntlet-resume/reference/brief-contract.md +7 -7
  26. package/skills/gauntlet-resume/reference/reconstruction.md +28 -6
  27. package/skills/loop-implementation/SKILL.md +48 -32
  28. package/skills/loop-implementation/reference/coordinated-legs.md +8 -18
  29. package/skills/verification-before-completion/reference/conformance-check.md +9 -3
package/CHANGELOG.md CHANGED
@@ -1,5 +1,14 @@
1
1
  # Changelog
2
2
 
3
+ ## v8.1.1 - 2026-11-01
4
+
5
+ - Loop verification and finish run the deduplicated gate set of recorded row checks in their owning checkouts; CI on push owns the full suite. Gate output stays in session logs, repair children receive log paths, and the driver reads only a 20-line tail at the verify-step-1 human stop. Workers record selective commands with TDD and test-economy rules; ledgers without `checks:` stop instead of falling back to a full run. Loop PR summaries identify scoped verification; unmatched gate commands can leave `verification_runs` at zero and trigger the expected ship advisory without authorizing a full local run. (#109)
6
+
7
+ ## v8.1.0 - 2026-11-01
8
+
9
+ - Both implement engines use the task tracker. Loop derives its view from the ledger with `plan_tracker refresh`, including in-flight rows, baseline skips and preserved conformance tasks; handoff and resume restore that view. Implement completes when every ledger row is closed, and verify step 1 runs the ledger's gate once per leg. Re-bounded rows during verify require an authorized ledger return, not a tracker-only close. (#106)
10
+ - Telemetry counts name-based task reopens and parent verification-command matcher hits from implement start through seal. The performance digest adds `disp/task` (implement dispatch-result events / non-skipped tasks, `n/a` for missing or incomplete data) and `coord` (non-empty sealed companion deliveries). (#106)
11
+
3
12
  ## v8.0.2 - 2026-10-31
4
13
 
5
14
  - **Implementer identity and guard exception.** Structured returns copy the dispatched `anchor` or `anchors` so failed tasks remain identifiable. Explicit `completionGuard: false` preserves edit-less `NEEDS_CONTEXT` and `BLOCKED` returns; isolated waves reject edit-less `DONE` through their patch requirement, while in-place and non-wave callers retain gate/review coverage.
package/README.md CHANGED
@@ -39,9 +39,9 @@ Concretely, one change through the gauntlet:
39
39
  1. Opt in with `/skill:brainstorming` and describe the change. The command is the reliable discovery-and-entry path; explicit prose requesting brainstorming/the gauntlet also supplies consent when the skill is already loaded or an available instruction or route identifies it. An ordinary implementation request or an agent recommendation is not consent. A leading run of GitHub references (`#N`, `owner/repo#N`, or a `/pull/N` or `/issues/N` URL), separated by whitespace or commas, resolves each reference as a PR first: related open, same-repository, non-fork PRs on one base form one rework subject, including drafts. Admission runs in listed order and relatedness is checked before provisioning; the first retained PR is home, whose existing branch rebases onto the fetched PR base by default, then member commits replay onto it in listed order. The first intent word `merge-forward` opts out and keeps the merge path. An unresolved conflict after its scoped helper aborts that step and continues to the spec with the repair as Task 1. A single non-PR reference follows the ticket path. The first non-reference token starts intent text; mid-sentence references and other ticket/free-text subjects keep their existing paths. The combined code, PR-carried specs, claims, and review threads are the brainstorm input, with each PR's provenance retained; only home's bound ticket supplies acceptance criteria, while member tickets supply context. A single PR retains the same rework path without member replays or a close offer. The combined state is judged against constraints from the fetched current base HEAD, not the fork point or the PRs' proposed policy edits; see the [PR-subject contract](skills/brainstorming/reference/pr-subject.md) and [PR-head worktree owner](skills/using-git-worktrees/reference/pr-head.md). **`brainstorming`** sets up an isolated worktree (on the existing branch for PR rework), explores the codebase, and turns the subject into a written spec - when its recon finds a cited case that the ask is the wrong change (misdiagnosed, wrong-sized, already solved, a band-aid, or cutting across a sound pattern - including a repo convention that is itself debt) and can name the alternative, question one is a framing question (`as framed`, `pivot: <alternative>`, or `smaller or none`); otherwise it opens with one sentence naming what it checked, and every approach it proposes ends with a `Pattern:` line naming what it follows or cuts across; when the run starts from a ticket, the spec carries the ticket's acceptance criteria verbatim, each with a disposition (`in-scope`, `deviates:`, `deferred:`, `venue:`, `elsewhere:` for a row another repo's part delivers), scope cuts come only from your answer to a cut question and are listed at the gate, the conformance gate checks the in-scope ones, and `/skill:check-delivery` verifies `venue:` rows before the prod deploy. A multi-model critique runs on it automatically. If the spec replaces a known prior spec, brainstorming marks the predecessor with a `> **Superseded by:**` banner under its title (default format, syntax overridable via `.pi/gauntlet-overrides.md`; candidates come from brainstorming's scout recon, never a mechanical sweep). **You read and approve the spec - human gate 1.** Options 1/2/4/5 approve and pick an engine; option 6 approves the content only: it writes `**Engine:** deferred` under `**Amend-grant:** none`, pushes the branch, opens a draft PR carrying just the spec, prints the URL, and ends the flow - no plan, no loop, no handoff brief. For a coordinated spec, it first pushes every companion branch to origin, then opens one home draft PR; it creates no companion PRs. A later `brainstorming #N` reconstructs the recorded companions from their origin branches before re-presenting the implementation gate; the [publication and re-gate owner](skills/brainstorming/reference/spec-pr.md) defines all-member preflight, recovery, and cleanup. The branch stays; a gauntlet-owned worktree is removed, a host-owned checkout is reported and left. Option 6 is withheld on a PR subject (`Withheld: draft PR - #N already exists`) and when the branch's diff against the default branch is not spec-only. A later `brainstorming #N` on that PR classifies it from the diff alone: one added spec carrying the marker plus banner-only predecessor edits re-presents the gate with 1/2/4/5 (3 only when the spec commit carries a council audit) and skips gather, questionary, council, and scout; a spec-only diff without the marker prints `spec-only PR without the gate marker - running lint and council` and takes the normal brainstorm path with the carried spec as the draft; any other diff takes the rework path. Approval at that re-gate writes `**Engine:**`, `**Amend-grant:**`, and `**PR:** <owner/repo#N>` as a new commit on the PR branch and chains into planning or the loop as usual. Before approval, PR rework's sole write/run exception outside the spec directories permits only the base sync (rebase or merge), each member replay or merge, their conflict helpers' edits to the conflicted files, and their scoped conflict tests; other subjects have no implementation code yet.
40
40
  2. By default, **`writing-plans`** decomposes the approved spec into atomic, independently-verifiable tasks, grouped into parallel waves where they don't touch the same files. Experimental approval options 4/5 instead select **`loop-implementation`**: the plan phase is skipped and a stamped acceptance-criteria ledger replaces the task plan. Option 4 retains amendment review; option 5 grants auto-apply amends, with redraws still stopping. See [Loop engine operations](doc/configuration.md#loop-engine-operations).
41
41
  3. **`subagent-driven-development`** executes the plan in waves, with task implementers followed by one integrated `wave-reviewer` after the scoped test gate passes. The reviewer checks anchored spec clauses and code quality together; repairs stay on the current tree and pass the same gate before re-review. TDD-locked: red, green, refactor. Independent implementation, review, and council batches remain parallel, but their dispatches are foreground: the orchestrator waits for terminal results before accepting work or advancing a phase.
42
- The loop alternative to step 3 runs one fresh `loop-implementer` per seed or row iteration, without implement-phase reviewers. Stagnation stops for a human decision. Under amend permission, row-allowance exhaustion and failed exit verification each get one bounded escalated-model round within the existing cap; passing full verification with every row closed enters the same verify gates.
42
+ The loop alternative to step 3 runs one fresh `loop-implementer` per seed or row iteration, without implement-phase reviewers. Stagnation stops for a human decision. Under amend permission, row-allowance exhaustion and a failed verify-step gate each get one bounded escalated-model round within the existing cap. Ledger refresh completes implement when every row is closed; verify step 1 runs the gate set: deduplicated (owning checkout, command) pairs from done rows' recorded checks, with ownership from the ledger. Gate output goes to logs; the driver keeps exit codes and paths, not test output. CI on push owns the full run.
43
43
 
44
- 4. **verify**: the parent first runs the plan's full verification command set (the ledger's `verification:` command for loop); only a passing result permits whole-diff code review, then the **conformance gate**. A review fix invalidates that result, so the parent reruns full verification before the next review or conformance gate. When the overrides file declares a happy path for the touched paths (see [Project-specific overrides](#project-specific-overrides)), the parent runs it once here, between code review and the conformance gate, and the reviewer reads its transcript as runtime evidence - a hang or a failed request shows up in the audit with the transcript quoted - as a gap when an origin clause ties to it, otherwise as a `happy-path: failed - unattributable` line - never as a silent green. The conformance subagent reads the finished code and docs against your *original words* from step 1, not the plan, and reports per-requirement: delivered, partial, missing, drifted, or unauthorized. Unauthorized rows - shipped surface no human input asked for, whether it crept in or was laundered through the spec - follow their recommendation like every other row: a contained removal auto-runs, anything another requirement leans on is deferred with a plain-language "I'd cut it / I'd keep it" recommendation. Inside a brainstorming-entered flow this gate is machine-blocked from being skipped. Compatible executable recommendations auto-run through an isolated fix-and-re-audit loop with no prompt; anything still open surfaces as a numbered decision brief - one item per decision, the problem in plain words first, lettered options each with a one-line consequence, and `Recommended: <letter> - <reason>` or `No recommendation - pick one`. Reply with one line per item (`1. B`), or bare `1` to take every recommendation (no-recommendation items are re-asked by number); a current `CONFORMS` / no-concerns result goes straight to the branch options with no extra conformance sign-off.
44
+ 4. **verify**: the parent first runs the plan's full verification command set for waves, or the ledger gate set for loop; only a passing result permits whole-diff code review, then the **conformance gate**. A review fix invalidates that result, so the parent reruns the same set before the next review or conformance gate. Loop conformance repairs and finish also use the gate set, never the ledger's full `verification:` command. When the overrides file declares a happy path for the touched paths (see [Project-specific overrides](#project-specific-overrides)), the parent runs it once here, between code review and the conformance gate, and the reviewer reads its transcript as runtime evidence - a hang or a failed request shows up in the audit with the transcript quoted - as a gap when an origin clause ties to it, otherwise as a `happy-path: failed - unattributable` line - never as a silent green. The conformance subagent reads the finished code and docs against your *original words* from step 1, not the plan, and reports per-requirement: delivered, partial, missing, drifted, or unauthorized. Unauthorized rows - shipped surface no human input asked for, whether it crept in or was laundered through the spec - follow their recommendation like every other row: a contained removal auto-runs, anything another requirement leans on is deferred with a plain-language "I'd cut it / I'd keep it" recommendation. Inside a brainstorming-entered flow this gate is machine-blocked from being skipped. Compatible executable recommendations auto-run through an isolated fix-and-re-audit loop with no prompt; anything still open surfaces as a numbered decision brief - one item per decision, the problem in plain words first, lettered options each with a one-line consequence, and `Recommended: <letter> - <reason>` or `No recommendation - pick one`. Reply with one line per item (`1. B`), or bare `1` to take every recommendation (no-recommendation items are re-asked by number); a current `CONFORMS` / no-concerns result goes straight to the branch options with no extra conformance sign-off.
45
45
  5. **`finishing-a-development-branch`**: PR, draft PR, squash, keep, or discard; PR rework's row 1 offers a lease push of the rebased home branch to the existing PR (`Base:` records in `merge:` form push non-force). A spec whose header carries `**PR:**` (approved at a spec-PR re-gate) and no `## Rework record` replaces the two create rows with `Push to PR #N and mark it ready for review` and `Push to PR #N, keep it a draft` (`Push to PR #N (already ready for review)` when the PR is already ready): both push without force after strip-and-seal, the first then runs `gh pr ready`, neither rewrites the PR body, and both are withheld when origin's branch head is no longer an ancestor of local HEAD. For coordinated specs, these [header-linked rows 1/2](skills/finishing-a-development-branch/reference/spec-pr.md#push-rows-12) deliver companions in recorded order before one home seal carrying every companion PR URL, then push home and apply its readiness choice. Row 1 makes companion drafts ready too; row 2 creates drafts but never converts an already-ready PR back to draft. Companion URLs appear in the shipped summary, companions first and home last, not in the unchanged home PR body; keep that summary for cross-repo review because successful home delivery removes receipts. A spec still carrying `**Engine:** deferred` stops finishing (and `gauntlet-resume`) with `spec approved for handoff only - run brainstorming #N` before any phase change, verification, or push. The PR-rework push appends a `## Rework` block naming absorbed PRs while retaining the original description bytes; the header-linked `**PR:**` push leaves the body untouched. Only after a successful rework push and home-description edit does one gated confirmation show the exact close comments and eligible member targets, linking home; declining or any other finish option leaves members open. Closing never deletes member branches, and finish posts no review-thread replies. Its `Before:` handle includes the full portable pre-rework SHA and the local `refs/gauntlet/pre-rework/pr-<N>` ref in the rework checkout. The [finishing owner](skills/finishing-a-development-branch/SKILL.md#pr-subject-row-1-push-to-the-pr-branch) defines the landing and body-update contract. Once a PR exists, run `/skill:gatekeep-pr <pr>` to verify it before merging - it judges the diff against the spec at the PR's head (the acceptance-criteria dispositions and Design you approved; the PR description when no spec applies) - rows the spec defers or deviates are settled, every other row needs its mechanism (code, tests, docs) - and against the rubric; whatever can be seen only after merge is left to step 6. **Human gate 2** - the only other decision you make.
46
46
  6. *(Optional)* Once the merge lands, `/skill:check-delivery <ref>` can prove delivery - default-branch landing, delivery target, per-AC evidence - before the tracker status advances. Explicit ask only, no auto-chain: deploys commonly lag merges by minutes to hours, so an auto-run would routinely check too early.
47
47
 
@@ -97,7 +97,7 @@ pi-gauntlet ships three kinds of pieces, layered on top of pi-cohort's dispatch:
97
97
 
98
98
  - **21 skills** - the workflow logic. Sixteen activate automatically when pi sees the matching kind of task, and the workflow skills chain through their gates: `shape-ticket` (creates one tracker issue or repairs a comma/whitespace list of refs against a Context/Problem/Idea/Acceptance-Criteria template, with a grounded repo audit, an AC integrity check, a cheap council roast, and sequential per-ticket exact-diff write confirmations; model-invocable so other skills and commands can route "file a ticket" to it, the gate still needs your yes), `gatekeep-pr` (model-invocable on an explicit ask that names the PR and the verb; the gates still need your selection - is consent-gated pre-merge verification of a PR against the scope contract at its head - the approved spec's acceptance-criteria dispositions and Design when the PR carries one, else the PR's stated intent; the ticket is cross-checked, never judged here (`check-delivery` owns it) - read-only gathering, verification evidence resolved CI-first (green checks on the exact assessed head count as evidence; a pending commit status that GitHub does not report as blocking the gating actor (`mergeStateStatus` not `BLOCKED`) is not waited for - external-CI commit statuses included - while a running Actions check, an unknown merge state, and every other merge precondition still withhold; the project's verification command runs only as fallback), a rubric-based review, a compact bottom-up report (what the PR delivers, including rows deferred or deviating per spec, PR comments, nits, blockers, then a `mergeable` / `fixable` verdict) and course and terminal menus whose rows derive from the actor's permissions on the PR; nothing mutates (repairs, pushes, reviews, merges, tracker comments) until you pick a course or terminal row, except the opt-in `--rebase [base]` sync that rebases, reviews, and lease-pushes the head before verification, or holds the rebased head locally when a conflict stop's scoped tests fail, so a selected delivery course resolves it before the lease push; after your own merge two opt-in rows post the `AC coverage` block to the ticket or the PR; own-merge, approved, and closed-PR menus offer a recommended `cleanup` row that tears down the gate's worktree and branch and runs the overrides file's `## cleanup` steps - run it with `/skill:gatekeep-pr [<pr>] [--rebase [base]]` (no argument: your open PRs)), `check-delivery` (model-invocable on an explicit ask that names the ticket and the verb; the gates still need your selection - is a post-merge detective control: proves an issue actually shipped (default-branch landing, delivery target, per-AC evidence) before its tracker status advances; it never writes a terminal status - run it with `/skill:check-delivery <ref>`), `writing-plans`, `roasting-the-spec`, `test-driven-development`, `subagent-driven-development`, `loop-implementation` (the plan-free implement driver that iterates a fresh `loop-implementer` over the spec's acceptance-criteria ledger when approval picks the loop engine), `dispatching-parallel-agents`, `verification-before-completion`, `requesting-code-review`, `receiving-code-review`, `using-git-worktrees`, `finishing-a-development-branch`, `forge-skill`, `linear` (reads/searches/comments on/manages Linear tickets via the `linearis` CLI; owns all linearis mechanics and the `## Issue tracker` overrides schema; tracker-facing skills route to it). Five more require explicit entry (`disable-model-invocation: true`): `brainstorming` starts the gated workflow on explicit consent - use `/skill:brainstorming` for reliable discovery, or opt in through reachable explicit prose or a human-selected handoff; after spec approval, downstream planning and implementation still chain automatically. `chase-bug` is human-only bug triage: read-only root-cause discovery to an evidenced verdict menu (real bug -> ticket/brainstorm/hotfix/respond; five negative verdicts), then a gated response to the reporter for addressable origins (GitHub issue / tracker ticket) and a rendered verdict summary otherwise - it never fixes during triage; the hotfix row hands off to `skills/chase-bug/hotfix.md` after the menu - run it with `/skill:chase-bug`. `gauntlet-performance` reads the committed run telemetry (current repo by default, `--dir <path>` adds others) through the parse-only `gauntlet-performance` CLI and answers with one example-led recommendation, 3-5 cornerstone numbers, a council assessment when available, and a menu of at most three actions (render a report file, open the recommendation as a ticket via `shape-ticket`, drill into one run); it writes nothing unless you pick render - run it with `/skill:gauntlet-performance [--dir <path>]... [--since <version>]`. `gauntlet-handoff` ends a session whose flow a fresh session will continue: it invokes pi-cohort's `handoff` skill (`--out <path>` or `--key <name>`, default key = the run worktree's branch with `/` flattened to `-`, default file `<tmpdir>/pi-handoff/<key>.md`), then appends the gauntlet process-state section (phase/plan tracker status) per the shared contract `skills/gauntlet-resume/reference/brief-contract.md` - run it with `/skill:gauntlet-handoff [--out <path> | --key <name>]`. `gauntlet-resume` is the only way back into an interrupted flow from a fresh session: it takes a `gauntlet-handoff` brief (a file path, pasted text, or - with no arguments - a pick from the briefs under `<tmpdir>/pi-handoff/`), a bare worktree that already holds a spec, or a spec tracked on main under a configured spec dir (a spec seed: it creates or reuses the worktree through `using-git-worktrees` and pins the spec), restores phase/plan tracker state through the legal arming sequence (`start brainstorm`, `skip` with `resume:` reasons, `plan_check` before implement-or-later), and never infers approval from artifacts - run it with `/skill:gauntlet-resume [<brief-file> | <spec>.md | <worktree-name-or-path>]`.
99
99
  - **10 subagent personas** - the specialized child agents the skills dispatch via pi-cohort: `implementer`, `loop-implementer`, `wave-reviewer`, `conformance-reviewer`, `spec-summarizer`, `plan-writer`, `spec-council-member`, `spec-council-synthesizer`, `spec-council-applier`, `pr-gatekeeper`. See [doc/personas.md](./doc/personas.md) for what each one does and why its permissions are scoped the way they are.
100
- - **4 runtime extensions** - the enforcement layer. `plan-tracker` and `phase-tracker` are tools skills call to track progress (with a TUI widget); `verify-before-ship` is a hook that warns if you push or open a PR without a passing test run since your last edit; a phase-tracker flow guard reminds on implement-phase commits missing spec/code review. In a brainstorming-entered flow, phase-tracker rejects `implement` or `verify` completion while tracker tasks remain pending or in progress; see [its configuration reference](./doc/configuration.md#phase-tracker). phase-tracker also registers `plan_check`, which verifies a plan against its spec and against the grammar in [skills/writing-plans/reference/plan-contract.md](./skills/writing-plans/reference/plan-contract.md), including that each task's `Tests:` commands are selective and never the full suite, and that every test command carries a passing base-commit result; a pass stamps the plan and spec content hashes for implementation and returns a bounded task manifest; see the [manifest contract and size limit](./doc/configuration.md#phase-tracker). `telemetry` records one YAML record per gauntlet run (phase timing, models, personas, gate/fix rounds, diff at ship) that ships in the squash beside the spec - it is written only inside a brainstorming-entered flow, kept out of history until `finishing-a-development-branch` seals it once with `gauntlet-telemetry-seal` (a `telemetry:` commit under git; the working-copy snapshot in a plain jj workspace) - and blocks a brainstorm `write` into an already-shipped spec; see [its configuration reference](./doc/configuration.md#telemetry). See [doc/configuration.md](./doc/configuration.md) for the settings each one reads.
100
+ - **4 runtime extensions** - the enforcement layer. `plan-tracker` and `phase-tracker` are tools skills call to track progress (with a TUI widget); `verify-before-ship` is a hook that warns if you push or open a PR without a passing test run since your last edit (expected when a loop gate set does not match its patterns; the scoped `Verification:` line answers it, not a full local run); a phase-tracker flow guard reminds on implement-phase commits missing spec/code review. Both engines use the tracker: waves manages plan tasks, while loop calls `plan_tracker refresh` to project ledger rows and preserve conformance tasks. Loop implement completes when every row is closed, then verify step 1 runs the ledger's gate set in the owning checkouts. In a brainstorming-entered flow, phase-tracker rejects `implement` or `verify` completion while tracker tasks remain pending or in progress; see [its configuration reference](./doc/configuration.md#phase-tracker). phase-tracker also registers `plan_check`, which verifies a plan against its spec and against the grammar in [skills/writing-plans/reference/plan-contract.md](./skills/writing-plans/reference/plan-contract.md), including that each task's `Tests:` commands are selective and never the full suite, and that every test command carries a passing base-commit result; a pass stamps the plan and spec content hashes for implementation and returns a bounded task manifest; see the [manifest contract and size limit](./doc/configuration.md#phase-tracker). `telemetry` records one YAML record per gauntlet run (phase timing, models, personas, gate/fix rounds, diff at ship) that ships in the squash beside the spec - it is written only inside a brainstorming-entered flow, kept out of history until `finishing-a-development-branch` seals it once with `gauntlet-telemetry-seal` (a `telemetry:` commit under git; the working-copy snapshot in a plain jj workspace) - and blocks a brainstorm `write` into an already-shipped spec; see [its configuration reference](./doc/configuration.md#telemetry). See [doc/configuration.md](./doc/configuration.md) for the settings each one reads.
101
101
 
102
102
  pi-gauntlet is **opinionated**: it recommends this pipeline for non-trivial changes, but starts `brainstorming` only after explicit user consent. Enforcement is opt-in by entry, not ambient: once brainstorming starts a flow, the phase-tracker extension mechanically blocks a phase from closing before its gate runs, and warns once if the main loop writes code during implement (subagents own implement-phase edits). A change made *without* entering the flow (a typo, a formatting run, a dependency bump - see "When to use / when NOT to use") is not gated; the discipline of routing real work through the pipeline is a convention the tooling supports, not a trap it springs on every edit.
103
103
 
@@ -140,7 +140,7 @@ Run by a human against pi-cohort >= 7.1.0, before a pi-gauntlet release claims t
140
140
  ## Requirements
141
141
 
142
142
  - [pi-coding-agent](https://github.com/earendil-works/pi) ≥ 0.85.1 - tested minimum for sequential `phase_tracker` tool execution, which persists a tracker update before a later tool call in the same model message.
143
- - [pi-cohort](https://github.com/jjuraszek/pi-cohort) >= 7.6.5 - required peer package, including binary-safe worktree patches; it also lets a persona with `subagent` in `tools:` start with inherited extensions ([pi-cohort #26](https://github.com/jjuraszek/pi-cohort/issues/26)), which multi-PR `gatekeep-pr` needs. Skills that dispatch agents (`requesting-code-review`, `subagent-driven-development`, `dispatching-parallel-agents`, `writing-plans`, `forge-skill`, `shape-ticket`, `roasting-the-spec`) call `subagent({})`, which pi-cohort provides. pi-gauntlet does not vendor the dispatch tool; without pi-cohort those skills have nothing to call. Gauntlet execution dispatches explicitly use `async: false`; do not enable pi-cohort's incompatible `forceTopLevelAsync` setting. See pi-cohort's [dispatch configuration](https://github.com/jjuraszek/pi-cohort/blob/main/doc/configuration.md) for that setting's owner and semantics. A wave of N >= 1 tasks dispatches its implementers and first spec reviews as one chain call. `subagent-driven-development` stops before dispatch on unsupported installs; there is no older-runtime fallback.
143
+ - [pi-cohort](https://github.com/jjuraszek/pi-cohort) >= 7.6.6 - required peer package, including binary-safe worktree patches; it also lets a persona with `subagent` in `tools:` start with inherited extensions ([pi-cohort #26](https://github.com/jjuraszek/pi-cohort/issues/26)), which multi-PR `gatekeep-pr` needs. Skills that dispatch agents (`requesting-code-review`, `subagent-driven-development`, `dispatching-parallel-agents`, `writing-plans`, `forge-skill`, `shape-ticket`, `roasting-the-spec`) call `subagent({})`, which pi-cohort provides. pi-gauntlet does not vendor the dispatch tool; without pi-cohort those skills have nothing to call. Gauntlet execution dispatches explicitly use `async: false`; do not enable pi-cohort's incompatible `forceTopLevelAsync` setting. See pi-cohort's [dispatch configuration](https://github.com/jjuraszek/pi-cohort/blob/main/doc/configuration.md) for that setting's owner and semantics. A wave of N >= 1 tasks dispatches its implementers and first spec reviews as one chain call. `subagent-driven-development` stops before dispatch on unsupported installs; there is no older-runtime fallback.
144
144
 
145
145
  Both packages must be listed in your `.pi/settings.json#packages` array (pi adds them automatically when you `pi install`). pi-gauntlet and pi-cohort are versioned independently but release together whenever dispatch semantics change - pin compatible versions of both. At each pi-gauntlet release, set the pi-cohort peer minimum and README floor together to the newest released pi-cohort version; `/skill:release` checks the registry before proceeding.
146
146
 
@@ -168,7 +168,9 @@ Pin an exact release with `npm:pi-gauntlet@X.Y.Z`. See [doc/install-internals.md
168
168
 
169
169
  ## Performance digest
170
170
 
171
- `gauntlet-performance` digests telemetry records without judging them: `node <pi-gauntlet-package>/bin/gauntlet-performance.mjs [--dir <repo root or telemetry dir>]... [--since <version>] [--json]` prints a `corpus:` line, one row per run (`run_id`, repo, spec, version, status - `shipped*` when the record has no ship phase - wall, per-phase minutes, tokens, cost, models, dispatches, `fix_round_grants`, reopens, conformance loops, reviewer findings, council dispatches, and `big` - the count of parent calls whose cache write exceeded 50k tokens with their share of the run's parent cache-write tokens (`-` for records without `parent_calls`)), a `by version` block of p50/max over shipped, non-truncated runs, a `council` block grouped by member roster, and `skipped:` lines for files it could not use. Each council table shows member `total`, `unique`, `applied`, `uniq_appl`, `deferred`, `rejected` blocker/major/minor counts and `uniq_appl_nonminor/run`; `chair:` shows runs, average clusters and members reported, and retried runs. Roster flags (JSON `flags[].rule`: `low_unique_applied` - fewer than 0.2 unique-and-applied blocker/major findings per run; `high_rejection` - over 50% rejected while another member is below 25%) require five runs and print as `flag: <member> - <detail>`. Smaller groups print `not enough runs to assess (N of 5)`; absent council data prints `no council data in selected records`. JSON includes `council: [{ roster, runs, members, chairs, flags }]`. Default corpus is the current repo's telemetry dir; each `--dir` adds a repo root (its own `telemetry.dir` setting is honoured) or a bare telemetry dir. `--since` compares versions numerically. Exit 0 always except usage errors. `/skill:gauntlet-performance` is the reasoning layer on top.
171
+ `gauntlet-performance` digests telemetry records without judging them: `node <pi-gauntlet-package>/bin/gauntlet-performance.mjs [--dir <repo root or telemetry dir>]... [--since <version>] [--json]` prints a `corpus:` line, one row per run (`run_id`, repo, spec, version, status - `shipped*` when the record has no ship phase - wall, per-phase minutes, tokens, cost, models, dispatches, `disp/task`, `coord`, `fix_round_grants`, reopens, conformance loops, reviewer findings, council dispatches, and `big` - the count of parent calls whose cache write exceeded 50k tokens with their share of the run's parent cache-write tokens (`-` for records without `parent_calls`)), a `by version` block of p50/max over shipped, non-truncated runs, a `council` block grouped by member roster, and `skipped:` lines for files it could not use. Each council table shows member `total`, `unique`, `applied`, `uniq_appl`, `deferred`, `rejected` blocker/major/minor counts and `uniq_appl_nonminor/run`; `chair:` shows runs, average clusters and members reported, and retried runs. Roster flags (JSON `flags[].rule`: `low_unique_applied` - fewer than 0.2 unique-and-applied blocker/major findings per run; `high_rejection` - over 50% rejected while another member is below 25%) require five runs and print as `flag: <member> - <detail>`. Smaller groups print `not enough runs to assess (N of 5)`; absent council data prints `no council data in selected records`. JSON includes `council: [{ roster, runs, members, chairs, flags }]`. Default corpus is the current repo's telemetry dir; each `--dir` adds a repo root (its own `telemetry.dir` setting is honoured) or a bare telemetry dir. `--since` compares versions numerically. Exit 0 always except usage errors. `/skill:gauntlet-performance` is the reasoning layer on top.
172
+
173
+ `disp/task` counts recorded implement-phase dispatch-result events divided by `plan.tasks - plan.skipped`; missing plan data, zero denominator or dropped events print `n/a`. `coord` is `yes` only for a non-empty sealed `legs:` list. Telemetry also records `verification_runs`: parent bash matcher hits from implement start through seal, not child checks or an assertion that verification passed. See [the telemetry reference](doc/configuration.md#telemetry) for the counter window and name-based reopen rules.
172
174
 
173
175
  For local development against a checkout instead of npm:
174
176
 
@@ -27,35 +27,59 @@ Read additive inputs `Leg:`, `Rules:`, mapped original AC and, for companions, e
27
27
 
28
28
  ## Procedure
29
29
 
30
- 1. Read the given spec in full before reading the ledger or code. Read the given ledger next.
30
+ 1. Read the given spec in full before reading the ledger or code. Read the given ledger next. Save the exact `pending:` tuple and bumped `iterations:` count before editing.
31
31
  2. Work only on the `target` row. Use `previous decision` and `hint` to avoid repeating work and to try a different approach; keep the target unchanged.
32
32
  3. Run `git diff --stat <base>`. Open full diff hunks only for files the target row's change touches, using `git diff <base> -- <target files>`; read the relevant code before editing it.
33
- 4. Implement the target row. Write or run its evidence check: a test you wrote, or a named command or file assertion for a row you cannot automate. Run only that targeted check, never the ledger's project verification command or the full suite.
34
- 5. Update only the target row's mutable cells: set `status` to `done` on a passing check or `failed` on a failing check, increment `attempts` by 1, and set `evidence` to one physical Markdown table cell containing the check name plus at most three result lines separated by semicolons. Escape pipes as `\|` and use plain text without angle-bracket markup. On failure, record the concrete cause in plain words: the spec-vs-code contradiction, failing assertion, or missing credential, tool, or fixture. Keep reasoning and transcripts out of evidence.
33
+ 4. Implement the target row using Three-scenario TDD and Test economy below. Run only its targeted check under Evidence command, never the ledger's project verification command or the full suite.
34
+ 5. Update only the target row's mutable cells: set `status` to `done` on a passing check or `failed` on a failing check, increment `attempts` by 1, and set `evidence` to one physical Markdown table cell opening with the command you ran under Evidence command, up to the first unescaped semicolon, plus at most three result lines separated by semicolons. Escape pipes as `\|` and use plain text without angle-bracket markup. On failure, record the concrete cause in plain words: the spec-vs-code contradiction, failing assertion, or missing credential, tool, or fixture. Keep reasoning and transcripts out of evidence.
35
35
  6. Append the decision line defined in Output contract.
36
- 7. Stage your target changes and the ledger explicitly. Commit exactly once with subject `loop: row <n> <status>`. Check `git status --porcelain` and leave the working tree clean.
36
+ 7. Remove the `pending:` line from the ledger in your commit. Stage your target changes and the ledger explicitly. Commit exactly once with subject `loop: row <n> <status>`. Check `git status --porcelain` and leave the working tree clean.
37
37
  8. Reply with the appended decision line only.
38
38
 
39
+ For every home row or seed commit, put the saved `pending:` tuple and consumed `iteration:` on separate body lines immediately after the subject's blank separator, matching the companion attribution prefix below. Read that count from the ledger's bumped `iterations:` header, not the task's pre-bump `iteration`. Keep result rows and the decision in the ledger; write no sidecar or extra commit.
40
+
41
+ ## Three-scenario TDD
42
+
43
+ | Row behavior | Procedure |
44
+ |---|---|
45
+ | New behavior | RED: run the test seed wrote for this row, or write a new test when seed named a check instead; confirm it fails for the row's stated reason. GREEN: make the smallest change that passes it. REFACTOR: keep the test selecting the row. |
46
+ | Modifying tested behavior | Read and extend the existing test first with a parameter row or a stronger assertion; run it and see it fail, then change the code and rerun. |
47
+ | Trivial or assertion-only row | Use the asserting command as the check, such as `test -f doc/plans/x.md`; write no test file for a doc sentence, file existence or token assertion. |
48
+
49
+ Keep and extend the test from earlier attempts on a retry; never start it over.
50
+
51
+ ## Test economy
52
+
53
+ - Prefer a parameter row or a stronger assertion on an existing test over a new test. Make near-identical inputs table-driven cases; keep one scenario per operation.
54
+ - Use the cheapest layer that proves the row. Put expensive tests in the repo's slow tier by the convention `AGENTS.md`, the overrides file or the task names. No convention named or discoverable: place the test beside its siblings and say so in your report.
55
+ - Merge only tests the target row's own recorded command selects. Leave a test selected by another closed row's recorded command unchanged, because you cannot update a foreign row's command. After a merge, keep the target row's command selecting its test and re-record that command whenever it changes.
56
+
57
+ ## Evidence command
58
+
59
+ Record one selective invocation: a test file, test id or filter, or an asserting command. Use no `;`, `|` or `&&` in that command; use the runner's filter flags rather than a shell pipeline. Never record the owning checkout's full verification command (the home ledger's `verification:` header or a leg's recorded command), or a runner with no file or filter. Keep the command first and results bounded as in step 5: `node --test extensions/x.test.ts; 3 passed; 0 failed`.
60
+
61
+ Require the recorded command to exit nonzero when it selects no test. Before recording a filter, verify its runner's behavior once: run it with a name that matches nothing and require a nonzero exit; otherwise record the file. For Node, record the test file: `node --test` with a missing file exits nonzero, but `--test-name-pattern` with no matching name exits 0 and reports `tests 0`.
62
+
39
63
  ## Target seed
40
64
 
41
65
  Use this contract for `target: seed`, normally iteration 0. After a human-authorized recovery discards an interrupted seed, repeat seeding at the consumed iteration count; keep the ledger's `iterations` unchanged.
42
66
 
43
67
  Replace steps 2-6 with evidence seeding across all `open` rows. Keep the spec-first read order and scope diff hunks to the seeded rows' files.
44
68
 
45
- For every `open` row, write its failing test or name its check and record the check in `evidence`. Close a row already satisfied by the codebase as `done` only with passing baseline evidence. Keep other open rows `open`. Keep every row's `attempts` at 0; never set `failed`. Preserve `cut` rows and their disposition evidence. Bound evidence as in step 5.
69
+ For every `open` row, write or identify its failing test (use an asserting command for a trivial or assertion-only row) and record the command you ran in `evidence` under Evidence command. Close a row already satisfied by the codebase as `done` only with passing baseline evidence; record the command of the passing check for every baseline closure. Keep other open rows `open`. Keep every row's `attempts` at 0; never set `failed`. Preserve `cut` rows and their disposition evidence. Bound evidence as in step 5.
46
70
 
47
71
  Append exactly one decision line: `row 0 done | seeded <k> rows, closed <m> on baseline | <first row to take>`.
48
72
 
49
- Stage the seeded evidence changes and ledger, commit exactly once with subject `loop: seed evidence`, check for a clean tree, and reply with the appended seed decision only.
73
+ Stage the seeded evidence changes and ledger, remove `pending:` in that same commit, commit exactly once with subject `loop: seed evidence`, check for a clean tree, and reply with the appended seed decision only.
50
74
 
51
75
  ## Companion result commit
52
76
 
53
77
  For a companion `pending:` input, replace ledger mutation/staging and decision appending in the procedure and output contract with the commit-body path below; leave the home ledger untouched. Limit work and seed checks to mapped entries for this leg. Apply the same one-commit and bounded evidence contracts. Prepare the resulting four-column `row | status | attempts | evidence` target row in memory, not a sidecar. Serialize it in the companion commit body instead of editing the home ledger; for seed serialize all and only this leg's non-cut rows, preserving their attempts and proven baseline closures. For ordinary work serialize only its target, attempts +1 and status done/failed.
54
78
 
55
- Use the existing subject `loop: row <n> <status>` or `loop: seed evidence`. Put the exact dispatched `pending:` tuple and consumed `iteration:` on separate body lines, then a blank line, then the resulting pipe-delimited table rows without a header, then a blank line and the existing bounded decision line. Include the check name plus at most three semicolon-separated result lines and escaped pipes in each evidence cell. Compute seed decision counts from this leg only. Stage code/tests explicitly and commit exactly once in the companion; check clean status. Reply with the existing decision line only. Let the parent validate the commit and write the home ledger.
79
+ Use the existing subject `loop: row <n> <status>` or `loop: seed evidence`. Put the exact dispatched `pending:` tuple and consumed `iteration:` on separate body lines, then a blank line, then the resulting pipe-delimited table rows without a header, then a blank line and the existing bounded decision line. Include the evidence command from Evidence command plus at most three semicolon-separated result lines and escaped pipes in each evidence cell. Compute seed decision counts from this leg only. Stage code/tests explicitly and commit exactly once in the companion; check clean status. Reply with the existing decision line only. Let the parent validate the commit and write the home ledger.
56
80
 
57
- For coordinated home work, edit only mapped home entries and its seeded state under the ordinary ledger contract; preserve foreign entries, mappings, bases and all other headers.
81
+ For coordinated home work, edit only mapped home entries and its seeded state under the ordinary ledger contract; preserve foreign entries, mappings, bases and all other headers except removal of `pending:`.
58
82
 
59
83
  ## Boundaries
60
84
 
61
- Edit code and tests only for the target row, or the open rows during seed. Outside `target: seed`, never edit a foreign ledger row. Preserve row identifiers, row text, ledger headers, and existing decisions. Never edit the spec or the plan directory outside the supplied ledger. Never touch `main`. Leave whole-project verification and reviews to the parent.
85
+ Edit code and tests only for the target row, or the open rows during seed. Outside `target: seed`, never edit a foreign ledger row. Preserve row identifiers, row text, ledger headers (except removal of `pending:`), and existing decisions. Never edit the spec or the plan directory outside the supplied ledger. Never touch `main`. Leave whole-project verification and reviews to the parent.
@@ -92,7 +92,7 @@ __export(runs_exports, {
92
92
  renderText: () => renderText
93
93
  });
94
94
  var key = "runs";
95
- var HEADER = ["run_id", "repo", "spec", "version", "status", "engine", "iterations", "wall", "b/p/i/v/s min", "tokens", "cost", "models", "disp", "grants", "reopens", "loops", "findings", "council", "big"];
95
+ var HEADER = ["run_id", "repo", "spec", "version", "status", "engine", "iterations", "wall", "b/p/i/v/s min", "tokens", "cost", "models", "disp", "disp/task", "coord", "grants", "reopens", "loops", "findings", "council", "big"];
96
96
  var runRow = (r) => [
97
97
  r.run_id.slice(0, 8),
98
98
  r.repo,
@@ -107,6 +107,8 @@ var runRow = (r) => [
107
107
  fmtCost(r.cost),
108
108
  r.models.join(",") || "-",
109
109
  dash(r.dispatches),
110
+ r.dispatches_per_task === null ? "n/a" : r.dispatches_per_task.toFixed(2),
111
+ r.coordinated ? "yes" : "no",
110
112
  dash(r.grants),
111
113
  dash(r.reopens),
112
114
  dash(r.loops),
@@ -376,6 +378,11 @@ function loadRecord(file, label) {
376
378
  return [severity, counters.includes(null) ? null : counters.reduce((a, b) => a + b, 0)];
377
379
  }));
378
380
  const truncated = ph("ship") === null;
381
+ const plan = obj(derived.plan);
382
+ const tasks = num(plan?.tasks);
383
+ const skipped = num(plan?.skipped);
384
+ const inScope = tasks === null || skipped === null ? null : tasks - skipped;
385
+ const implementDispatches = Array.isArray(d.events) ? d.events.filter((e) => e?.kind === "dispatch" && e.phase === "implement").length : 0;
379
386
  return {
380
387
  row: {
381
388
  run_id: d.run_id,
@@ -396,6 +403,8 @@ function loadRecord(file, label) {
396
403
  cost: sumOrNull(PHASES.map((p) => num(tok(p)?.cost))),
397
404
  models: [...new Set(PHASES.map((p) => str(ph(p)?.model)).filter((m) => m !== null))].sort(),
398
405
  dispatches: sumOrNull(Object.values(personas).map((p) => num(obj(p)?.dispatches))),
406
+ dispatches_per_task: inScope === null || inScope === 0 || (num(derived.events_dropped) ?? 0) > 0 ? null : implementDispatches / inScope,
407
+ coordinated: Array.isArray(d.legs) && d.legs.length > 0,
399
408
  grants: num(gates.fix_round_grants),
400
409
  reopens: num(gates.task_reopens),
401
410
  spec_rounds: num(gates.spec_rounds),
@@ -1,6 +1,7 @@
1
+ import { parse as parseYaml } from "yaml";
1
2
  import { test } from "node:test";
2
3
  import assert from "node:assert/strict";
3
- import { mkdtempSync, mkdirSync, writeFileSync, rmSync } from "node:fs";
4
+ import { mkdtempSync, mkdirSync, writeFileSync, readFileSync, rmSync } from "node:fs";
4
5
  import { join, dirname } from "node:path";
5
6
  import { tmpdir } from "node:os";
6
7
  import { spawnSync } from "node:child_process";
@@ -20,8 +21,9 @@ const gitRepo = (prefix = "gp-") => {
20
21
  spawnSync("git", ["init", "-q"], { cwd: root });
21
22
  return root;
22
23
  };
23
- const invoke = (cwd, args = [], env = {}) => {
24
- const r = spawnSync(process.execPath, [CLI, ...args], {
24
+ const SOURCE_CLI = join(dirname(CLI), "../src/bins/gauntlet-performance.mjs");
25
+ const invoke = (cwd, args = [], env = {}, cli = CLI) => {
26
+ const r = spawnSync(process.execPath, [cli, ...args], {
25
27
  cwd, encoding: "utf8", env: { ...process.env, PI_CODING_AGENT_DIR: EMPTY_AGENT, ...env },
26
28
  });
27
29
  return { status: r.status, stdout: r.stdout, stderr: r.stderr.trim(), lines: r.stdout.split("\n").filter(Boolean) };
@@ -320,12 +322,12 @@ test("text output: header, shipped* marker, - cells, by version block, skipped l
320
322
  assert.equal(r.status, 0, r.stderr);
321
323
  assert.equal(r.lines[0], `corpus: ${root.split("/").pop()}=8`);
322
324
  assert.equal(r.lines[1], "runs (8)");
323
- assert.match(r.lines[2], /^run_id\s+repo\s+spec\s+version\s+status\s+engine\s+iterations\s+wall\s+b\/p\/i\/v\/s min\s+tokens\s+cost\s+models\s+disp\s+grants\s+reopens\s+loops\s+findings\s+council\s+big$/);
325
+ assert.match(r.lines[2], /^run_id\s+repo\s+spec\s+version\s+status\s+engine\s+iterations\s+wall\s+b\/p\/i\/v\/s min\s+tokens\s+cost\s+models\s+disp\s+disp\/task\s+coord\s+grants\s+reopens\s+loops\s+findings\s+council\s+big$/);
324
326
  const bRow = r.lines.find((l) => l.startsWith("bbbbbbbb"));
325
327
  assert.match(bRow, /\s5\.9\.1\s+shipped\*\s+waves\s+-\s+10\/-\/-\/-\/-\s/);
326
328
  assert.match(bRow, /\s-\s*$/);
327
329
  const aRow = r.lines.find((l) => l.startsWith("aaaaaaaa"));
328
- assert.match(aRow, /\sshipped\s+waves\s+60m\s+10\/10\/10\/10\/10\s+6\.6k\s+2\.25\s+m\/alpha,m\/beta\s+7\s+1\s+0\s+2\s+1\/2\/3\s+4\s+-$/);
330
+ assert.match(aRow, /\sshipped\s+waves\s+60m\s+10\/10\/10\/10\/10\s+6\.6k\s+2\.25\s+m\/alpha,m\/beta\s+7\s+n\/a\s+no\s+1\s+0\s+2\s+1\/2\/3\s+4\s+-$/);
329
331
  assert.ok(r.lines.includes("by version"));
330
332
  const v = r.lines.find((l) => l.startsWith("5.9.1"));
331
333
  assert.match(v, /^5\.9\.1\s+4\s+3\s+1\s+60m\/120m\s/);
@@ -500,6 +502,51 @@ test("council section: --since and no council data", (t) => {
500
502
  assert.equal(none.lines.length, 2);
501
503
  });
502
504
 
505
+ test("source and shipped runs report implement dispatch ratios and sealed coordination", (t) => {
506
+ const root = gitRepo(); cleanup(t, root);
507
+ const results = [
508
+ { ts: "2026-09-01T00:00:00Z", session: "s", phase: "implement", kind: "dispatch", agent: "worker" },
509
+ { ts: "2026-09-01T00:00:01Z", session: "s", phase: "implement", kind: "dispatch", agent: "worker", exit: 1 },
510
+ { ts: "2026-09-01T00:00:02Z", session: "s", phase: "implement", kind: "dispatch", agent: "worker" },
511
+ { ts: "2026-09-01T00:00:03Z", session: "s", phase: "verify", kind: "dispatch", agent: "reviewer" },
512
+ { ts: "2026-09-01T00:00:04Z", session: "s", phase: "implement", kind: "phase", action: "complete", name: "implement" },
513
+ ];
514
+ const cases = [
515
+ { name: "waves", engine: "waves", plan: { tasks: 4, skipped: 2 }, events: results, legs: [], ratio: 1.5 },
516
+ { name: "loop", engine: "loop", plan: { tasks: 4, skipped: 2 }, events: results, legs: [{ repo: "example/companion", sha: "abc123" }], ratio: 1.5 },
517
+ { name: "no-plan", events: results, ratio: null },
518
+ { name: "all-skipped", plan: { tasks: 2, skipped: 2 }, events: results, ratio: null },
519
+ { name: "dropped", plan: { tasks: 4, skipped: 2 }, events: results, events_dropped: 1, ratio: null },
520
+ { name: "zero-dispatches", plan: { tasks: 4, skipped: 2 }, events: [], ratio: 0 },
521
+ { name: "legacy", ratio: null },
522
+ { name: "unrounded", plan: { tasks: 10, skipped: 1 }, events: results, ratio: 1 / 3 },
523
+ ];
524
+ for (const c of cases) write(root, `${TDIR}/${c.name}.yaml`, JSON.stringify({
525
+ schema: 1, spec: `doc/specs/${c.name}.md`, run_id: c.name,
526
+ status: "shipped", legs: c.legs,
527
+ derived: { engine: c.engine, plan: c.plan, events_dropped: c.events_dropped,
528
+ personas: { worker: { dispatches: 3 }, reviewer: { dispatches: 1 } } },
529
+ events: c.events,
530
+ }));
531
+ for (const cli of [SOURCE_CLI, CLI]) {
532
+ const output = invoke(root, ["--json"], {}, cli);
533
+ assert.equal(output.status, 0, output.stderr);
534
+ const rows = JSON.parse(output.stdout).runs;
535
+ const rendered = invoke(root, [], {}, cli);
536
+ assert.equal(rendered.status, 0, rendered.stderr);
537
+ assert.match(rendered.lines[2], /disp\/task\s+coord/);
538
+ for (const c of cases) {
539
+ const row = rows.find((r) => r.run_id === c.name);
540
+ assert.equal(row.dispatches_per_task, c.ratio, c.name);
541
+ assert.equal(row.coordinated, Boolean(c.legs?.length), c.name);
542
+ assert.equal(row.dispatches, 4, c.name);
543
+ const textRow = rendered.lines.find((line) => line.startsWith(c.name.slice(0, 8) + " "));
544
+ const ratioCell = c.ratio === null ? "n/a" : c.ratio.toFixed(2).replace(".", "\\.");
545
+ assert.match(textRow, new RegExp(`\\s${ratioCell}\\s+${c.legs?.length ? "yes" : "no"}\\s`), c.name);
546
+ }
547
+ }
548
+ });
549
+
503
550
  test("runs expose engine and iterations for loop and legacy records in JSON and text", (t) => {
504
551
  const root = gitRepo();
505
552
  cleanup(t, root);
@@ -515,3 +562,36 @@ test("runs expose engine and iterations for loop and legacy records in JSON and
515
562
  assert.match(text, /^aaaaaaaa\s+.*\sshipped\s+waves\s+60m\s/m);
516
563
  assert.match(text, /^bbbbbbbb\s+.*\sshipped\*\s+loop\s+4\s+-\s/m);
517
564
  });
565
+
566
+
567
+ test("shipped seal preserves verification runs and plan totals for performance", (t) => {
568
+ const root = gitRepo(); cleanup(t, root);
569
+ const git = (args) => {
570
+ const result = spawnSync("git", ["-c", "user.name=t", "-c", "user.email=t@t",
571
+ "-c", "commit.gpgsign=false", ...args], { cwd: root, encoding: "utf8" });
572
+ assert.equal(result.status, 0, result.stderr);
573
+ return result.stdout.trim();
574
+ };
575
+ write(root, "README.md", "# Test\n");
576
+ git(["add", "README.md"]); git(["commit", "-qm", "Base"]);
577
+ const base = git(["rev-parse", "HEAD"]);
578
+ write(root, "doc/specs/seal.md", "# Seal\n");
579
+ git(["add", "doc/specs/seal.md"]); git(["commit", "-qm", "Spec"]);
580
+ const path = `${TDIR}/seal.yaml`;
581
+ write(root, path, JSON.stringify({
582
+ schema: 1, spec: "doc/specs/seal.md", run_id: "seal", status: "in_progress",
583
+ created_at: "2026-09-01T00:00:00Z", sessions: ["s"], accumulators: {},
584
+ derived: { verification_runs: 2, plan: { tasks: 2, complete: 1, skipped: 1, failed: 0 } },
585
+ events: [{ ts: "2026-09-01T00:00:00Z", session: "s", phase: "implement", kind: "dispatch", agent: "worker" }],
586
+ }));
587
+ const result = invoke(root, ["--worktree", root, "--option", "squash", "--base", base,
588
+ "--spec", "doc/specs/seal.md"], {
589
+ GIT_AUTHOR_NAME: "t", GIT_AUTHOR_EMAIL: "t@t", GIT_COMMITTER_NAME: "t", GIT_COMMITTER_EMAIL: "t@t",
590
+ }, join(dirname(CLI), "gauntlet-telemetry-seal.mjs"));
591
+ assert.equal(result.status, 0, result.stderr);
592
+ const sealed = parseYaml(readFileSync(join(root, path), "utf8"));
593
+ assert.equal(sealed.status, "shipped");
594
+ assert.equal(sealed.derived.verification_runs, 2);
595
+ assert.deepEqual(sealed.derived.plan, { tasks: 2, complete: 1, skipped: 1, failed: 0 });
596
+ assert.equal(json(root).runs[0].dispatches_per_task, 1);
597
+ });
@@ -346,7 +346,7 @@ function derive(rec, now) {
346
346
  }
347
347
  for (const k of Object.keys(acc.phases)) phases[k] = compact({ ...phases[k] ?? {}, ...acc.phases[k] });
348
348
  const live = liveShipEvent(rec.events);
349
- return compact({ engine: rec.events.findLast((e) => e.kind === "engine")?.engine ?? "waves", iterations: acc.personas["loop-implementer"]?.dispatches, duration_s: seconds(rec.created_at, rec.shipped_at ?? rec.abandoned_at ?? now), phases, personas: acc.personas, reviews: Object.keys(acc.reviews).length ? acc.reviews : void 0, conformance_loops: acc.conformance_loops, conformance_open_gaps: acc.conformance_open_gaps, gates: compact({ ...acc.gates, ship_option: live?.option }), plan: rec.derived.plan, tests: rec.derived.tests, amendments: acc.amendments, spec_edits_after_ship: acc.spec_edits_after_ship, diff: rec.derived.diff, modified_files: rec.derived.modified_files, council: rec.derived.council, spec_writes: acc.spec_writes, events_dropped: acc.events_dropped });
349
+ return compact({ engine: rec.events.findLast((e) => e.kind === "engine")?.engine ?? "waves", iterations: acc.personas["loop-implementer"]?.dispatches, duration_s: seconds(rec.created_at, rec.shipped_at ?? rec.abandoned_at ?? now), phases, personas: acc.personas, reviews: Object.keys(acc.reviews).length ? acc.reviews : void 0, conformance_loops: acc.conformance_loops, conformance_open_gaps: acc.conformance_open_gaps, gates: compact({ ...acc.gates, ship_option: live?.option }), plan: rec.derived.plan, tests: rec.derived.tests, verification_runs: rec.derived.verification_runs, amendments: acc.amendments, spec_edits_after_ship: acc.spec_edits_after_ship, diff: rec.derived.diff, modified_files: rec.derived.modified_files, council: rec.derived.council, spec_writes: acc.spec_writes, events_dropped: acc.events_dropped });
350
350
  }
351
351
  var stripUndefined = (v) => {
352
352
  if (Array.isArray(v)) return v.map(stripUndefined);
@@ -1283,14 +1283,16 @@ for (const [name, change] of Object.entries({
1283
1283
  foreignBase: (s: string) => s + "base stranger: abc1234\n",
1284
1284
  missingSeed: (s: string) => s.replace("seeded repo-b: no\n", ""),
1285
1285
  invalidCap: (s: string) => s.replace("cap: 20", "cap: 0"),
1286
- homePending: (s: string) => s + "pending: 1 | repo-a | abc1234\n",
1286
+ wrongHomeMapping: (s: string) => s + "pending: 2 | repo-a | abc1234\n",
1287
1287
  foreignTarget: (s: string) => s + "pending: 3 | repo-b | abc1234\n",
1288
1288
  duplicatePending: (s: string) => s + "pending: seed | repo-b | abc1234\npending: seed | repo-b | abc1234\n",
1289
1289
  })) test(`coordinated ledger rejects ${name}`, () => {
1290
1290
  assert.ok(checkPlan(change(coordinatedLedger()), coordinatedSpec, emptyPort).findings.length);
1291
1291
  });
1292
1292
  test("coordinated ledger admits partial seeds, recovery tuples and later cap grants", () => {
1293
- for (const target of ["2", "seed"]) assert.deepEqual(checkPlan(coordinatedLedger().replace("seeded repo-a: no", "seeded repo-a: yes").replace("cap: 20", "cap: 99") + `pending: ${target} | REPO-B | abc1234\n`, coordinatedSpec, emptyPort).findings, []);
1293
+ for (const [target, repo] of [["1", "REPO-A"], ["seed", "REPO-A"], ["2", "REPO-B"], ["seed", "REPO-B"]]) {
1294
+ assert.deepEqual(checkPlan(coordinatedLedger().replace("seeded repo-a: no", "seeded repo-a: yes").replace("cap: 20", "cap: 99") + `pending: ${target} | ${repo} | abc1234\n`, coordinatedSpec, emptyPort).findings, [], `${target} ${repo}`);
1295
+ }
1294
1296
  });
1295
1297
 
1296
1298
  const legPlanSpec = SPEC_TEXT + "\n## Legs\n### Leg: repo-a\nscope: none\n### Leg: repo-b\nscope: none\n";
@@ -1227,7 +1227,7 @@ function coordinatedMetadata(lines: string[], legs: ReturnType<typeof parseLegs>
1227
1227
  for (const row of pending) {
1228
1228
  const m = /^(seed|[1-9]\d*) \| (\S+) \| ([a-f0-9]{7,40})$/.exec(row.value);
1229
1229
  const repo = m?.[2].toLowerCase();
1230
- if (!m || !legs.slice(1).some(l => l.repo === repo) || (m[1] !== "seed" && pairs[Number(m[1]) - 1]?.leg !== repo)) add(row.i, "invalid pending tuple or target leg");
1230
+ if (!m || !legs.some(l => l.repo === repo) || (m[1] !== "seed" && pairs[Number(m[1]) - 1]?.leg !== repo)) add(row.i, "invalid pending tuple or target leg");
1231
1231
  }
1232
1232
  const mapping = lines.findIndex(l => l === "## Leg mapping");
1233
1233
  if (lines.filter(l => l === "## Leg mapping").length > 1) add(mapping, "duplicate leg mapping section");
@@ -1,6 +1,6 @@
1
1
  import assert from "node:assert/strict";
2
2
  import { test } from "node:test";
3
- import { COUNTED_USER_PHASES, REVIEWER_AGENTS, countFindings, countOpenGaps, hasReopen, insertedText, planTotals, textOf, usageToTokens } from "./telemetry-collect.ts";
3
+ import { COUNTED_USER_PHASES, REVIEWER_AGENTS, countFindings, countOpenGaps, countReopens, insertedText, planTotals, textOf, usageToTokens } from "./telemetry-collect.ts";
4
4
 
5
5
  const usage = (input: number, output = 0, cost = 0) => ({ input, output, cacheRead: 0, cacheWrite: 0, totalTokens: input + output, cost: { input: cost, output: 0, cacheRead: 0, cacheWrite: 0, total: cost } });
6
6
 
@@ -32,11 +32,18 @@ test("planTotals counts tasks by terminal status", () => {
32
32
  assert.deepEqual(planTotals([{ status: "complete" }, { status: "skipped" }, { status: "failed" }, { status: "complete" }]), { tasks: 4, complete: 2, failed: 1, skipped: 1 });
33
33
  });
34
34
 
35
- test("hasReopen detects complete -> in_progress at the same index only", () => {
36
- assert.equal(hasReopen([{ status: "complete" }, { status: "pending" }], [{ status: "in_progress" }, { status: "pending" }]), true);
37
- assert.equal(hasReopen([{ status: "complete" }], [{ status: "complete" }]), false);
38
- assert.equal(hasReopen(undefined, [{ status: "in_progress" }]), false);
39
- assert.equal(hasReopen([{ status: "pending" }], [{ status: "in_progress" }]), false);
35
+ test("reopens match distinct string names, not position", () => {
36
+ for (const [from, to, expected] of [
37
+ ["complete", "failed", 1], ["complete", "in_progress", 1],
38
+ ["failed", "in_progress", 0], ["complete", "skipped", 0],
39
+ ["complete", "pending", 0], ["complete", "complete", 0],
40
+ ] as const) assert.equal(countReopens([{ name: "a", status: from }], [{ name: "a", status: to }]), expected, `${from} -> ${to}`);
41
+ assert.equal(countReopens([{ name: "a", status: "complete" }, { name: "b", status: "complete" }], [{ name: "b", status: "failed" }, { name: "a", status: "in_progress" }]), 2);
42
+ assert.equal(countReopens([{ name: "a", status: "complete" }, { name: "b", status: "pending" }], [{ name: "b", status: "pending" }, { name: "a", status: "complete" }]), 0);
43
+ assert.equal(countReopens([{ name: "a", status: "complete" }], [{ name: "a", status: "failed" }, { name: "a", status: "in_progress" }]), 1);
44
+ assert.equal(countReopens(undefined, [{ name: "a", status: "in_progress" }]), 0);
45
+ assert.equal(countReopens([{ name: "a", status: "complete" }], [{ name: "b", status: "failed" }]), 0);
46
+ assert.equal(countReopens([{ status: "complete" }], [{ status: "in_progress" }]), 0);
40
47
  });
41
48
 
42
49
  test("insertedText joins edit newText values or returns write content", () => {
@@ -71,9 +71,10 @@ export const planTotals = (tasks: { status: string }[]): PlanTotals => ({
71
71
  skipped: tasks.filter((t) => t.status === "skipped").length,
72
72
  });
73
73
 
74
- // A plan_tracker update that moves a task from complete back to in_progress.
75
- export const hasReopen = (prev: { status: string }[] | undefined, next: { status: string }[]): boolean =>
76
- !!prev && next.some((t, i) => prev[i]?.status === "complete" && t.status === "in_progress");
74
+ export const countReopens = (prev: { name?: string; status: string }[] | undefined, next: { name?: string; status: string }[]): number => {
75
+ const complete = new Set(prev?.filter((t) => typeof t.name === "string" && t.status === "complete").map((t) => t.name));
76
+ return new Set(next.filter((t) => typeof t.name === "string" && complete.has(t.name) && (t.status === "failed" || t.status === "in_progress")).map((t) => t.name)).size;
77
+ };
77
78
 
78
79
  // The text an edit/write call inserts: joined newText values, or the write body.
79
80
  export const insertedText = (input: unknown): string => {
@@ -239,3 +239,17 @@ test("legs round-trip and malformed stored legs are rejected", () => {
239
239
  assert.equal(parseRecord(serializeRecord({ ...rec, legs } as any)), undefined);
240
240
  }
241
241
  });
242
+
243
+ test("derive and YAML preserve verification runs and legacy absence", () => {
244
+ for (const count of [undefined, 0, 4]) {
245
+ const rec = newRecord({ spec: "doc/specs/a.md", session: "s1", now: "2026-09-17T10:00:00Z", runId: "r1" });
246
+ if (count !== undefined) rec.derived.verification_runs = count;
247
+ rec.derived = derive(rec, "2026-09-17T10:01:00Z");
248
+ const loaded = parseRecord(serializeRecord(rec))!;
249
+ assert.equal(loaded.derived.verification_runs, count);
250
+ loaded.status = "shipped";
251
+ loaded.shipped_at = "2026-09-17T10:02:00Z";
252
+ loaded.derived = derive(loaded, loaded.shipped_at);
253
+ assert.equal(parseRecord(serializeRecord(loaded))!.derived.verification_runs, count);
254
+ }
255
+ });
@@ -93,7 +93,7 @@ export interface Derived {
93
93
  engine: "loop" | "waves"; iterations?: number;
94
94
  duration_s: number; phases: Partial<Record<PhaseKey, PhaseAcc & { started_at?: string; completed_at?: string; duration_s?: number; model?: string; thinking?: string }>>;
95
95
  personas: Record<string, PersonaAcc>; reviews?: Record<string, ReviewAcc>; conformance_loops: number; conformance_open_gaps?: number; gates: Gates & { ship_option?: string };
96
- plan?: { tasks: number; complete: number; failed: number; skipped: number }; tests?: { command: string; result: "pass" | "fail" }; amendments: number;
96
+ plan?: { tasks: number; complete: number; failed: number; skipped: number }; tests?: { command: string; result: "pass" | "fail" }; verification_runs?: number; amendments: number;
97
97
  spec_edits_after_ship: number; diff?: DiffSummary; modified_files?: string[]; council?: CouncilBlock; spec_writes: Accumulators["spec_writes"]; events_dropped: number;
98
98
  }
99
99
  export type RecordStatus = "in_progress" | "shipped" | "abandoned";
@@ -130,7 +130,7 @@ export function derive(rec: TelemetryRecord, now: string): Derived {
130
130
  }
131
131
  for (const k of Object.keys(acc.phases) as PhaseKey[]) phases[k] = compact({ ...(phases[k] ?? {}), ...acc.phases[k] });
132
132
  const live = liveShipEvent(rec.events);
133
- return compact({ engine: rec.events.findLast((e) => e.kind === "engine")?.engine ?? "waves", iterations: acc.personas["loop-implementer"]?.dispatches, duration_s: seconds(rec.created_at, rec.shipped_at ?? rec.abandoned_at ?? now), phases, personas: acc.personas, reviews: Object.keys(acc.reviews).length ? acc.reviews : undefined, conformance_loops: acc.conformance_loops, conformance_open_gaps: acc.conformance_open_gaps, gates: compact({ ...acc.gates, ship_option: live?.option }), plan: rec.derived.plan, tests: rec.derived.tests, amendments: acc.amendments, spec_edits_after_ship: acc.spec_edits_after_ship, diff: rec.derived.diff, modified_files: rec.derived.modified_files, council: rec.derived.council, spec_writes: acc.spec_writes, events_dropped: acc.events_dropped }) as Derived;
133
+ return compact({ engine: rec.events.findLast((e) => e.kind === "engine")?.engine ?? "waves", iterations: acc.personas["loop-implementer"]?.dispatches, duration_s: seconds(rec.created_at, rec.shipped_at ?? rec.abandoned_at ?? now), phases, personas: acc.personas, reviews: Object.keys(acc.reviews).length ? acc.reviews : undefined, conformance_loops: acc.conformance_loops, conformance_open_gaps: acc.conformance_open_gaps, gates: compact({ ...acc.gates, ship_option: live?.option }), plan: rec.derived.plan, tests: rec.derived.tests, verification_runs: rec.derived.verification_runs, amendments: acc.amendments, spec_edits_after_ship: acc.spec_edits_after_ship, diff: rec.derived.diff, modified_files: rec.derived.modified_files, council: rec.derived.council, spec_writes: acc.spec_writes, events_dropped: acc.events_dropped }) as Derived;
134
134
  }
135
135
 
136
136
  const stripUndefined = (v: unknown): unknown => {
@@ -8,6 +8,7 @@ import { createHash } from "node:crypto";
8
8
  import { tmpdir } from "node:os";
9
9
  import { isAbsolute, join } from "node:path";
10
10
  import registerPhaseTracker from "./phase-tracker.ts";
11
+ import registerPlanTracker from "./plan-tracker.ts";
11
12
 
12
13
  const originalSubagentDepth = process.env.PI_SUBAGENT_DEPTH;
13
14
  process.env.PI_SUBAGENT_DEPTH = "0";
@@ -531,6 +532,50 @@ test("plan activity auto-completes implement with complete+skipped, not with fai
531
532
  assert.equal(await implementStatus([{ status: "complete" }, { status: "skipped" }], true), "in_progress");
532
533
  });
533
534
 
535
+ test("ledger refresh completes only started implement and keeps failed, in-flight and Gn work open", async () => {
536
+ for (const allCut of [true, false]) {
537
+ const dir = tempCwd({ piGauntlet: { flowGuards: { enforce: false } } });
538
+ const path = join(dir, "ledger.md");
539
+ const save = (status: string, pending = "") => writeFileSync(path, `# Ledger - loop engine\n${pending}\n| row | status | attempts | evidence |\n|---|---|---|---|\n| 1 | ${status} | 0 | check |\n\n## Decisions\n`);
540
+ save(allCut ? "cut" : "failed");
541
+ const branch = implementBranch().slice(0, -1);
542
+ const h = harness({ cwd: dir, branch });
543
+ registerPlanTracker(h.pi as any);
544
+ await h.emit("session_start");
545
+ const tracker = h.tools.find(t => t.name === "plan_tracker")!;
546
+ const phase = h.tools.find(t => t.name === "phase_tracker")!;
547
+ const status = async () => (await phase.execute("status", { action: "status" }, undefined, undefined, h.ctx) as any).details.phases.implement.status;
548
+ const refresh = async (target = path) => {
549
+ const result = await tracker.execute("refresh", { action: "refresh", path: target }, undefined, undefined, h.ctx) as any;
550
+ branch.push({ type: "message", message: { role: "toolResult", toolName: "plan_tracker", details: result.details } } as any);
551
+ await h.emitEvent("tool_execution_end", { toolName: "plan_tracker", result });
552
+ return result;
553
+ };
554
+ assert.equal((await refresh()).details.error, undefined);
555
+ assert.equal(await status(), "pending", "refresh before start cannot complete implement");
556
+ assert.equal((await startPhase(h, "implement")).status, "in_progress");
557
+ await refresh();
558
+ if (!allCut) {
559
+ assert.equal(await status(), "in_progress", "failed row keeps implement active");
560
+ save("failed", "pending: 1 | home | abc1234");
561
+ const inFlight = await refresh();
562
+ assert.equal(inFlight.details.tasks[0].status, "in_progress");
563
+ assert.equal(await status(), "in_progress");
564
+ assert.ok((await refresh(join(dir, "missing.md"))).details.error);
565
+ assert.equal(await status(), "in_progress", "invalid refresh cannot complete implement");
566
+ await tracker.execute("add", { action: "add", tasks: ["G1: conformance"] }, undefined, undefined, h.ctx);
567
+ save("done");
568
+ const withGap = await refresh();
569
+ assert.equal(withGap.details.tasks[1].status, "pending");
570
+ assert.equal(await status(), "in_progress", "kept Gn task blocks completion");
571
+ await tracker.execute("update", { action: "update", index: 1, status: "complete" }, undefined, undefined, h.ctx);
572
+ await refresh();
573
+ }
574
+ assert.equal(await status(), "complete");
575
+ assert.equal((await startPhase(h, "verify")).status, "in_progress");
576
+ }
577
+ });
578
+
534
579
  test("cold implement and verify completions ignore unfinished snapshots", async () => {
535
580
  for (const phase of ["implement", "verify"] as const) {
536
581
  const h = harness({