pi-gauntlet 8.2.1 → 8.4.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +8 -0
- package/README.md +5 -5
- package/agents/loop-implementer.md +19 -4
- package/bin/gauntlet-performance.mjs +477 -64
- package/bin/gauntlet-performance.test.mjs +27 -27
- package/bin/gauntlet-spec-index.mjs +290 -47
- package/bin/gauntlet-spec-index.test.mjs +42 -18
- package/bin/gauntlet-telemetry-seal.mjs +220 -77
- package/bin/gauntlet-telemetry-seal.test.mjs +222 -457
- package/extensions/lib/gauntlet-settings.test.ts +12 -14
- package/extensions/lib/gauntlet-settings.ts +7 -16
- package/extensions/lib/plan-check.test.ts +10 -0
- package/extensions/lib/plan-check.ts +2 -2
- package/extensions/lib/telemetry-branch.test.ts +102 -0
- package/extensions/lib/telemetry-branch.ts +145 -0
- package/extensions/lib/telemetry-diff.test.ts +2 -2
- package/extensions/lib/telemetry-diff.ts +1 -1
- package/extensions/lib/telemetry-leg.ts +163 -0
- package/extensions/lib/telemetry-record.ts +2 -0
- package/extensions/lib/vcs.ts +6 -23
- package/extensions/telemetry-leg-failures.test.ts +138 -0
- package/extensions/telemetry-leg-fields.test.ts +130 -0
- package/extensions/telemetry-leg-no-write.test.ts +90 -0
- package/extensions/telemetry-leg-rounds.test.ts +157 -0
- package/extensions/telemetry-leg-trackers.test.ts +114 -0
- package/extensions/telemetry-leg.test.ts +151 -0
- package/extensions/telemetry-settlement-contract.test.ts +67 -0
- package/extensions/telemetry.test.ts +44 -27
- package/extensions/telemetry.ts +68 -48
- package/extensions/test-support/git-env.mjs +13 -0
- package/package.json +1 -1
- package/skills/brainstorming/SKILL.md +3 -1
- package/skills/check-delivery/SKILL.md +2 -0
- package/skills/finishing-a-development-branch/SKILL.md +13 -13
- package/skills/finishing-a-development-branch/reference/verification-set.md +2 -2
- package/skills/gatekeep-pr/reference/batch.md +3 -1
- package/skills/gatekeep-pr/reference/cleanup.md +1 -1
- package/skills/gatekeep-pr/reference/post-selection-loop.md +7 -1
- package/skills/gauntlet-resume/reference/reconstruction.md +6 -6
- package/skills/gauntlet-resume/reference/seed.md +1 -6
- package/skills/loop-implementation/SKILL.md +40 -21
- package/skills/loop-implementation/reference/action-spend.md +10 -0
- package/skills/loop-implementation/reference/coordinated-legs.md +5 -5
- package/skills/loop-implementation/reference/interrupted-recovery.md +10 -0
- package/skills/loop-implementation/reference/unreachable-returns.md +13 -0
- package/skills/shape-ticket/SKILL.md +1 -1
- package/skills/shape-ticket/reference/repo-audit.md +2 -2
- package/skills/verification-before-completion/reference/conformance-check.md +1 -1
package/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,13 @@
|
|
|
1
1
|
# Changelog
|
|
2
2
|
|
|
3
|
+
## v8.4.0 - 2026-11-07
|
|
4
|
+
|
|
5
|
+
- Telemetry publishes run and per-session skill records on the orphan `gauntlet/telemetry` branch, outside product commits. `telemetry.remote` replaces `telemetry.dir`; readers refresh the branch and cost digests allocate ticket and PR rounds, including unattributed shares. Finish and resume use branch records and full-checkout verification proofs. (#124)
|
|
6
|
+
|
|
7
|
+
## v8.3.0 - 2026-11-06
|
|
8
|
+
|
|
9
|
+
- Loop execution uses the ledger allowance for interrupted child and seed recovery, exit-verification repairs and grant-funded conformance rounds. Recovery requires checkout ownership and confirmed job exit; stagnation supplies a row-local hint rather than stopping early. Unreachable target failures become audited scope cuts under amend permission, while exhausted named-action bounds still stop. Loop workers bound blocking calls and track longer jobs in isolated POSIX sessions; design approval authorizes named paid or external actions only within their flow-wide bounds.
|
|
10
|
+
|
|
3
11
|
## v8.2.1 - 2026-11-06
|
|
4
12
|
|
|
5
13
|
- Shape-ticket's AC integrity gate drops workflow-stage proof after splitting bundled observations, retains retired proof in the Reporter note, and routes an emptied AC list to recovery or park.
|
package/README.md
CHANGED
|
@@ -37,9 +37,9 @@ Concretely, one change through the gauntlet:
|
|
|
37
37
|
|
|
38
38
|
0. *(Optional)* Before there's even a spec, `/skill:shape-ticket` can create one tracker issue or repair a comma/whitespace list of issue refs - shaping each into a Context/Problem/Idea/Acceptance Criteria ticket, grounded by a repository audit, an AC integrity check, and a cheap council roast. Repair finishes preparation for all tickets before presenting exact-diff confirmations sequentially in ref order; each approval writes that ticket before the next gate, and may include one optional gated comment for a Reporter note or overlap evidence. The audit checks factual claims, Idea fit, AC mechanisms against cited repo guidance, and already-stated observable legs in configured sibling checkouts. It reports in-motion work and attempts a best-effort scenario through authorized current-system reads, without promising reproduction or a fix; missing access or nothing to observe is reported, not a new approval gate. Implementer detail moves into the gated Reporter note; brainstorming reads that note with the ticket so the spec retains the detail. A failed roast is retried once, then surfaced inline at the gate if it fails again. It's a tool, not a phase: no worktree, no plan/phase tracker, runs from any repo state. The agent may start it on its own when the ask is "file a ticket" or another skill routes there; the write still waits for your yes on the presented diff. Similarly, `/skill:chase-bug` triages a raw bug report into an evidenced verdict - and can hand off to shape-ticket, brainstorming, or a bounded hotfix - before any spec exists.
|
|
39
39
|
1. Opt in with `/skill:brainstorming` and describe the change. The command is the reliable discovery-and-entry path; explicit prose requesting brainstorming/the gauntlet also supplies consent when the skill is already loaded or an available instruction or route identifies it. An ordinary implementation request or an agent recommendation is not consent. A leading run of GitHub references (`#N`, `owner/repo#N`, or a `/pull/N` or `/issues/N` URL), separated by whitespace or commas, resolves each reference as a PR first: related open, same-repository, non-fork PRs on one base form one rework subject, including drafts. Admission runs in listed order and relatedness is checked before provisioning; the first retained PR is home, whose existing branch rebases onto the fetched PR base by default, then member commits replay onto it in listed order. The first intent word `merge-forward` opts out and keeps the merge path. An unresolved conflict after its scoped helper aborts that step and continues to the spec with the repair as Task 1. A single non-PR reference follows the ticket path. The first non-reference token starts intent text; mid-sentence references and other ticket/free-text subjects keep their existing paths. The combined code, PR-carried specs, claims, and review threads are the brainstorm input, with each PR's provenance retained; only home's bound ticket supplies acceptance criteria, while member tickets supply context. A single PR retains the same rework path without member replays or a close offer. The combined state is judged against constraints from the fetched current base HEAD, not the fork point or the PRs' proposed policy edits; see the [PR-subject contract](skills/brainstorming/reference/pr-subject.md) and [PR-head worktree owner](skills/using-git-worktrees/reference/pr-head.md). **`brainstorming`** sets up an isolated worktree (on the existing branch for PR rework), explores the codebase, and turns the subject into a written spec - when its recon finds a cited case that the ask is the wrong change (misdiagnosed, wrong-sized, already solved, a band-aid, or cutting across a sound pattern - including a repo convention that is itself debt) and can name the alternative, question one is a framing question (`as framed`, `pivot: <alternative>`, or `smaller or none`); otherwise it opens with one sentence naming what it checked, and every approach it proposes ends with a `Pattern:` line naming what it follows or cuts across; when the run starts from a ticket, the spec carries the ticket's acceptance criteria verbatim, each with a disposition (`in-scope`, `deviates:`, `deferred:`, `venue:`, `elsewhere:` for a row another repo's part delivers), scope cuts come only from your answer to a cut question and are listed at the gate, the conformance gate checks the in-scope ones, and `/skill:check-delivery` verifies `venue:` rows before the prod deploy. A multi-model critique runs on it automatically. If the spec replaces a known prior spec, brainstorming marks the predecessor with a `> **Superseded by:**` banner under its title (default format, syntax overridable via `.pi/gauntlet-overrides.md`; candidates come from brainstorming's scout recon, never a mechanical sweep). **You read and approve the spec - human gate 1.** Options 1/2/4/5 approve and pick an engine; option 6 approves the content only: it writes `**Engine:** deferred` under `**Amend-grant:** none`, pushes the branch, opens a draft PR carrying just the spec, prints the URL, and ends the flow - no plan, no loop, no handoff brief. For a coordinated spec, it first pushes every companion branch to origin, then opens one home draft PR; it creates no companion PRs. A later `brainstorming #N` reconstructs the recorded companions from their origin branches before re-presenting the implementation gate; the [publication and re-gate owner](skills/brainstorming/reference/spec-pr.md) defines all-member preflight, recovery, and cleanup. The branch stays; a gauntlet-owned worktree is removed, a host-owned checkout is reported and left. Option 6 is withheld on a PR subject (`Withheld: draft PR - #N already exists`) and when the branch's diff against the default branch is not spec-only. A later `brainstorming #N` on that PR classifies it from the diff alone: one added spec carrying the marker plus banner-only predecessor edits re-presents the gate with 1/2/4/5 (3 only when the spec commit carries a council audit) and skips gather, questionary, council, and scout; a spec-only diff without the marker prints `spec-only PR without the gate marker - running lint and council` and takes the normal brainstorm path with the carried spec as the draft; any other diff takes the rework path. Approval at that re-gate writes `**Engine:**`, `**Amend-grant:**`, and `**PR:** <owner/repo#N>` as a new commit on the PR branch and chains into planning or the loop as usual. Before approval, PR rework's sole write/run exception outside the spec directories permits only the base sync (rebase or merge), each member replay or merge, their conflict helpers' edits to the conflicted files, and their scoped conflict tests; other subjects have no implementation code yet.
|
|
40
|
-
2. By default, **`writing-plans`** decomposes the approved spec into atomic, independently-verifiable tasks, grouped into parallel waves where they don't touch the same files. Experimental approval options 4/5 instead select **`loop-implementation`**: the plan phase is skipped and a stamped acceptance-criteria ledger replaces the task plan. Option 4 retains amendment review; option 5
|
|
40
|
+
2. By default, **`writing-plans`** decomposes the approved spec into atomic, independently-verifiable tasks, grouped into parallel waves where they don't touch the same files. Experimental approval options 4/5 instead select **`loop-implementation`**: the plan phase is skipped and a stamped acceptance-criteria ledger replaces the task plan. Option 4 retains amendment review; option 5 auto-applies amends, including unreachable-row cuts, and funds conformance rounds up to the allowance when the cap blocks. Redraws, exhausted bounds and unproven recovery still stop. See [Loop engine operations](doc/configuration.md#loop-engine-operations).
|
|
41
41
|
3. **`subagent-driven-development`** executes the plan in waves, with task implementers followed by one integrated `wave-reviewer` after the scoped test gate passes. The reviewer checks anchored spec clauses and code quality together; repairs stay on the current tree and pass the same gate before re-review. TDD-locked: red, green, refactor. Independent implementation, review, and council batches remain parallel, but their dispatches are foreground: the orchestrator waits for terminal results before accepting work or advancing a phase.
|
|
42
|
-
The loop alternative to step 3 runs one fresh `loop-implementer` per seed or row iteration, without implement-phase reviewers. Each row starts with the shared allowance (default 4); approval replies such as `4; allowance 6` override it for one run. Under either loop approval option, attempt `escalateAfter + 1` (default 4), clamped to the last allowed attempt, escalates once per row on `escalationLoop.implModel` (the main-loop model when unset); later attempts use the ordinary worker model
|
|
42
|
+
The loop alternative to step 3 runs one fresh `loop-implementer` per seed or row iteration, without implement-phase reviewers. Each row starts with the shared allowance (default 4); approval replies such as `4; allowance 6` override it for one run. Under either loop approval option, attempt `escalateAfter + 1` (default 4), clamped to the last allowed attempt, escalates once per row on `escalationLoop.implModel` (the main-loop model when unset); later attempts use the ordinary worker model. The allowance also bounds seed and exit-verification repair counts; verification repairs use the same escalation model and count home repair commits plus an unmatched `pending: verify` reservation. Stagnation supplies a row-local hint, not an early stop. Interrupted returns spend attempts after ownership-fenced recovery: the driver confirms owned jobs exited before resetting the checkout; unknown ownership or failed cleanup requires a human decision. An admitted `unreachable:` target failure becomes an audited scope cut under amend permission, never a passing row. At a row stop, `retry <n> | <hint>` adds one attempt to that row, while `rows +N` raises every row's allowance. Attempts never reset and there is no total iteration cap. Ledger refresh completes implement when every row is closed; verify step 1 runs the gate set: deduplicated (owning checkout, command) pairs from done rows' recorded checks, with ownership from the ledger. Gate output goes to logs; the driver keeps exit codes and paths, not test output. CI on push owns the full run.
|
|
43
43
|
|
|
44
44
|
4. **verify**: the parent first runs the plan's full verification command set for waves, or the ledger gate set for loop; only a passing result permits whole-diff code review, then the **conformance gate**. A review fix invalidates that result, so the parent reruns the same set before the next review or conformance gate. Loop conformance repairs and finish also use the gate set, never the ledger's full `verification:` command. When the overrides file declares a happy path for the touched paths (see [Project-specific overrides](#project-specific-overrides)), the parent runs it once here, between code review and the conformance gate, and the reviewer reads its transcript as runtime evidence - a hang or a failed request shows up in the audit with the transcript quoted - as a gap when an origin clause ties to it, otherwise as a `happy-path: failed - unattributable` line - never as a silent green. The conformance subagent reads the finished code and docs against your *original words* from step 1, not the plan, and reports per-requirement: delivered, partial, missing, drifted, or unauthorized. Unauthorized rows - shipped surface no human input asked for, whether it crept in or was laundered through the spec - follow their recommendation like every other row: a contained removal auto-runs, anything another requirement leans on is deferred with a plain-language "I'd cut it / I'd keep it" recommendation. Inside a brainstorming-entered flow this gate is machine-blocked from being skipped. Compatible executable recommendations auto-run through an isolated fix-and-re-audit loop with no prompt; anything still open surfaces as a numbered decision brief - one item per decision, the problem in plain words first, lettered options each with a one-line consequence, and `Recommended: <letter> - <reason>` or `No recommendation - pick one`. Reply with one line per item (`1. B`), or bare `1` to take every recommendation (no-recommendation items are re-asked by number); a current `CONFORMS` / no-concerns result goes straight to the branch options with no extra conformance sign-off.
|
|
45
45
|
5. **`finishing-a-development-branch`**: PR, draft PR, squash, keep, or discard; PR rework's row 1 offers a lease push of the rebased home branch to the existing PR (`Base:` records in `merge:` form push non-force). A spec whose header carries `**PR:**` (approved at a spec-PR re-gate) and no `## Rework record` replaces the two create rows with `Push to PR #N and mark it ready for review` and `Push to PR #N, keep it a draft` (`Push to PR #N (already ready for review)` when the PR is already ready): both push without force after strip-and-seal, the first then runs `gh pr ready`, neither rewrites the PR body, and both are withheld when origin's branch head is no longer an ancestor of local HEAD. For coordinated specs, these [header-linked rows 1/2](skills/finishing-a-development-branch/reference/spec-pr.md#push-rows-12) deliver companions in recorded order before one home seal carrying every companion PR URL, then push home and apply its readiness choice. Row 1 makes companion drafts ready too; row 2 creates drafts but never converts an already-ready PR back to draft. Companion URLs appear in the shipped summary, companions first and home last, not in the unchanged home PR body; keep that summary for cross-repo review because successful home delivery removes receipts. A spec still carrying `**Engine:** deferred` stops finishing (and `gauntlet-resume`) with `spec approved for handoff only - run brainstorming #N` before any phase change, verification, or push. The PR-rework push appends a `## Rework` block naming absorbed PRs while retaining the original description bytes; the header-linked `**PR:**` push leaves the body untouched. Only after a successful rework push and home-description edit does one gated confirmation show the exact close comments and eligible member targets, linking home; declining or any other finish option leaves members open. Closing never deletes member branches, and finish posts no review-thread replies. Its `Before:` handle includes the full portable pre-rework SHA and the local `refs/gauntlet/pre-rework/pr-<N>` ref in the rework checkout. The [finishing owner](skills/finishing-a-development-branch/SKILL.md#pr-subject-row-1-push-to-the-pr-branch) defines the landing and body-update contract. Once a PR exists, run `/skill:gatekeep-pr <pr>` to verify it before merging - it judges the diff against the spec at the PR's head (the acceptance-criteria dispositions and Design you approved; the PR description when no spec applies) - rows the spec defers or deviates are settled, every other row needs its mechanism (code, tests, docs) - and against the rubric; whatever can be seen only after merge is left to step 6. **Human gate 2** - the only other decision you make.
|
|
@@ -99,7 +99,7 @@ pi-gauntlet ships three kinds of pieces, layered on top of pi-cohort's dispatch:
|
|
|
99
99
|
|
|
100
100
|
- **21 skills** - the workflow logic. Sixteen activate automatically when pi sees the matching kind of task, and the workflow skills chain through their gates: `shape-ticket` (creates one tracker issue or repairs a comma/whitespace list of refs against a Context/Problem/Idea/Acceptance-Criteria template, with a grounded repo audit, an AC integrity check, a cheap council roast, and sequential per-ticket exact-diff write confirmations; model-invocable so other skills and commands can route "file a ticket" to it, the gate still needs your yes), `gatekeep-pr` (model-invocable on an explicit ask that names the PR and the verb; the gates still need your selection - is consent-gated pre-merge verification of a PR against the scope contract at its head - the approved spec's acceptance-criteria dispositions and Design when the PR carries one, else the PR's stated intent; the ticket is cross-checked, never judged here (`check-delivery` owns it) - read-only gathering, verification evidence resolved CI-first (green checks on the exact assessed head count as evidence; a pending commit status that GitHub does not report as blocking the gating actor (`mergeStateStatus` not `BLOCKED`) is not waited for - external-CI commit statuses included - while a running Actions check, an unknown merge state, and every other merge precondition still withhold; the project's verification command runs only as fallback), a rubric-based review, a compact bottom-up report (what the PR delivers, including rows deferred or deviating per spec, PR comments, nits, blockers, then a `mergeable` / `fixable` verdict) and course and terminal menus whose rows derive from the actor's permissions on the PR; nothing mutates (repairs, pushes, reviews, merges, tracker comments) until you pick a course or terminal row, except the opt-in `--rebase [base]` sync that rebases, reviews, and lease-pushes the head before verification, or holds the rebased head locally when a conflict stop's scoped tests fail, so a selected delivery course resolves it before the lease push; after your own merge two opt-in rows post the `AC coverage` block to the ticket or the PR; own-merge, approved, and closed-PR menus offer a recommended `cleanup` row that tears down the gate's worktree and branch and runs the overrides file's `## cleanup` steps - run it with `/skill:gatekeep-pr [<pr>] [--rebase [base]]` (no argument: your open PRs)), `check-delivery` (model-invocable on an explicit ask that names the ticket and the verb; the gates still need your selection - is a post-merge detective control: proves an issue actually shipped (default-branch landing, delivery target, per-AC evidence) before its tracker status advances; it never writes a terminal status - run it with `/skill:check-delivery <ref>` or, in pi, `/skill:check-delivery #12 #15` for fresh parallel verification and one gate and write per ticket in turn), `writing-plans`, `roasting-the-spec`, `test-driven-development`, `subagent-driven-development`, `loop-implementation` (the plan-free implement driver that iterates a fresh `loop-implementer` over the spec's acceptance-criteria ledger when approval picks the loop engine), `dispatching-parallel-agents`, `verification-before-completion`, `requesting-code-review`, `receiving-code-review`, `using-git-worktrees`, `finishing-a-development-branch`, `forge-skill`, `linear` (reads/searches/comments on/manages Linear tickets via the `linearis` CLI; owns all linearis mechanics and the `## Issue tracker` overrides schema; tracker-facing skills route to it). Five more require explicit entry (`disable-model-invocation: true`): `brainstorming` starts the gated workflow on explicit consent - use `/skill:brainstorming` for reliable discovery, or opt in through reachable explicit prose or a human-selected handoff; after spec approval, downstream planning and implementation still chain automatically. `chase-bug` is human-only bug triage: read-only root-cause discovery to an evidenced verdict menu (real bug -> ticket/brainstorm/hotfix/respond; five negative verdicts), then a gated response to the reporter for addressable origins (GitHub issue / tracker ticket) and a rendered verdict summary otherwise - it never fixes during triage; the hotfix row hands off to `skills/chase-bug/hotfix.md` after the menu - run it with `/skill:chase-bug`. `gauntlet-performance` reads the committed run telemetry (current repo by default, `--dir <path>` adds others) through the parse-only `gauntlet-performance` CLI and answers with one example-led recommendation, 3-5 cornerstone numbers, a council assessment when available, and a menu of at most three actions (render a report file, open the recommendation as a ticket via `shape-ticket`, drill into one run); it writes nothing unless you pick render - run it with `/skill:gauntlet-performance [--dir <path>]... [--since <version>]`. `gauntlet-handoff` ends a session whose flow a fresh session will continue: it invokes pi-cohort's `handoff` skill (`--out <path>` or `--key <name>`, default key = the run worktree's branch with `/` flattened to `-`, default file `<tmpdir>/pi-handoff/<key>.md`), then appends the gauntlet process-state section (phase/plan tracker status) per the shared contract `skills/gauntlet-resume/reference/brief-contract.md` - run it with `/skill:gauntlet-handoff [--out <path> | --key <name>]`. `gauntlet-resume` is the only way back into an interrupted flow from a fresh session: it takes a `gauntlet-handoff` brief (a file path, pasted text, or - with no arguments - a pick from the briefs under `<tmpdir>/pi-handoff/`), a bare worktree that already holds a spec, or a spec tracked on main under a configured spec dir (a spec seed: it creates or reuses the worktree through `using-git-worktrees` and pins the spec), restores phase/plan tracker state through the legal arming sequence (`start brainstorm`, `skip` with `resume:` reasons, `plan_check` before implement-or-later), and never infers approval from artifacts - run it with `/skill:gauntlet-resume [<brief-file> | <spec>.md | <worktree-name-or-path>]`.
|
|
101
101
|
- **11 subagent personas** - the specialized child agents the skills dispatch via pi-cohort: `implementer`, `loop-implementer`, `wave-reviewer`, `conformance-reviewer`, `spec-summarizer`, `plan-writer`, `spec-council-member`, `spec-council-synthesizer`, `spec-council-applier`, `pr-gatekeeper`, `delivery-checker`. See [doc/personas.md](./doc/personas.md) for what each one does and why its permissions are scoped the way they are.
|
|
102
|
-
- **4 runtime extensions** - the enforcement layer. `plan-tracker` and `phase-tracker` are tools skills call to track progress (with a TUI widget); `verify-before-ship` is a hook that warns if you push or open a PR without a passing test run since your last edit (expected when a loop gate set does not match its patterns; the scoped `Verification:` line answers it, not a full local run); a phase-tracker flow guard reminds on implement-phase commits missing spec/code review. Both engines use the tracker: waves manages plan tasks, while loop calls `plan_tracker refresh` to project ledger rows and preserve conformance tasks. Loop implement completes when every row is closed, then verify step 1 runs the ledger's gate set in the owning checkouts. In a brainstorming-entered flow, phase-tracker rejects `implement` or `verify` completion while tracker tasks remain pending or in progress; see [its configuration reference](./doc/configuration.md#phase-tracker). phase-tracker also registers `plan_check`, which verifies a plan against its spec and against the grammar in [skills/writing-plans/reference/plan-contract.md](./skills/writing-plans/reference/plan-contract.md), including that each task's `Tests:` commands are selective and never the full suite, and that every test command carries a passing base-commit result; a pass stamps the plan and spec content hashes for implementation and returns a bounded task manifest; see the [manifest contract and size limit](./doc/configuration.md#phase-tracker). `telemetry`
|
|
102
|
+
- **4 runtime extensions** - the enforcement layer. `plan-tracker` and `phase-tracker` are tools skills call to track progress (with a TUI widget); `verify-before-ship` is a hook that warns if you push or open a PR without a passing test run since your last edit (expected when a loop gate set does not match its patterns; the scoped `Verification:` line answers it, not a full local run); a phase-tracker flow guard reminds on implement-phase commits missing spec/code review. Both engines use the tracker: waves manages plan tasks, while loop calls `plan_tracker refresh` to project ledger rows and preserve conformance tasks. Loop implement completes when every row is closed, then verify step 1 runs the ledger's gate set in the owning checkouts. In a brainstorming-entered flow, phase-tracker rejects `implement` or `verify` completion while tracker tasks remain pending or in progress; see [its configuration reference](./doc/configuration.md#phase-tracker). phase-tracker also registers `plan_check`, which verifies a plan against its spec and against the grammar in [skills/writing-plans/reference/plan-contract.md](./skills/writing-plans/reference/plan-contract.md), including that each task's `Tests:` commands are selective and never the full suite, and that every test command carries a passing base-commit result; a pass stamps the plan and spec content hashes for implementation and returns a bounded task manifest; see the [manifest contract and size limit](./doc/configuration.md#phase-tracker). `telemetry` publishes YAML records on the orphan `gauntlet/telemetry` branch, outside the product diff: one sealed brainstorming run plus one session leg per settled `shape-ticket`, `gatekeep-pr` or `check-delivery` invocation. In-progress runs live in Git common-dir metadata; `gauntlet-telemetry-seal` publishes before landing without changing checkout HEAD or index. Publication failure warns without blocking ship; plain jj keeps local metadata and reports no Git remote. A fetched shipped record blocks a brainstorm `write` into that spec. See [the record layout, settings and migration guide](./doc/configuration.md#telemetry). See [doc/configuration.md](./doc/configuration.md) for the settings each one reads.
|
|
103
103
|
|
|
104
104
|
pi-gauntlet is **opinionated**: it recommends this pipeline for non-trivial changes, but starts `brainstorming` only after explicit user consent. Enforcement is opt-in by entry, not ambient: once brainstorming starts a flow, the phase-tracker extension mechanically blocks a phase from closing before its gate runs, and warns once if the main loop writes code during implement (subagents own implement-phase edits). A change made *without* entering the flow (a typo, a formatting run, a dependency bump - see "When to use / when NOT to use") is not gated; the discipline of routing real work through the pipeline is a convention the tooling supports, not a trap it springs on every edit.
|
|
105
105
|
|
|
@@ -170,7 +170,7 @@ Pin an exact release with `npm:pi-gauntlet@X.Y.Z`. See [doc/install-internals.md
|
|
|
170
170
|
|
|
171
171
|
## Performance digest
|
|
172
172
|
|
|
173
|
-
`gauntlet-performance` digests telemetry records without judging them: `node <pi-gauntlet-package>/bin/gauntlet-performance.mjs [--dir <
|
|
173
|
+
`gauntlet-performance` digests telemetry records without judging them: `node <pi-gauntlet-package>/bin/gauntlet-performance.mjs [--dir <telemetry dir>]... [--since <version>] [--json]` prints a `corpus:` line, one row per run (`run_id`, repo, spec, version, status - `shipped*` when the record has no ship phase - wall, per-phase minutes, tokens, cost, models, dispatches, `disp/task`, `coord`, `fix_round_grants`, reopens, conformance loops, reviewer findings, council dispatches, and `big` - the count of parent calls whose cache write exceeded 50k tokens with their share of the run's parent cache-write tokens (`-` for records without `parent_calls`)), a `by version` block of p50/max over shipped, non-truncated runs, a `council` block grouped by member roster, and `skipped:` lines for files it could not use. Each council table shows member `total`, `unique`, `applied`, `uniq_appl`, `deferred`, `rejected` blocker/major/minor counts and `uniq_appl_nonminor/run`; `chair:` shows runs, average clusters and members reported, and retried runs. Roster flags (JSON `flags[].rule`: `low_unique_applied` - fewer than 0.2 unique-and-applied blocker/major findings per run; `high_rejection` - over 50% rejected while another member is below 25%) require five runs and print as `flag: <member> - <detail>`. Smaller groups print `not enough runs to assess (N of 5)`; absent council data prints `no council data in selected records`. JSON includes `council: [{ roster, runs, members, chairs, flags }]`. By default it refreshes the configured `gauntlet/telemetry` branch and reads under the current repository's own `<owner>/<name>/` prefix; `--dir <path>` reads a bare directory of YAML records without fetching. Network failure warns and serves the last fetched ref, or an empty corpus when none exists. The `tickets`, `prs` and `unattributed` tables include session-leg costs and split shared batch costs evenly; see [allocation and record fields](doc/configuration.md#telemetry). `--since` compares versions numerically. Exit 0 always except usage errors. `/skill:gauntlet-performance` is the reasoning layer on top.
|
|
174
174
|
|
|
175
175
|
`disp/task` counts recorded implement-phase dispatch-result events divided by `plan.tasks - plan.skipped`; missing plan data, zero denominator or dropped events print `n/a`. `coord` is `yes` only for a non-empty sealed `legs:` list. Telemetry also records `verification_runs`: parent bash matcher hits from implement start through seal, not child checks or an assertion that verification passed. See [the telemetry reference](doc/configuration.md#telemetry) for the counter window and name-based reopen rules.
|
|
176
176
|
|
|
@@ -624,7 +624,7 @@ The conformance gate's model, the spec council's roster, and the phase-tracker's
|
|
|
624
624
|
| `verifyBeforeShip.testCommands` | verify-before-ship | the `DEFAULT_TEST_COMMANDS` patterns in `extensions/lib/gauntlet-settings.ts` (make, npm, pnpm, yarn, pytest, rspec, cargo, go) | patterns that count as a test run - [detail](./doc/configuration.md#verify-before-ship) |
|
|
625
625
|
| `verifyBeforeShip.warningReference` | verify-before-ship | unset | pointer shown with the warning - [detail](./doc/configuration.md#verify-before-ship) |
|
|
626
626
|
| `telemetry.enabled` | telemetry | `true` | record each flow - [detail](./doc/configuration.md#telemetry) |
|
|
627
|
-
| `telemetry.
|
|
627
|
+
| `telemetry.remote` | telemetry and readers | `origin` | separate telemetry repository URL - [detail](./doc/configuration.md#telemetry) |
|
|
628
628
|
| `telemetry.buckets` | telemetry | test / docs / config globs (`DEFAULT_TELEMETRY_BUCKETS`) | file classification - [detail](./doc/configuration.md#telemetry) |
|
|
629
629
|
|
|
630
630
|
For Markdown instructions rather than runtime settings, use [Project-specific overrides](#project-specific-overrides). Repo-local settings replace each configured second-level object wholesale; see the [deep reference](./doc/configuration.md).
|
|
@@ -17,7 +17,7 @@ Append one decision line per changed row to the ledger's `## Decisions` section
|
|
|
17
17
|
|
|
18
18
|
`row <n> <done|failed> | <one sentence: what was done> | <one sentence: reusable artifacts for done, retry guidance for failed>`
|
|
19
19
|
|
|
20
|
-
For `done`, name what later rows can build on (a file, helper, interface or fixture) and how to reuse it; write `nothing new to build on` when the row added no reusable artifact, including an extra closed by a check without further edits. For `failed`, state what the next iteration should not redo or should try differently. Limit each sentence to 200 characters. Use the seed contract below for `target: seed`.
|
|
20
|
+
For `done`, name what later rows can build on (a file, helper, interface or fixture) and how to reuse it; write `nothing new to build on` when the row added no reusable artifact, including an extra closed by a check without further edits. For `failed`, state what the next iteration should not redo or should try differently. Limit each sentence to 200 characters. Put action accounting inside the first sentence, not in a third slot. Use the seed contract below for `target: seed`.
|
|
21
21
|
|
|
22
22
|
## Inputs
|
|
23
23
|
|
|
@@ -30,8 +30,15 @@ Read additive inputs `Leg:`, `Rules:`, mapped original AC and, for companions, e
|
|
|
30
30
|
1. Read the given spec in full before reading the ledger or code. Read the given ledger next. Save the exact `pending:` tuple and bumped `iterations:` count before editing.
|
|
31
31
|
2. Close the `target` row first. Read every `row <n> ...` line for the target row from `## Decisions`, oldest first, before choosing an approach. Treat them as ordered guidance: avoid what a `failed` line's third slot says not to redo, follow what it says to try differently, and never repeat an approach a prior line of this row rules out. Exclude other rows' lines from guidance for this target. Read `retry` and `escalated` lines in the same pass: use the retry hint as guidance; treat the escalation marker as historical evidence that escalation is spent, and never infer the current attempt's model from it. Keep `previous decision` as the latest admitted target line and the tie-break for most recent. Use `previous decision` and `hint` to avoid repeating work and to try a different approach; keep the target unchanged.
|
|
32
32
|
3. Run `git diff --stat <base>`. Open full diff hunks only for files the row being worked on touches, using `git diff <base> -- <row files>`; read the relevant code before editing it.
|
|
33
|
-
4. Implement the target row using Three-scenario TDD and Test economy below. Treat empty evidence on an unreached row as a normal state after partial seeding, not a seed failure; create its first check before implementing it, then close or fail the row with command-first evidence under step 5. Run only its targeted check under Evidence command, never the owning checkout's full verification command or the full suite.
|
|
34
|
-
5. Update the target row's mutable cells: set `status` to `done` on a passing check or `failed` on a failing check, increment `attempts` by 1, and set `evidence` to one physical Markdown table cell opening with the command you ran under Evidence command, up to the first unescaped semicolon, plus at most three result lines separated by semicolons. Escape pipes as `\|` and use plain text without angle-bracket markup. On failure, record the concrete cause in plain words: the spec-vs-code contradiction, failing assertion, or missing credential, tool, or fixture. Keep reasoning and transcripts out of evidence.
|
|
33
|
+
4. Implement the target row using Three-scenario TDD and Test economy below. Apply step 5's action authorization before running a check. Treat empty evidence on an unreached row as a normal state after partial seeding, not a seed failure; create its first check before implementing it, then close or fail the row with command-first evidence under step 5. Run only its targeted check under Evidence command, never the owning checkout's full verification command or the full suite.
|
|
34
|
+
5. Before replacing an evidence cell, repeat each carried `ran <action> (<k> of <bound>)` and `bound spent: <action> ...` record verbatim in that row's own decision sentence before any new spend, including records from a successful seed; retain their ordinals and count copies only once. Treat a dropped record as a validation miss even when the replacement check passes. Update the target row's mutable cells: set `status` to `done` on a passing check or `failed` on a failing check, increment `attempts` by 1, and set `evidence` to one physical Markdown table cell opening with the command you ran under Evidence command, up to the first unescaped semicolon, plus at most three result lines separated by semicolons. Escape pipes as `\|` and use plain text without angle-bracket markup. On failure, record the concrete cause in plain words: the spec-vs-code contradiction, failing assertion, or missing credential, tool, or fixture. Keep reasoning and transcripts out of evidence. Run ordinary local checks as today. Apply the action rules below before any paid or external action; an approved spec clause in an acceptance row or Design must name its bound (runner, inputs, count per flow). Read all earlier decision lines, evidence cells, `loop: repair` commit bodies since base and caller-supplied home repair records in `hint` for `ran <action> (<k> of <bound>)` and `bound spent: <action> (<k> of <bound>)`, including other rows, across seed, rows, repairs and finish. Use the highest recorded k for each action, counting duplicate records of one spend once; use these records only for accounting, not target retry guidance. Before launching an action, append `bound spent: <action> (<k> of <bound>)` to the jobs file named under Evidence command and retain it through return; append `ran <action> (<k> of <bound>)` after confirmed execution. Preserve action records when removing exited process lines. Treat a reservation without confirmation as spent, not permission to rerun. With uncertain prior execution and neither a reservation nor affirmative no-action provenance, stop for parent recovery rather than infer zero spend.
|
|
35
|
+
|
|
36
|
+
| Observed action state | Result |
|
|
37
|
+
|---|---|
|
|
38
|
+
| Named, prerequisites present, count below bound | Run inside that bound without asking; append `ran <action> (<k> of <bound>)` within the row's decision sentence, with k one more than the highest recorded k (zero with no spend history). Put the same record in the evidence result line. Reuse its output for local rechecks rather than spending it again. |
|
|
39
|
+
| Concrete prerequisite no permitted attempt in this checkout can supply: credential, environment or data not present, or action the spec does not name | Record `unreachable: <what is missing> - <where it can run>` as the first result line after the check name; set the target `failed` and make no further attempt in this session. Name the missing prerequisite and runnable venue; do not ask the driver for permission. |
|
|
40
|
+
| Named action's count exhausted | Record `bound spent: <action> <k> of <k>` as the first result line and set the target `failed`; do not run the action or mark it unreachable. |
|
|
41
|
+
|
|
35
42
|
6. Append the target's decision line defined in Output contract. Only after the target is `done`, consider further open rows of your leg whose work sits in the files or helper just changed: a check already passing, or a small step on the same code. Take extras one at a time; write or name each row's selective check before implementing it, apply Three-scenario TDD, make it pass, record command-first evidence bounded as in step 5, and append its decision line. Set each extra `done` with attempts unchanged. Do not look elsewhere to pad the return or start a row you cannot finish in this session. Revert all code, test, cell and decision changes for an unfinished extra before committing; preserve the completed rows' changes.
|
|
36
43
|
7. Remove the `pending:` line from the ledger in your commit. Stage your row changes and the ledger explicitly. Commit exactly once with subject `loop: row <n> <status>`, naming the target only. Check `git status --porcelain` and leave the working tree clean.
|
|
37
44
|
8. Reply with the appended decision lines only, target first.
|
|
@@ -56,15 +63,23 @@ Keep and extend the test from earlier attempts on a retry; never start it over.
|
|
|
56
63
|
|
|
57
64
|
## Evidence command
|
|
58
65
|
|
|
66
|
+
Bound every blocking tool call with `timeout -k 30 <n>`, n at most 600 seconds: a foreground command, a poll, a wait. Include bookkeeping commands such as git reads, staging, commits and status, not just checks. Wrap shell builtins through `timeout -k 30 <n> bash -c '...'`. Keep the tool's own deadline above n + 30 and retain the executed timeout wrapper in evidence. Record a timeout as a failed attempt with `timed out after <n> s` in evidence; do not retry the timed-out check in this iteration.
|
|
67
|
+
|
|
68
|
+
Preflight python3 and timeout before launching longer work; missing facilities stop before spawning, without dependency installs, with the executed preflight as the evidence command. Start the scratch Python controller from a bounded launcher that returns immediately: use `subprocess.Popen(["python3", controller_path], start_new_session=True, stdout=controller_log, stderr=controller_log)`. Let the controller outlive the launcher's short timeout rather than wrapping the controller's lifetime in that timeout. Inside the controller use `subprocess.Popen(["timeout", "-k", "30", "3600", ...command], start_new_session=True)` and redirect stdout and stderr to a log under `${TMPDIR:-/tmp}`. Keep the controller alive to wait, reap and record the exit code. Immediately append `<pid> <process-start-identity> <command>` to `${TMPDIR:-/tmp}/loop-<base SHA>-<iteration>.jobs`, using the leg's base and the pre-bump task iteration, not the consumed ledger count. Capture the OS-observed process start identity and full command at launch; retain launch provenance, including the controller identity, for the parent. Preserve existing jobs lines. Run each poll in a separate tool call bounded as above, never a long polling loop inside one call.
|
|
69
|
+
|
|
70
|
+
Wait for affirmative exit evidence from the controller before removing a jobs line. For cancellation, require the live start identity and command to match the launch record and PGID to equal the recorded pid. Recheck identity and command immediately before signaling the owned group with `kill -TERM -- -<pid>`; wait at most 10 s, then recheck ownership before `kill -KILL -- -<pid>` on a surviving group. Require the controller to reap the job, `kill -0 <pid>` to fail and the group to be absent; remove only that job's line and confirm the controller exited with a bounded process check before ordinary completion. A 3600-second timeout records `timed out after 3600 s`. Missing launch provenance, unavailable identity or a mismatch means cleanup incomplete: signal nothing, preserve records and changes, and stop safely for parent recovery rather than claim completion. Never treat a missing identity as proof of exit or reply as completed with a job running.
|
|
71
|
+
|
|
59
72
|
Record one selective invocation: a test file, test id or filter, or an asserting command. Use no `;`, `|` or `&&` in that command; use the runner's filter flags rather than a shell pipeline. Never record the owning checkout's full verification command (the home ledger's `verification:` header or a leg's recorded command), or a runner with no file or filter. Keep the command first and results bounded as in step 5: `node --test extensions/x.test.ts; 3 passed; 0 failed`.
|
|
60
73
|
|
|
74
|
+
Never record a paid or external action as the evidence's first segment: verify, fix rounds, conformance and finish rerun it. Record the local replay or fixture check as the command and the action in a result line.
|
|
75
|
+
|
|
61
76
|
Require the recorded command to exit nonzero when it selects no test. Before recording a filter, verify its runner's behavior once: run it with a name that matches nothing and require a nonzero exit; otherwise record the file. For Node, record the test file: `node --test` with a missing file exits nonzero, but `--test-name-pattern` with no matching name exits 0 and reports `tests 0`.
|
|
62
77
|
|
|
63
78
|
## Target seed
|
|
64
79
|
|
|
65
80
|
Use this contract for `target: seed`, normally iteration 0. After a human-authorized recovery discards an interrupted seed, repeat seeding at the consumed iteration count; keep the ledger's `iterations` unchanged.
|
|
66
81
|
|
|
67
|
-
Replace steps 2-6 with evidence seeding for at least one non-cut row; seed a subset or all open rows. Keep the spec-first read order and scope diff hunks to the seeded rows' files. Leave unreached rows unchanged.
|
|
82
|
+
Replace steps 2-6 with evidence seeding for at least one non-cut row; seed a subset or all open rows. Apply step 5's action authorization and durable accounting before every seed check or action; record action spends in the seeded evidence, preserving the fixed seed decision below. Keep the spec-first read order and scope diff hunks to the seeded rows' files. Leave unreached rows unchanged.
|
|
68
83
|
|
|
69
84
|
For each selected `open` row, write or identify its failing test (use an asserting command for a trivial or assertion-only row) and record the command you ran in `evidence` under Evidence command. Close a row already satisfied by the codebase as `done` only with passing baseline evidence; record the command of the passing check for every baseline closure. Keep other open rows `open`. Keep every row's `attempts` at 0; never set `failed`. Preserve `cut` rows and their disposition evidence. Bound evidence as in step 5.
|
|
70
85
|
|