pi-gauntlet 8.3.0 → 8.4.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (38) hide show
  1. package/CHANGELOG.md +4 -0
  2. package/README.md +3 -3
  3. package/bin/gauntlet-performance.mjs +477 -64
  4. package/bin/gauntlet-performance.test.mjs +27 -27
  5. package/bin/gauntlet-spec-index.mjs +290 -47
  6. package/bin/gauntlet-spec-index.test.mjs +42 -18
  7. package/bin/gauntlet-telemetry-seal.mjs +220 -77
  8. package/bin/gauntlet-telemetry-seal.test.mjs +222 -457
  9. package/extensions/lib/gauntlet-settings.test.ts +12 -14
  10. package/extensions/lib/gauntlet-settings.ts +7 -16
  11. package/extensions/lib/telemetry-branch.test.ts +102 -0
  12. package/extensions/lib/telemetry-branch.ts +145 -0
  13. package/extensions/lib/telemetry-diff.test.ts +2 -2
  14. package/extensions/lib/telemetry-diff.ts +1 -1
  15. package/extensions/lib/telemetry-leg.ts +163 -0
  16. package/extensions/lib/telemetry-record.ts +2 -0
  17. package/extensions/lib/vcs.ts +6 -23
  18. package/extensions/telemetry-leg-failures.test.ts +138 -0
  19. package/extensions/telemetry-leg-fields.test.ts +130 -0
  20. package/extensions/telemetry-leg-no-write.test.ts +90 -0
  21. package/extensions/telemetry-leg-rounds.test.ts +157 -0
  22. package/extensions/telemetry-leg-trackers.test.ts +114 -0
  23. package/extensions/telemetry-leg.test.ts +151 -0
  24. package/extensions/telemetry-settlement-contract.test.ts +67 -0
  25. package/extensions/telemetry.test.ts +44 -27
  26. package/extensions/telemetry.ts +68 -48
  27. package/extensions/test-support/git-env.mjs +13 -0
  28. package/package.json +1 -1
  29. package/skills/brainstorming/SKILL.md +1 -1
  30. package/skills/check-delivery/SKILL.md +2 -0
  31. package/skills/finishing-a-development-branch/SKILL.md +13 -13
  32. package/skills/finishing-a-development-branch/reference/verification-set.md +2 -2
  33. package/skills/gatekeep-pr/reference/batch.md +3 -1
  34. package/skills/gatekeep-pr/reference/cleanup.md +1 -1
  35. package/skills/gatekeep-pr/reference/post-selection-loop.md +7 -1
  36. package/skills/gauntlet-resume/reference/seed.md +1 -6
  37. package/skills/shape-ticket/SKILL.md +1 -1
  38. package/skills/shape-ticket/reference/repo-audit.md +2 -2
package/CHANGELOG.md CHANGED
@@ -1,5 +1,9 @@
1
1
  # Changelog
2
2
 
3
+ ## v8.4.0 - 2026-11-07
4
+
5
+ - Telemetry publishes run and per-session skill records on the orphan `gauntlet/telemetry` branch, outside product commits. `telemetry.remote` replaces `telemetry.dir`; readers refresh the branch and cost digests allocate ticket and PR rounds, including unattributed shares. Finish and resume use branch records and full-checkout verification proofs. (#124)
6
+
3
7
  ## v8.3.0 - 2026-11-06
4
8
 
5
9
  - Loop execution uses the ledger allowance for interrupted child and seed recovery, exit-verification repairs and grant-funded conformance rounds. Recovery requires checkout ownership and confirmed job exit; stagnation supplies a row-local hint rather than stopping early. Unreachable target failures become audited scope cuts under amend permission, while exhausted named-action bounds still stop. Loop workers bound blocking calls and track longer jobs in isolated POSIX sessions; design approval authorizes named paid or external actions only within their flow-wide bounds.
package/README.md CHANGED
@@ -99,7 +99,7 @@ pi-gauntlet ships three kinds of pieces, layered on top of pi-cohort's dispatch:
99
99
 
100
100
  - **21 skills** - the workflow logic. Sixteen activate automatically when pi sees the matching kind of task, and the workflow skills chain through their gates: `shape-ticket` (creates one tracker issue or repairs a comma/whitespace list of refs against a Context/Problem/Idea/Acceptance-Criteria template, with a grounded repo audit, an AC integrity check, a cheap council roast, and sequential per-ticket exact-diff write confirmations; model-invocable so other skills and commands can route "file a ticket" to it, the gate still needs your yes), `gatekeep-pr` (model-invocable on an explicit ask that names the PR and the verb; the gates still need your selection - is consent-gated pre-merge verification of a PR against the scope contract at its head - the approved spec's acceptance-criteria dispositions and Design when the PR carries one, else the PR's stated intent; the ticket is cross-checked, never judged here (`check-delivery` owns it) - read-only gathering, verification evidence resolved CI-first (green checks on the exact assessed head count as evidence; a pending commit status that GitHub does not report as blocking the gating actor (`mergeStateStatus` not `BLOCKED`) is not waited for - external-CI commit statuses included - while a running Actions check, an unknown merge state, and every other merge precondition still withhold; the project's verification command runs only as fallback), a rubric-based review, a compact bottom-up report (what the PR delivers, including rows deferred or deviating per spec, PR comments, nits, blockers, then a `mergeable` / `fixable` verdict) and course and terminal menus whose rows derive from the actor's permissions on the PR; nothing mutates (repairs, pushes, reviews, merges, tracker comments) until you pick a course or terminal row, except the opt-in `--rebase [base]` sync that rebases, reviews, and lease-pushes the head before verification, or holds the rebased head locally when a conflict stop's scoped tests fail, so a selected delivery course resolves it before the lease push; after your own merge two opt-in rows post the `AC coverage` block to the ticket or the PR; own-merge, approved, and closed-PR menus offer a recommended `cleanup` row that tears down the gate's worktree and branch and runs the overrides file's `## cleanup` steps - run it with `/skill:gatekeep-pr [<pr>] [--rebase [base]]` (no argument: your open PRs)), `check-delivery` (model-invocable on an explicit ask that names the ticket and the verb; the gates still need your selection - is a post-merge detective control: proves an issue actually shipped (default-branch landing, delivery target, per-AC evidence) before its tracker status advances; it never writes a terminal status - run it with `/skill:check-delivery <ref>` or, in pi, `/skill:check-delivery #12 #15` for fresh parallel verification and one gate and write per ticket in turn), `writing-plans`, `roasting-the-spec`, `test-driven-development`, `subagent-driven-development`, `loop-implementation` (the plan-free implement driver that iterates a fresh `loop-implementer` over the spec's acceptance-criteria ledger when approval picks the loop engine), `dispatching-parallel-agents`, `verification-before-completion`, `requesting-code-review`, `receiving-code-review`, `using-git-worktrees`, `finishing-a-development-branch`, `forge-skill`, `linear` (reads/searches/comments on/manages Linear tickets via the `linearis` CLI; owns all linearis mechanics and the `## Issue tracker` overrides schema; tracker-facing skills route to it). Five more require explicit entry (`disable-model-invocation: true`): `brainstorming` starts the gated workflow on explicit consent - use `/skill:brainstorming` for reliable discovery, or opt in through reachable explicit prose or a human-selected handoff; after spec approval, downstream planning and implementation still chain automatically. `chase-bug` is human-only bug triage: read-only root-cause discovery to an evidenced verdict menu (real bug -> ticket/brainstorm/hotfix/respond; five negative verdicts), then a gated response to the reporter for addressable origins (GitHub issue / tracker ticket) and a rendered verdict summary otherwise - it never fixes during triage; the hotfix row hands off to `skills/chase-bug/hotfix.md` after the menu - run it with `/skill:chase-bug`. `gauntlet-performance` reads the committed run telemetry (current repo by default, `--dir <path>` adds others) through the parse-only `gauntlet-performance` CLI and answers with one example-led recommendation, 3-5 cornerstone numbers, a council assessment when available, and a menu of at most three actions (render a report file, open the recommendation as a ticket via `shape-ticket`, drill into one run); it writes nothing unless you pick render - run it with `/skill:gauntlet-performance [--dir <path>]... [--since <version>]`. `gauntlet-handoff` ends a session whose flow a fresh session will continue: it invokes pi-cohort's `handoff` skill (`--out <path>` or `--key <name>`, default key = the run worktree's branch with `/` flattened to `-`, default file `<tmpdir>/pi-handoff/<key>.md`), then appends the gauntlet process-state section (phase/plan tracker status) per the shared contract `skills/gauntlet-resume/reference/brief-contract.md` - run it with `/skill:gauntlet-handoff [--out <path> | --key <name>]`. `gauntlet-resume` is the only way back into an interrupted flow from a fresh session: it takes a `gauntlet-handoff` brief (a file path, pasted text, or - with no arguments - a pick from the briefs under `<tmpdir>/pi-handoff/`), a bare worktree that already holds a spec, or a spec tracked on main under a configured spec dir (a spec seed: it creates or reuses the worktree through `using-git-worktrees` and pins the spec), restores phase/plan tracker state through the legal arming sequence (`start brainstorm`, `skip` with `resume:` reasons, `plan_check` before implement-or-later), and never infers approval from artifacts - run it with `/skill:gauntlet-resume [<brief-file> | <spec>.md | <worktree-name-or-path>]`.
101
101
  - **11 subagent personas** - the specialized child agents the skills dispatch via pi-cohort: `implementer`, `loop-implementer`, `wave-reviewer`, `conformance-reviewer`, `spec-summarizer`, `plan-writer`, `spec-council-member`, `spec-council-synthesizer`, `spec-council-applier`, `pr-gatekeeper`, `delivery-checker`. See [doc/personas.md](./doc/personas.md) for what each one does and why its permissions are scoped the way they are.
102
- - **4 runtime extensions** - the enforcement layer. `plan-tracker` and `phase-tracker` are tools skills call to track progress (with a TUI widget); `verify-before-ship` is a hook that warns if you push or open a PR without a passing test run since your last edit (expected when a loop gate set does not match its patterns; the scoped `Verification:` line answers it, not a full local run); a phase-tracker flow guard reminds on implement-phase commits missing spec/code review. Both engines use the tracker: waves manages plan tasks, while loop calls `plan_tracker refresh` to project ledger rows and preserve conformance tasks. Loop implement completes when every row is closed, then verify step 1 runs the ledger's gate set in the owning checkouts. In a brainstorming-entered flow, phase-tracker rejects `implement` or `verify` completion while tracker tasks remain pending or in progress; see [its configuration reference](./doc/configuration.md#phase-tracker). phase-tracker also registers `plan_check`, which verifies a plan against its spec and against the grammar in [skills/writing-plans/reference/plan-contract.md](./skills/writing-plans/reference/plan-contract.md), including that each task's `Tests:` commands are selective and never the full suite, and that every test command carries a passing base-commit result; a pass stamps the plan and spec content hashes for implementation and returns a bounded task manifest; see the [manifest contract and size limit](./doc/configuration.md#phase-tracker). `telemetry` records one YAML record per gauntlet run (phase timing, models, personas, gate/fix rounds, diff at ship) that ships in the squash beside the spec - it is written only inside a brainstorming-entered flow, kept out of history until `finishing-a-development-branch` seals it once with `gauntlet-telemetry-seal` (a `telemetry:` commit under git; the working-copy snapshot in a plain jj workspace) - and blocks a brainstorm `write` into an already-shipped spec; see [its configuration reference](./doc/configuration.md#telemetry). See [doc/configuration.md](./doc/configuration.md) for the settings each one reads.
102
+ - **4 runtime extensions** - the enforcement layer. `plan-tracker` and `phase-tracker` are tools skills call to track progress (with a TUI widget); `verify-before-ship` is a hook that warns if you push or open a PR without a passing test run since your last edit (expected when a loop gate set does not match its patterns; the scoped `Verification:` line answers it, not a full local run); a phase-tracker flow guard reminds on implement-phase commits missing spec/code review. Both engines use the tracker: waves manages plan tasks, while loop calls `plan_tracker refresh` to project ledger rows and preserve conformance tasks. Loop implement completes when every row is closed, then verify step 1 runs the ledger's gate set in the owning checkouts. In a brainstorming-entered flow, phase-tracker rejects `implement` or `verify` completion while tracker tasks remain pending or in progress; see [its configuration reference](./doc/configuration.md#phase-tracker). phase-tracker also registers `plan_check`, which verifies a plan against its spec and against the grammar in [skills/writing-plans/reference/plan-contract.md](./skills/writing-plans/reference/plan-contract.md), including that each task's `Tests:` commands are selective and never the full suite, and that every test command carries a passing base-commit result; a pass stamps the plan and spec content hashes for implementation and returns a bounded task manifest; see the [manifest contract and size limit](./doc/configuration.md#phase-tracker). `telemetry` publishes YAML records on the orphan `gauntlet/telemetry` branch, outside the product diff: one sealed brainstorming run plus one session leg per settled `shape-ticket`, `gatekeep-pr` or `check-delivery` invocation. In-progress runs live in Git common-dir metadata; `gauntlet-telemetry-seal` publishes before landing without changing checkout HEAD or index. Publication failure warns without blocking ship; plain jj keeps local metadata and reports no Git remote. A fetched shipped record blocks a brainstorm `write` into that spec. See [the record layout, settings and migration guide](./doc/configuration.md#telemetry). See [doc/configuration.md](./doc/configuration.md) for the settings each one reads.
103
103
 
104
104
  pi-gauntlet is **opinionated**: it recommends this pipeline for non-trivial changes, but starts `brainstorming` only after explicit user consent. Enforcement is opt-in by entry, not ambient: once brainstorming starts a flow, the phase-tracker extension mechanically blocks a phase from closing before its gate runs, and warns once if the main loop writes code during implement (subagents own implement-phase edits). A change made *without* entering the flow (a typo, a formatting run, a dependency bump - see "When to use / when NOT to use") is not gated; the discipline of routing real work through the pipeline is a convention the tooling supports, not a trap it springs on every edit.
105
105
 
@@ -170,7 +170,7 @@ Pin an exact release with `npm:pi-gauntlet@X.Y.Z`. See [doc/install-internals.md
170
170
 
171
171
  ## Performance digest
172
172
 
173
- `gauntlet-performance` digests telemetry records without judging them: `node <pi-gauntlet-package>/bin/gauntlet-performance.mjs [--dir <repo root or telemetry dir>]... [--since <version>] [--json]` prints a `corpus:` line, one row per run (`run_id`, repo, spec, version, status - `shipped*` when the record has no ship phase - wall, per-phase minutes, tokens, cost, models, dispatches, `disp/task`, `coord`, `fix_round_grants`, reopens, conformance loops, reviewer findings, council dispatches, and `big` - the count of parent calls whose cache write exceeded 50k tokens with their share of the run's parent cache-write tokens (`-` for records without `parent_calls`)), a `by version` block of p50/max over shipped, non-truncated runs, a `council` block grouped by member roster, and `skipped:` lines for files it could not use. Each council table shows member `total`, `unique`, `applied`, `uniq_appl`, `deferred`, `rejected` blocker/major/minor counts and `uniq_appl_nonminor/run`; `chair:` shows runs, average clusters and members reported, and retried runs. Roster flags (JSON `flags[].rule`: `low_unique_applied` - fewer than 0.2 unique-and-applied blocker/major findings per run; `high_rejection` - over 50% rejected while another member is below 25%) require five runs and print as `flag: <member> - <detail>`. Smaller groups print `not enough runs to assess (N of 5)`; absent council data prints `no council data in selected records`. JSON includes `council: [{ roster, runs, members, chairs, flags }]`. Default corpus is the current repo's telemetry dir; each `--dir` adds a repo root (its own `telemetry.dir` setting is honoured) or a bare telemetry dir. `--since` compares versions numerically. Exit 0 always except usage errors. `/skill:gauntlet-performance` is the reasoning layer on top.
173
+ `gauntlet-performance` digests telemetry records without judging them: `node <pi-gauntlet-package>/bin/gauntlet-performance.mjs [--dir <telemetry dir>]... [--since <version>] [--json]` prints a `corpus:` line, one row per run (`run_id`, repo, spec, version, status - `shipped*` when the record has no ship phase - wall, per-phase minutes, tokens, cost, models, dispatches, `disp/task`, `coord`, `fix_round_grants`, reopens, conformance loops, reviewer findings, council dispatches, and `big` - the count of parent calls whose cache write exceeded 50k tokens with their share of the run's parent cache-write tokens (`-` for records without `parent_calls`)), a `by version` block of p50/max over shipped, non-truncated runs, a `council` block grouped by member roster, and `skipped:` lines for files it could not use. Each council table shows member `total`, `unique`, `applied`, `uniq_appl`, `deferred`, `rejected` blocker/major/minor counts and `uniq_appl_nonminor/run`; `chair:` shows runs, average clusters and members reported, and retried runs. Roster flags (JSON `flags[].rule`: `low_unique_applied` - fewer than 0.2 unique-and-applied blocker/major findings per run; `high_rejection` - over 50% rejected while another member is below 25%) require five runs and print as `flag: <member> - <detail>`. Smaller groups print `not enough runs to assess (N of 5)`; absent council data prints `no council data in selected records`. JSON includes `council: [{ roster, runs, members, chairs, flags }]`. By default it refreshes the configured `gauntlet/telemetry` branch and reads under the current repository's own `<owner>/<name>/` prefix; `--dir <path>` reads a bare directory of YAML records without fetching. Network failure warns and serves the last fetched ref, or an empty corpus when none exists. The `tickets`, `prs` and `unattributed` tables include session-leg costs and split shared batch costs evenly; see [allocation and record fields](doc/configuration.md#telemetry). `--since` compares versions numerically. Exit 0 always except usage errors. `/skill:gauntlet-performance` is the reasoning layer on top.
174
174
 
175
175
  `disp/task` counts recorded implement-phase dispatch-result events divided by `plan.tasks - plan.skipped`; missing plan data, zero denominator or dropped events print `n/a`. `coord` is `yes` only for a non-empty sealed `legs:` list. Telemetry also records `verification_runs`: parent bash matcher hits from implement start through seal, not child checks or an assertion that verification passed. See [the telemetry reference](doc/configuration.md#telemetry) for the counter window and name-based reopen rules.
176
176
 
@@ -624,7 +624,7 @@ The conformance gate's model, the spec council's roster, and the phase-tracker's
624
624
  | `verifyBeforeShip.testCommands` | verify-before-ship | the `DEFAULT_TEST_COMMANDS` patterns in `extensions/lib/gauntlet-settings.ts` (make, npm, pnpm, yarn, pytest, rspec, cargo, go) | patterns that count as a test run - [detail](./doc/configuration.md#verify-before-ship) |
625
625
  | `verifyBeforeShip.warningReference` | verify-before-ship | unset | pointer shown with the warning - [detail](./doc/configuration.md#verify-before-ship) |
626
626
  | `telemetry.enabled` | telemetry | `true` | record each flow - [detail](./doc/configuration.md#telemetry) |
627
- | `telemetry.dir` | telemetry | `.pi/gauntlet/telemetry` | record location - [detail](./doc/configuration.md#telemetry) |
627
+ | `telemetry.remote` | telemetry and readers | `origin` | separate telemetry repository URL - [detail](./doc/configuration.md#telemetry) |
628
628
  | `telemetry.buckets` | telemetry | test / docs / config globs (`DEFAULT_TELEMETRY_BUCKETS`) | file classification - [detail](./doc/configuration.md#telemetry) |
629
629
 
630
630
  For Markdown instructions rather than runtime settings, use [Project-specific overrides](#project-specific-overrides). Repo-local settings replace each configured second-level object wholesale; see the [deep reference](./doc/configuration.md).