pi-gauntlet 7.2.0 → 7.3.0

package/CHANGELOG.md

@@ -1,5 +1,19 @@
 # Changelog
 
+## v7.3.0 - 2026-10-05
+
+- eval/brainstorming/replay: the live-tool replay harness code and verdict tables are committed; private inputs and transcripts are documented as non-committable.
+
+### Added
+
+- **forge-skill persona rules and eval harness.** `skills/forge-skill/SKILL.md` gains `## Persona rules` (the body is the whole system prompt, `description` is a routing label, frontmatter stays byte-identical unless a knob is named, tools stay as narrow as the job, no `/skill:` or `reference/` machinery, no reload step in pi) and marks its three skill-only authoring rules `Skills only:`. `eval/forge-skill/` is a fact-judged eval driver (worker edit, two reviewers, arithmetic kept/disputed/lost, refusal-guarded `compare`) with ten samples, six of them persona edits (four from `agents/` history, two constructed to exercise the persona rules); it ships outside the package; `scripts/ci.mjs` runs its deterministic driver tests and checks sample shape (exactly `case.md` + `expected.md`). The sample files are `case.md` (fixture files plus the edit request, fenced) and `expected.md`, and raw results stay under `$TMPDIR` - both deliberate deviations from `eval/README.md`, recorded in the spec. Baseline-vs-candidate judgment is recorded in the spec's `## Eval judgment`.
+
+## v7.2.1 - 2026-10-04
+
+### Changed
+
+- **brainstorming: challenges the ask instead of hardening it.** The scout's recon carries a `Framing:` finding (the strongest cited case that the request as framed is the wrong change, or `no objection`); the main loop verifies it without skipping premise verification and, with a cited concern and a named alternative, asks one framing question (`A) as framed`, `B) pivot: <alternative> - <reason>`, `C) smaller or none`) before any other question - otherwise it opens with a one-sentence holds-statement naming what was checked. Approaches must differ on a named axis, consider a reuse-only option and one that drops an assumption from the ask, and each ends with a `Pattern:` line cited to a repo location; a repo convention that is itself debt is named and is never the recommendation unless the user or an adopted ticket decision kept the ask as framed. New Red Flag for approaches that skip the framing question or the holds-statement. Spec council unchanged.
+
 ## v7.2.0 - 2026-10-04
 
 ### Changed

package/README.md

@@ -36,7 +36,7 @@ pi-gauntlet's only hard dependency is pi-cohort - every gate that dispatches a r
 Concretely, one change through the gauntlet:
 
 0. *(Optional)* Before there's even a spec, `/skill:shape-ticket` can create or repair a single tracker issue - shaping a raw ask into a Context/Problem/Idea/Acceptance Criteria ticket, gated by an AC integrity check, a cheap council roast, and one human-confirmed write that may include one optional gated Reporter-note comment. A failed roast is retried once, then surfaced inline at the gate if it fails again. It's a tool, not a phase: no worktree, no plan/phase tracker, runs from any repo state. It never activates on its own (`disable-model-invocation: true`) - invoke it explicitly. Similarly, `/skill:chase-bug` triages a raw bug report into an evidenced verdict - and can hand off to shape-ticket, brainstorming, or a bounded hotfix - before any spec exists.
-1. Opt in with `/skill:brainstorming` and describe the change. The command is the reliable discovery-and-entry path; explicit prose requesting brainstorming/the gauntlet also supplies consent when the skill is already loaded or an available instruction or route identifies it. An ordinary implementation request or an agent recommendation is not consent. **`brainstorming`** sets up an isolated worktree, explores the codebase, and turns your description into a written spec; when the run starts from a ticket, the spec carries the ticket's acceptance criteria verbatim, each with a disposition (`in-scope`, `deviates:`, `deferred:`, `venue:`), the conformance gate checks the in-scope ones, and `/skill:check-delivery` verifies `venue:` rows after deploy. A multi-model critique runs on it automatically. If the spec replaces a known prior spec, brainstorming marks the predecessor with a `> **Superseded by:**` banner under its title (default format, syntax overridable via `.pi/gauntlet-overrides.md`; candidates come from brainstorming's scout recon, never a mechanical sweep). **You read and approve the spec - human gate 1.** No implementation code exists yet.
+1. Opt in with `/skill:brainstorming` and describe the change. The command is the reliable discovery-and-entry path; explicit prose requesting brainstorming/the gauntlet also supplies consent when the skill is already loaded or an available instruction or route identifies it. An ordinary implementation request or an agent recommendation is not consent. **`brainstorming`** sets up an isolated worktree, explores the codebase, and turns your description into a written spec - when its recon finds a cited case that the ask is the wrong change (misdiagnosed, wrong-sized, already solved, a band-aid, or cutting across a sound pattern - including a repo convention that is itself debt) and can name the alternative, question one is a framing question (`as framed`, `pivot: <alternative>`, or `smaller or none`); otherwise it opens with one sentence naming what it checked, and every approach it proposes ends with a `Pattern:` line naming what it follows or cuts across; when the run starts from a ticket, the spec carries the ticket's acceptance criteria verbatim, each with a disposition (`in-scope`, `deviates:`, `deferred:`, `venue:`), the conformance gate checks the in-scope ones, and `/skill:check-delivery` verifies `venue:` rows after deploy. A multi-model critique runs on it automatically. If the spec replaces a known prior spec, brainstorming marks the predecessor with a `> **Superseded by:**` banner under its title (default format, syntax overridable via `.pi/gauntlet-overrides.md`; candidates come from brainstorming's scout recon, never a mechanical sweep). **You read and approve the spec - human gate 1.** No implementation code exists yet.
 2. **`writing-plans`** decomposes the approved spec into atomic, independently-verifiable tasks, grouped into parallel waves where they don't touch the same files.
 3. **`subagent-driven-development`** executes the plan one task at a time, each in a fresh subagent, behind spec-compliance review then code-quality review. TDD-locked: red, green, refactor. Independent implementation, review, and council batches remain parallel, but their dispatches are foreground: the orchestrator waits for terminal results before accepting work or advancing a phase.
 4. **verify**: the parent first runs the plan's full verification command set; only a passing result permits whole-diff code review, then the **conformance gate**. A review fix invalidates that result, so the parent reruns full verification before the next review or conformance gate. When the overrides file declares a happy path for the touched paths (see [Project-specific overrides](#project-specific-overrides)), the parent runs it once here, between code review and the conformance gate, and the reviewer reads its transcript as runtime evidence - a hang or a failed request shows up in the audit with the transcript quoted - as a gap when an origin clause ties to it, otherwise as a `happy-path: failed - unattributable` line - never as a silent green. The conformance subagent reads the finished code and docs against your *original words* from step 1, not the plan, and reports per-requirement: delivered, partial, missing, drifted, or unauthorized. Unauthorized rows - shipped surface no human input asked for, whether it crept in or was laundered through the spec - follow their recommendation like every other row: a contained removal auto-runs, anything another requirement leans on is deferred with a plain-language "I'd cut it / I'd keep it" recommendation. Inside a brainstorming-entered flow this gate is machine-blocked from being skipped. Compatible executable recommendations auto-run through an isolated fix-and-re-audit loop with no prompt; anything still open surfaces as a dense list - one line per decision, plain-language, with its recommended choice inline. Reply `1` to take every recommendation, or `2:` with per-item overrides; a current `CONFORMS` / no-concerns result goes straight to the branch options with no extra conformance sign-off.
@@ -159,6 +159,8 @@ Evals compare prompt wording on fixed inputs against human-approved must-hold fa
 
 Run an arm with externally supplied models: `node eval/spec-summarizer/run.mjs run --arm candidate --persona agents/spec-summarizer.md --candidate-model <candidate-id> --reviewers <reviewer-a>,<reviewer-b>`. The [eval convention](eval/README.md) owns the layout, baseline/candidate process, and publication rules; the [spec-summarizer eval](eval/spec-summarizer/README.md) is the worked example with eight samples and committed results. Model calls run manually, never in CI; CI runs deterministic eval tests and hygiene checks. `eval/` is checkout-only and excluded from the npm tarball.
 
+`eval/forge-skill/` measures an edit to `skills/forge-skill/SKILL.md`: `node eval/forge-skill/run.mjs --skill-dir <path> --candidate-model <id> --reviewers <id>,<id>` runs a worker over ten samples (four skill fixtures, six persona fixtures - four from this repo's `agents/` history, two constructed to exercise the persona rules), has two reviewer models vote per must-hold fact, and writes records under `$TMPDIR`; `run.mjs compare <baseline> <candidate>` refuses mismatched inputs and fails on any fact that moved from kept to lost. Models are arguments, never committed. Its samples are `case.md` + `expected.md` and its raw results stay under `$TMPDIR` - the two places it deviates from the convention. Process and sample grammar: [`eval/forge-skill/README.md`](eval/forge-skill/README.md).
+
 ## Use from Claude Code
 
 Six skills are exposed to Claude Code via the plugin marketplace at

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "pi-gauntlet",
-  "version": "7.2.0",
+  "version": "7.3.0",
   "description": "Opinionated, gated workflow skills, subagent personas, and runtime extensions for the pi coding agent.",
   "author": "Jacek Juraszek",
   "type": "module",

package/skills/brainstorming/SKILL.md

@@ -100,13 +100,22 @@ Append only citable findings - schemas, hard constraints, contradictions, and sc
 
 Before approaches, state in chat the supported, disproved, corrected, and unverified premises with sources and attempted lookups, in full sentences - no status-keyword lists, no template; if the design depends on no claims, one sentence says so. An unverified claim is not a stop: put it in Open Questions or a stated assumption. If a load-bearing claim is contradicted, the premise note is your next message - even as question one - and states the corrected fact with its source; the design continues on it and the user overrides in reply. Record the corrected fact, and any override, in the draft for `## Problem` or the relevant design decision.
 
+Before question one, verify the draft's `Framing:` line against primary code and add your own recon. This check adds to the premise verification above and never replaces it: trace every consumer, caller, and data path the ask's constraints name before writing the holds-statement or the question, and name those traces in the holds-statement, not only the files read. A framing concern exists when you can state a cited case that the ask is misdiagnosed, wrong-sized, already solved, a band-aid where the root cause is reachable, or cuts across a sound pattern - including a repo convention that is itself the debt - and can name the alternative it implies; an objection without an alternative is a lead to investigate, not a concern. With a concern, question one is the framing question in the questionary format: `A) as framed`, `B) pivot: <named alternative> - <cited reason>`, `C) smaller or none: <what ships, or what happens if nothing ships>`, `Recommendation: <letter> - <why>`; for a band-aid concern, B carries the root-cause alternative. Without a concern, open the first message with one sentence naming what was checked - `The framing holds: checked <x>, <y>` - and continue to the first ordinary question; a scout objection that yielded no alternative is named among the things checked.
+
+After any premise correction in the same message, ask the framing question or give the holds-statement. Ask the framing question at most once per brainstorm, including after pushback; the user's reply is final, and later doubts on the same point go to Open Questions or a `Pattern:` line. Recommend; never pivot without the reply. A ticket-recorded decision on the exact point follows the ticket rule above; an adopted decision records `Framing: kept (ticket <id>)`. A pivot never rewrites AC rows: rows it no longer satisfies take `deviates:` dispositions, and a pivot that changes the topic re-mints the slug.
+
+Record the outcome in the draft under `## Appended during questionary` as `Framing: kept` or `Framing: pivoted to <x> - <why>`; spec-writing carries that line into `## Problem`. On redraw entry (checklist step 4) the count restarts: run this evaluation once against the new draft. On redraw the draft is the approved spec: its `## Problem` `Framing:` line is context, not scout evidence, and the new outcome is recorded directly in `## Problem`. When the scout failed there is no `Framing:` line; the holds-statement or the question still comes from your own recon.
+
+Bad: `The ask adds another settings path - should we? Recommendation: no`
+Good: `The ask adds a second settings path beside the resolver that already owns every setting (settings/resolver.ts:11). A) as framed B) pivot: a key under the existing resolver - one read path, no new boundary C) smaller or none: document the default and ship no setting. Recommendation: B - the owning schema already exists.`
+
 ### 4. Explore approaches
 
-Propose 2-3 approaches with trade-offs; lead with the recommendation and explain it. Use conversational prose unless the user asks for a table.
+Propose 2-3 genuinely distinct approaches and name what separates them (the axis), comparing them on explicit trade-offs. Consider two candidates in every set: the minimal reuse-only option (no new construct) and an option that drops an assumption the ask makes which the framing reply, if any, did not settle; list each when viable, otherwise state in one sentence with evidence why not (`reuse-only: not viable - the only existing path is the one identified as debt`); a non-viable candidate is not an approach and does not count toward the 2-3. End each approach with `Pattern: <what it follows or cuts across, cited to a repo location>` - judge against sound engineering practice first, then repo rules; when the repo's own convention is the debt, say so, and never recommend an approach that extends it - unless the framing reply or an adopted ticket decision kept the ask as framed: then recommend the best as-framed approach and name the debt in its `Pattern:` line. A `Pattern:` line names things like a band-aid over a shared contract, a second settings path beside the owning one, or a layer bypassed for one caller - the judgment comes from the repo and your own engineering knowledge, not a list. Lead with the recommendation and end with the condition that would flip it. Use conversational prose unless the user asks for a table.
 
 ### 5. Design for clarity and isolation
 
-Prefer clear testable boundaries, YAGNI, existing conventions, the owning schema/contract rather than parallel state, and explicit errors and edge cases.
+Prefer clear testable boundaries, YAGNI, existing conventions when they are sound - when the convention is the debt, say so and do not extend it - the owning schema/contract rather than parallel state, and explicit errors and edge cases.
 
 ### 6. Present the design in two rounds
 
@@ -168,6 +177,7 @@ Redraw: keep the worktree and the approved spec file. `plan_tracker({ action: "c
 - Missing predecessor banner; invalid multi-spec split ([owner](#spec-self-review-before-user-review-gate); [owner](#2-scope-check)).
 - Gate reached without the second predecessor pass ([owner](#spec-self-review-before-user-review-gate)).
 - Approaches before the premise note states a contradicted claim's correction ([owner](#3-understand-the-idea)).
+- Approaches before the framing question or the holds-statement (either may follow the premise correction in the same message); a holds-statement that names nothing checked ([owner](#3-understand-the-idea)).
 - Amend without `reference/amendment-surface.md`; waiting after an amend grant; auto-applying a redraw ([owner](#amending-an-approved-spec)).
 
 ## Project overrides

package/skills/brainstorming/gatherer.md

@@ -47,7 +47,10 @@ Scout (always dispatched):
 > Recon for an upcoming design discussion. The request: `<initial prompt verbatim>`.
 > Map the territory this change touches: relevant files with line ranges, existing
 > patterns and conventions the change must match, test conventions, integration
-> points, and whether the codebase or ecosystem already solves any of this. Cite
+> points, and whether the codebase or ecosystem already solves any of this. Then one required line:
+> `Framing: <the strongest cited case that the request as framed is the wrong change - a path that already covers it, a sound convention it cuts across, an existing convention it would extend that is itself the debt, a band-aid where the root cause is reachable, or a smaller change with the same outcome>`
+> or `Framing: no objection - checked <what was read>`; this line is
+> evidence for the design discussion, not a verdict - the main loop decides. Cite
 > exact paths and line ranges. If a spec you cite carries a supersession marker
 > (default: a `> **Superseded by:**` banner; the project's overrides may define
 > another format), follow the successor for the superseded scope and cite it

package/skills/forge-skill/SKILL.md

@@ -11,6 +11,8 @@ This skill governs every file a model reads as instructions: `SKILL.md` and its
 
 ## Conventions
 
+Skills only; persona conventions are in ## Persona rules.
+
 Sources: pi - `docs/skills` in `@earendil-works/pi-coding-agent`; Claude Code - `code.claude.com/docs/en/skills`.
 
 | Item | pi | Claude Code |
@@ -31,24 +33,36 @@ Sources: pi - `docs/skills` in `@earendil-works/pi-coding-agent`; Claude Code -
 These rules bind the lines an edit adds or changes. Leave other lines alone.
 
 - Imperative voice. Write instructions as commands. Never "should", "consider", "you may want to", "it is recommended".
-- Low conditionality. One path per step: branch only on a runtime fact the model can observe - a tool result, a file's presence, a settings value - never on the reader's judgment. Two branches is the ceiling; a third goes into a table or a `reference/` file.
+- Low conditionality. One path per step: branch only on a runtime fact the model can observe - a tool result, a file's presence, a settings value - never on the reader's judgment. Two branches is the ceiling; a third goes into a table, or for a skill a reference/ file.
 - Minimal diff. A rule change touches the sentence that owns the rule, not the section. Never restate a rule in a second place; link the owner.
-- Oversized file. A change touching a file over 500 lines extracts the concern it touches - or the largest self-contained `##` section - into `reference/<topic>.md` in the same change, and leaves a one-line "read X now" pointer at the step that needs it.
+- Skills only: Oversized file. A change touching a file over 500 lines extracts the concern it touches - or the largest self-contained `##` section - into `reference/<topic>.md` in the same change, and leaves a one-line "read X now" pointer at the step that needs it.
 - Recipe over prohibition. A shape or quality problem gets a statement of what the output is; a discipline problem gets one explicit rule. Never a list of things to avoid.
 - No nuance clauses. "Unless it matters", "when appropriate", "if needed" reopen the decision; delete them or name the observable condition.
-- Description is a trigger. State when to load the skill (symptoms, file types, situations); never summarize its steps.
+- Skills only: Description is a trigger. State when to load the skill (symptoms, file types, situations); never summarize its steps.
 - One example. One concrete example per rule, never three.
 - Skip what the model knows. No explanations of git, shell, or the harness.
-- Never force-load. Cross-reference a skill by its invocation name (pi `/skill:name`, Claude Code `/name` or `/plugin:name`); never with `@`.
+- Skills only: Never force-load. Cross-reference a skill by its invocation name (pi `/skill:name`, Claude Code `/name` or `/plugin:name`); never with `@`.
 - ASCII punctuation, compact `|---|` table separators, present tense.
 
+## Persona rules
+
+A persona is an `agents/<name>.md` file dispatched by name (pi-cohort `subagent({ agent })`, Claude Code subagents); its body is the child's entire system prompt. Frontmatter fields: the "Agent frontmatter" table in pi-cohort's agents-and-chains doc; a project's pins: its personas doc ("Frontmatter knobs" in this repo's `doc/personas`). Every `## Authoring rules` bullet without a `Skills only:` marker binds here too.
+
+- Whole prompt. pi-cohort assembles no base prompt, AGENTS.md, or skills catalog around the body unless `inheritProjectContext` or `inheritSkills` opts in; Claude Code replaces its default prompt with it. Open with the role and why it exists, state the output contract the caller parses before any method; when the caller parses a closing line, keep it last.
+- Routing label. `description` is the one-sentence dispatch label the orchestrator's agent list shows: name when to dispatch, and write `Not for direct dispatch` when a skill owns the call. Never a summary of the body.
+- Frontmatter pin. Leave every line the edit does not touch byte-identical, and leave frontmatter untouched unless the request names a knob - a pinned field removes the matching `agentOverrides` knob, so the request states the pin; a line the edit rewrites follows the ASCII rule like any other. Never pin `model`.
+- Narrow tools. `tools` and `inherit*` list the operations the job performs and nothing else; a reviewer keeps `bash` when its caller injects a scoped command. A non-implementing persona with `bash` sets `completionGuard: false`; an implementing persona keeps the guard.
+- Contract strings. A sentence a caller parses - closing line, mode discriminator, verdict word - changes only together with its caller; the project's validator pins them. Example: `lean: nothing to cut` stays verbatim.
+- No skill machinery. No `/skill:` cross-reference and no `reference/` file: the body is read whole and nothing beside it is loaded (frontmatter `skills:` injects named skills; it is a knob, not a file split). A persona over 500 lines splits by job into a second persona.
+- Reload. pi-cohort re-reads `agents/` on every dispatch, so an edit is live at the next call; Claude Code needs a new session.
+
 ## Edit procedure
 
 1. Read the whole file.
 2. Locate and edit the sentence that owns the rule you are changing (see Minimal diff).
 3. Read the changed lines back against `## Authoring rules`.
-4. Run `wc -l` on the file; over 500 lines, extract per the oversized-file rule.
-5. pi: run `/reload`. Claude Code: start a new session.
+4. Run wc -l; over 500 lines, a skill extracts per the oversized-file rule, a persona splits by job.
+5. Skill, pi: run /reload. Persona, pi: nothing. Claude Code: start a new session.
 
 ## Test (optional)
 
@@ -62,6 +76,7 @@ Run after adding a rule or rewording a discipline rule.
 |---|---|
 | pi | `subagent({ agent: "worker", context: "fresh", task })` twice, one task with the body appended |
 | Claude Code | The harness's subagent dispatch tool, twice, same split |
+| pi, persona | baseline: `subagent({ agent: "worker", context: "fresh", task })` with the pre-edit body appended; loaded: `subagent({ agent: "<name>", context: "fresh", task })` after the edit - pi-cohort re-reads `agents/` per dispatch, so capture the pre-edit body first |
 
 ## Project overrides