pi-gauntlet 7.21.0 → 7.22.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +18 -0
- package/README.md +1 -1
- package/agents/plan-writer.md +7 -6
- package/extensions/lib/plan-check.test.ts +105 -2
- package/extensions/lib/plan-check.ts +112 -3
- package/extensions/phase-tracker.test.ts +35 -1
- package/package.json +1 -1
- package/skills/brainstorming/SKILL.md +5 -3
- package/skills/brainstorming/gatherer.md +8 -1
- package/skills/brainstorming/reference/amendment-surface.md +1 -1
- package/skills/gauntlet-handoff/SKILL.md +11 -7
- package/skills/gauntlet-resume/reference/brief-contract.md +21 -1
- package/skills/subagent-driven-development/SKILL.md +6 -4
- package/skills/subagent-driven-development/implementer-prompt.md +2 -2
- package/skills/subagent-driven-development/stop-note.md +9 -4
- package/skills/writing-plans/SKILL.md +32 -4
- package/skills/writing-plans/reference/plan-contract.md +13 -3
package/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,23 @@
|
|
|
1
1
|
# Changelog
|
|
2
2
|
|
|
3
|
+
## v7.22.1 - 2026-10-20
|
|
4
|
+
|
|
5
|
+
### Fixed
|
|
6
|
+
|
|
7
|
+
- **`/skill:gauntlet-handoff` finds cohort's `handoff` skill by path.** pi-cohort >= 7.4.3 marks `handoff` command-only, which drops it from the session skill list the skill used to search, so every run stopped with "not in the session skill list". Step 2 now resolves the pi-cohort package root from `pi list` and reads `<root>/skills/handoff/SKILL.md` directly.
|
|
8
|
+
|
|
9
|
+
## v7.22.0 - 2026-10-20
|
|
10
|
+
|
|
11
|
+
### Added
|
|
12
|
+
|
|
13
|
+
- **Plans carry base-commit test evidence.** The `plan-writer` persona gains `bash`, runs every task's `Tests:` command on the base commit, and records a `- baseline: <sha> pass | pass via <base command> | skipped - <paths> created by this task` bullet; `plan_check` gains the `baseline-line` and `open-questions` findings and refuses to stamp without that evidence or while `## Open Questions` holds a substantive row. A failing base reaches the human once, with the command and output, through the fix-first / narrow / amend menu. A boundary the spec leaves open is settled from the caller or harness contract as a `Decision:` line in the task. `/skill:gauntlet-resume` re-baselines pre-release plans without replacing task statuses. (#72)
|
|
14
|
+
- **Brainstorming traces implicit dependencies.** The scout reports every unnamed file, binary, extension, or fixture the design relies on; a missing one lands in the premise note and the spec as a corrected fact instead of a stop. (#72)
|
|
15
|
+
- **Lookup before asking in the implement loop.** A worker's `NEEDS_CONTEXT`/`BLOCKED` question is classified as a fact ask or a decision; fact asks are looked up in the documented spec, plan, and repo locations before any stop, and the stop note carries `Searched: <where> - <result>` lines. (#72)
|
|
16
|
+
|
|
17
|
+
### Changed
|
|
18
|
+
|
|
19
|
+
- **Eval runner accepts unplanned improvements.** An unlisted `fails -> holds` fact is labeled `unplanned improvement` and counts as `improved`; only same-fact regressions are confirmed; cached cells are reclassified locally with zero model calls and the report regenerates even on a zero-call run. (#72)
|
|
20
|
+
|
|
3
21
|
## v7.21.0 - 2026-10-19
|
|
4
22
|
|
|
5
23
|
### Changed
|
package/README.md
CHANGED
|
@@ -79,7 +79,7 @@ pi-gauntlet ships three kinds of pieces, layered on top of pi-cohort's dispatch:
|
|
|
79
79
|
|
|
80
80
|
- **20 skills** - the workflow logic. Fifteen activate automatically when pi sees the matching kind of task, and the workflow skills chain through their gates: `shape-ticket` (creates or repairs one tracker issue per run against a Context/Problem/Idea/Acceptance-Criteria template, gated by an AC integrity check, a cheap council roast, and a single human-confirmed write; model-invocable so other skills and commands can route "file a ticket" to it, the gate still needs your yes), `gatekeep-pr` (model-invocable on an explicit ask that names the PR and the verb; the gates still need your selection - is consent-gated pre-merge verification of a PR against the scope contract at its head - the approved spec's acceptance-criteria dispositions and Design when the PR carries one, else the PR's stated intent; the ticket is cross-checked, never judged here (`check-delivery` owns it) - read-only gathering, verification evidence resolved CI-first (green checks on the exact assessed head count as evidence; a pending commit status that GitHub does not report as blocking the gating actor (`mergeStateStatus` not `BLOCKED`) is not waited for - external-CI commit statuses included - while a running Actions check, an unknown merge state, and every other merge precondition still withhold; the project's verification command runs only as fallback), a rubric-based review, a compact bottom-up report (what the PR delivers, including rows deferred or deviating per spec, PR comments, nits, blockers, then a `mergeable` / `fixable` verdict) and a verb menu whose rows derive from the actor's permissions on the PR; nothing mutates (fixes, pushes, reviews, merges, tracker comments) until you pick a row, except the opt-in `--rebase [base]` sync that rebases, reviews, and lease-pushes the head before verification, or holds the rebased head locally when a conflict stop's scoped tests fail, so the normal `fix` row resolves it before the lease push; after your own merge two opt-in rows post the `AC coverage` block to the ticket or the PR; every terminal menu offers a recommended `cleanup` row that tears down the gate's worktree and branch and runs the overrides file's `## cleanup` steps - run it with `/skill:gatekeep-pr <pr> [--rebase [base]]`), `check-delivery` (model-invocable on an explicit ask that names the ticket and the verb; the gates still need your selection - is a post-merge detective control: proves an issue actually shipped (default-branch landing, delivery target, per-AC evidence) before its tracker status advances; it never writes a terminal status - run it with `/skill:check-delivery <ref>`), `writing-plans`, `roasting-the-spec`, `test-driven-development`, `subagent-driven-development`, `dispatching-parallel-agents`, `verification-before-completion`, `requesting-code-review`, `receiving-code-review`, `using-git-worktrees`, `finishing-a-development-branch`, `forge-skill`, `linear` (reads/searches/comments on/manages Linear tickets via the `linearis` CLI; owns all linearis mechanics and the `## Issue tracker` overrides schema; tracker-facing skills route to it). Five more require explicit entry (`disable-model-invocation: true`): `brainstorming` starts the gated workflow on explicit consent - use `/skill:brainstorming` for reliable discovery, or opt in through reachable explicit prose or a human-selected handoff; after spec approval, downstream planning and implementation still chain automatically. `chase-bug` is human-only bug triage: read-only root-cause discovery to an evidenced verdict menu (real bug -> ticket/brainstorm/hotfix/respond; five negative verdicts), then a gated response to the reporter for addressable origins (GitHub issue / tracker ticket) and a rendered verdict summary otherwise - it never fixes during triage; the hotfix row hands off to `skills/chase-bug/hotfix.md` after the menu - run it with `/skill:chase-bug`. `gauntlet-performance` reads the committed run telemetry (current repo by default, `--dir <path>` adds others) through the parse-only `gauntlet-performance` CLI and answers with one example-led recommendation, 3-5 cornerstone numbers, a council assessment when available, and a menu of at most three actions (render a report file, open the recommendation as a ticket via `shape-ticket`, drill into one run); it writes nothing unless you pick render - run it with `/skill:gauntlet-performance [--dir <path>]... [--since <version>]`. `gauntlet-handoff` ends a session whose flow a fresh session will continue: it invokes pi-cohort's `handoff` skill (`--out <path>` or `--key <name>`, default key = the run worktree's branch with `/` flattened to `-`, default file `<tmpdir>/pi-handoff/<key>.md`), then appends the gauntlet process-state section (phase/plan tracker status) per the shared contract `skills/gauntlet-resume/reference/brief-contract.md` - run it with `/skill:gauntlet-handoff [--out <path> | --key <name>]`. `gauntlet-resume` is the only way back into an interrupted flow from a fresh session: it takes a `gauntlet-handoff` brief (a file path, pasted text, or - with no arguments - a pick from the briefs under `<tmpdir>/pi-handoff/`), a bare worktree that already holds a spec, or a spec tracked on main under a configured spec dir (a spec seed: it creates or reuses the worktree through `using-git-worktrees` and pins the spec), restores phase/plan tracker state through the legal arming sequence (`start brainstorm`, `skip` with `resume:` reasons, `plan_check` before implement-or-later), and never infers approval from artifacts - run it with `/skill:gauntlet-resume [<brief-file> | <spec>.md | <worktree-name-or-path>]`.
|
|
81
81
|
- **9 subagent personas** - the specialized child agents the skills dispatch via pi-cohort: `implementer`, `code-reviewer`, `spec-reviewer`, `conformance-reviewer`, `spec-summarizer`, `plan-writer`, `spec-council-member`, `spec-council-synthesizer`, `spec-council-applier`. See [doc/personas.md](./doc/personas.md) for what each one does and why its permissions are scoped the way they are.
|
|
82
|
-
- **4 runtime extensions** - the enforcement layer. `plan-tracker` and `phase-tracker` are tools skills call to track progress (with a TUI widget); `verify-before-ship` is a hook that warns if you push or open a PR without a passing test run since your last edit; a phase-tracker flow guard reminds on implement-phase commits missing spec/code review. In a brainstorming-entered flow, phase-tracker rejects `implement` or `verify` completion while tracker tasks remain pending or in progress; see [its configuration reference](./doc/configuration.md#phase-tracker). phase-tracker also registers `plan_check`, which verifies a plan against its spec and against the grammar in [skills/writing-plans/reference/plan-contract.md](./skills/writing-plans/reference/plan-contract.md), including that each task's `Tests:` commands are selective and never the full suite; a pass stamps the plan for implementation. `telemetry` records one YAML record per gauntlet run (phase timing, models, personas, gate/fix rounds, diff at ship) that ships in the squash beside the spec - it is written only inside a brainstorming-entered flow, kept out of history until `finishing-a-development-branch` seals it once with `gauntlet-telemetry-seal` (a `telemetry:` commit under git; the working-copy snapshot in a plain jj workspace) - and blocks a brainstorm `write` into an already-shipped spec; see [its configuration reference](./doc/configuration.md#telemetry). See [doc/configuration.md](./doc/configuration.md) for the settings each one reads.
|
|
82
|
+
- **4 runtime extensions** - the enforcement layer. `plan-tracker` and `phase-tracker` are tools skills call to track progress (with a TUI widget); `verify-before-ship` is a hook that warns if you push or open a PR without a passing test run since your last edit; a phase-tracker flow guard reminds on implement-phase commits missing spec/code review. In a brainstorming-entered flow, phase-tracker rejects `implement` or `verify` completion while tracker tasks remain pending or in progress; see [its configuration reference](./doc/configuration.md#phase-tracker). phase-tracker also registers `plan_check`, which verifies a plan against its spec and against the grammar in [skills/writing-plans/reference/plan-contract.md](./skills/writing-plans/reference/plan-contract.md), including that each task's `Tests:` commands are selective and never the full suite, and that every test command carries a passing base-commit result; a pass stamps the plan for implementation. `telemetry` records one YAML record per gauntlet run (phase timing, models, personas, gate/fix rounds, diff at ship) that ships in the squash beside the spec - it is written only inside a brainstorming-entered flow, kept out of history until `finishing-a-development-branch` seals it once with `gauntlet-telemetry-seal` (a `telemetry:` commit under git; the working-copy snapshot in a plain jj workspace) - and blocks a brainstorm `write` into an already-shipped spec; see [its configuration reference](./doc/configuration.md#telemetry). See [doc/configuration.md](./doc/configuration.md) for the settings each one reads.
|
|
83
83
|
|
|
84
84
|
pi-gauntlet is **opinionated**: it recommends this pipeline for non-trivial changes, but starts `brainstorming` only after explicit user consent. Enforcement is opt-in by entry, not ambient: once brainstorming starts a flow, the phase-tracker extension mechanically blocks a phase from closing before its gate runs, and warns once if the main loop writes code during implement (subagents own implement-phase edits). A change made *without* entering the flow (a typo, a formatting run, a dependency bump - see "When to use / when NOT to use") is not gated; the discipline of routing real work through the pipeline is a convention the tooling supports, not a trap it springs on every edit.
|
|
85
85
|
|
package/agents/plan-writer.md
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: plan-writer
|
|
3
3
|
description: Writes, checks, and repairs the gauntlet implementation plan for one approved spec in a fresh context; the plan file is its only output. Dispatched only by the writing-plans skill; not for direct dispatch.
|
|
4
|
-
tools: read, grep, find, ls, write, plan_check
|
|
4
|
+
tools: read, grep, find, ls, bash, write, plan_check
|
|
5
5
|
inheritProjectContext: false
|
|
6
6
|
inheritSkills: false
|
|
7
7
|
completionGuard: true
|
|
@@ -23,7 +23,7 @@ No plan content, no summary, no commentary.
|
|
|
23
23
|
|
|
24
24
|
The task names five absolute paths: the worktree, the approved spec, the plan path, the plan skill file, and the plan contract file. It may also carry one block after them:
|
|
25
25
|
|
|
26
|
-
- `Findings:` - the verbatim output of the orchestrator's own `plan_check` on the plan at the plan path, or a human's answers to the plan's open questions. Read the existing plan first and keep everything the block does not touch. Treat each answer as the resolution of the matching
|
|
26
|
+
- `Findings:` - the verbatim output of the orchestrator's own `plan_check` on the plan at the plan path, or a human's answers to the plan's open questions. Read the existing plan first and keep everything the block does not touch, except baseline rows, which Baseline owns and regenerates. Treat each answer as the resolution of the matching contract question; remove the section when no questions remain. `Findings: rerun baseline` reruns every task command.
|
|
27
27
|
- `Amend:` - the amended spec sections and the rule for the edit. Read the existing plan first; preserve every task title and wave position the amendment does not touch; edit only the affected anchors and tasks.
|
|
28
28
|
|
|
29
29
|
## Procedure
|
|
@@ -32,12 +32,13 @@ The task names five absolute paths: the worktree, the approved spec, the plan pa
|
|
|
32
32
|
2. Read the plan skill file (`SKILL.md`) from its `## File Structure` heading through the end of its `## Self-Review` section, and the whole plan contract file (`plan-contract.md`). That text is the plan contract; follow it as written.
|
|
33
33
|
3. Read the spec in full. Recon the code the spec touches by reading it directly: exact paths to create, modify, or delete; call sites and tests with line ranges; the scoped command that runs one test file; scoped and repo-wide lint and format commands; the full-suite verification entrypoint and whether it bundles style checks. Dispatch nothing.
|
|
34
34
|
4. Write the plan at the plan path with `write` as a full replacement. The plan path is the only file you write; never create or edit any other file.
|
|
35
|
-
5. Run `plan_check({ planPath })
|
|
36
|
-
6.
|
|
37
|
-
7. Stop when `plan_check` passes and
|
|
35
|
+
5. Run Baseline from the skill's Self-Review before `plan_check({ planPath })`. On Findings/Amend re-dispatch, read provenance once for the current base; retain evidence only for unchanged commands on that same base, rerun missing, invalid, changed-command or changed-SHA evidence, and rerun every command for `Findings: rerun baseline`. Never rewrite SHAs without execution. Baseline owns and regenerates its rows; preserve unresolved contract rows.
|
|
36
|
+
6. Run the deterministic checker and judgment checks in Self-Review, including Contract decisions. Repair `baseline-line` and substantive baseline `open-questions` findings only by Baseline execution, never by authoring evidence or deleting/rewording a row. Normalize malformed Open Questions into rows without changing substantive text. Fix every other finding; with substantive rows remaining, reply `OPEN QUESTIONS <abs plan path>` immediately, without a passing check or repair rounds for those rows. Any ordinary edit returns to step 5.
|
|
37
|
+
7. Stop when `plan_check` passes and judgment checks cause no edit. Apply the 3-round budget only to ordinary findings: convert an exhausted finding to an Open Questions row and stop. Findings blocks continue that finding's count across dispatches. Omit the section when no rows remain and reply per the output contract.
|
|
38
38
|
|
|
39
39
|
## Rules
|
|
40
40
|
|
|
41
|
-
-
|
|
41
|
+
- Use `bash` only for Baseline provenance (`git rev-parse`, `git status --porcelain`, or the contract's jj equivalents) and the plan's own `Tests:` commands in the worktree. Write only the plan. Never edit tracked source, commit, run `stash`, `checkout`, `restore`, `clean`, or mutate jj's working copy or revisions (`new`, `commit`, `abandon`, `restore`, `squash`).
|
|
42
|
+
- Never leave a placeholder in authored plan prose; retain verbatim failure evidence in row continuation fences. Derive decisions from source per Contract decisions; name only an unsettled decision as an open question.
|
|
42
43
|
- Never call `plan_tracker`, `phase_tracker`, or `subagent`; the orchestrator owns them.
|
|
43
44
|
- Never reply with more than the one line.
|
|
@@ -42,6 +42,7 @@ const VALID_PLAN = `# Fixture Plan
|
|
|
42
42
|
|
|
43
43
|
**Tests:**
|
|
44
44
|
- \`node --test extensions/lib/fixture-task1.test.ts\`
|
|
45
|
+
- baseline: abc1234 pass
|
|
45
46
|
- via: \`helperFn()\`
|
|
46
47
|
|
|
47
48
|
This task implements helperFn() for parsing.
|
|
@@ -57,6 +58,7 @@ This task implements helperFn() for parsing.
|
|
|
57
58
|
|
|
58
59
|
**Tests:**
|
|
59
60
|
- \`node --test extensions/lib/fixture-task2.test.ts\`
|
|
61
|
+
- baseline: abc1234 pass
|
|
60
62
|
|
|
61
63
|
This task handles naming details.
|
|
62
64
|
|
|
@@ -72,6 +74,7 @@ This task handles naming details.
|
|
|
72
74
|
|
|
73
75
|
**Tests:**
|
|
74
76
|
- \`node --test extensions/lib/fixture-task3.test.ts\`
|
|
77
|
+
- baseline: abc1234 pass
|
|
75
78
|
|
|
76
79
|
The literal TODO is intentionally documented here per spec quote-integrity requirement.
|
|
77
80
|
|
|
@@ -528,6 +531,7 @@ const ENTRYPOINT_PLAN = `# Fixture Plan
|
|
|
528
531
|
|
|
529
532
|
**Tests:**
|
|
530
533
|
- \`node --test extensions/lib/plan-check.test.ts\`
|
|
534
|
+
- baseline: abc1234 pass
|
|
531
535
|
|
|
532
536
|
Run node --test extensions/lib/plan-check.test.ts and confirm green.
|
|
533
537
|
|
|
@@ -546,7 +550,7 @@ test("quote-integrity: header entrypoint literal on an anchored line is satisfie
|
|
|
546
550
|
|
|
547
551
|
test("quote-integrity: scoped command on the same anchored line is still required in the task body", () => {
|
|
548
552
|
const mutated = ENTRYPOINT_PLAN.replace(
|
|
549
|
-
"- Test: extensions/lib/plan-check.test.ts\n\n**Tests:**\n- `node --test extensions/lib/plan-check.test.ts`\n\nRun node --test extensions/lib/plan-check.test.ts and confirm green.",
|
|
553
|
+
"- Test: extensions/lib/plan-check.test.ts\n\n**Tests:**\n- `node --test extensions/lib/plan-check.test.ts`\n- baseline: abc1234 pass\n\nRun node --test extensions/lib/plan-check.test.ts and confirm green.",
|
|
550
554
|
"- Test: extensions/lib/other.test.ts\n\n**Tests:**\n- `node --test extensions/lib/other.test.ts`\n\nRun the scoped test and confirm green.",
|
|
551
555
|
);
|
|
552
556
|
const qi = findingsFor(checkPlan(mutated, ENTRYPOINT_SPEC, alwaysTruePort()), "quote-integrity");
|
|
@@ -645,7 +649,7 @@ test("check 4 fail-closed: invalid glob (port throws)", () => {
|
|
|
645
649
|
|
|
646
650
|
test("check 4 fail-closed: task missing **Files:** block", () => {
|
|
647
651
|
const mutated = VALID_PLAN.replace(
|
|
648
|
-
"**Files:**\n- Modify: extensions/lib/fixture-task3.ts\n- Test: extensions/lib/fixture-task3.test.ts\n\n**Tests:**\n- `node --test extensions/lib/fixture-task3.test.ts`\n\n",
|
|
652
|
+
"**Files:**\n- Modify: extensions/lib/fixture-task3.ts\n- Test: extensions/lib/fixture-task3.test.ts\n\n**Tests:**\n- `node --test extensions/lib/fixture-task3.test.ts`\n- baseline: abc1234 pass\n\n",
|
|
649
653
|
"",
|
|
650
654
|
);
|
|
651
655
|
const findings = checkPlan(mutated, SPEC_TEXT, alwaysTruePort());
|
|
@@ -753,6 +757,7 @@ const EXEMPTION_PLAN = `# Exemption Plan
|
|
|
753
757
|
|
|
754
758
|
**Tests:**
|
|
755
759
|
- \`node --test extensions/lib/exemption-task1.test.ts\`
|
|
760
|
+
- baseline: abc1234 pass
|
|
756
761
|
|
|
757
762
|
The literal TODO is intentionally documented here per spec quote-integrity requirement.
|
|
758
763
|
|
|
@@ -1043,6 +1048,104 @@ test("tests-block: no path normalization - ./x and x differ, as in Files:", () =
|
|
|
1043
1048
|
assert.ok(tb(mutated).some((x) => x.reason.includes("not anchored")));
|
|
1044
1049
|
});
|
|
1045
1050
|
|
|
1051
|
+
const BASELINE = "- baseline: abc1234 pass";
|
|
1052
|
+
const T1_COMMAND = "node --test extensions/lib/fixture-task1.test.ts";
|
|
1053
|
+
const T1_PATH = "extensions/lib/fixture-task1.test.ts";
|
|
1054
|
+
const baselineFindings = (plan: string) => findingsFor(checkPlan(plan, SPEC_TEXT, alwaysTruePort()), "baseline-line");
|
|
1055
|
+
|
|
1056
|
+
for (const [name, evidence] of [
|
|
1057
|
+
["pass", BASELINE],
|
|
1058
|
+
["pass artifacts", BASELINE + " - artifacts: coverage.json, report.txt"],
|
|
1059
|
+
["via", BASELINE + " via `" + T1_COMMAND + "`"],
|
|
1060
|
+
["via artifacts", BASELINE + " via `" + T1_COMMAND + "` - artifacts: coverage.json"],
|
|
1061
|
+
] ) {
|
|
1062
|
+
test(`baseline-line accepts ${name}`, () => {
|
|
1063
|
+
assert.deepEqual(baselineFindings(VALID_PLAN.replace(BASELINE, evidence)), []);
|
|
1064
|
+
});
|
|
1065
|
+
}
|
|
1066
|
+
|
|
1067
|
+
for (const [name, evidence, reason] of [
|
|
1068
|
+
["missing", "", `Task 1: command \`${T1_COMMAND}\` has no baseline: bullet`],
|
|
1069
|
+
["bad sha", "- baseline: ABC1234 pass", "Task 1: baseline: bullet malformed:"],
|
|
1070
|
+
["short sha", "- baseline: abc123 pass", "Task 1: baseline: bullet malformed:"],
|
|
1071
|
+
["long sha", "- baseline: " + "a".repeat(41) + " pass", "Task 1: baseline: bullet malformed:"],
|
|
1072
|
+
["bad result", "- baseline: abc1234 fail", "Task 1: baseline: bullet malformed:"],
|
|
1073
|
+
["empty base command", BASELINE + " via ` `", "Task 1: baseline: bullet malformed:"],
|
|
1074
|
+
["separator-only base command", BASELINE + " via ` ; `", "Task 1: baseline: bullet malformed:"],
|
|
1075
|
+
["misplaced", "- via: seam\n" + BASELINE, "Task 1: baseline: not directly after its command"],
|
|
1076
|
+
["duplicate", BASELINE + "\n" + BASELINE, "Task 1: baseline: not directly after its command"],
|
|
1077
|
+
["sha mismatch", "- baseline: def5678 pass", "baseline SHAs differ: def5678, abc1234"],
|
|
1078
|
+
["skip existing", `- baseline: abc1234 skipped - ${T1_PATH} created by this task`, `Task 1: baseline: skipped but \`${T1_PATH}\` is not a Create: path`],
|
|
1079
|
+
["via unanchored", BASELINE + " via `echo hi`", `Task 1: baseline: pass via \`echo hi\` segment \`echo hi\` anchors no existing Test: path`],
|
|
1080
|
+
["via unanchored compound", BASELINE + " via `" + T1_COMMAND + " && echo hi`", "anchors no existing Test: path"],
|
|
1081
|
+
]) {
|
|
1082
|
+
test(`baseline-line rejects ${name}`, () => {
|
|
1083
|
+
const findings = baselineFindings(VALID_PLAN.replace(BASELINE, evidence));
|
|
1084
|
+
assert.ok(findings.some(f => f.reason.includes(reason)), JSON.stringify(findings));
|
|
1085
|
+
});
|
|
1086
|
+
}
|
|
1087
|
+
|
|
1088
|
+
test("baseline-line accepts 40-character SHAs and introduced-selector base commands", () => {
|
|
1089
|
+
assert.deepEqual(baselineFindings(VALID_PLAN.replaceAll("abc1234", "a".repeat(40))), []);
|
|
1090
|
+
const plan = VALID_PLAN.replace(`- \`${T1_COMMAND}\``, `- \`${T1_COMMAND}::new_case\``)
|
|
1091
|
+
.replace(BASELINE, BASELINE + " via `" + T1_COMMAND + "`");
|
|
1092
|
+
assert.deepEqual(baselineFindings(plan), []);
|
|
1093
|
+
});
|
|
1094
|
+
|
|
1095
|
+
test("baseline-line pass via rejects created tokens outside this task's Test paths", () => {
|
|
1096
|
+
const plan = VALID_PLAN.replace(BASELINE, BASELINE + " via `" + T1_COMMAND + " extensions/lib/fixture-task2.ts`");
|
|
1097
|
+
assert.ok(baselineFindings(plan).some(f => f.reason.includes("anchors Create: path extensions/lib/fixture-task2.ts")));
|
|
1098
|
+
});
|
|
1099
|
+
|
|
1100
|
+
test("baseline-line validates skipped anchor sets and derived surviving segments", () => {
|
|
1101
|
+
const created = VALID_PLAN.replace("- Create: extensions/lib/fixture-task1.ts", `- Create: ${T1_PATH}`);
|
|
1102
|
+
const skipped = `- baseline: abc1234 skipped - ${T1_PATH} created by this task`;
|
|
1103
|
+
assert.deepEqual(baselineFindings(created.replace(BASELINE, skipped)), []);
|
|
1104
|
+
assert.ok(baselineFindings(created.replace(BASELINE, BASELINE + " via `" + T1_COMMAND + "`" )).some(f => f.reason.includes("anchors Create: path " + T1_PATH)));
|
|
1105
|
+
for (const list of ["other.test.ts", `${T1_PATH}, other.test.ts`]) {
|
|
1106
|
+
assert.ok(baselineFindings(created.replace(BASELINE, `- baseline: abc1234 skipped - ${list} created by this task`)).length);
|
|
1107
|
+
}
|
|
1108
|
+
const compound = created.replace(`- \`${T1_COMMAND}\``, `- \`${T1_COMMAND} && node --test extensions/lib/fixture-task2.test.ts\``)
|
|
1109
|
+
.replace(`- Test: ${T1_PATH}`, `- Test: ${T1_PATH}\n- Test: extensions/lib/fixture-task2.test.ts`);
|
|
1110
|
+
assert.deepEqual(baselineFindings(compound.replace(BASELINE, BASELINE + " via `node --test extensions/lib/fixture-task2.test.ts`")), []);
|
|
1111
|
+
assert.ok(baselineFindings(compound.replace(BASELINE, skipped)).length);
|
|
1112
|
+
const allCreated = compound.replace("- Create: extensions/lib/fixture-task1.ts", "- Create: extensions/lib/fixture-task2.test.ts")
|
|
1113
|
+
.replace("- Create: extensions/lib/fixture-task2.ts", "- Create: extensions/lib/fixture-task2.test.ts");
|
|
1114
|
+
const bothSkipped = `- baseline: abc1234 skipped - ${T1_PATH}, extensions/lib/fixture-task2.test.ts created by this task`;
|
|
1115
|
+
assert.deepEqual(baselineFindings(allCreated.replace(BASELINE, bothSkipped)), []);
|
|
1116
|
+
assert.ok(baselineFindings(allCreated.replace(BASELINE, skipped)).length);
|
|
1117
|
+
const none = VALID_PLAN.replace(`- \`${T1_COMMAND}\``, "- none: docs");
|
|
1118
|
+
assert.ok(baselineFindings(none).some(f => f.reason === "Task 1: baseline: under a none: task"));
|
|
1119
|
+
});
|
|
1120
|
+
|
|
1121
|
+
const FAILURE_ROW = `- Task 1: \`${T1_COMMAND}\` fails on abc1234 (exit 2): assertion failed`;
|
|
1122
|
+
for (const [name, body, count, malformed] of [
|
|
1123
|
+
["absent", "", 0, false],
|
|
1124
|
+
["sentinel", "- none", 0, false],
|
|
1125
|
+
["failure", FAILURE_ROW, 1, false],
|
|
1126
|
+
["global then failure", "- Baseline: worktree dirty on abc1234: source.ts\n" + FAILURE_ROW, 2, false],
|
|
1127
|
+
["contract", "- Task 1: unresolved caller contract", 1, false],
|
|
1128
|
+
["prose", "Not resolved", 1, true],
|
|
1129
|
+
["table", "| question | owner |", 1, true],
|
|
1130
|
+
["continuations", FAILURE_ROW + "\n ran: `node --test original.ts`\n blocked: Task 2 `node --test next.ts`\n ```text\n TODO etc.\n ```", 1, false],
|
|
1131
|
+
] as const) {
|
|
1132
|
+
test(`open-questions ${name}`, () => {
|
|
1133
|
+
const plan = VALID_PLAN + (body ? "\n## Open Questions\n\n" + body + "\n" : "");
|
|
1134
|
+
const findings = checkPlan(plan, SPEC_TEXT, alwaysTruePort());
|
|
1135
|
+
const questions = findingsFor(findings, "open-questions");
|
|
1136
|
+
assert.equal(questions.length, count);
|
|
1137
|
+
if (malformed) assert.equal(questions[0].reason, "## Open Questions malformed: expected `- ` rows or `- none`");
|
|
1138
|
+
if (body.includes(FAILURE_ROW)) assert.ok(questions.some(f => f.reason === "plan has open questions: " + FAILURE_ROW));
|
|
1139
|
+
assert.deepEqual(findingsFor(findings, "placeholder-scan"), []);
|
|
1140
|
+
});
|
|
1141
|
+
}
|
|
1142
|
+
|
|
1143
|
+
test("placeholder-scan masks only valid question output, not authored fences or continuations", () => {
|
|
1144
|
+
for (const text of ["```\nTODO\n```", "## Open Questions\n ```\n TODO\n ```", "## Open Questions\n- TODO unresolved", "## Open Questions\n" + FAILURE_ROW + "\n ```\n TODO", "## Open Questions\n" + FAILURE_ROW + "\n ```\nTODO\n ```", "## Open Questions\n" + FAILURE_ROW + "\n ran: `TODO`", "## Open Questions\n" + FAILURE_ROW + "\n blocked: TODO"]) {
|
|
1145
|
+
assert.ok(findingsFor(checkPlan(VALID_PLAN + "\n" + text, SPEC_TEXT, alwaysTruePort()), "placeholder-scan").length);
|
|
1146
|
+
}
|
|
1147
|
+
});
|
|
1148
|
+
|
|
1046
1149
|
test("tests-block: fenced **Tests:** lines are ignored", () => {
|
|
1047
1150
|
const mutated = VALID_PLAN.replace(
|
|
1048
1151
|
"This task handles naming details.",
|
|
@@ -58,6 +58,7 @@ interface Task {
|
|
|
58
58
|
testsMisplacedLines: number[];
|
|
59
59
|
tests: TestsBullet[];
|
|
60
60
|
testsVia: TestsBullet[];
|
|
61
|
+
testsBaseline: TestsBullet[];
|
|
61
62
|
testsNone: TestsBullet[];
|
|
62
63
|
testsMalformed: TestsBullet[];
|
|
63
64
|
}
|
|
@@ -101,6 +102,7 @@ interface ParsedPlan {
|
|
|
101
102
|
tasks: Task[];
|
|
102
103
|
coverageTableFound: boolean;
|
|
103
104
|
coverageRows: CoverageRow[];
|
|
105
|
+
openQuestions: { findings: PlanCheckFinding[]; output: Set<number> };
|
|
104
106
|
}
|
|
105
107
|
|
|
106
108
|
interface SpecHeading {
|
|
@@ -250,6 +252,7 @@ function parsePlan(planText: string): ParsedPlan {
|
|
|
250
252
|
const testsMisplacedLines: number[] = [];
|
|
251
253
|
const tests: TestsBullet[] = [];
|
|
252
254
|
const testsVia: TestsBullet[] = [];
|
|
255
|
+
const testsBaseline: TestsBullet[] = [];
|
|
253
256
|
const testsNone: TestsBullet[] = [];
|
|
254
257
|
const testsMalformed: TestsBullet[] = [];
|
|
255
258
|
|
|
@@ -282,10 +285,12 @@ function parsePlan(planText: string): ParsedPlan {
|
|
|
282
285
|
const cmd = /^- `([^`]+)`$/.exec(l);
|
|
283
286
|
const via = /^- via: (\S.*)$/.exec(l);
|
|
284
287
|
const none = /^- none: (\S.*)$/.exec(l);
|
|
285
|
-
const
|
|
288
|
+
const baseline = /^- baseline: (\S.*)$/.exec(l);
|
|
289
|
+
const bullet = { line: p + 1, text: l, value: (cmd ?? via ?? none ?? baseline)?.[1] ?? "" };
|
|
286
290
|
if (cmd) tests.push(bullet);
|
|
287
291
|
else if (via) testsVia.push(bullet);
|
|
288
292
|
else if (none) testsNone.push(bullet);
|
|
293
|
+
else if (baseline) testsBaseline.push(bullet);
|
|
289
294
|
else testsMalformed.push(bullet);
|
|
290
295
|
}
|
|
291
296
|
}
|
|
@@ -306,6 +311,7 @@ function parsePlan(planText: string): ParsedPlan {
|
|
|
306
311
|
testsMisplacedLines,
|
|
307
312
|
tests,
|
|
308
313
|
testsVia,
|
|
314
|
+
testsBaseline,
|
|
309
315
|
testsNone,
|
|
310
316
|
testsMalformed,
|
|
311
317
|
});
|
|
@@ -367,8 +373,59 @@ function parsePlan(planText: string): ParsedPlan {
|
|
|
367
373
|
break;
|
|
368
374
|
}
|
|
369
375
|
|
|
376
|
+
const questionFindings: PlanCheckFinding[] = [];
|
|
377
|
+
const questionOutput = new Set<number>();
|
|
378
|
+
let inSection = false;
|
|
379
|
+
let row = false;
|
|
380
|
+
let inOutput = false;
|
|
381
|
+
let sentinel: number | undefined;
|
|
382
|
+
let contentCount = 0;
|
|
383
|
+
const malformed = (idx: number) => questionFindings.push({ check: "open-questions", line: idx + 1, text: lines[idx], reason: "## Open Questions malformed: expected `- ` rows or `- none`" });
|
|
384
|
+
for (let idx = 0; idx < lines.length; idx++) {
|
|
385
|
+
const line = lines[idx];
|
|
386
|
+
if (!mask[idx] && /^## /.test(line)) {
|
|
387
|
+
if (inSection && sentinel !== undefined && contentCount !== 1) malformed(sentinel);
|
|
388
|
+
inSection = line === "## Open Questions";
|
|
389
|
+
row = false;
|
|
390
|
+
inOutput = false;
|
|
391
|
+
sentinel = undefined;
|
|
392
|
+
contentCount = 0;
|
|
393
|
+
continue;
|
|
394
|
+
}
|
|
395
|
+
if (!inSection) continue;
|
|
396
|
+
if (inOutput) {
|
|
397
|
+
questionOutput.add(idx);
|
|
398
|
+
if (/^\s+```/.test(line)) inOutput = false;
|
|
399
|
+
continue;
|
|
400
|
+
}
|
|
401
|
+
if (!line.trim()) continue;
|
|
402
|
+
contentCount++;
|
|
403
|
+
if (/^- \S/.test(line)) {
|
|
404
|
+
row = line !== "- none";
|
|
405
|
+
if (row) questionFindings.push({ check: "open-questions", line: idx + 1, text: line, reason: `plan has open questions: ${line}` });
|
|
406
|
+
else sentinel = idx;
|
|
407
|
+
} else if (row && /^\s+\S/.test(line)) {
|
|
408
|
+
if (/^\s+```/.test(line)) {
|
|
409
|
+
let end = idx + 1;
|
|
410
|
+
while (end < lines.length && !/^\s+```/.test(lines[end])) {
|
|
411
|
+
if (lines[end].trim() && !/^\s+/.test(lines[end])) break;
|
|
412
|
+
end++;
|
|
413
|
+
}
|
|
414
|
+
if (end < lines.length && /^\s+```/.test(lines[end])) {
|
|
415
|
+
inOutput = true;
|
|
416
|
+
questionOutput.add(idx);
|
|
417
|
+
}
|
|
418
|
+
}
|
|
419
|
+
} else {
|
|
420
|
+
malformed(idx);
|
|
421
|
+
row = false;
|
|
422
|
+
}
|
|
423
|
+
}
|
|
424
|
+
if (inSection && sentinel !== undefined && contentCount !== 1) malformed(sentinel);
|
|
425
|
+
|
|
370
426
|
return {
|
|
371
427
|
lines,
|
|
428
|
+
openQuestions: { findings: questionFindings, output: questionOutput },
|
|
372
429
|
header: { specPathLine, specPath, verificationLine, verificationText, happyPathLine, happyPathText, happyPathCommand, separatorLine },
|
|
373
430
|
waves,
|
|
374
431
|
tasks,
|
|
@@ -474,7 +531,7 @@ function checkTestsBlock(parsed: ParsedPlan, fs: FsPort): PlanCheckFinding[] {
|
|
|
474
531
|
|
|
475
532
|
const headingText = parsed.lines[task.testsHeadingLine - 1];
|
|
476
533
|
if (task.tests.length === 0 && task.testsNone.length === 0) push(task, task.testsHeadingLine, headingText, "block empty: no command bullet and no `none:`");
|
|
477
|
-
for (const b of task.testsMalformed) push(task, b.line, b.text, "malformed bullet: expected `- \\`command\\``, `- via: <seam>`, or `- none: <category>`");
|
|
534
|
+
for (const b of task.testsMalformed) push(task, b.line, b.text, "malformed bullet: expected `- \\`command\\``, `- baseline: <evidence>`, `- via: <seam>`, or `- none: <category>`");
|
|
478
535
|
if (task.testsNone.length > 1 || (task.testsNone.length > 0 && (task.tests.length > 0 || task.testsVia.length > 0))) {
|
|
479
536
|
push(task, task.testsNone[0].line, task.testsNone[0].text, "contradictory block: `none:` with a command or `via:`, or more than one `none:`");
|
|
480
537
|
}
|
|
@@ -502,6 +559,55 @@ function checkTestsBlock(parsed: ParsedPlan, fs: FsPort): PlanCheckFinding[] {
|
|
|
502
559
|
return findings;
|
|
503
560
|
}
|
|
504
561
|
|
|
562
|
+
function checkBaselineLines(parsed: ParsedPlan, fs: FsPort): PlanCheckFinding[] {
|
|
563
|
+
const findings: PlanCheckFinding[] = [];
|
|
564
|
+
const shas = new Set<string>();
|
|
565
|
+
const creates = new Set(parsed.tasks.flatMap(t => t.files.filter(f => f.kind === "create").map(f => stripLineSuffix(f.path))));
|
|
566
|
+
for (const task of parsed.tasks) {
|
|
567
|
+
const push = (bullet: TestsBullet, reason: string) => findings.push({ check: "baseline-line", line: bullet.line, text: bullet.text, reason: `Task ${task.number}: ${reason}` });
|
|
568
|
+
const paths = task.files.filter(f => f.kind === "test").map(f => stripLineSuffix(f.path)).filter(p => !isBroadening(p));
|
|
569
|
+
const anchors = (command: string) => commandSegments(command).flatMap(segment => segment.split(" ").flatMap(token => paths.filter(p => token === p || token.startsWith(p + "::") || token.startsWith(p + "#") || token.startsWith(p + ":"))));
|
|
570
|
+
for (const command of task.tests) {
|
|
571
|
+
if (!task.testsBaseline.some(b => b.line === command.line + 1)) push(command, `command \`${command.value}\` has no baseline: bullet`);
|
|
572
|
+
}
|
|
573
|
+
for (const bullet of task.testsBaseline) {
|
|
574
|
+
const command = task.tests.find(c => c.line + 1 === bullet.line);
|
|
575
|
+
if (task.testsNone.length) push(bullet, "baseline: under a none: task");
|
|
576
|
+
else if (!command) push(bullet, "baseline: not directly after its command");
|
|
577
|
+
const match = /^([a-f0-9]{7,40}) (?:pass(?: via `([^`]+)`)?(?: - artifacts: (\S.*))?|skipped - (\S.*) created by this task)$/.exec(bullet.value);
|
|
578
|
+
if (!match || (match[2] !== undefined && !match[2].trim()) || (match[3] !== undefined && !match[3].split(", ").every(p => p.trim().length > 0))) {
|
|
579
|
+
push(bullet, `baseline: bullet malformed: ${bullet.text}`);
|
|
580
|
+
continue;
|
|
581
|
+
}
|
|
582
|
+
shas.add(match[1]);
|
|
583
|
+
if (match[4] !== undefined && command) {
|
|
584
|
+
const listed = match[4].split(", ");
|
|
585
|
+
const expected = new Set(anchors(command.value));
|
|
586
|
+
for (const path of new Set([...expected, ...listed])) {
|
|
587
|
+
if (!creates.has(path)) push(bullet, `baseline: skipped but \`${path}\` is not a Create: path`);
|
|
588
|
+
else if (!expected.has(path) || !listed.includes(path)) push(bullet, `baseline: bullet malformed: ${bullet.text}`);
|
|
589
|
+
}
|
|
590
|
+
if (!expected.size || listed.length !== new Set(listed).size) push(bullet, `baseline: bullet malformed: ${bullet.text}`);
|
|
591
|
+
}
|
|
592
|
+
if (match[2] !== undefined) {
|
|
593
|
+
const segments = commandSegments(match[2]);
|
|
594
|
+
if (!segments.length) {
|
|
595
|
+
push(bullet, `baseline: bullet malformed: ${bullet.text}`);
|
|
596
|
+
continue;
|
|
597
|
+
}
|
|
598
|
+
for (const segment of segments) {
|
|
599
|
+
const found = anchors(segment);
|
|
600
|
+
const created = [...creates].filter(p => segment.split(" ").some(token => token === p || token.startsWith(p + "::") || token.startsWith(p + "#") || token.startsWith(p + ":")));
|
|
601
|
+
for (const path of created) push(bullet, `baseline: pass via \`${match[2]}\` segment \`${segment}\` anchors Create: path ${path}`);
|
|
602
|
+
if (!found.some(p => !creates.has(p) && fs.exists(p))) push(bullet, `baseline: pass via \`${match[2]}\` segment \`${segment}\` anchors no existing Test: path`);
|
|
603
|
+
}
|
|
604
|
+
}
|
|
605
|
+
}
|
|
606
|
+
}
|
|
607
|
+
if (shas.size > 1) findings.push({ check: "baseline-line", line: 0, text: "", reason: `baseline SHAs differ: ${[...shas].join(", ")}` });
|
|
608
|
+
return findings;
|
|
609
|
+
}
|
|
610
|
+
|
|
505
611
|
function taskBodyText(task: Task, lines: string[]): string {
|
|
506
612
|
return lines.slice(task.bodyStartLine - 1, task.bodyEndLine).join("\n");
|
|
507
613
|
}
|
|
@@ -809,6 +915,7 @@ function checkPlaceholderScan(parsed: ParsedPlan, requiredLiterals: Map<number,
|
|
|
809
915
|
for (let ln = t.bodyStartLine; ln <= t.bodyEndLine; ln++) lineTask.set(ln, t);
|
|
810
916
|
}
|
|
811
917
|
parsed.lines.forEach((line, idx) => {
|
|
918
|
+
if (parsed.openQuestions.output.has(idx)) return;
|
|
812
919
|
const lineNo = idx + 1;
|
|
813
920
|
const lower = line.toLowerCase();
|
|
814
921
|
const task = lineTask.get(lineNo);
|
|
@@ -962,7 +1069,7 @@ function checkHeaderEntrypoint(parsed: ParsedPlan): PlanCheckFinding[] {
|
|
|
962
1069
|
const entrypoints = [entrypoint, parsed.header.happyPathCommand].filter((e): e is string => Boolean(e));
|
|
963
1070
|
const executable = new Set<number>();
|
|
964
1071
|
for (const task of parsed.tasks) {
|
|
965
|
-
for (const bullet of [...task.tests, ...task.testsVia, ...task.testsNone, ...task.testsMalformed]) {
|
|
1072
|
+
for (const bullet of [...task.tests, ...task.testsVia, ...task.testsBaseline, ...task.testsNone, ...task.testsMalformed]) {
|
|
966
1073
|
executable.add(bullet.line);
|
|
967
1074
|
}
|
|
968
1075
|
}
|
|
@@ -1057,6 +1164,8 @@ export function checkPlan(planText: string, specText: string, fs: FsPort): PlanC
|
|
|
1057
1164
|
findings.push(...checkAnchorResolution(parsed, specLines));
|
|
1058
1165
|
findings.push(...checkPathsExist(parsed, fs));
|
|
1059
1166
|
findings.push(...checkTestsBlock(parsed, fs));
|
|
1167
|
+
findings.push(...checkBaselineLines(parsed, fs));
|
|
1168
|
+
findings.push(...parsed.openQuestions.findings);
|
|
1060
1169
|
const requiredLiterals = computeRequiredLiteralsPerTask(parsed, specLines);
|
|
1061
1170
|
findings.push(...checkPlaceholderScan(parsed, requiredLiterals));
|
|
1062
1171
|
findings.push(...checkWaveFileDisjointness(parsed, fs));
|
|
@@ -874,6 +874,7 @@ const FIXTURE_PLAN = `# Fixture Plan
|
|
|
874
874
|
|
|
875
875
|
**Tests:**
|
|
876
876
|
- \`node --test lib/task1.test.ts\`
|
|
877
|
+
- baseline: abc1234 skipped - lib/task1.test.ts created by this task
|
|
877
878
|
|
|
878
879
|
This task implements helperFn() for parsing.
|
|
879
880
|
|
|
@@ -889,6 +890,7 @@ This task implements helperFn() for parsing.
|
|
|
889
890
|
|
|
890
891
|
**Tests:**
|
|
891
892
|
- \`node --test lib/task2.test.ts\`
|
|
893
|
+
- baseline: abc1234 skipped - lib/task2.test.ts created by this task
|
|
892
894
|
|
|
893
895
|
This task handles naming details.
|
|
894
896
|
|
|
@@ -968,7 +970,13 @@ test("Guard 2 evaluates the command's target checkout (AC 3)", async () => {
|
|
|
968
970
|
|
|
969
971
|
test("plan_check pass: text, details, and stamp arming when a flow is entered", async () => {
|
|
970
972
|
const dir = gitPrimary().primary;
|
|
971
|
-
const { planPath, specPath, planText, specText } = writePlanFixture(dir
|
|
973
|
+
const { planPath, specPath, planText, specText } = writePlanFixture(dir, {
|
|
974
|
+
mutatePlan: t => t.replace(/- Create: lib\/task[12]\.test\.ts\n/g, "")
|
|
975
|
+
.replace(/skipped - lib\/task[12]\.test\.ts created by this task/g, "pass"),
|
|
976
|
+
});
|
|
977
|
+
mkdirSync(join(dir, "lib"), { recursive: true });
|
|
978
|
+
writeFileSync(join(dir, "lib/task1.test.ts"), "");
|
|
979
|
+
writeFileSync(join(dir, "lib/task2.test.ts"), "");
|
|
972
980
|
const h = harness({ cwd: dir, branch: readyForImplementBranch });
|
|
973
981
|
await h.emit("session_start");
|
|
974
982
|
const tool = h.tools.find((t) => t.name === "plan_check")!;
|
|
@@ -996,6 +1004,32 @@ test("plan_check pass: text, details, and stamp arming when a flow is entered",
|
|
|
996
1004
|
void specText;
|
|
997
1005
|
});
|
|
998
1006
|
|
|
1007
|
+
for (const failure of [false, true]) {
|
|
1008
|
+
test(`plan_check missing baseline${failure ? " with base failure" : ""} prevents implement stamp`, async () => {
|
|
1009
|
+
const dir = gitPrimary().primary;
|
|
1010
|
+
const command = "node --test lib/task1.test.ts";
|
|
1011
|
+
const row = `- Task 1: \`${command}\` fails on abc1234 (exit 2): assertion failed`;
|
|
1012
|
+
const { planPath } = writePlanFixture(dir, {
|
|
1013
|
+
mutatePlan: t => t.replace("- baseline: abc1234 skipped - lib/task1.test.ts created by this task\n", "") + (failure ? "\n## Open Questions\n" + row + "\n" : ""),
|
|
1014
|
+
});
|
|
1015
|
+
const h = harness({ cwd: dir, branch: readyForImplementBranch });
|
|
1016
|
+
await h.emit("session_start");
|
|
1017
|
+
const tool = h.tools.find(t => t.name === "plan_check")!;
|
|
1018
|
+
const result = await tool.execute("baseline", { planPath }, undefined, undefined, h.ctx) as {
|
|
1019
|
+
details: { status: string; findings: { check: string; reason: string }[] };
|
|
1020
|
+
};
|
|
1021
|
+
assert.equal(result.details.status, "fail");
|
|
1022
|
+
assert.ok(result.details.findings.some(f => f.check === "baseline-line" && f.reason.includes("Task 1")));
|
|
1023
|
+
if (failure) assert.ok(result.details.findings.some(f => f.reason === "plan has open questions: " + row));
|
|
1024
|
+
const phase = h.tools.find(t => t.name === "phase_tracker")!;
|
|
1025
|
+
const start = await phase.execute("start", { action: "start", phase: "implement" }, undefined, undefined, h.ctx) as {
|
|
1026
|
+
details: { error?: string; phases: { implement: { status: string } } };
|
|
1027
|
+
};
|
|
1028
|
+
assert.match(start.details.error ?? "", /plan_check/);
|
|
1029
|
+
assert.notEqual(start.details.phases.implement.status, "in_progress");
|
|
1030
|
+
});
|
|
1031
|
+
}
|
|
1032
|
+
|
|
999
1033
|
test("plan_check pass outside a flow: no stamp, plain-linter text", async () => {
|
|
1000
1034
|
const dir = gitPrimary().primary;
|
|
1001
1035
|
const { planPath } = writePlanFixture(dir);
|
package/package.json
CHANGED
|
@@ -105,9 +105,9 @@ Recommendation: A - the extension already holds the poll loop, so no new entrypo
|
|
|
105
105
|
|
|
106
106
|
Append only citable findings - schemas, hard constraints, contradictions, and scope-changing answers - to `## Appended during questionary` using `edit`.
|
|
107
107
|
|
|
108
|
-
Before approaches, state in chat the supported, disproved, corrected, and unverified premises with sources and attempted lookups, in full sentences - no status-keyword lists, no template; if the design depends on no claims, one sentence says so. An unverified claim is not a stop: put it in Open Questions or a stated assumption. If a load-bearing claim is contradicted, the premise note is your next message - even as question one - and states the corrected fact with its source; the design continues on it and the user overrides in reply. Record the corrected fact, and any override, in the draft for `## Problem` or the relevant design decision.
|
|
108
|
+
Before approaches, state in chat the supported, disproved, corrected, and unverified premises with sources and attempted lookups, in full sentences - no status-keyword lists, no template; if the design depends on no claims, one sentence says so. Treat an implicit dependency found missing as a corrected premise and one not checkable as unverified; state each in the premise note with the lookup the repo or spec names and its result, and carry it into `## Problem`. An unverified claim is not a stop: put it in Open Questions or a stated assumption. If a load-bearing claim is contradicted, the premise note is your next message - even as question one - and states the corrected fact with its source; the design continues on it and the user overrides in reply. Record the corrected fact, and any override, in the draft for `## Problem` or the relevant design decision.
|
|
109
109
|
|
|
110
|
-
Before question one, verify the draft's `Framing:` line against primary code and add your own recon. This check adds to the premise verification above and never replaces it: trace every consumer, caller, and data path the ask's constraints name before writing the holds-statement or the question, and name
|
|
110
|
+
Before question one, verify the draft's `Framing:` line against primary code and add your own recon. This check adds to the premise verification above and never replaces it: trace every consumer, caller, and data path the ask's constraints name, including retained loaders and every repro and verification step's implicit dependencies, before writing the holds-statement or the question; check those dependencies with the lookup the repo or spec names, and name the traces and dependencies in the holds-statement, not only the files read. A framing concern exists when you can state a cited case that the ask is misdiagnosed, wrong-sized, already solved, a band-aid where the root cause is reachable, or cuts across a sound pattern - including a repo convention that is itself the debt - and can name the alternative it implies; an objection without an alternative is a lead to investigate, not a concern. With a concern, question one is the framing question in the questionary format:
|
|
111
111
|
|
|
112
112
|
```text
|
|
113
113
|
A) as framed
|
|
@@ -120,7 +120,7 @@ For a band-aid concern, put the root-cause alternative in B. Without a concern,
|
|
|
120
120
|
|
|
121
121
|
After any premise correction in the same message, ask the framing question or give the holds-statement. Ask the framing question at most once per brainstorm, including after pushback; the user's reply is final, and later doubts on the same point go to Open Questions or a `Pattern:` line. Recommend; never pivot without the reply. A ticket-recorded decision on the exact point follows the ticket rule above; an adopted decision records `Framing: kept (ticket <id>)`. A pivot never rewrites AC rows: rows it no longer satisfies take `deviates:` dispositions, and a pivot that changes the topic re-mints the slug.
|
|
122
122
|
|
|
123
|
-
Record the outcome in the draft under `## Appended during questionary` as `Framing: kept` or `Framing: pivoted to <x> - <why>`; spec-writing carries that line into `## Problem`. On redraw entry (checklist step 4) the count restarts: run this evaluation once against the new draft. On redraw the draft is the approved spec: its `## Problem` `Framing:` line is context, not scout evidence, and the new outcome is recorded directly in `## Problem`. When the scout failed there is no `Framing:` line; the holds-statement or
|
|
123
|
+
Record the outcome in the draft under `## Appended during questionary` as `Framing: kept` or `Framing: pivoted to <x> - <why>`; spec-writing carries that line into `## Problem`. On redraw entry (checklist step 4) the count restarts: run this evaluation once against the new draft. On redraw the draft is the approved spec: its `## Problem` `Framing:` line is context, not scout evidence, and the new outcome is recorded directly in `## Problem`. When the scout failed there is no `Framing:` line; supply the implicit-dependency check and the holds-statement or question from your own recon.
|
|
124
124
|
|
|
125
125
|
Bad: `The ask adds another settings path - should we? Recommendation: no`
|
|
126
126
|
Good:
|
|
@@ -134,6 +134,8 @@ Recommendation: B - the owning schema already exists.
|
|
|
134
134
|
|
|
135
135
|
### 4. Explore approaches
|
|
136
136
|
|
|
137
|
+
In a separate message after the premise note, restate each implicit dependency recorded as missing or not checkable, with its lookup and result, before presenting any approaches.
|
|
138
|
+
|
|
137
139
|
Propose 2-3 genuinely distinct approaches and name what separates them (the axis), comparing them on explicit trade-offs. Consider two candidates in every set: the minimal reuse-only option (no new construct) and an option that drops an assumption the ask makes which the framing reply, if any, did not settle; list each when viable, otherwise state in one sentence with evidence why not (`reuse-only: not viable - the only existing path is the one identified as debt`); a non-viable candidate is not an approach and does not count toward the 2-3. End each approach with `Pattern: <what it follows or cuts across, cited to a repo location>` - judge against sound engineering practice first, then repo rules; when the repo's own convention is the debt, say so, and never recommend an approach that extends it - unless the framing reply or an adopted ticket decision kept the ask as framed: then recommend the best as-framed approach and name the debt in its `Pattern:` line. A `Pattern:` line names things like a band-aid over a shared contract, a second settings path beside the owning one, or a layer bypassed for one caller - the judgment comes from the repo and your own engineering knowledge, not a list. Lead with the recommendation and end with the condition that would flip it. Use conversational prose unless the user asks for a table.
|
|
138
140
|
|
|
139
141
|
### 5. Design for clarity and isolation
|
|
@@ -47,7 +47,14 @@ Scout (always dispatched):
|
|
|
47
47
|
> Recon for an upcoming design discussion. The request: `<initial prompt verbatim>`.
|
|
48
48
|
> Map the territory this change touches: relevant files with line ranges, existing
|
|
49
49
|
> patterns and conventions the change must match, test conventions, integration
|
|
50
|
-
> points, and whether the codebase or ecosystem already solves any of this.
|
|
50
|
+
> points, and whether the codebase or ecosystem already solves any of this. Trace retained
|
|
51
|
+
> loaders and call paths, and every repro and verification step, for dependencies the
|
|
52
|
+
> request relies on without naming: files, binaries, extensions, fixtures, and data
|
|
53
|
+
> subsets. Check each with the lookup the repo or spec names; a named path or
|
|
54
|
+
> capability is a claim to check, not proof it exists. Emit one line per item:
|
|
55
|
+
> `Implicit dependencies: <item> - <how checked> - <found | not found | not checkable: <why>>`
|
|
56
|
+
> or `Implicit dependencies: none found - traced <components, steps>`.
|
|
57
|
+
> Then one required line:
|
|
51
58
|
> `Framing: <the strongest cited case that the request as framed is the wrong change - a path that already covers it, a sound convention it cuts across, an existing convention it would extend that is itself the debt, a band-aid where the root cause is reachable, or a smaller change with the same outcome>`
|
|
52
59
|
> or `Framing: no objection - checked <what was read>`; this line is
|
|
53
60
|
> evidence for the design discussion, not a verdict - the main loop decides. Cite
|
|
@@ -140,7 +140,7 @@ Before the aftermath, `read` this section again; never run it from memory.
|
|
|
140
140
|
|
|
141
141
|
Apply accepted items only: reviewer clears, `accept`, `alt-n`, and state-changing `custom`. Print one line per applied item: `amended the spec: <title> - <what>`.
|
|
142
142
|
|
|
143
|
-
Aftermath, once per batch, only when at least one item applied (nothing applied -> skip to the commit below). No plan yet -> commit the spec; continue. Plan exists ->
|
|
143
|
+
Aftermath, once per batch, only when at least one item applied (nothing applied -> skip to the commit below). No plan yet -> commit the spec; continue. Plan exists -> read `git -C "<abs worktree path>" status --porcelain` (plain jj: `jj -R "<abs worktree path>" diff --summary`); collect the parent's in-flight workflow edits as worktree-relative paths, including the amended spec and staged in-place wave-repair deliverables. Dispatch `plan-writer` per `writing-plans` `## Author the plan` step 3 with an `Amend:` block naming the amended spec sections, an `In-flight:` list of those paths (the parent supplies the set; the child never infers it), and the rule "preserve every task title and wave position the amendment does not touch; edit only affected anchors and tasks", then run that section's steps 5-6 (the parent `plan_check` and its branches), then read the amended plan once (step 7's read, without its phase completion or `plan_tracker` init) - the main loop never edits the plan text; then update the tracker: `plan_tracker` `add` for new tasks; anchor-changed completed tasks are reopened as `in_progress` and re-run the task loop (`update` never sets `pending`). A removed task is deleted from the plan; then re-`init` the tracker with `{ name, status }` elements: preserved tasks keep their order and statuses, reopened tasks are `in_progress` in place, every still-`pending` task (including newly added ones, whatever wave label they carry) trails the non-pending ones, removed tasks are the only deletions (the only permitted `init` after handoff; never `clear`). Commit spec + plan together; continue. A task reopened while `verify` or `ship` is in progress stays in that phase - no `phase_tracker` call. Repair it through the current phase's fix loop and `plan_tracker({ action: "update", index: N, status: "complete" })` it once that repair's review or re-audit accepts; open tasks block `complete verify`, and in `ship` nothing checks them, so close them explicitly.
|
|
144
144
|
|
|
145
145
|
One commit per batch:
|
|
146
146
|
|
|
@@ -22,7 +22,8 @@ loop: tracker state is extension state no subagent can read.
|
|
|
22
22
|
## Boundaries
|
|
23
23
|
|
|
24
24
|
- Reads: `../gauntlet-resume/reference/brief-contract.md` in full (resolve against this
|
|
25
|
-
skill's directory) before any other action; the transcript; git metadata by path
|
|
25
|
+
skill's directory) before any other action; the transcript; git metadata by path;
|
|
26
|
+
`pi list` output; cohort's `skills/handoff/SKILL.md`.
|
|
26
27
|
- Writes: only an append to the brief cohort wrote, wherever the resolved path puts it
|
|
27
28
|
(a repository-local `--out` included). Never a second file, never the trackers, never
|
|
28
29
|
any other repository file.
|
|
@@ -58,12 +59,15 @@ loop: tracker state is extension state no subagent can read.
|
|
|
58
59
|
`node -p "require('os').tmpdir()"`; the authoritative path is the one cohort
|
|
59
60
|
reports in step 2.
|
|
60
61
|
2. **Run cohort's handoff procedure.** A skill cannot expand `/skill:handoff` (pi expands
|
|
61
|
-
skill commands on typed input only)
|
|
62
|
-
|
|
63
|
-
|
|
64
|
-
|
|
65
|
-
|
|
66
|
-
|
|
62
|
+
skill commands on typed input only), and cohort's `handoff` is command-only
|
|
63
|
+
(`disable-model-invocation: true`), so it is absent from the session skill list.
|
|
64
|
+
Locate it by path: run `pi list`; the indented path under the `pi-cohort` entry
|
|
65
|
+
(`npm:pi-cohort`, or a local path ending in `/pi-cohort`) is the package root. `Read` `<root>/skills/handoff/SKILL.md` and follow its procedure with the
|
|
66
|
+
output option resolved in step 1 - the brief's core (six headings, ending at
|
|
67
|
+
`## Skills loaded`) is cohort's to write, by cohort's rules. No `pi-cohort` entry, or
|
|
68
|
+
the file missing -> STOP: "cohort `handoff` skill not found at
|
|
69
|
+
`<root>/skills/handoff/SKILL.md` - install or upgrade to pi-cohort >= 7.1.0". No
|
|
70
|
+
fallback to the `/handoff` prompt.
|
|
67
71
|
3. **Take the path from the report.** Cohort's procedure ends with `Handoff written: <abs
|
|
68
72
|
path>` or `Handoff not written: <reason>`. `Handoff not written` -> STOP with that reason.
|
|
69
73
|
Otherwise that path is the brief; line 1 not starting `# Handoff:` -> STOP with the path.
|
|
@@ -145,6 +145,26 @@ Zero pairs -> stop; more than one -> human picks.
|
|
|
145
145
|
|
|
146
146
|
Entered from the call table's `implement` row and from `reconstruction.md` "Spec with plan" step 2 when `plan_check` returns FAIL. Inputs: the findings, and the task list with statuses (the brief's, or the confirmed reconstruction snapshot) - read it for evidence, never rewrite it.
|
|
147
147
|
|
|
148
|
+
Before citation-table classification, route `baseline-line` and substantive `open-questions` findings through the continuation below. Keep the brief's or confirmed reconstruction's task snapshot (names, order, statuses) verbatim throughout; never take writing-plans step 7's fresh-plan initialization.
|
|
149
|
+
|
|
150
|
+
### Baseline continuation
|
|
151
|
+
|
|
152
|
+
On `baseline-line` or a substantive baseline-form `open-questions` finding, bring a landed fix named by the brief or human into the worktree through `/skill:using-git-worktrees` "the base branch advances" (rebase, or merge for a shared branch). Without a named fix, proceed directly to the rerun. Dispatch `plan-writer` per `/skill:writing-plans` "Author the plan" step 3: `agent: "plan-writer"`, `context: "fresh"`, `async: false`, `cwd: <absolute worktree>`, `phase: "plan-write"`, `acceptance: { level: "none", reason: "the parent plan_check owns acceptance" }`. Fill its fixed task template with the five absolute paths (worktree, spec, plan, writing-plans SKILL.md, its reference/plan-contract.md), then append `Findings: rerun baseline`. Let the writer run Baseline and regenerate its rows; never synthesize evidence or SHAs. On dispatch error, async handle, or missing/empty plan, stop and report without a parent-authored fallback.
|
|
153
|
+
|
|
154
|
+
Run `plan_check({ planPath })` in the parent after either child reply. Route the result:
|
|
155
|
+
|
|
156
|
+
| Result / substantive row | Continuation |
|
|
157
|
+
|---|---|
|
|
158
|
+
| `PLAN WRITTEN` + `PASS`, section absent or exactly `- none` | Return to the originating restore's remaining steps: call table from `skip plan R` through the appropriate implement/verify continuation (ship restores at verify), or reconstruction step 3. Initialize `plan_tracker` only from the saved task snapshot, never the generated fresh-plan list. |
|
|
159
|
+
| `- Task N: ... fails on <sha>` | Present the task, command, base SHA and exit code, then writing-plans step 6's three options, one per line: **fix-first** - `/skill:shape-ticket` for the base failure (its gate applies), then pause; **narrow** - human supplies filter or path text, re-dispatch with `Findings: Task N Tests: narrow to <text>`; **amend** - brainstorming's `reference/amendment-surface.md` section 5. Print a `Recommendation:` line and wait for selection. |
|
|
160
|
+
| Global or contamination rows (`Baseline: worktree dirty`, `Baseline: in-flight source edits`, `dirtied tracked source`, `moved HEAD`) | Take writing-plans step 6's restore-and-rerun path, not the menu. Name paths and causes; the parent's own in-flight work uses its own commit step (doc-only paths can be listed under `In-flight:`); ask the human to commit, discard, or move other paths and test-produced dirt. Never clean automatically. After restoration re-dispatch with `Findings: rerun baseline`, then parent check. |
|
|
161
|
+
| Other substantive `open-questions` rows (contract questions) | Present the rows, wait for answers, re-dispatch per step 3 with answers in `Findings:`, then parent check and this same return route. |
|
|
162
|
+
| Other checks | Use the citation table and its three-round rollback below. |
|
|
163
|
+
|
|
164
|
+
End a turn awaiting selection, restoration, or answers with phase `plan` retained in_progress and no tracker initialization of any kind. Keep the snapshot for the eventual return. Resolved failure rows disappear only through the writer's Baseline rerun; resume writes no Open Questions rows. Plans without `baseline:` bullets intentionally enter this continuation on their `baseline-line` finding.
|
|
165
|
+
|
|
166
|
+
### Citation repair
|
|
167
|
+
|
|
148
168
|
Record the plan's state before any write: tracked and clean when `git -C <worktree> ls-files --error-unmatch -- <plan path>` exits 0 and `git -C <worktree> status --porcelain -- <plan path>` prints nothing (index and working copy both match HEAD; a staged edit alone makes it dirty); otherwise copy the file to a temp path outside the worktree.
|
|
149
169
|
|
|
150
170
|
Classify every finding against the table by its `check` and `reason`, gathering the row's evidence first. One finding with no row -> stop (below) before any write.
|
|
@@ -175,7 +195,7 @@ Stop (a finding with no row, or a finding surviving the third round): when a rou
|
|
|
175
195
|
- no row: `Repair stopped: <check> - <reason> is outside citation repair; the plan file is unchanged. Edit the plan (task changes allowed), then re-run /skill:gauntlet-resume.`
|
|
176
196
|
- third round: `Repair stopped: <check> - <reason> survived 3 rounds. Edit the plan, then re-run /skill:gauntlet-resume.`
|
|
177
197
|
|
|
178
|
-
Resume never edits the spec
|
|
198
|
+
Resume never edits the spec or writes an Open Question itself; the dispatched plan-writer owns generated baseline rows.
|
|
179
199
|
|
|
180
200
|
"Exact" restoration binds: the active phase identity and substep, and the plan task list
|
|
181
201
|
(names, order, statuses) verbatim. Prior phases show `⊘ (resume: ...)` regardless of the
|
|
@@ -43,7 +43,7 @@ Periodic "should I continue?" prompts add latency without adding safety. The pla
|
|
|
43
43
|
- Approved plan or clear task scope
|
|
44
44
|
- `plan_tracker` list initialized at writing-plans handoff with the full wave-ordered task list. Consume and preserve it on continuation; only direct recovery with no tracker initializes the full plan list once, before dispatch, and never over an existing list.
|
|
45
45
|
|
|
46
|
-
Extract **`SCOPED_TEST_COMMANDS`** from the task's `**Tests:**` command bullets, backticks stripped, one per line; `- none:` -> `none`; a wave = the union of its tasks' commands. Treat `via:` bullets and `- Test:` paths as contract, not commands: carry them with the task text, never in `SCOPED_TEST_COMMANDS`.
|
|
46
|
+
Extract **`SCOPED_TEST_COMMANDS`** from the task's `**Tests:**` command bullets, backticks stripped, one per line; `- none:` -> `none`; a wave = the union of its tasks' commands. Treat `via:` and `baseline:` bullets and `- Test:` paths as contract, not commands: carry them with the task text, never in `SCOPED_TEST_COMMANDS`.
|
|
47
47
|
|
|
48
48
|
Keep the orchestrator as the spec's only writer during execution. On implementer `BLOCKED` citing a spec defect, or a review finding showing the spec (not the code) is wrong, pause the fix loop, read and execute brainstorming's [amendment path](../brainstorming/SKILL.md#amending-an-approved-spec) in place (it reviews first, stops only on escalation or redraw), then resume. Keep code-vs-spec mismatch in the SR loop. Require an SR unable to read the spec at a cited anchor (missing file, unresolvable heading/range) to return a blocking finding - the contract is spec+task or stop, never a silent fallback to task-only review.
|
|
49
49
|
|
|
@@ -103,8 +103,10 @@ Every implementer dispatch returns one of:
|
|
|
103
103
|
|---|---|---|
|
|
104
104
|
| `DONE` | Implemented + tests pass + self-reviewed; work present in the worker's tree; the wave commit is step 7 | Proceed to spec review |
|
|
105
105
|
| `DONE_WITH_CONCERNS` | Implemented + tests pass, but flagged a deviation or risk | Read concerns. If acceptable, proceed. If not, dispatch fix subagent. |
|
|
106
|
-
| `NEEDS_CONTEXT` | Implementer needs information you have |
|
|
107
|
-
| `BLOCKED` | Cannot proceed (missing dependency, spec contradicts code, environment broken) |
|
|
106
|
+
| `NEEDS_CONTEXT` | Implementer needs information you have | Resolve fact asks per [lookup handling](./stop-note.md). For decisions, answer and re-dispatch with answers in the task text, or escalate to the user. |
|
|
107
|
+
| `BLOCKED` | Cannot proceed (missing dependency, spec contradicts code, environment broken) | Resolve fact asks per [lookup handling](./stop-note.md). For decisions, stop and diagnose: fix the orchestration-level issue and re-dispatch, or escalate to the user. |
|
|
108
|
+
|
|
109
|
+
Classify each question: artifact location, value, or existence is a fact ask; scope choices, option choices, and another-round approval are decisions.
|
|
108
110
|
|
|
109
111
|
Implementer prompt templates must instruct subagents to return one of these statuses explicitly.
|
|
110
112
|
|
|
@@ -261,7 +263,7 @@ Route the result already read in step 3; retain all successful sibling patches a
|
|
|
261
263
|
|
|
262
264
|
| Outcome | Recovery |
|
|
263
265
|
|---|---|
|
|
264
|
-
| Exit-0 stop (`## Stop`) with no failed siblings | Siblings finished, statuses remain in `## Outputs`, and no SR step ran. Judge `DONE_WITH_CONCERNS`,
|
|
266
|
+
| Exit-0 stop (`## Stop`) with no failed siblings | Siblings finished, statuses remain in `## Outputs`, and no SR step ran. Judge `DONE_WITH_CONCERNS`, handle `NEEDS_CONTEXT` and `BLOCKED` per the status table, then snapshot accepted patches and use the ordinary SR fan-out below. |
|
|
265
267
|
| Chain error, with or without `## Stop` | Identify and diagnose every failed task from the result, including prose `**Status:**` / `Verdict:` recovered through the session-artifact rule above. Completion-guard failure with no edits, rejected/missing `structured_output`, failed `treePath`, and sibling process failure all take this route. Handle a recoverable `BLOCKED` or `NEEDS_CONTEXT` through its status matrix even when that unaccepted task has no patch. Route a task with no recoverable status, or an accepted task with a missing/empty patch, to [When a Subagent Fails](#when-a-subagent-fails) with the visible diagnostic; never invent a status or review an unchanged tree. Retain exit-0 SR verdicts obtained in a materialized tree; dispatch only accepted patches without a verdict. Successful siblings are never re-run. |
|
|
266
268
|
| Completed chain with `NEEDS_REWORK` or `OUT_OF_SCOPE_CHANGES` | Repair only those tasks using the spec-gap loop below. Preserve passing sibling patches/verdicts. |
|
|
267
269
|
| Older-runtime fallback | Parse the plain fan-out statuses, then use the same snapshot-backed SR and fix loop below. |
|
|
@@ -38,13 +38,13 @@ Dispatch a subagent with this prompt:
|
|
|
38
38
|
|
|
39
39
|
Work from: [directory]
|
|
40
40
|
|
|
41
|
-
SCOPED_TEST_COMMANDS: [the task's Tests: bullets, backticks stripped, verbatim | none]
|
|
41
|
+
SCOPED_TEST_COMMANDS: [only the task's Tests: command bullets, backticks stripped, verbatim; exclude via:, baseline:, and none: metadata | none]
|
|
42
42
|
|
|
43
43
|
Run ONLY these commands for verification. Never run a repo-wide suite,
|
|
44
44
|
linter, or type-checker. If the value is `none`, run nothing and say so
|
|
45
45
|
in your report.
|
|
46
46
|
|
|
47
|
-
TEST_CONTRACT: [the task's **Tests:** block verbatim (commands, via:, none:) and its Files: Create:/Test: paths]
|
|
47
|
+
TEST_CONTRACT: [the task's **Tests:** block verbatim (commands, via:, baseline: bullets verbatim, none:) and its Files: Create:/Test: paths]
|
|
48
48
|
|
|
49
49
|
Tests call the via: seam directly - not a wrapper, not the internals behind it.
|
|
50
50
|
Create every Create: path at exactly that path.
|
|
@@ -1,13 +1,16 @@
|
|
|
1
1
|
# Stop note (subagent-driven-development companion)
|
|
2
2
|
|
|
3
|
-
|
|
3
|
+
For a `NEEDS_CONTEXT`/`BLOCKED` fact ask, search the locations documented by the spec, plan, or repo: one bounded lookup per location in documented order, stopping at the first hit. Record each lookup and result, including earlier misses; re-dispatch with the found result in the task text.
|
|
4
|
+
|
|
5
|
+
Emit when no escalation model resolves, the escalated dispatch errors, its one escalated round fails (see `SKILL.md` "Fix-Loop Rounds"), or a `NEEDS_CONTEXT`/`BLOCKED` fact ask exhausts documented locations, needs credentials for a remaining location, or lacks its named lookup tool; an unavailable `gauntlet_setting` tool is a configuration error - stop and report, no stop note. Inline in the reply; the turn ends; phase stays `implement`, the task stays `in_progress`; no further tasks start. Emit one note per stalled task, after the current batch returns.
|
|
4
6
|
|
|
5
7
|
## Template
|
|
6
8
|
|
|
7
9
|
```
|
|
8
10
|
Stopped on task <n> (<title>)[; escalated round on <implModel> did not resolve it].
|
|
9
11
|
|
|
10
|
-
Problem: <one sentence: what is wrong and why
|
|
12
|
+
Problem: <one sentence: what is wrong and why it remains unresolved>
|
|
13
|
+
Searched: <where> - <result> [fact asks only; repeat in lookup order]
|
|
11
14
|
<file:line> - <quoted finding from the final review>
|
|
12
15
|
<failing test/command + 1-3 line output snippet, when present>
|
|
13
16
|
|
|
@@ -22,8 +25,10 @@ Pick one, or give another fix.
|
|
|
22
25
|
## Rules
|
|
23
26
|
|
|
24
27
|
- Bracketed header clause only when an escalated round actually ran; omit it when no model was resolvable or the dispatch errored.
|
|
25
|
-
-
|
|
26
|
-
-
|
|
28
|
+
- For review stalls, quote only residual issues from the final review report; do not summarize history.
|
|
29
|
+
- For fact asks, put one `Searched: <where> - <result>` line per lookup directly under `Problem:`, in lookup order. For an unavailable named tool, write `Searched: <tool> - not available`. Omit `Searched:` for decisions, including extra-round approval.
|
|
30
|
+
- For fact asks, offer supplying the artifact or access, or amending the spec or plan task to drop the requirement. Never ask the human where the artifact is.
|
|
31
|
+
- When no escalated round ran, state the reason in `Problem:`: exhausted or inaccessible fact lookups, "no escalation model resolvable", or the quoted implementer's non-DONE status text / dispatch error; skip the file:line and test lines.
|
|
27
32
|
- Options are actionable edits. Spec/plan amendment is first-class - stalls are usually a slightly contradictory spec, not a capability gap.
|
|
28
33
|
- Never offer "skip the task". If the task is genuinely droppable, say so and name the plan tasks that depend on it.
|
|
29
34
|
- No trajectory verdicts, round history, review counts, or paths to spec/plan/review reports.
|
|
@@ -72,9 +72,15 @@ The main loop never writes the plan. One fresh `plan-writer` child does the reco
|
|
|
72
72
|
5. `plan_check({ planPath: "<abs plan path>" })` in the parent. This is the stamping call; run it whichever line the child replied with.
|
|
73
73
|
6. Branch on the pair (child reply, parent check):
|
|
74
74
|
- `PLAN WRITTEN` + `PASS` -> step 7.
|
|
75
|
-
- `OPEN QUESTIONS` + any result ->
|
|
75
|
+
- `OPEN QUESTIONS` + any result -> read the section; absent or exactly `- none` takes the `PLAN WRITTEN` branch. For substantive rows, use the table below. On a pause retain phase `plan` as `in_progress` and its substep; no stamp or `plan_tracker` init.
|
|
76
|
+
|
|
77
|
+
| Row form | Parent handling |
|
|
78
|
+
|---|---|
|
|
79
|
+
| `- Task N: ... fails on <sha>` | Name task, original command, SHA, exit and summary. Present three options, one per line: `fix-first` - run `/skill:shape-ticket` for the base failure (its own gate applies), then report the plan paused and end the turn; `narrow` - the human supplies filter or path text; `amend` - use brainstorming's `reference/amendment-surface.md` section 5. Follow with `Recommendation: <option> - <reason>`, then wait. After fix-first lands, bring the advanced base into the worktree through `/skill:using-git-worktrees`' existing advanced-base procedure (rebase, or merge for a shared branch), re-dispatch per step 3 with `Findings: rerun baseline`, then step 5; after a session break use `/skill:gauntlet-resume`'s baseline continuation. For narrow, re-dispatch with a Findings row `Task N Tests: narrow to `<text>``; for amend, use section 5. Both run Baseline's rerun rule before step 5. |
|
|
80
|
+
| `- Baseline: worktree dirty`, `- Baseline: in-flight source edits`, `dirtied tracked source`, `moved HEAD` | Restore-and-rerun, not the menu: name paths and cause. Clear the parent's own wave/amendment source work through its own commit step; list doc-only work under `In-flight:` before re-dispatch. Take every other path, including every test-dirtied path, to the human to commit, discard, or move; never clean automatically. Once restored, re-dispatch per step 3 with `Findings: rerun baseline`, then step 5. Bullet `artifacts:` need no action. |
|
|
81
|
+
| Other substantive rows (contract questions) | Present the rows, wait for answers, re-dispatch per step 3 with the answers as the `Findings:` block, then step 5. |
|
|
76
82
|
- `PLAN WRITTEN` + `FAIL` -> re-dispatch per step 3 with the verbatim `FAIL (...)` output as the `Findings:` block, then step 5. Once per verbatim FAIL output: the same FAIL a second time goes to the user as an open question (the FAIL lines, verbatim), as an exhausted finding does.
|
|
77
|
-
- No child reply (the step 3 entry branch on a restored plan) -> read the result as if the reply were `OPEN QUESTIONS` when the plan carries
|
|
83
|
+
- No child reply (the step 3 entry branch on a restored plan) -> read the result as if the reply were `OPEN QUESTIONS` when the plan carries substantive Open Questions rows, else `PLAN WRITTEN` (section absent or exactly `- none`), and take that reply's branch above; a `FAIL` re-dispatch overwrites the restored plan with a repaired one.
|
|
78
84
|
7. Clear the substep (`phase_tracker({ action: "substep", phase: "plan", substep: null })`). Read the plan once, then `## Execution Handoff` as written below: `phase_tracker({ action: "complete", phase: "plan" })`, `plan_tracker({ action: "init", ... })` from the `W<k>: <title>` list, `/skill:subagent-driven-development`.
|
|
79
85
|
|
|
80
86
|
The parent holds no run id and never calls `resume` or `status`; every dispatch here is `async: false`. Re-entering this section without a restored plan overwrites whatever the plan path holds - a committed prior plan (recoverable from git history) or an uncommitted one (destroyed); re-planning is a deliberate overwrite. An amendment of the approved spec after the plan exists re-enters at step 3 with an `Amend:` block (brainstorming's `reference/amendment-surface.md` section 5 owns that call).
|
|
@@ -192,6 +198,7 @@ Each task uses `- [ ]` checkbox steps so execution tools (and humans) can track
|
|
|
192
198
|
|
|
193
199
|
**Tests:**
|
|
194
200
|
- `uv run pytest tests/exact/path/to/test.py`
|
|
201
|
+
- baseline: abc1234 pass
|
|
195
202
|
- via: `function()`
|
|
196
203
|
|
|
197
204
|
- [ ] **Step 1: Write the failing test**
|
|
@@ -266,9 +273,30 @@ If a decision is genuinely open, put it in an explicit **Open Questions** sectio
|
|
|
266
273
|
|
|
267
274
|
## Self-Review (Before Handoff)
|
|
268
275
|
|
|
269
|
-
The plan-writer runs
|
|
276
|
+
The plan-writer runs Baseline, the deterministic checker, then the judgment checks before replying.
|
|
277
|
+
|
|
278
|
+
- **Baseline.** Execute the numbered procedure below for task `Tests:` commands only; leave header `**Verification:**` to verify. Never author baseline evidence without execution.
|
|
279
|
+
1. Read provenance: `BASE=$(git -C "<abs worktree path>" rev-parse --short HEAD)` and `git -C "<abs worktree path>" status --porcelain --untracked-files=all`. Match every `XY <path>` entry (including `??`) against the worktree-relative plan and spec paths, plus only the parent-supplied `In-flight:` paths on an `Amend:` re-dispatch; never infer that set. Also allow untracked paths already recorded in existing baseline bullets' ` - artifacts:` suffixes in the plan at the plan path, excluding every plan `Create:` or `Test:` path. Keep those suffixes on retained bullets. Any other entry blocks all commands: collect `- Baseline: worktree dirty on <BASE>: <paths>`. Allowed in-flight paths outside the spec and plan directories are source edits: collect `- Baseline: in-flight source edits on <BASE>: <paths>` and run nothing. Allow doc-only workflow edits. For plain jj (no `.git`, VCS kind `jj`), use `BASE=$(jj -R "<abs worktree path>" log -r @- --no-graph -T 'commit_id.short()')` and `jj -R "<abs worktree path>" diff --summary`; classify one-letter status and relative paths identically (`A` includes new files). Never revert or clean anything.
|
|
280
|
+
2. On Findings/Amend, retain a bullet only for unchanged command text on the same base; rerun commands with missing/invalid bullets, changed text, or different recorded SHA. `Findings: rerun baseline` reruns all commands. Never rewrite recorded SHAs without execution. For each command, split on the contract's separators (`&&`, `||`, `;`, `|`) and classify anchor tokens (a `Test:` path alone or with `::`/`#`/`:` filters) and selectors independently:
|
|
281
|
+
|
|
282
|
+
| Command anchors/selectors | Baseline action |
|
|
283
|
+
|---|---|
|
|
284
|
+
| Every anchor in every segment is a `Create:` path of some task | Append `- baseline: <BASE> skipped - <path>[, <path>] created by this task`, one path per created anchor. Only all-created anchors skip. |
|
|
285
|
+
| No segment has a created anchor or introduced selector | Run `(cd "<abs worktree path>" && <command>)` unchanged. |
|
|
286
|
+
| Mixed anchors or introduced selectors | Derive per segment: drop each created-anchor token; strip an introduced attached filter to its path; drop an introduced flag selector and its value. Drop a segment with no existing `Test:` anchor of this task. Join surviving segments with their original separators. Keep at least one segment, each anchored to an existing `Test:` path of this task and no `Create:` path; otherwise skip only for all-created anchors as above. Run the derived base command. |
|
|
287
|
+
|
|
288
|
+
An introduced selector is an attached filter or flag/value pair (`-k <expr>`, `-t <name>`, `-e <name>`, `-n <name>`, `--grep <name>`, `--test-name-pattern <name>`) whose filter/name appears verbatim in a test name the task's steps say they add. Keep selectors the steps do not name and run unchanged. Pass requires exit 0 and a runner summary reporting at least one test executed. Append `- baseline: <BASE> pass` for unchanged runs or `- baseline: <BASE> pass via `<base command>`` for derived runs. Treat non-zero exits, host allowlist/runtime errors, and exit-0 summaries with zero tests as failures; collect task, original command, exit code, summary and last 20 output lines, plus the derived command when used. Record a zero-test result as `exit <n>, no tests ran`, never a harmless skip. `- none:` tasks get no baseline.
|
|
289
|
+
3. After each run reread status and BASE with step 1's commands. New tracked status outside the allowed set, a new `??` (jj `A`) at any plan `Create:` or `Test:` path, or moved BASE contaminates the run: collect the task, original command and `dirtied tracked source: <paths>` or `HEAD moved to <sha>`; stop further runs and record every unrun command. Any other new untracked entry is a disposable generated artifact: keep the result and append ` - artifacts: <path>[, <path>]` to its `pass` or `pass via` bullet. For each rerun (changed command text, missing/invalid bullet, different base, or `Findings: rerun baseline`), carry its recorded artifact paths still present as untracked onto the replacement bullet's ` - artifacts:` suffix, merged with new untracked entries from that run and excluding every plan `Create:` or `Test:` path. Record artifacts, never delete them. Do not give a contaminated command a passing bullet.
|
|
290
|
+
4. Regenerate every baseline-form Open Questions row from this pass's collection; resolved rows disappear and `blocked:` lines are recomputed. Preserve unresolved contract rows until Findings answers them. Put `## Open Questions` at the top when any rows remain; omit it otherwise. Use these exact forms:
|
|
291
|
+
|
|
292
|
+
- Failure: `- Task N: `<command>` fails on <BASE> (exit <n>): <one-line result summary>`, quoting the original Tests command. Indent a `ran: `<base command>`` continuation for a derived run, then a fenced block containing the last 20 output lines (or all output when shorter).
|
|
293
|
+
- Contamination: `- Task N: `<command>` dirtied tracked source on <BASE>: <paths>` or `- Task N: `<command>` moved HEAD to <sha>`.
|
|
294
|
+
- Global: `- Baseline: worktree dirty on <BASE>: <paths>` or `- Baseline: in-flight source edits on <BASE>: <paths>`.
|
|
295
|
+
- Under each contamination/global row indent `blocked: Task M `<command>`` for every command it blocked. Keep failure output verbatim, including `TODO` or `etc.` inside valid continuation fences; only those evidence fences are exempt from placeholder scanning.
|
|
270
296
|
|
|
271
|
-
|
|
297
|
+
With substantive rows remaining reply `OPEN QUESTIONS <abs plan path>` immediately; do not spend repair rounds on them.
|
|
298
|
+
- **Deterministic checker.** Run `plan_check({ planPath: "<abs plan path>" })` on the saved plan. Repair every `baseline-line` finding (missing, malformed, misplaced, invalid skipped/pass via or differing SHAs) and substantive baseline `open-questions` finding only through Baseline, never by authoring `pass`/`skipped` evidence or deleting/rewording a row. Normalize `## Open Questions malformed` into `- ` rows without changing substantive text. Fix every other finding yourself and rerun. With substantive rows remaining, reply `OPEN QUESTIONS <abs plan path>` without a passing check. Apply the 3-round budget only to ordinary findings: convert a finding surviving 3 rounds to an Open Questions row and stop. A child pass validates; the parent's call writes the execution stamp. Omit an empty questions section and reply `PLAN WRITTEN <abs plan path>` after all checks pass. Findings are defined in [reference/plan-contract.md](reference/plan-contract.md).
|
|
299
|
+
- **Contract decisions.** For every task crossing a boundary the spec leaves unsettled (a host-forwarded value, a caller's error type), derive the decision from code, the harness contract, or a sibling task. State `Decision: <what> - <caller/contract/sibling task cited>` in the task text. Ask only when the spec is silent or wrong and no source settles it; put that unresolved contract in Open Questions. Add no per-task `Decisions:` field.
|
|
272
300
|
- **Code-vs-anchor sanity.** For each task-owned requirement row, re-read the anchored spec lines and confirm the owner tasks' bodies do what they say - mechanism present, not just the quoted literal. For each `Verification` row, confirm the header command exercises the anchored requirement. Fix the task, don't annotate.
|
|
273
301
|
- **Type / API consistency.** Function signatures and field names that appear in multiple tasks must match exactly. The plan is its own contract — internal contradictions surface as bugs during execution.
|
|
274
302
|
- **Test contract.** Every code task's `Tests:` commands are anchored to its `Test:` path(s); `none:` only where no tests apply; a spec-named seam appears as `via:`, a spec-named fixture path as `Create:`. Prefer the existing test file for the behavior as `Test:`; a new test file is declared as both `Create:` and `Test:`.
|
|
@@ -20,14 +20,24 @@ A wave is a maximal set of tasks that have no ordering dependency on each other,
|
|
|
20
20
|
|
|
21
21
|
## Tests (`tests-block`)
|
|
22
22
|
|
|
23
|
-
Every `### Task N` carries a `**Tests:**` block: the bare line `**Tests:**` directly after the last `Files:` entry (blank lines allowed).
|
|
23
|
+
Every `### Task N` carries a `**Tests:**` block: the bare line `**Tests:**` directly after the last `Files:` entry (blank lines allowed). Use command groups or the no-tests form:
|
|
24
24
|
|
|
25
|
-
- `- ` + backtick + command + backtick - one scoped test command; `- via: <entry point>`
|
|
25
|
+
- `- ` + backtick + command + backtick - one scoped test command; directly after it exactly one `- baseline:` bullet, then zero or more `- via: <entry point>` bullets naming the seam tests call (function, route, CLI, module).
|
|
26
26
|
- `- none: <category>` - the task runs no tests; `<category>` is free text derived from the project (docs, config, fixtures, generated assets, ...); exactly one, and no `- Test:` path in `Files:`
|
|
27
27
|
|
|
28
28
|
Absence is never valid. A `- [ ]` step or any non-bullet line ends the block. `via:` and `none:` are unchecked beyond form.
|
|
29
29
|
|
|
30
|
-
|
|
30
|
+
Baseline forms (`baseline-line`): `- baseline: <sha> pass`, `- baseline: <sha> pass via `<base command>`` (both allow ` - artifacts: <path>[, <path>]`), or `- baseline: <sha> skipped - <path>[, <path>] created by this task`. Use 7-40 lowercase hex characters for `<sha>`; every baseline SHA in the plan must match. Never put a baseline under `none:`. `skipped` is valid only when every command anchor is a listed `Create:` path and every listed path is one of those anchors. Every segment of a `pass via` base command must anchor an existing `Test:` path of the task and no `Create:` path. Obtain evidence through writing-plans Self-Review Baseline, never by editing evidence to satisfy a finding.
|
|
31
|
+
|
|
32
|
+
Commands are written repo-relative and run in the worktree via `(cd "<abs worktree path>" && <command>)` - the checker never runs them; Baseline and the executor do. Each command is split into segments on `&&`, `||`, `;`, `|`; every segment must contain, as a whitespace-delimited token, a `Test:` path of the same task (the path alone, or followed by `::`, `#`, or `:` and a filter). A `Test:` value containing `*`, `?`, `[` or ending in `/` never anchors; any other argument token with those shapes is a broadening selector and fails. `cd `, `sh -c`, `bash -c`, `eval `, `$(` are unsupported. Each `Test:` path must exist or be a `Create:` path of some task. A segment equal to a header `**Verification:**` or `**Happy path:**` segment is a full-suite command and fails. Runners with no file-addressable form are out of scope (`go test ./pkg -run X`, `mvn -Dtest=`).
|
|
33
|
+
|
|
34
|
+
## Open Questions (`open-questions`)
|
|
35
|
+
|
|
36
|
+
Bound the section from `## Open Questions` to the next `## ` heading or EOF. Use `- ` rows with indented continuations (`ran:`, `blocked:`, fenced output). The empty sentinel is exactly one line `- none`; an absent section also passes. Every substantive row produces one finding quoting its first line. Prose and tables are malformed: normalize them to rows without changing substantive content.
|
|
37
|
+
|
|
38
|
+
Baseline failure: `- Task N: `<command>` fails on <sha> (exit <n>): <summary>`. Derived runs add an indented `ran: `<base command>``; fence the last 20 output lines below it. Contamination: `- Task N: `<command>` dirtied tracked source on <sha>: <paths>` or `- Task N: `<command>` moved HEAD to <sha>`. Global rows: `- Baseline: worktree dirty on <sha>: <paths>` and `- Baseline: in-flight source edits on <sha>: <paths>`. Add indented `blocked: Task M `<command>`` continuations for every command each contamination/global row blocked. Baseline regenerates these rows; contract rows remain until answered.
|
|
39
|
+
|
|
40
|
+
Only fenced output continuing a well-formed row is exempt from `placeholder-scan` (verbatim tails can contain `TODO` or `etc.`). Row first lines, `ran:`/`blocked:` and authored task prose remain scanned.
|
|
31
41
|
|
|
32
42
|
## Header-only entrypoint (`header-entrypoint`)
|
|
33
43
|
|