pi-gauntlet 5.0.8 → 5.2.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +11 -0
- package/README.md +21 -6
- package/extensions/lib/plan-check.test.ts +485 -0
- package/extensions/lib/plan-check.ts +829 -0
- package/extensions/phase-tracker.test.ts +469 -2
- package/extensions/phase-tracker.ts +142 -3
- package/package.json +1 -1
- package/skills/brainstorming/gatherer.md +4 -1
- package/skills/chase-bug/SKILL.md +16 -2
- package/skills/check-delivery/SKILL.md +18 -7
- package/skills/dispatching-parallel-agents/SKILL.md +8 -1
- package/skills/linear/SKILL.md +188 -0
- package/skills/requesting-code-review/SKILL.md +1 -1
- package/skills/shape-ticket/SKILL.md +9 -9
- package/skills/subagent-driven-development/SKILL.md +1 -1
- package/skills/writing-plans/SKILL.md +6 -9
package/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,16 @@
|
|
|
1
1
|
# Changelog
|
|
2
2
|
|
|
3
|
+
## v5.2.0 - 2026-08-31
|
|
4
|
+
|
|
5
|
+
- **`plan_check` deterministic plan checker (new tool).** `phase-tracker` now registers `plan_check({ planPath })`, running 8 mechanical plan-vs-spec checks (table closure, quote integrity, anchor resolution, path existence, placeholder scan, wave file-disjointness, solo-line presence, header-only entrypoint) from a pi-free library at `extensions/lib/plan-check.ts`. Fail-closed: it never throws, and findings are returned for the main model to fix autonomously.
|
|
6
|
+
- **Implement-start gate (enforcing by default).** Inside a gauntlet flow, `phase_tracker({ action: "start", phase: "implement" })` is rejected unless a passing `plan_check` stamp exists and both the plan and spec files still match their stamped content hashes. Sixth flow guard, block-only, no new settings key - disable with `piGauntlet.flowGuards.enforce: false`. Outside a flow, `plan_check` acts as a plain linter.
|
|
7
|
+
- **`writing-plans` Self-Review** is now one deterministic-checker bullet plus seven judgment bullets; the mechanical checks are no longer LLM-attested.
|
|
8
|
+
- **`Parallel-safe:` structural probe.** The three consumer skills (`dispatching-parallel-agents`, `requesting-code-review`, `subagent-driven-development`) replace silent degradation with an explicit probe -> one reviewer re-ask -> explicit sequential fallback, with the `<group>` production now defined at the canonical consumer. Prose-only; the five pinned producer grammar copies are untouched. Fixes #19. Spec: `doc/specs/2026-08-31-gh-19-deterministic-plan-checker.md`.
|
|
9
|
+
|
|
10
|
+
## v5.1.0 - 2026-08-31
|
|
11
|
+
|
|
12
|
+
- New skill `linear`: package-owned home for all linearis mechanics (verb reference, six gotchas, multi-line body pattern, ID-cache convention, failure modes, MCP fallback) and the five-key overrides `## Issue tracker` schema, including the explicit `tracker:` off-switch (`linear` / `github` / `none` - exclusive selection, zero probing when non-Linear). shape-ticket, check-delivery, and chase-bug slim to detection + a `/skill:linear` routing line (chase-bug's fabricated `linearis comment` example fixed); brainstorming's gatherer fetch-path definition honors the off-switch; `./skills/linear` joins the Claude Code marketplace allowlist (five exposed skills). Fixes #17. Spec: `doc/specs/2026-08-31-gh-17-linear-skill.md`.
|
|
13
|
+
|
|
3
14
|
## v5.0.8 - 2026-08-31
|
|
4
15
|
|
|
5
16
|
- shape-ticket: ticket bodies must be self-contained - new wording contract in `skills/shape-ticket/reference/ticket-wording.md` (repo-stranger reader, strip-test, plain-words lead + example per asserted failure, jargon defined, references demoted to deletable parenthetical pointers, big chunks linkable with a summary line - no spiderman tickets). Wired into the draft step, every roast brief (members, worker fallback, runtime conditional; fidelity exempts contract-driven unpacking from `added`), and the repair-mode no-op check (a violating body is non-conforming and gets a proposed rewrite). Generic repo density norms can no longer override ticket prose; only an explicit ticket-wording overrides section can. Fixes #18. Spec: `doc/specs/2026-08-31-gh-18-self-contained-ticket-wording.md`.
|
package/README.md
CHANGED
|
@@ -69,9 +69,9 @@ Everything between gate 1 and gate 2 - task breakdown, implementation, both revi
|
|
|
69
69
|
|
|
70
70
|
pi-gauntlet ships three kinds of pieces, layered on top of pi-cohort's dispatch:
|
|
71
71
|
|
|
72
|
-
- **
|
|
72
|
+
- **17 skills** - the workflow logic. Thirteen activate automatically when pi sees the matching kind of task, and each one gates the next: `brainstorming`, `writing-plans`, `roasting-the-spec`, `test-driven-development`, `subagent-driven-development`, `dispatching-parallel-agents`, `verification-before-completion`, `requesting-code-review`, `receiving-code-review`, `using-git-worktrees`, `finishing-a-development-branch`, `writing-skills`, `linear` (reads/searches/comments on/manages Linear tickets via the `linearis` CLI; owns all linearis mechanics and the `## Issue tracker` overrides schema; tracker-facing skills route to it). Four more are explicit-invocation-only (`disable-model-invocation: true`): `shape-ticket` creates or repairs one tracker issue per run against a Context/Problem/Idea/Acceptance-Criteria template, gated by an AC integrity check, a cheap council roast, and a single human-confirmed write - run it with `/skill:shape-ticket`. `gatekeep-pr` is consent-gated pre-merge verification of a PR against its issue - read-only gathering, verification evidence resolved CI-first (green checks on the exact assessed head count as evidence; the project's verification command runs only as fallback), a rubric-based review, then a deterministic authorship-aware menu with stable finding IDs (P#/L#/C#/F#) and numbered pre-composed courses (fixes execute as a single parallel-safe wave: one gate run, one re-review, one push); nothing mutates (fixes, pushes, reviews, merges) until you pick a row - run it with `/skill:gatekeep-pr <pr>`. `check-delivery` is a post-merge detective control: proves an issue actually shipped (default-branch landing, delivery target, per-AC evidence) before its tracker status advances; it never writes a terminal status - run it with `/skill:check-delivery <ref>`. `chase-bug` is human-only bug triage: read-only root-cause discovery to an evidenced verdict menu (real bug -> ticket/brainstorm/respond; five negative verdicts), then a gated response to the reporter for addressable origins (GitHub issue / tracker ticket) and a rendered verdict summary otherwise - it never fixes during triage - run it with `/skill:chase-bug`.
|
|
73
73
|
- **7 subagent personas** - the specialized child agents the skills dispatch via pi-cohort: `implementer`, `code-reviewer`, `spec-reviewer`, `conformance-reviewer`, `spec-summarizer`, `spec-council-member`, `spec-council-synthesizer`. See [doc/personas.md](./doc/personas.md) for what each one does and why its permissions are scoped the way they are.
|
|
74
|
-
- **3 runtime extensions** - the enforcement layer. `plan-tracker` and `phase-tracker` are tools skills call to track progress (with a TUI widget); `verify-before-ship` is a hook that warns if you push or open a PR without a passing test run since your last edit; a phase-tracker flow guard reminds on implement-phase commits missing spec/code review. See [doc/configuration.md](./doc/configuration.md) for the settings each one reads.
|
|
74
|
+
- **3 runtime extensions** - the enforcement layer. `plan-tracker` and `phase-tracker` are tools skills call to track progress (with a TUI widget); `verify-before-ship` is a hook that warns if you push or open a PR without a passing test run since your last edit; a phase-tracker flow guard reminds on implement-phase commits missing spec/code review. phase-tracker also registers plan_check, a deterministic plan linter (8 mechanical plan-vs-spec checks) whose pass stamp gates implement-start inside a gauntlet flow. See [doc/configuration.md](./doc/configuration.md) for the settings each one reads.
|
|
75
75
|
|
|
76
76
|
pi-gauntlet is **opinionated**: every non-trivial change is *meant* to ride this one pipeline, entered through `brainstorming`. Enforcement is opt-in by entry, not ambient: once brainstorming starts a flow, the phase-tracker extension mechanically blocks a phase from closing before its gate runs, and warns once if the main loop writes code during implement (subagents own implement-phase edits). A change made *without* entering the flow (a typo, a formatting run, a dependency bump - see "When to use / when NOT to use") is not gated; the discipline of routing real work through the pipeline is a convention the tooling supports, not a trap it springs on every edit.
|
|
77
77
|
|
|
@@ -126,9 +126,9 @@ cd ~/repos/pi-gauntlet && npm run link-agents # local-path installs skip npm i
|
|
|
126
126
|
|
|
127
127
|
## Use from Claude Code
|
|
128
128
|
|
|
129
|
-
|
|
129
|
+
Five skills are exposed to Claude Code via the plugin marketplace at
|
|
130
130
|
`.claude-plugin/marketplace.json`: **shape-ticket**, **gatekeep-pr**,
|
|
131
|
-
**check-delivery**,
|
|
131
|
+
**check-delivery**, **chase-bug**, and **linear**. They are harness-portable by design - every pi-specific
|
|
132
132
|
mechanic they touch (`plan_tracker`, `gauntlet_setting`, `subagent()`) carries
|
|
133
133
|
an inline fallback, so they run on Claude Code's native facilities. This is the
|
|
134
134
|
supported set. Not exposed, in two classes: (a) genuinely pi-bound surface -
|
|
@@ -202,8 +202,9 @@ exact repo folder* in interactive Claude Code. Trusting a parent folder,
|
|
|
202
202
|
4. Run `/plugin` and confirm: marketplace `pi-gauntlet` is listed, plugin
|
|
203
203
|
`gauntlet` is enabled. If it shows as known but not installed, run
|
|
204
204
|
`/plugin install gauntlet@pi-gauntlet` and re-check.
|
|
205
|
-
5. Confirm exactly
|
|
206
|
-
`/plugin` details view): shape-ticket, gatekeep-pr, check-delivery, chase-bug
|
|
205
|
+
5. Confirm exactly five skills are registered under the plugin (via the
|
|
206
|
+
`/plugin` details view): shape-ticket, gatekeep-pr, check-delivery, chase-bug,
|
|
207
|
+
linear.
|
|
207
208
|
6. Invoke `/gauntlet:shape-ticket` with a deliberately two-concern ask (e.g.
|
|
208
209
|
"shape a ticket: CSV import for operators, plus a partner-facing status
|
|
209
210
|
API") so the skill deterministically consults its
|
|
@@ -261,6 +262,20 @@ Use the `jira` CLI (authenticated via `jira login`), not `gh` or `linearis`.
|
|
|
261
262
|
- post comment (Reporter note only): `jira issue comment ABC-123 --body "<text>"`
|
|
262
263
|
```
|
|
263
264
|
|
|
265
|
+
**`linear` setup:** mandatory - `linearis` installed and authenticated (or a Linear
|
|
266
|
+
MCP server as a fallback when `linearis` is unavailable). Optional - the five
|
|
267
|
+
`## Issue tracker` override keys (`tracker`, `workspace urlKey`, `default team`,
|
|
268
|
+
`self`, `id cache`); the full schema is documented once, in
|
|
269
|
+
[skills/linear/SKILL.md](./skills/linear/SKILL.md) - not restated here. Off switch:
|
|
270
|
+
set `tracker: github` or `tracker: none` in `## Issue tracker` to disable Linear
|
|
271
|
+
entirely - no `linearis` probing, no prompts.
|
|
272
|
+
|
|
273
|
+
**Coexistence:** the five `## Issue tracker` keys compose with the free-form
|
|
274
|
+
command-mapping convention above, they don't replace it. `tracker:` adds exclusive
|
|
275
|
+
tracker selection; free-form verb mappings keep working both without a `tracker:`
|
|
276
|
+
key (ladder rung 1, as today) and as the mechanics source when `tracker:` names an
|
|
277
|
+
unknown value.
|
|
278
|
+
|
|
264
279
|
**`## Deployment` section:** `shape-ticket` (split rule), `writing-plans` (scope check), and `brainstorming` (scope check) read deploy topology from this section: what ships together, what ships independently, and the mechanism. It is a fact to look up, never to infer - when the section is absent, or when it documents a monolithic topology (like the example below), the "separable release timing" split axis is unavailable and splits fail closed to one artifact.
|
|
265
280
|
|
|
266
281
|
```markdown
|
|
@@ -0,0 +1,485 @@
|
|
|
1
|
+
import assert from "node:assert/strict";
|
|
2
|
+
import test from "node:test";
|
|
3
|
+
import { checkPlan, sha256, type FsPort, type PlanCheckFinding } from "./plan-check.ts";
|
|
4
|
+
|
|
5
|
+
const SPEC_TEXT = [
|
|
6
|
+
"# Fixture Spec", // 1
|
|
7
|
+
"", // 2
|
|
8
|
+
"## Design", // 3
|
|
9
|
+
"Line A.", // 4
|
|
10
|
+
"This part defines `helperFn()` config.", // 5
|
|
11
|
+
"Another line here.", // 6
|
|
12
|
+
"", // 7
|
|
13
|
+
"## Testing", // 8
|
|
14
|
+
"This mentions `TODO` as a banned token example for exemption checking.", // 9
|
|
15
|
+
"More text.", // 10
|
|
16
|
+
"", // 11
|
|
17
|
+
"## Other", // 12
|
|
18
|
+
"Stuff.", // 13
|
|
19
|
+
].join("\n");
|
|
20
|
+
|
|
21
|
+
const VALID_PLAN = `# Fixture Plan
|
|
22
|
+
|
|
23
|
+
**Spec:** \`doc/specs/fixture-spec.md\`
|
|
24
|
+
|
|
25
|
+
**Verification:** npm run fixture-verify
|
|
26
|
+
|
|
27
|
+
---
|
|
28
|
+
|
|
29
|
+
## Wave 1 — Two parallel tasks
|
|
30
|
+
|
|
31
|
+
### Task 1: Implement helper
|
|
32
|
+
|
|
33
|
+
**Spec:** doc/specs/fixture-spec.md § "Design" L4-L6
|
|
34
|
+
|
|
35
|
+
**Files:**
|
|
36
|
+
- Create: extensions/lib/fixture-task1.ts
|
|
37
|
+
- Modify: extensions/lib/fixture-shared.ts
|
|
38
|
+
|
|
39
|
+
This task implements helperFn() for parsing.
|
|
40
|
+
|
|
41
|
+
### Task 2: Implement naming
|
|
42
|
+
|
|
43
|
+
**Spec:** doc/specs/fixture-spec.md § "Design" L4-L4
|
|
44
|
+
|
|
45
|
+
**Files:**
|
|
46
|
+
- Create: extensions/lib/fixture-task2.ts
|
|
47
|
+
- Modify: extensions/lib/fixture-other.ts
|
|
48
|
+
|
|
49
|
+
This task handles naming details.
|
|
50
|
+
|
|
51
|
+
## Wave 2 — Solo task
|
|
52
|
+
|
|
53
|
+
Solo: lone remaining task
|
|
54
|
+
|
|
55
|
+
### Task 3: Document banned token handling
|
|
56
|
+
|
|
57
|
+
**Spec:** doc/specs/fixture-spec.md § "Testing" L9
|
|
58
|
+
|
|
59
|
+
**Files:**
|
|
60
|
+
- Modify: extensions/lib/fixture-task3.ts
|
|
61
|
+
|
|
62
|
+
The literal TODO is intentionally documented here per spec quote-integrity requirement.
|
|
63
|
+
|
|
64
|
+
## Spec coverage
|
|
65
|
+
|
|
66
|
+
| anchor | requirement | owner |
|
|
67
|
+
|---|---|---|
|
|
68
|
+
| § "Design" L4-L6 | parser grammar basics | Task 1 |
|
|
69
|
+
| § "Design" L4-L4 | helper naming | Task 2 |
|
|
70
|
+
| § "Testing" L9-L9 | banned token literal handling | Task 3 |
|
|
71
|
+
| - | mechanical: wire test into CI | Task 2 |
|
|
72
|
+
| § "Other" L12-L13 | out of scope thing | waived: out of scope per spec |
|
|
73
|
+
`;
|
|
74
|
+
|
|
75
|
+
function alwaysTruePort(): FsPort {
|
|
76
|
+
return {
|
|
77
|
+
exists: () => true,
|
|
78
|
+
glob: () => [],
|
|
79
|
+
};
|
|
80
|
+
}
|
|
81
|
+
|
|
82
|
+
function lineOf(text: string, needle: string): number {
|
|
83
|
+
const idx = text.split("\n").findIndex((l) => l.includes(needle));
|
|
84
|
+
assert.notEqual(idx, -1, `expected to find a line containing ${JSON.stringify(needle)}`);
|
|
85
|
+
return idx + 1;
|
|
86
|
+
}
|
|
87
|
+
|
|
88
|
+
function findingsFor(findings: PlanCheckFinding[], check: string): PlanCheckFinding[] {
|
|
89
|
+
return findings.filter((f) => f.check === check);
|
|
90
|
+
}
|
|
91
|
+
|
|
92
|
+
test("valid fixture: checkPlan returns no findings (also covers the placeholder exemption case)", () => {
|
|
93
|
+
const findings = checkPlan(VALID_PLAN, SPEC_TEXT, alwaysTruePort());
|
|
94
|
+
assert.deepEqual(findings, []);
|
|
95
|
+
});
|
|
96
|
+
|
|
97
|
+
test("no gauntlet grammar at all: emits both an absent-wave-headers and an absent-task-headers finding", () => {
|
|
98
|
+
const plan = "# Just a plan\n\nSome prose with no wave or task headers at all.\n";
|
|
99
|
+
const findings = checkPlan(plan, SPEC_TEXT, alwaysTruePort());
|
|
100
|
+
const waveFindings = findings.filter((f) => f.reason.includes("wave headers"));
|
|
101
|
+
const taskFindings = findings.filter((f) => f.reason.includes("task headers"));
|
|
102
|
+
assert.equal(waveFindings.length, 1, "expected exactly one absent-wave-headers finding");
|
|
103
|
+
assert.equal(taskFindings.length, 1, "expected exactly one absent-task-headers finding");
|
|
104
|
+
});
|
|
105
|
+
|
|
106
|
+
test("check 1 table-closure: task not covered by any Spec coverage row", () => {
|
|
107
|
+
const mutated = VALID_PLAN.replace('| § "Testing" L9-L9 | banned token literal handling | Task 3 |\n', "");
|
|
108
|
+
const findings = checkPlan(mutated, SPEC_TEXT, alwaysTruePort());
|
|
109
|
+
const tc = findingsFor(findings, "table-closure");
|
|
110
|
+
assert.ok(
|
|
111
|
+
tc.some((f) => f.reason.includes("Task 3") && f.reason.includes("does not appear as an owner")),
|
|
112
|
+
`expected a table-closure finding about Task 3 not covered, got: ${JSON.stringify(tc)}`,
|
|
113
|
+
);
|
|
114
|
+
assert.ok(
|
|
115
|
+
tc.some(
|
|
116
|
+
(f) =>
|
|
117
|
+
f.reason.includes("Task 3") &&
|
|
118
|
+
f.line === lineOf(mutated, "### Task 3: Document banned token handling"),
|
|
119
|
+
),
|
|
120
|
+
`expected the table-closure finding's line to point at Task 3's header, got: ${JSON.stringify(tc)}`,
|
|
121
|
+
);
|
|
122
|
+
});
|
|
123
|
+
|
|
124
|
+
test("check 1 fail-closed: missing '## Spec coverage' table entirely", () => {
|
|
125
|
+
const mutated = VALID_PLAN.slice(0, VALID_PLAN.indexOf("## Spec coverage"));
|
|
126
|
+
const findings = checkPlan(mutated, SPEC_TEXT, alwaysTruePort());
|
|
127
|
+
const tc = findingsFor(findings, "table-closure");
|
|
128
|
+
assert.equal(tc.length, 1);
|
|
129
|
+
assert.match(tc[0].reason, /no '## Spec coverage' table found/);
|
|
130
|
+
assert.equal(tc[0].line, 0);
|
|
131
|
+
});
|
|
132
|
+
|
|
133
|
+
test("check 2 quote-integrity: required verbatim literal missing from owner task body", () => {
|
|
134
|
+
const mutated = VALID_PLAN.replace(
|
|
135
|
+
"This task implements helperFn() for parsing.",
|
|
136
|
+
"This task implements the helper for parsing.",
|
|
137
|
+
);
|
|
138
|
+
const findings = checkPlan(mutated, SPEC_TEXT, alwaysTruePort());
|
|
139
|
+
const qi = findingsFor(findings, "quote-integrity");
|
|
140
|
+
assert.ok(
|
|
141
|
+
qi.some((f) => f.reason.includes("helperFn()") && f.reason.includes("Task 1")),
|
|
142
|
+
`expected a quote-integrity finding naming the missing literal, got: ${JSON.stringify(qi)}`,
|
|
143
|
+
);
|
|
144
|
+
assert.ok(
|
|
145
|
+
qi.some((f) => f.line === lineOf(mutated, "parser grammar basics")),
|
|
146
|
+
`expected the quote-integrity finding's line to point at the offending Spec coverage row, got: ${JSON.stringify(qi)}`,
|
|
147
|
+
);
|
|
148
|
+
});
|
|
149
|
+
|
|
150
|
+
test("check 1 owner-cell grammar: trailing junk after a Task <n> list is malformed, not accepted", () => {
|
|
151
|
+
const mutated = VALID_PLAN.replace(
|
|
152
|
+
'| § "Design" L4-L6 | parser grammar basics | Task 1 |',
|
|
153
|
+
'| § "Design" L4-L6 | parser grammar basics | Task 1 (see note) |',
|
|
154
|
+
);
|
|
155
|
+
const findings = checkPlan(mutated, SPEC_TEXT, alwaysTruePort());
|
|
156
|
+
const tc = findingsFor(findings, "table-closure");
|
|
157
|
+
assert.ok(
|
|
158
|
+
tc.some(
|
|
159
|
+
(f) =>
|
|
160
|
+
f.reason.includes("owner cell is not a 'Task <n>' list or 'waived: <reason>'") &&
|
|
161
|
+
f.text.includes("Task 1 (see note)"),
|
|
162
|
+
),
|
|
163
|
+
`expected an owner-cell malformed finding, got: ${JSON.stringify(tc)}`,
|
|
164
|
+
);
|
|
165
|
+
assert.ok(
|
|
166
|
+
tc.some(
|
|
167
|
+
(f) =>
|
|
168
|
+
f.reason.includes("owner cell is not a 'Task <n>' list or 'waived: <reason>'") &&
|
|
169
|
+
f.line === lineOf(mutated, '| § "Design" L4-L6 | parser grammar basics | Task 1 (see note) |'),
|
|
170
|
+
),
|
|
171
|
+
`expected the owner-cell malformed finding's line to point at the offending row, got: ${JSON.stringify(tc)}`,
|
|
172
|
+
);
|
|
173
|
+
});
|
|
174
|
+
|
|
175
|
+
test("check 1 owner-cell grammar: empty waiver reason is malformed, not accepted as a waiver", () => {
|
|
176
|
+
const mutated = VALID_PLAN.replace(
|
|
177
|
+
'| § "Other" L12-L13 | out of scope thing | waived: out of scope per spec |',
|
|
178
|
+
'| § "Other" L12-L13 | out of scope thing | waived: |',
|
|
179
|
+
);
|
|
180
|
+
const findings = checkPlan(mutated, SPEC_TEXT, alwaysTruePort());
|
|
181
|
+
const tc = findingsFor(findings, "table-closure");
|
|
182
|
+
assert.ok(
|
|
183
|
+
tc.some((f) => f.reason.includes("owner cell is not a 'Task <n>' list or 'waived: <reason>'")),
|
|
184
|
+
`expected an owner-cell malformed finding for empty waiver reason, got: ${JSON.stringify(tc)}`,
|
|
185
|
+
);
|
|
186
|
+
assert.ok(
|
|
187
|
+
tc.some(
|
|
188
|
+
(f) =>
|
|
189
|
+
f.reason.includes("owner cell is not a 'Task <n>' list or 'waived: <reason>'") &&
|
|
190
|
+
f.line === lineOf(mutated, '| § "Other" L12-L13 | out of scope thing | waived: |'),
|
|
191
|
+
),
|
|
192
|
+
`expected the owner-cell malformed finding's line to point at the offending row, got: ${JSON.stringify(tc)}`,
|
|
193
|
+
);
|
|
194
|
+
});
|
|
195
|
+
|
|
196
|
+
test("check 3 anchor-resolution: ambiguous heading match (duplicate spec heading)", () => {
|
|
197
|
+
const dupSpec = SPEC_TEXT.replace('## Testing', '## Design\n\nduplicate section body.\n\n## Testing');
|
|
198
|
+
const findings = checkPlan(VALID_PLAN, dupSpec, alwaysTruePort());
|
|
199
|
+
const ar = findingsFor(findings, "anchor-resolution");
|
|
200
|
+
assert.ok(
|
|
201
|
+
ar.some((f) => f.reason.includes("ambiguous") && f.reason.includes('"Design"')),
|
|
202
|
+
`expected an ambiguous-heading finding, got: ${JSON.stringify(ar)}`,
|
|
203
|
+
);
|
|
204
|
+
assert.ok(
|
|
205
|
+
ar.some(
|
|
206
|
+
(f) =>
|
|
207
|
+
f.reason.includes("ambiguous") &&
|
|
208
|
+
f.line === lineOf(VALID_PLAN, '**Spec:** doc/specs/fixture-spec.md § "Design" L4-L6'),
|
|
209
|
+
),
|
|
210
|
+
`expected an ambiguous-heading finding whose line points at Task 1's **Spec:** anchor line, got: ${JSON.stringify(ar)}`,
|
|
211
|
+
);
|
|
212
|
+
});
|
|
213
|
+
|
|
214
|
+
test("check 3 fail-closed: unparseable **Spec:** anchor line", () => {
|
|
215
|
+
const mutated = VALID_PLAN.replace(
|
|
216
|
+
'**Spec:** doc/specs/fixture-spec.md § "Design" L4-L6',
|
|
217
|
+
"**Spec:** doc/specs/fixture-spec.md § Design without quotes or line numbers",
|
|
218
|
+
);
|
|
219
|
+
const findings = checkPlan(mutated, SPEC_TEXT, alwaysTruePort());
|
|
220
|
+
const ar = findingsFor(findings, "anchor-resolution");
|
|
221
|
+
assert.equal(ar.length, 1);
|
|
222
|
+
assert.match(ar[0].reason, /unparseable/);
|
|
223
|
+
assert.equal(ar[0].line, lineOf(mutated, "**Spec:** doc/specs/fixture-spec.md § Design without quotes"));
|
|
224
|
+
});
|
|
225
|
+
|
|
226
|
+
test("check 4 paths-exist: Modify: literal path does not exist", () => {
|
|
227
|
+
const port: FsPort = {
|
|
228
|
+
exists: (p) => p !== "extensions/lib/fixture-shared.ts",
|
|
229
|
+
glob: () => [],
|
|
230
|
+
};
|
|
231
|
+
const findings = checkPlan(VALID_PLAN, SPEC_TEXT, port);
|
|
232
|
+
const pe = findingsFor(findings, "paths-exist");
|
|
233
|
+
assert.ok(
|
|
234
|
+
pe.some((f) => f.reason.includes("fixture-shared.ts") && f.reason.includes("does not exist")),
|
|
235
|
+
`expected a paths-exist finding, got: ${JSON.stringify(pe)}`,
|
|
236
|
+
);
|
|
237
|
+
assert.ok(
|
|
238
|
+
pe.some((f) => f.line === lineOf(VALID_PLAN, "- Modify: extensions/lib/fixture-shared.ts")),
|
|
239
|
+
`expected the paths-exist finding's line to point at the offending Modify: entry, got: ${JSON.stringify(pe)}`,
|
|
240
|
+
);
|
|
241
|
+
});
|
|
242
|
+
|
|
243
|
+
test("check 4 paths-exist: backtick-wrapped Modify: path is stripped before the existence check", () => {
|
|
244
|
+
const mutated = VALID_PLAN.replace(
|
|
245
|
+
"- Modify: extensions/lib/fixture-shared.ts",
|
|
246
|
+
"- Modify: `extensions/lib/fixture-shared.ts`",
|
|
247
|
+
);
|
|
248
|
+
const findings = checkPlan(mutated, SPEC_TEXT, alwaysTruePort());
|
|
249
|
+
assert.deepEqual(findingsFor(findings, "paths-exist"), []);
|
|
250
|
+
});
|
|
251
|
+
|
|
252
|
+
test("check 4 fail-closed: invalid glob (port throws)", () => {
|
|
253
|
+
const mutated = VALID_PLAN.replace(
|
|
254
|
+
"- Modify: extensions/lib/fixture-shared.ts",
|
|
255
|
+
"- Modify: extensions/lib/fixture-*.ts",
|
|
256
|
+
);
|
|
257
|
+
const port: FsPort = {
|
|
258
|
+
exists: () => true,
|
|
259
|
+
glob: () => {
|
|
260
|
+
throw new Error("bad pattern");
|
|
261
|
+
},
|
|
262
|
+
};
|
|
263
|
+
const findings = checkPlan(mutated, SPEC_TEXT, port);
|
|
264
|
+
const pe = findingsFor(findings, "paths-exist");
|
|
265
|
+
assert.ok(
|
|
266
|
+
pe.some((f) => f.reason.includes("invalid") && f.reason.includes("bad pattern")),
|
|
267
|
+
`expected an invalid-glob finding, got: ${JSON.stringify(pe)}`,
|
|
268
|
+
);
|
|
269
|
+
assert.ok(
|
|
270
|
+
pe.some(
|
|
271
|
+
(f) =>
|
|
272
|
+
f.reason.includes("invalid") &&
|
|
273
|
+
f.line === lineOf(mutated, "- Modify: extensions/lib/fixture-*.ts"),
|
|
274
|
+
),
|
|
275
|
+
`expected the invalid-glob finding's line to point at the offending Modify: entry, got: ${JSON.stringify(pe)}`,
|
|
276
|
+
);
|
|
277
|
+
});
|
|
278
|
+
|
|
279
|
+
test("check 4 fail-closed: task missing **Files:** block", () => {
|
|
280
|
+
const mutated = VALID_PLAN.replace(
|
|
281
|
+
"**Files:**\n- Modify: extensions/lib/fixture-task3.ts\n\n",
|
|
282
|
+
"",
|
|
283
|
+
);
|
|
284
|
+
const findings = checkPlan(mutated, SPEC_TEXT, alwaysTruePort());
|
|
285
|
+
const pe = findingsFor(findings, "paths-exist");
|
|
286
|
+
assert.ok(
|
|
287
|
+
pe.some((f) => f.reason.includes("missing a **Files:** block") && f.text.includes("Task 3")),
|
|
288
|
+
`expected a missing-Files-block finding, got: ${JSON.stringify(pe)}`,
|
|
289
|
+
);
|
|
290
|
+
assert.ok(
|
|
291
|
+
pe.some(
|
|
292
|
+
(f) =>
|
|
293
|
+
f.reason.includes("missing a **Files:** block") &&
|
|
294
|
+
f.line === lineOf(mutated, "### Task 3: Document banned token handling"),
|
|
295
|
+
),
|
|
296
|
+
`expected the missing-Files-block finding's line to point at Task 3's header, got: ${JSON.stringify(pe)}`,
|
|
297
|
+
);
|
|
298
|
+
});
|
|
299
|
+
|
|
300
|
+
test("check 5 placeholder-scan: banned token flagged outside a required-literal span", () => {
|
|
301
|
+
const mutated = VALID_PLAN.replace(
|
|
302
|
+
"This task handles naming details.",
|
|
303
|
+
"This task handles naming details. TODO clean this up.",
|
|
304
|
+
);
|
|
305
|
+
const findings = checkPlan(mutated, SPEC_TEXT, alwaysTruePort());
|
|
306
|
+
const ps = findingsFor(findings, "placeholder-scan");
|
|
307
|
+
assert.ok(
|
|
308
|
+
ps.some((f) => f.reason.includes('"TODO"') && f.line === lineOf(mutated, "TODO clean this up")),
|
|
309
|
+
`expected a placeholder-scan finding for the unexempted TODO, got: ${JSON.stringify(ps)}`,
|
|
310
|
+
);
|
|
311
|
+
});
|
|
312
|
+
|
|
313
|
+
test("check 6 wave-file-disjointness: two tasks in the same wave declare the same file", () => {
|
|
314
|
+
const mutated = VALID_PLAN.replace(
|
|
315
|
+
"- Modify: extensions/lib/fixture-other.ts",
|
|
316
|
+
"- Modify: extensions/lib/fixture-shared.ts",
|
|
317
|
+
);
|
|
318
|
+
const findings = checkPlan(mutated, SPEC_TEXT, alwaysTruePort());
|
|
319
|
+
const wfd = findingsFor(findings, "wave-file-disjointness");
|
|
320
|
+
assert.ok(
|
|
321
|
+
wfd.some((f) => f.reason.includes("Task 1") && f.reason.includes("Task 2") && f.reason.includes("fixture-shared.ts")),
|
|
322
|
+
`expected a wave-file-disjointness finding, got: ${JSON.stringify(wfd)}`,
|
|
323
|
+
);
|
|
324
|
+
assert.ok(
|
|
325
|
+
wfd.some((f) => f.line === lineOf(mutated, "## Wave 1 — Two parallel tasks")),
|
|
326
|
+
`expected the wave-file-disjointness finding's line to point at the offending wave header, got: ${JSON.stringify(wfd)}`,
|
|
327
|
+
);
|
|
328
|
+
});
|
|
329
|
+
|
|
330
|
+
test("check 7 solo-line: single-task wave missing its Solo: line", () => {
|
|
331
|
+
const mutated = VALID_PLAN.replace("Solo: lone remaining task\n\n", "");
|
|
332
|
+
const findings = checkPlan(mutated, SPEC_TEXT, alwaysTruePort());
|
|
333
|
+
const sl = findingsFor(findings, "solo-line");
|
|
334
|
+
assert.equal(sl.length, 1);
|
|
335
|
+
assert.match(sl[0].reason, /missing a 'Solo: <reason>' line/);
|
|
336
|
+
assert.equal(sl[0].line, lineOf(mutated, "## Wave 2 — Solo task"));
|
|
337
|
+
});
|
|
338
|
+
|
|
339
|
+
test("check 8 header-entrypoint: verification entrypoint string reused outside the header", () => {
|
|
340
|
+
const mutated = VALID_PLAN.replace(
|
|
341
|
+
"This task handles naming details.",
|
|
342
|
+
"This task handles naming details. Run `npm run fixture-verify` here too.",
|
|
343
|
+
);
|
|
344
|
+
const findings = checkPlan(mutated, SPEC_TEXT, alwaysTruePort());
|
|
345
|
+
const he = findingsFor(findings, "header-entrypoint");
|
|
346
|
+
assert.ok(
|
|
347
|
+
he.some((f) => f.line === lineOf(mutated, "Run `npm run fixture-verify` here too")),
|
|
348
|
+
`expected a header-entrypoint finding, got: ${JSON.stringify(he)}`,
|
|
349
|
+
);
|
|
350
|
+
});
|
|
351
|
+
|
|
352
|
+
test("check 8 header-entrypoint: entrypoint text inside the '## Spec coverage' table is out of scope (not a wave/task body)", () => {
|
|
353
|
+
const mutated = VALID_PLAN.replace(
|
|
354
|
+
"parser grammar basics",
|
|
355
|
+
"parser grammar basics npm run fixture-verify",
|
|
356
|
+
);
|
|
357
|
+
const findings = checkPlan(mutated, SPEC_TEXT, alwaysTruePort());
|
|
358
|
+
assert.deepEqual(findingsFor(findings, "header-entrypoint"), []);
|
|
359
|
+
});
|
|
360
|
+
|
|
361
|
+
const EXEMPTION_SPEC_TEXT = [
|
|
362
|
+
"# Exemption Spec", // 1
|
|
363
|
+
"", // 2
|
|
364
|
+
"## Notes", // 3
|
|
365
|
+
"This mentions `TODO` as a banned placeholder example.", // 4
|
|
366
|
+
].join("\n");
|
|
367
|
+
|
|
368
|
+
const EXEMPTION_PLAN = `# Exemption Plan
|
|
369
|
+
|
|
370
|
+
**Spec:** \`doc/specs/exemption-spec.md\`
|
|
371
|
+
|
|
372
|
+
**Verification:** npm run exemption-verify
|
|
373
|
+
|
|
374
|
+
---
|
|
375
|
+
|
|
376
|
+
## Wave 1 — Solo task
|
|
377
|
+
|
|
378
|
+
Solo: only task in this wave
|
|
379
|
+
|
|
380
|
+
### Task 1: Document banned token handling
|
|
381
|
+
|
|
382
|
+
**Spec:** doc/specs/exemption-spec.md § "Notes" L4
|
|
383
|
+
|
|
384
|
+
**Files:**
|
|
385
|
+
- Create: extensions/lib/exemption-task1.ts
|
|
386
|
+
|
|
387
|
+
The literal TODO is intentionally documented here per spec quote-integrity requirement.
|
|
388
|
+
|
|
389
|
+
## Spec coverage
|
|
390
|
+
|
|
391
|
+
| anchor | requirement | owner |
|
|
392
|
+
|---|---|---|
|
|
393
|
+
| § "Notes" L4-L4 | banned token literal handling | Task 1 |
|
|
394
|
+
`;
|
|
395
|
+
|
|
396
|
+
test("check 2/5 exemption: task body carrying the required verbatim TODO literal satisfies quote-integrity and exempts placeholder-scan", () => {
|
|
397
|
+
const findings = checkPlan(EXEMPTION_PLAN, EXEMPTION_SPEC_TEXT, alwaysTruePort());
|
|
398
|
+
assert.deepEqual(findings, []);
|
|
399
|
+
});
|
|
400
|
+
|
|
401
|
+
test("check 2/5 exemption negative: without the task body carrying the literal, quote-integrity fails and an unexempted TODO elsewhere trips placeholder-scan", () => {
|
|
402
|
+
const mutated = EXEMPTION_PLAN.replace("## Wave 1 — Solo task", "## Wave 1 — TODO task").replace(
|
|
403
|
+
"The literal TODO is intentionally documented here per spec quote-integrity requirement.",
|
|
404
|
+
"This task implements the handling here.",
|
|
405
|
+
);
|
|
406
|
+
const findings = checkPlan(mutated, EXEMPTION_SPEC_TEXT, alwaysTruePort());
|
|
407
|
+
const qi = findingsFor(findings, "quote-integrity");
|
|
408
|
+
const ps = findingsFor(findings, "placeholder-scan");
|
|
409
|
+
assert.ok(
|
|
410
|
+
qi.some((f) => f.reason.includes("TODO") && f.reason.includes("Task 1")),
|
|
411
|
+
`expected a quote-integrity finding for the missing literal, got: ${JSON.stringify(qi)}`,
|
|
412
|
+
);
|
|
413
|
+
assert.ok(
|
|
414
|
+
qi.some(
|
|
415
|
+
(f) =>
|
|
416
|
+
f.reason.includes("TODO") &&
|
|
417
|
+
f.line === lineOf(mutated, '| § "Notes" L4-L4 | banned token literal handling | Task 1 |'),
|
|
418
|
+
),
|
|
419
|
+
`expected the quote-integrity finding's line to point at the offending Spec coverage row, got: ${JSON.stringify(qi)}`,
|
|
420
|
+
);
|
|
421
|
+
assert.ok(
|
|
422
|
+
ps.some((f) => f.reason.includes('"TODO"') && f.line === lineOf(mutated, "## Wave 1 — TODO task")),
|
|
423
|
+
`expected an unexempted placeholder-scan finding on the wave header, got: ${JSON.stringify(ps)}`,
|
|
424
|
+
);
|
|
425
|
+
});
|
|
426
|
+
|
|
427
|
+
test("aggregate: independent mutations across three checks are all reported together", () => {
|
|
428
|
+
let mutated = VALID_PLAN;
|
|
429
|
+
mutated = mutated.replace("Solo: lone remaining task\n\n", "");
|
|
430
|
+
mutated = mutated.replace(
|
|
431
|
+
"This task implements helperFn() for parsing.",
|
|
432
|
+
"This task implements the helper for parsing.",
|
|
433
|
+
);
|
|
434
|
+
mutated = mutated.replace(
|
|
435
|
+
"- Modify: extensions/lib/fixture-other.ts",
|
|
436
|
+
"- Modify: extensions/lib/fixture-shared.ts",
|
|
437
|
+
);
|
|
438
|
+
const findings = checkPlan(mutated, SPEC_TEXT, alwaysTruePort());
|
|
439
|
+
assert.ok(findingsFor(findings, "solo-line").length > 0, "expected a solo-line finding");
|
|
440
|
+
assert.ok(findingsFor(findings, "quote-integrity").length > 0, "expected a quote-integrity finding");
|
|
441
|
+
assert.ok(findingsFor(findings, "wave-file-disjointness").length > 0, "expected a wave-file-disjointness finding");
|
|
442
|
+
});
|
|
443
|
+
|
|
444
|
+
test("grammarless plan: absent-structure findings do not short-circuit the catalog (placeholder-scan still runs)", () => {
|
|
445
|
+
const grammarless = "# Not a plan\n\nJust some prose with a TODO in it.\n";
|
|
446
|
+
const findings = checkPlan(grammarless, SPEC_TEXT, alwaysTruePort());
|
|
447
|
+
const input = findingsFor(findings, "input");
|
|
448
|
+
assert.ok(
|
|
449
|
+
input.some((f) => f.reason.includes("no wave headers found")),
|
|
450
|
+
`expected an absent-wave-structure finding, got: ${JSON.stringify(input)}`,
|
|
451
|
+
);
|
|
452
|
+
assert.ok(
|
|
453
|
+
input.some((f) => f.reason.includes("no task headers found")),
|
|
454
|
+
`expected an absent-task-structure finding, got: ${JSON.stringify(input)}`,
|
|
455
|
+
);
|
|
456
|
+
const ps = findingsFor(findings, "placeholder-scan");
|
|
457
|
+
assert.ok(
|
|
458
|
+
ps.some((f) => f.reason.includes('"TODO"')),
|
|
459
|
+
`expected placeholder-scan to still run over a grammarless plan, got: ${JSON.stringify(ps)}`,
|
|
460
|
+
);
|
|
461
|
+
});
|
|
462
|
+
|
|
463
|
+
test("fail-closed: checkPlan never throws on an empty string", () => {
|
|
464
|
+
assert.doesNotThrow(() => checkPlan("", SPEC_TEXT, alwaysTruePort()));
|
|
465
|
+
});
|
|
466
|
+
|
|
467
|
+
test("fail-closed: checkPlan never throws on a non-markdown garbage blob", () => {
|
|
468
|
+
const garbage = "\u0000\u0001binary\ngarbage\tstuff \u{1F4A9} \"\"\"\n```unterminated fence";
|
|
469
|
+
assert.doesNotThrow(() => checkPlan(garbage, SPEC_TEXT, alwaysTruePort()));
|
|
470
|
+
});
|
|
471
|
+
|
|
472
|
+
test("sha256 is deterministic and content-sensitive", () => {
|
|
473
|
+
const a = sha256(new TextEncoder().encode("hello"));
|
|
474
|
+
const b = sha256(new TextEncoder().encode("hello"));
|
|
475
|
+
const c = sha256(new TextEncoder().encode("hello!"));
|
|
476
|
+
assert.equal(a, b);
|
|
477
|
+
assert.notEqual(a, c);
|
|
478
|
+
assert.equal(a, "2cf24dba5fb0a30e26e83b2ac5b9e29e1b161e5c1fa7425e73043362938b9824");
|
|
479
|
+
});
|
|
480
|
+
|
|
481
|
+
test("sha256 returns lowercase hex of the expected length", () => {
|
|
482
|
+
const digest = sha256(new TextEncoder().encode(""));
|
|
483
|
+
assert.equal(digest.length, 64);
|
|
484
|
+
assert.match(digest, /^[0-9a-f]+$/);
|
|
485
|
+
});
|