prizmkit 1.1.86 → 1.1.87
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/bundled/VERSION.json +3 -3
- package/bundled/dev-pipeline/scripts/generate-bootstrap-prompt.py +462 -4
- package/bundled/dev-pipeline/templates/bootstrap-prompt.md +1 -1
- package/bundled/dev-pipeline/templates/bootstrap-tier1.md +14 -0
- package/bundled/dev-pipeline/templates/bootstrap-tier2.md +18 -3
- package/bundled/dev-pipeline/templates/bootstrap-tier3.md +18 -3
- package/bundled/dev-pipeline/templates/sections/phase-commit-full.md +2 -2
- package/bundled/dev-pipeline/templates/sections/phase-commit.md +2 -2
- package/bundled/dev-pipeline/templates/sections/phase-prizmkit-test.md +260 -0
- package/bundled/dev-pipeline/templates/sections/phase-review-agent.md +5 -1
- package/bundled/dev-pipeline/templates/sections/phase-review-full.md +5 -1
- package/bundled/dev-pipeline/tests/test_generate_bootstrap_prompt.py +429 -0
- package/bundled/dev-pipeline-windows/scripts/generate-bootstrap-prompt.py +462 -4
- package/bundled/dev-pipeline-windows/templates/bootstrap-prompt.md +1 -1
- package/bundled/dev-pipeline-windows/templates/bootstrap-tier1.md +14 -0
- package/bundled/dev-pipeline-windows/templates/bootstrap-tier2.md +18 -3
- package/bundled/dev-pipeline-windows/templates/bootstrap-tier3.md +18 -3
- package/bundled/dev-pipeline-windows/templates/sections/phase-commit-full.md +2 -2
- package/bundled/dev-pipeline-windows/templates/sections/phase-commit.md +2 -2
- package/bundled/dev-pipeline-windows/templates/sections/phase-prizmkit-test.md +186 -0
- package/bundled/dev-pipeline-windows/templates/sections/phase-review-agent.md +5 -1
- package/bundled/dev-pipeline-windows/templates/sections/phase-review-full.md +5 -1
- package/bundled/skills/_metadata.json +1 -1
- package/bundled/skills/prizmkit-code-review/SKILL.md +7 -2
- package/bundled/skills/prizmkit-code-review/references/reviewer-agent-prompt.md +6 -2
- package/bundled/skills/prizmkit-test/SKILL.md +106 -14
- package/bundled/skills/prizmkit-test/references/boundary-coverage-protocol.md +220 -0
- package/bundled/skills/prizmkit-test/references/examples.md +82 -9
- package/bundled/skills/prizmkit-test/references/test-generation-steps.md +68 -12
- package/bundled/skills/prizmkit-test/references/test-report-template.md +73 -9
- package/bundled/skills/prizmkit-test/scripts/validate_boundary_report.py +347 -0
- package/bundled/skills-windows/prizmkit-code-review/SKILL.md +7 -2
- package/bundled/skills-windows/prizmkit-code-review/references/reviewer-agent-prompt.md +6 -2
- package/bundled/skills-windows/prizmkit-test/SKILL.md +106 -14
- package/bundled/skills-windows/prizmkit-test/references/boundary-coverage-protocol.md +220 -0
- package/bundled/skills-windows/prizmkit-test/references/examples.md +82 -9
- package/bundled/skills-windows/prizmkit-test/references/service-boundary-test-catalog.md +4 -4
- package/bundled/skills-windows/prizmkit-test/references/test-generation-steps.md +68 -12
- package/bundled/skills-windows/prizmkit-test/references/test-report-template.md +73 -9
- package/bundled/skills-windows/prizmkit-test/scripts/validate_boundary_report.py +347 -0
- package/package.json +1 -1
|
@@ -0,0 +1,186 @@
|
|
|
1
|
+
### Scoped Feature Test Gate — PrizmKit Test
|
|
2
|
+
|
|
3
|
+
**Goal**: Generate and verify tests for this feature's changed scope before code review.
|
|
4
|
+
|
|
5
|
+
Create a start marker immediately before invoking the skill so the gate can reject stale reports from older sessions:
|
|
6
|
+
|
|
7
|
+
```powershell
|
|
8
|
+
New-Item -ItemType Directory -Force -Path ".prizmkit/specs/{{FEATURE_SLUG}}" | Out-Null
|
|
9
|
+
$testGateMarker = ".prizmkit/specs/{{FEATURE_SLUG}}/.prizmkit-test-started"
|
|
10
|
+
New-Item -ItemType File -Force -Path $testGateMarker | Out-Null
|
|
11
|
+
```
|
|
12
|
+
|
|
13
|
+
Run `/prizmkit-test` with the feature artifact directory:
|
|
14
|
+
|
|
15
|
+
```text
|
|
16
|
+
/prizmkit-test scope=this-change artifact_dir=.prizmkit/specs/{{FEATURE_SLUG}}/
|
|
17
|
+
```
|
|
18
|
+
|
|
19
|
+
Rules:
|
|
20
|
+
- This gate is feature-pipeline only. Do NOT use `.prizmkit/bugfix/` or `.prizmkit/refactor/` artifact directories here.
|
|
21
|
+
- Generate or update tests only for this feature's changed files, changed public interfaces, and acceptance criteria.
|
|
22
|
+
- Record unrelated historical coverage gaps under `Out of Scope Gaps`; do not generate tests for them and do not fail this feature because of them.
|
|
23
|
+
- If generated tests fail because implementation is wrong, fix implementation within this feature scope and rerun the affected tests.
|
|
24
|
+
- If failures are baseline or unrelated to this feature, document them in the report and continue only when no in-scope failure remains unexplained.
|
|
25
|
+
|
|
26
|
+
**Gate Check — Test Report**:
|
|
27
|
+
After `/prizmkit-test` returns, locate the newest test report created after the start marker and validate that it belongs to this feature scope, has a PASS verdict, and has passed the boundary completion/validation gates before writing the checkpoint pointer:
|
|
28
|
+
```powershell
|
|
29
|
+
$featureArtifactDir = ".prizmkit/specs/{{FEATURE_SLUG}}/"
|
|
30
|
+
$latestTestReport = Get-ChildItem -Path ".prizmkit/test" -Filter "test-report.md" -Recurse -ErrorAction SilentlyContinue |
|
|
31
|
+
Where-Object { $_.LastWriteTime -ge (Get-Item -LiteralPath $testGateMarker).LastWriteTime } |
|
|
32
|
+
Sort-Object FullName |
|
|
33
|
+
Select-Object -Last 1
|
|
34
|
+
|
|
35
|
+
if ($latestTestReport) {
|
|
36
|
+
$lines = Get-Content -LiteralPath $latestTestReport.FullName -ErrorAction SilentlyContinue
|
|
37
|
+
$inScope = $false
|
|
38
|
+
$reportMode = ""
|
|
39
|
+
$reportArtifactDir = ""
|
|
40
|
+
$reportVerdict = ""
|
|
41
|
+
$inBoundaryGate = $false
|
|
42
|
+
$inBoundaryValidation = $false
|
|
43
|
+
$inBoundaryMatrix = $false
|
|
44
|
+
$boundaryGatePassed = ""
|
|
45
|
+
$boundaryMissing = ""
|
|
46
|
+
$happyPathOnly = ""
|
|
47
|
+
$boundaryValidationResult = ""
|
|
48
|
+
$boundaryColumnsOk = $false
|
|
49
|
+
$boundaryIncomplete = $false
|
|
50
|
+
for ($i = 0; $i -lt $lines.Count; $i++) {
|
|
51
|
+
$line = $lines[$i]
|
|
52
|
+
if ($line -match '^## Scope\s*$') { $inScope = $true; $inBoundaryGate = $false; $inBoundaryValidation = $false; $inBoundaryMatrix = $false; continue }
|
|
53
|
+
if ($line -match '^## Boundary Matrix\s*$') { $inBoundaryMatrix = $true; $inScope = $false; $inBoundaryGate = $false; $inBoundaryValidation = $false; continue }
|
|
54
|
+
if ($line -match '^## Boundary Completion Gate\s*$') { $inBoundaryGate = $true; $inScope = $false; $inBoundaryValidation = $false; $inBoundaryMatrix = $false; continue }
|
|
55
|
+
if ($line -match '^## Boundary Validation\s*$') { $inBoundaryValidation = $true; $inScope = $false; $inBoundaryGate = $false; $inBoundaryMatrix = $false; continue }
|
|
56
|
+
if ($line -match '^## ' -and $inScope) { $inScope = $false }
|
|
57
|
+
if ($line -match '^## ' -and $inBoundaryMatrix) { $inBoundaryMatrix = $false }
|
|
58
|
+
if ($line -match '^## ' -and $inBoundaryGate) { $inBoundaryGate = $false }
|
|
59
|
+
if ($line -match '^## ' -and $inBoundaryValidation) { $inBoundaryValidation = $false }
|
|
60
|
+
if ($inScope -and $line -match '^- Mode:\s*(.+)$') { $reportMode = $Matches[1].Trim() }
|
|
61
|
+
if ($inScope -and $line -match '^- Artifact Dir:\s*(.+)$') { $reportArtifactDir = $Matches[1].Trim() }
|
|
62
|
+
if ($inBoundaryMatrix -and $line -match '^\|') {
|
|
63
|
+
$hasRequiredColumns =
|
|
64
|
+
$line -match '\|\s*Module\s*\|' -and
|
|
65
|
+
$line -match '\|\s*Interface\s*\|' -and
|
|
66
|
+
$line -match '\|\s*Service Type\s*\|' -and
|
|
67
|
+
$line -match '\|\s*Happy Path\s*\|' -and
|
|
68
|
+
$line -match '\|\s*Request Validation\s*\|' -and
|
|
69
|
+
$line -match '\|\s*Auth / Permission / Ownership\s*\|' -and
|
|
70
|
+
$line -match '\|\s*Domain Invariants\s*\|' -and
|
|
71
|
+
$line -match '\|\s*Collection / Pagination\s*\|' -and
|
|
72
|
+
$line -match '\|\s*Date / Time\s*\|' -and
|
|
73
|
+
$line -match '\|\s*State Transitions\s*\|' -and
|
|
74
|
+
$line -match '\|\s*Dependency Failure\s*\|' -and
|
|
75
|
+
$line -match '\|\s*Response Contract\s*\|' -and
|
|
76
|
+
$line -match '\|\s*N/A Reasons\s*\|' -and
|
|
77
|
+
$line -match '\|\s*Final Status\s*\|'
|
|
78
|
+
if ($hasRequiredColumns) { $boundaryColumnsOk = $true }
|
|
79
|
+
if ($line -match 'boundary-missing|happy-path-only|unresolved') { $boundaryIncomplete = $true }
|
|
80
|
+
}
|
|
81
|
+
if ($inBoundaryGate -and $line -match '^-?\s*Completion gate passed:\s*yes\s*$') { $boundaryGatePassed = "yes" }
|
|
82
|
+
if ($inBoundaryGate -and $line -match '^-?\s*Boundary-missing:\s*(\d+)\s*$') { $boundaryMissing = $Matches[1].Trim() }
|
|
83
|
+
if ($inBoundaryGate -and $line -match '^-?\s*Happy-path-only:\s*(\d+)\s*$') { $happyPathOnly = $Matches[1].Trim() }
|
|
84
|
+
if ($inBoundaryValidation -and $line -match '^-?\s*Validator result:\s*(\S+)') { $boundaryValidationResult = $Matches[1].Trim().ToLowerInvariant() }
|
|
85
|
+
if ($line -match '^## Verdict\s*$') {
|
|
86
|
+
for ($j = $i + 1; $j -lt $lines.Count; $j++) {
|
|
87
|
+
if ($lines[$j].Trim()) { $reportVerdict = $lines[$j].Trim(); break }
|
|
88
|
+
}
|
|
89
|
+
break
|
|
90
|
+
}
|
|
91
|
+
}
|
|
92
|
+
$openapiFile = ""
|
|
93
|
+
$openapiCandidates = @(
|
|
94
|
+
"openapi.yaml", "openapi.yml", "swagger.yaml", "swagger.yml",
|
|
95
|
+
"docs/openapi.yaml", "docs/openapi.yml", "docs/swagger.yaml", "docs/swagger.yml",
|
|
96
|
+
"api/openapi.yaml", "api/openapi.yml", "api/swagger.yaml", "api/swagger.yml"
|
|
97
|
+
)
|
|
98
|
+
foreach ($openapiCandidate in $openapiCandidates) {
|
|
99
|
+
if (Test-Path -LiteralPath $openapiCandidate) { $openapiFile = $openapiCandidate; break }
|
|
100
|
+
}
|
|
101
|
+
$boundaryValidatorPassed = $false
|
|
102
|
+
$boundaryValidators = @(
|
|
103
|
+
".claude/command-assets/prizmkit-test/scripts/validate_boundary_report.py",
|
|
104
|
+
".codebuddy/skills/prizmkit-test/scripts/validate_boundary_report.py",
|
|
105
|
+
".agents/skills/prizmkit-test/scripts/validate_boundary_report.py"
|
|
106
|
+
)
|
|
107
|
+
foreach ($boundaryValidator in $boundaryValidators) {
|
|
108
|
+
if (Test-Path -LiteralPath $boundaryValidator) {
|
|
109
|
+
$boundaryValidatorArgs = @($boundaryValidator, "--report", $latestTestReport.FullName)
|
|
110
|
+
if ($openapiFile) { $boundaryValidatorArgs += @("--openapi", $openapiFile) }
|
|
111
|
+
Invoke-PrizmPython @boundaryValidatorArgs *> $null
|
|
112
|
+
if ($LASTEXITCODE -eq 0) { $boundaryValidatorPassed = $true; break }
|
|
113
|
+
}
|
|
114
|
+
}
|
|
115
|
+
$scopeFreshOk = $false
|
|
116
|
+
try {
|
|
117
|
+
$reportMtime = (Get-Item -LiteralPath $latestTestReport.FullName).LastWriteTimeUtc
|
|
118
|
+
$markerMtime = (Get-Item -LiteralPath $testGateMarker).LastWriteTimeUtc
|
|
119
|
+
if ($reportMtime -ge $markerMtime) {
|
|
120
|
+
function Get-ScopedFileList($sectionName) {
|
|
121
|
+
$inScopeSection = $false
|
|
122
|
+
$inList = $false
|
|
123
|
+
$values = @()
|
|
124
|
+
foreach ($rawLine in $lines) {
|
|
125
|
+
$line = $rawLine.Trim()
|
|
126
|
+
if ($line -match '^## Scope\s*$') { $inScopeSection = $true; continue }
|
|
127
|
+
if ($inScopeSection -and $line -match '^## ') { break }
|
|
128
|
+
if (-not $inScopeSection) { continue }
|
|
129
|
+
if ($line -eq "- ${sectionName}:") { $inList = $true; continue }
|
|
130
|
+
if ($inList) {
|
|
131
|
+
if ($rawLine -match '^\s{2,}-\s+(.+)$') {
|
|
132
|
+
$value = $Matches[1].Trim().Trim('`')
|
|
133
|
+
if ($value -and $value -ne '{...}' -and $value -ne '...') { $values += $value }
|
|
134
|
+
continue
|
|
135
|
+
}
|
|
136
|
+
if ($line -match '^- ') { break }
|
|
137
|
+
}
|
|
138
|
+
}
|
|
139
|
+
return $values
|
|
140
|
+
}
|
|
141
|
+
$sourceFiles = @(Get-ScopedFileList "In-Scope Source Files")
|
|
142
|
+
$testFiles = @(Get-ScopedFileList "In-Scope Test Files")
|
|
143
|
+
if ($sourceFiles.Count -gt 0 -and $testFiles.Count -gt 0) {
|
|
144
|
+
$scopeFreshOk = $true
|
|
145
|
+
$projectRoot = (Resolve-Path ".").Path
|
|
146
|
+
$ignored = @('none', 'n/a', 'not applicable', '(none)', '{...}', '...')
|
|
147
|
+
foreach ($listedFile in $sourceFiles + $testFiles) {
|
|
148
|
+
$normalized = $listedFile.Trim().Trim('`')
|
|
149
|
+
if (-not $normalized -or $ignored -contains $normalized.ToLowerInvariant()) { $scopeFreshOk = $false; break }
|
|
150
|
+
if ([System.IO.Path]::IsPathRooted($normalized)) { $candidate = $normalized } else { $candidate = Join-Path $projectRoot $normalized }
|
|
151
|
+
if (-not (Test-Path -LiteralPath $candidate -PathType Leaf)) { $scopeFreshOk = $false; break }
|
|
152
|
+
$candidatePath = (Resolve-Path -LiteralPath $candidate).Path
|
|
153
|
+
if (-not ($candidatePath -eq $projectRoot -or $candidatePath.StartsWith($projectRoot + [System.IO.Path]::DirectorySeparatorChar))) { $scopeFreshOk = $false; break }
|
|
154
|
+
if ((Get-Item -LiteralPath $candidatePath).LastWriteTimeUtc -gt $reportMtime) { $scopeFreshOk = $false; break }
|
|
155
|
+
}
|
|
156
|
+
}
|
|
157
|
+
}
|
|
158
|
+
} catch {
|
|
159
|
+
$scopeFreshOk = $false
|
|
160
|
+
}
|
|
161
|
+
$artifactOk = $reportArtifactDir -eq $featureArtifactDir -or $reportArtifactDir -eq $featureArtifactDir.TrimEnd('/')
|
|
162
|
+
$boundaryOk = $boundaryColumnsOk -and -not $boundaryIncomplete -and $boundaryGatePassed -eq "yes" -and $boundaryMissing -eq "0" -and $happyPathOnly -eq "0" -and $boundaryValidationResult -eq "passed" -and $boundaryValidatorPassed -and $scopeFreshOk
|
|
163
|
+
if ($reportMode -eq "this-change" -and $artifactOk -and $reportVerdict -eq "PASS" -and $boundaryOk) {
|
|
164
|
+
Set-Content -LiteralPath ".prizmkit/specs/{{FEATURE_SLUG}}/test-report-path.txt" -Value $latestTestReport.FullName
|
|
165
|
+
"GATE:PASS $($latestTestReport.FullName)"
|
|
166
|
+
} else {
|
|
167
|
+
"GATE:FAIL verdict=$reportVerdict mode=$reportMode artifact_dir=$reportArtifactDir boundary_gate=$boundaryGatePassed boundary_missing=$boundaryMissing happy_path_only=$happyPathOnly boundary_validation=$boundaryValidationResult validator_passed=$boundaryValidatorPassed scope_fresh=$scopeFreshOk openapi=$openapiFile report=$($latestTestReport.FullName)"
|
|
168
|
+
}
|
|
169
|
+
} else {
|
|
170
|
+
"GATE:MISSING"
|
|
171
|
+
}
|
|
172
|
+
```
|
|
173
|
+
|
|
174
|
+
Gate outcomes:
|
|
175
|
+
- `GATE:PASS` → append the report path and a 3-5 bullet summary to `context-snapshot.md` under `## PrizmKit Test Gate`, then update the checkpoint.
|
|
176
|
+
- `GATE:FAIL` with `NEEDS_FIXES` → read the report's `In-Scope Failures`, fix the feature implementation or generated tests within this feature scope, rerun `/prizmkit-test`, and repeat the gate check. Do not proceed to code review while the verdict is not `PASS`.
|
|
177
|
+
- `GATE:FAIL` with `BLOCKED`, wrong mode, wrong artifact dir, or unreadable report → write `.prizmkit/specs/{{FEATURE_SLUG}}/failure-log.md` with the scoped test failure and stop for recovery.
|
|
178
|
+
- `GATE:MISSING` → perform one bounded status check: inspect `/prizmkit-test` output and `.prizmkit/test/` for a report directory. If no current-run report exists, write `failure-log.md` and stop for recovery.
|
|
179
|
+
|
|
180
|
+
Only after `GATE:PASS`, run the update script to set step `prizmkit-test` to `"completed"`:
|
|
181
|
+
```powershell
|
|
182
|
+
Invoke-PrizmPython {{PIPELINE_DIR}}\scripts\update-checkpoint.py `
|
|
183
|
+
--checkpoint-path {{CHECKPOINT_PATH}} `
|
|
184
|
+
--step prizmkit-test `
|
|
185
|
+
--status completed
|
|
186
|
+
```
|
|
@@ -1,4 +1,8 @@
|
|
|
1
|
-
### Review
|
|
1
|
+
### Review — Code Review
|
|
2
|
+
|
|
3
|
+
Precondition: the `prizmkit-test` checkpoint is completed and `.prizmkit/specs/{{FEATURE_SLUG}}/test-report-path.txt` points to the scoped test report.
|
|
4
|
+
|
|
5
|
+
Before invoking review, read `.prizmkit/specs/{{FEATURE_SLUG}}/test-report-path.txt`, read the referenced `/prizmkit-test` report, and confirm the summary in `context-snapshot.md` under `## PrizmKit Test Gate` matches it. The code-review skill must receive this context through `artifact_dir=.prizmkit/specs/{{FEATURE_SLUG}}/` and the Reviewer must assess generated/updated test quality, `In-Scope Failures`, `Baseline Failures`, `Out of Scope Gaps`, boundary coverage if present, and the scoped report verdict.
|
|
2
6
|
|
|
3
7
|
Run `/prizmkit-code-review` with `artifact_dir=.prizmkit/specs/{{FEATURE_SLUG}}/`.
|
|
4
8
|
|
|
@@ -1,4 +1,8 @@
|
|
|
1
|
-
### Review
|
|
1
|
+
### Review — Code Review
|
|
2
|
+
|
|
3
|
+
Precondition: the `prizmkit-test` checkpoint is completed and `.prizmkit/specs/{{FEATURE_SLUG}}/test-report-path.txt` points to the scoped test report.
|
|
4
|
+
|
|
5
|
+
Before invoking review, read `.prizmkit/specs/{{FEATURE_SLUG}}/test-report-path.txt`, read the referenced `/prizmkit-test` report, and confirm the summary in `context-snapshot.md` under `## PrizmKit Test Gate` matches it. The code-review skill must receive this context through `artifact_dir=.prizmkit/specs/{{FEATURE_SLUG}}/` and the Reviewer must assess generated/updated test quality, `In-Scope Failures`, `Baseline Failures`, `Out of Scope Gaps`, boundary coverage if present, and the scoped report verdict.
|
|
2
6
|
|
|
3
7
|
Run `/prizmkit-code-review` with `artifact_dir=.prizmkit/specs/{{FEATURE_SLUG}}/`.
|
|
4
8
|
|
|
@@ -32,8 +32,13 @@ The loop repeats until the Reviewer finds no issues or the max round limit is re
|
|
|
32
32
|
|
|
33
33
|
1. **Read spec.md** from the artifact directory — extract goals and acceptance criteria.
|
|
34
34
|
2. **Read plan.md** from the artifact directory — extract architecture decisions and completed tasks.
|
|
35
|
-
3. **Read
|
|
36
|
-
|
|
35
|
+
3. **Read scoped test report when present**:
|
|
36
|
+
- If `{artifact_dir}/test-report-path.txt` exists, read the path, then read that `/prizmkit-test` report.
|
|
37
|
+
- Extract `Scope`, `Generated / Updated Tests`, `In-Scope Failures`, `Baseline Failures`, `Out of Scope Gaps`, `Boundary Matrix` / `Boundary Completion Gate` when present, and `Verdict`.
|
|
38
|
+
- If `context-snapshot.md` contains `## PrizmKit Test Gate`, read that summary too.
|
|
39
|
+
- If this is a feature artifact under `.prizmkit/specs/` and no test report pointer or summary exists, pass `No scoped /prizmkit-test report found` into the Reviewer context so the reviewer can flag the missing gate.
|
|
40
|
+
4. **Read dev rules** (if configured): If `.prizmkit/prizm-docs/root.prizm` exists, read it and check for a `RULES:` line. If present, read all referenced `.prizmkit/rules/<layer>-rules.md` files. If `root.prizm` or a referenced rules file does not exist, continue with "No custom dev rules configured — use general best practices."
|
|
41
|
+
5. **Capture workspace diff**: run `git diff` (unstaged) + `git diff --cached` (staged) + `git status` to understand the full scope of changes. For new files in git status, note their paths for the Reviewer to read.
|
|
37
42
|
- If no changes are detected, skip Phase 1 and proceed to Phase 2 with verdict PASS, rounds 0, and an empty findings list. Always write `review-report.md`; downstream pipeline steps use it as the review gate.
|
|
38
43
|
|
|
39
44
|
## Phase 1: Review-Fix Loop
|
|
@@ -11,6 +11,9 @@ You are a code reviewer. Review workspace changes against the spec goals, plan d
|
|
|
11
11
|
## Plan Decisions
|
|
12
12
|
{architecture decisions and task list from plan.md}
|
|
13
13
|
|
|
14
|
+
## Scoped Test Report
|
|
15
|
+
{summary from artifact_dir/test-report-path.txt and test-report.md, including Scope, Generated / Updated Tests, In-Scope Failures, Baseline Failures, Out of Scope Gaps, Boundary Matrix / Boundary Completion Gate if present, and Verdict. If no scoped report exists for a feature artifact, write "No scoped /prizmkit-test report found."}
|
|
16
|
+
|
|
14
17
|
## Dev Rules (per-layer conventions)
|
|
15
18
|
{rules from .prizmkit/rules/<layer>-rules.md, or "No custom dev rules configured — use general best practices."}
|
|
16
19
|
|
|
@@ -34,7 +37,8 @@ Evaluate the changes across these dimensions (focus on what's relevant):
|
|
|
34
37
|
3. **Completeness**: Files that should have been changed but weren't? Missing tests, types, imports, exports?
|
|
35
38
|
4. **Consistency**: Do changes follow the project's existing patterns, naming conventions, and code style?
|
|
36
39
|
5. **Security**: Hardcoded secrets, injection vulnerabilities, unsafe operations.
|
|
37
|
-
6. **
|
|
40
|
+
6. **Test quality**: If a scoped `/prizmkit-test` report is present, verify the report scope is `this-change`, the artifact dir matches the feature, generated/updated tests assert real behavior rather than mock success or empty assertions, in-scope failures are not mislabeled as baseline failures, baseline failures are genuinely unrelated or pre-existing, out-of-scope gaps were not turned into unrelated test changes, and the report verdict is compatible with proceeding.
|
|
41
|
+
7. **Rules compliance**: (Skip this dimension if no dev rules were provided.) Do changes follow the per-layer dev rules? Flag violations of framework conventions, naming patterns, state management, or other rules defined for that layer.
|
|
38
42
|
|
|
39
43
|
## Output Format
|
|
40
44
|
Respond with EXACTLY this format:
|
|
@@ -46,7 +50,7 @@ Respond with EXACTLY this format:
|
|
|
46
50
|
|
|
47
51
|
#### Finding N
|
|
48
52
|
- **Severity**: high | medium | low
|
|
49
|
-
- **Dimension**: goal-alignment | defect | completeness | consistency | security | rules-compliance
|
|
53
|
+
- **Dimension**: goal-alignment | defect | completeness | consistency | security | test-quality | rules-compliance
|
|
50
54
|
- **Location**: filepath:line (or "project-level")
|
|
51
55
|
- **Problem**: What is wrong and why it matters
|
|
52
56
|
- **Suggestion**: Recommended fix approach
|
|
@@ -1,11 +1,11 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: "prizmkit-test"
|
|
3
|
-
description: "Full-stack test generation and orchestration. Detects architecture, discovers/runs existing tests, analyzes coverage gaps, generates missing
|
|
3
|
+
description: "Full-stack test generation and orchestration. Detects architecture, discovers/runs existing tests, analyzes coverage gaps, generates missing unit/integration/E2E tests including business-specific boundary cases, validates every interface with a required boundary coverage matrix, and outputs a unified report. Use after completing development to verify quality before deploy. Trigger on: 'test', 'run tests', 'check quality', 'verify code', 'generate tests', 'test coverage', 'fill test gaps', 'boundary tests', 'edge cases', 'quality check', '测试', '验证', '跑测试', '补测试', '边界测试'. (project)"
|
|
4
4
|
---
|
|
5
5
|
|
|
6
6
|
# PrizmKit Test
|
|
7
7
|
|
|
8
|
-
A comprehensive test generation and orchestration skill. Discovers existing tests, runs them, compares coverage against spec acceptance criteria and module interfaces, generates missing tests at three levels (unit → integration → E2E), and produces a unified report.
|
|
8
|
+
A comprehensive test generation and orchestration skill. Discovers existing tests, runs them, compares coverage against spec acceptance criteria and module interfaces, generates missing tests at three levels (unit → integration → E2E), verifies business-specific boundary coverage for every in-scope interface, and produces a unified report.
|
|
9
9
|
|
|
10
10
|
### When to Use
|
|
11
11
|
- After completing one or more features/refactors/bugfixes
|
|
@@ -34,11 +34,26 @@ Before execution, load context once:
|
|
|
34
34
|
2. **Project config**: Read `.prizmkit/config.json` (tech stack, AI CLI config).
|
|
35
35
|
3. **Dependencies**: Read `package.json` or equivalent to detect test framework and project type.
|
|
36
36
|
|
|
37
|
+
For `scope=this-change`, load only:
|
|
38
|
+
- `.prizmkit/prizm-docs/root.prizm`
|
|
39
|
+
- relevant L1/L2 docs inferred from in-scope changed files
|
|
40
|
+
- `artifact_dir/spec.md`
|
|
41
|
+
- `artifact_dir/plan.md`
|
|
42
|
+
- `artifact_dir/context-snapshot.md` if present
|
|
43
|
+
- `artifact_dir/review-report.md` if present
|
|
44
|
+
- changed source files and directly related existing tests
|
|
45
|
+
|
|
46
|
+
Do not load all modules or all specs merely because they exist.
|
|
47
|
+
|
|
37
48
|
## Input
|
|
38
49
|
|
|
39
50
|
| Parameter | Required | Description |
|
|
40
51
|
|-----------|----------|-------------|
|
|
41
|
-
| `scope` | No |
|
|
52
|
+
| `scope` | No | `full-project`, `module:<name>`, `feature:<slug>`, or `this-change`. In headless mode, defaults to `this-change` only when `artifact_dir` points to `.prizmkit/specs/<FEATURE_SLUG>/`; otherwise defaults to full project. |
|
|
53
|
+
| `artifact_dir` | No | Feature artifact directory. For `scope=this-change`, must be `.prizmkit/specs/<FEATURE_SLUG>/`. Do not infer this-change scope from `.prizmkit/bugfix/` or `.prizmkit/refactor/`. |
|
|
54
|
+
| `diff_base` | No | Git ref used to compute changed files for this feature. |
|
|
55
|
+
| `changed_files` | No | Explicit caller-provided changed file list. Takes precedence over `diff_base`. |
|
|
56
|
+
| `generation_policy` | No | Defaults to `in-scope-only` for `scope=this-change`. Out-of-scope gaps are reported but not generated. |
|
|
42
57
|
|
|
43
58
|
## Execution
|
|
44
59
|
|
|
@@ -72,7 +87,14 @@ Before execution, load context once:
|
|
|
72
87
|
2. **Single module** — pick from L1 doc module names (e.g., "auth", "payment")
|
|
73
88
|
3. **Single feature** — pick from `.prizmkit/specs/###-*/` directories
|
|
74
89
|
|
|
75
|
-
**Headless mode
|
|
90
|
+
**Headless mode**:
|
|
91
|
+
1. If `scope=this-change`, require `artifact_dir=.prizmkit/specs/<FEATURE_SLUG>/` and run in feature-scoped mode.
|
|
92
|
+
2. If `artifact_dir=.prizmkit/specs/<FEATURE_SLUG>/` is provided without `scope`, treat it as `scope=this-change`.
|
|
93
|
+
3. If `scope=full-project` is explicitly provided, run full-project mode even if an artifact directory exists.
|
|
94
|
+
4. If `artifact_dir` points to `.prizmkit/bugfix/` or `.prizmkit/refactor/`, do not infer `scope=this-change`.
|
|
95
|
+
5. If neither `scope` nor feature `artifact_dir` is provided, default to full project.
|
|
96
|
+
|
|
97
|
+
`scope=this-change` is feature-only. It is valid only when `artifact_dir` is under `.prizmkit/specs/<FEATURE_SLUG>/`. Do not automatically apply `scope=this-change` to `.prizmkit/bugfix/<BUG_ID>/` or `.prizmkit/refactor/<REFACTOR_ID>/`. If the caller passes `scope=this-change` with a bugfix/refactor artifact directory, stop and report that this mode is unsupported for that artifact type. Bugfix and refactor workflows keep their existing reproduction-test and behavior-preservation test gates.
|
|
76
98
|
|
|
77
99
|
### Phase 2: Run Existing Tests
|
|
78
100
|
|
|
@@ -83,13 +105,37 @@ Before execution, load context once:
|
|
|
83
105
|
- Which test files exist (for gap analysis)
|
|
84
106
|
- Raw output (saved to report directory)
|
|
85
107
|
|
|
86
|
-
3.
|
|
108
|
+
3. Classify existing test failures before continuing:
|
|
109
|
+
- For `scope=this-change`, failures in changed tests, tests mapped to in-scope source files, or tests directly covering feature acceptance criteria are `In-Scope Failures` unless a captured pre-feature baseline proves they already failed before this feature.
|
|
110
|
+
- For `scope=this-change`, unrelated failures are `Baseline Failures`; record them without failing the scoped verdict.
|
|
111
|
+
- For other scopes, record failures as existing test failures and do not assume they are caused by this test-generation pass.
|
|
112
|
+
|
|
113
|
+
4. If `scope=this-change` has any unexplained `In-Scope Failures`, the final report verdict must be `NEEDS_FIXES` or `BLOCKED`, not `PASS`. Do not hide them as baseline failures.
|
|
87
114
|
|
|
88
115
|
### Phase 3: Coverage Gap Analysis
|
|
89
116
|
|
|
90
117
|
Staleness of `.prizmkit/prizm-docs/` was already checked during Context Loading (see Precondition). Gap analysis proceeds with the available data.
|
|
91
118
|
|
|
92
|
-
|
|
119
|
+
For `scope=this-change`:
|
|
120
|
+
|
|
121
|
+
1. Build the in-scope source set using this priority:
|
|
122
|
+
- `changed_files` parameter
|
|
123
|
+
- `git diff --name-only <diff_base>`
|
|
124
|
+
- files/tasks listed in `artifact_dir/plan.md`
|
|
125
|
+
- File Manifest in `artifact_dir/context-snapshot.md`
|
|
126
|
+
- direct file/module mapping from `artifact_dir/spec.md` acceptance criteria
|
|
127
|
+
2. Build the in-scope test set:
|
|
128
|
+
- tests already changed in this feature
|
|
129
|
+
- tests matching in-scope source files by project naming convention
|
|
130
|
+
- directly affected integration/E2E tests for feature acceptance criteria
|
|
131
|
+
3. Analyze only:
|
|
132
|
+
- changed/new public interfaces
|
|
133
|
+
- feature acceptance criteria from `artifact_dir/spec.md`
|
|
134
|
+
- integration boundaries directly affected by in-scope files
|
|
135
|
+
- UI/E2E flows directly affected by this feature
|
|
136
|
+
4. For unrelated historical gaps, record them under `Out of Scope Gaps`; do not generate tests and do not fail the feature verdict because of them.
|
|
137
|
+
|
|
138
|
+
For other scopes, compare what exists against what should exist, across three levels:
|
|
93
139
|
|
|
94
140
|
**Unit test gaps** — for each module in scope:
|
|
95
141
|
- Read the corresponding L2 `.prizm` doc INTERFACES section to get exported functions/classes. If no L2 doc exists for a module, analyze source files directly to identify exported functions/classes.
|
|
@@ -106,30 +152,76 @@ Compare what exists against what should exist, across three levels:
|
|
|
106
152
|
- Check existing E2E test files against these criteria
|
|
107
153
|
- Flag uncovered criteria
|
|
108
154
|
|
|
109
|
-
### Phase 4:
|
|
155
|
+
### Phase 4: Boundary Coverage Matrix
|
|
156
|
+
|
|
157
|
+
Phase 3 answers "which interfaces are missing tests?" Phase 4 answers "which business boundaries are missing for each interface?" Build this matrix before generating tests so the run does not mistake broad happy-path coverage for real safety coverage.
|
|
158
|
+
|
|
159
|
+
A path, method, function, or class is not considered covered merely because a happy-path test exists; happy path proves the interface works once, while boundary tests prove it behaves safely under realistic invalid, unauthorized, edge, and failure states.
|
|
160
|
+
|
|
161
|
+
Read `${SKILL_DIR}/references/boundary-coverage-protocol.md` for the required matrix format, category definitions, and completion gate. For every in-scope interface and API endpoint, classify applicable categories such as request validation, auth/permission/ownership, domain invariants, collection/pagination, date/time boundaries, state transitions, dependency failures, and response contract checks. In `scope=this-change`, build this matrix only for the in-scope feature interfaces/endpoints; unrelated interfaces belong in `Out of Scope Gaps`.
|
|
162
|
+
|
|
163
|
+
For each category, mark one of:
|
|
164
|
+
- `covered` — an existing test already asserts the boundary behavior
|
|
165
|
+
- `generated` — this run generated and passed the test
|
|
166
|
+
- `needs-review` — a valid generated test found behavior mismatch
|
|
167
|
+
- `skipped` — not applicable, with a concrete reason
|
|
110
168
|
|
|
111
|
-
|
|
169
|
+
If an interface has only happy-path coverage, mark it `boundary-missing` and generate applicable boundary tests in Phase 5. Do not mark happy-path-only coverage as `covered`.
|
|
112
170
|
|
|
113
|
-
|
|
171
|
+
### Phase 5: Generate Missing and Boundary Tests
|
|
114
172
|
|
|
115
|
-
|
|
173
|
+
Read `${SKILL_DIR}/references/test-generation-steps.md` for the detailed generation procedure — boundary-matrix-driven workflow, priority order (unit → integration → E2E), failure handling rules, and generation patterns for each test level.
|
|
174
|
+
|
|
175
|
+
When generating unit tests for service-like functions or API/integration tests for endpoints with business-specific behavior, use `${SKILL_DIR}/references/service-boundary-test-catalog.md` via the generation procedure so boundary cases reflect the interface's business responsibility, not only generic null/empty inputs.
|
|
176
|
+
|
|
177
|
+
For `scope=this-change`:
|
|
178
|
+
- Generate or update tests only for the in-scope source/test set.
|
|
179
|
+
- Check for equivalent existing tests before writing new tests.
|
|
180
|
+
- Do not create tests for unrelated modules just because coverage is missing.
|
|
181
|
+
- Do not refactor unrelated tests.
|
|
182
|
+
- Do not modify unrelated production code.
|
|
183
|
+
- Do not expand to full-project coverage unless the user explicitly requested `scope=full-project`.
|
|
184
|
+
- Generated tests must assert real behavior, boundaries, or integration contracts; do not add empty assertions or mock-success-only tests.
|
|
185
|
+
|
|
186
|
+
### Phase 6: Unified Report and Boundary Validation
|
|
116
187
|
|
|
117
188
|
Create `.prizmkit/test/{YYYY_MM_DD_HH_MM_SS}_testresult/` directory. Read `${SKILL_DIR}/references/test-report-template.md` for the full report format and artifact list.
|
|
118
189
|
|
|
190
|
+
For `scope=this-change`, the report must include:
|
|
191
|
+
- `## Scope` with mode, artifact dir, diff base, changed-files source, generation policy, in-scope source files, and in-scope test files
|
|
192
|
+
- `## Existing Tests Run`
|
|
193
|
+
- `## Generated / Updated Tests`
|
|
194
|
+
- `## In-Scope Failures`
|
|
195
|
+
- `## Baseline Failures`
|
|
196
|
+
- `## Out of Scope Gaps`
|
|
197
|
+
- `## Verdict` with `PASS`, `NEEDS_FIXES`, or `BLOCKED`
|
|
198
|
+
|
|
199
|
+
`Out of Scope Gaps` are informational only for `scope=this-change`: they must not trigger generated tests and must not fail the verdict.
|
|
200
|
+
|
|
201
|
+
After writing the report, run the deterministic boundary-report validator when the project has an OpenAPI file or the report contains a Boundary Matrix:
|
|
202
|
+
|
|
203
|
+
```bash
|
|
204
|
+
python3 ${SKILL_DIR}/scripts/validate_boundary_report.py \
|
|
205
|
+
--report .prizmkit/test/{timestamp}_testresult/test-report.md \
|
|
206
|
+
--openapi openapi.yaml
|
|
207
|
+
```
|
|
208
|
+
|
|
209
|
+
If the project has no OpenAPI file, omit `--openapi`. If validation fails, keep the report but do not claim the run is complete; report that tests may pass while boundary coverage remains incomplete.
|
|
210
|
+
|
|
119
211
|
## Output
|
|
120
212
|
|
|
121
213
|
- Test report: `.prizmkit/test/{timestamp}_testresult/test-report.md`
|
|
122
214
|
- Generated test files written to project test directories
|
|
123
|
-
- Existing test output, generated test copies,
|
|
215
|
+
- Existing test output, generated test copies, E2E artifacts, and boundary validation output in the report directory
|
|
124
216
|
|
|
125
217
|
## Recovery
|
|
126
218
|
|
|
127
219
|
If the session is interrupted:
|
|
128
220
|
- Check `.prizmkit/test/` for the most recent report directory — it contains what was completed before interruption
|
|
129
|
-
- Re-run `/prizmkit-test` — it starts fresh, but Phase 2 will skip tests that already pass, and Phase 3 will re-evaluate gaps
|
|
221
|
+
- Re-run `/prizmkit-test` — it starts fresh, but Phase 2 will skip tests that already pass, and Phase 3/4 will re-evaluate gaps and boundary coverage
|
|
130
222
|
|
|
131
223
|
## Examples
|
|
132
224
|
|
|
133
|
-
Read `${SKILL_DIR}/references/examples.md` for worked examples of full-project
|
|
225
|
+
Read `${SKILL_DIR}/references/examples.md` for worked examples of full-project, single-feature, and happy-path-only remediation runs.
|
|
134
226
|
|
|
135
|
-
**HANDOFF:** Independent skill — no handoff. User may proceed to `/prizmkit-committer` if
|
|
227
|
+
**HANDOFF:** Independent skill — no handoff. User may proceed to `/prizmkit-committer` only if tests pass and the boundary completion gate passes. If tests are marked `needs-review` or boundary validation fails, fix issues manually or run another targeted `/prizmkit-test` pass.
|