prizmkit 1.1.85 → 1.1.87
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/bundled/VERSION.json +3 -3
- package/bundled/dev-pipeline/scripts/generate-bootstrap-prompt.py +462 -4
- package/bundled/dev-pipeline/templates/bootstrap-prompt.md +1 -1
- package/bundled/dev-pipeline/templates/bootstrap-tier1.md +14 -0
- package/bundled/dev-pipeline/templates/bootstrap-tier2.md +18 -3
- package/bundled/dev-pipeline/templates/bootstrap-tier3.md +18 -3
- package/bundled/dev-pipeline/templates/sections/phase-commit-full.md +2 -2
- package/bundled/dev-pipeline/templates/sections/phase-commit.md +2 -2
- package/bundled/dev-pipeline/templates/sections/phase-prizmkit-test.md +260 -0
- package/bundled/dev-pipeline/templates/sections/phase-review-agent.md +5 -1
- package/bundled/dev-pipeline/templates/sections/phase-review-full.md +5 -1
- package/bundled/dev-pipeline/tests/test_generate_bootstrap_prompt.py +429 -0
- package/bundled/dev-pipeline-windows/scripts/generate-bootstrap-prompt.py +462 -4
- package/bundled/dev-pipeline-windows/templates/bootstrap-prompt.md +1 -1
- package/bundled/dev-pipeline-windows/templates/bootstrap-tier1.md +14 -0
- package/bundled/dev-pipeline-windows/templates/bootstrap-tier2.md +18 -3
- package/bundled/dev-pipeline-windows/templates/bootstrap-tier3.md +18 -3
- package/bundled/dev-pipeline-windows/templates/sections/phase-commit-full.md +2 -2
- package/bundled/dev-pipeline-windows/templates/sections/phase-commit.md +2 -2
- package/bundled/dev-pipeline-windows/templates/sections/phase-prizmkit-test.md +186 -0
- package/bundled/dev-pipeline-windows/templates/sections/phase-review-agent.md +5 -1
- package/bundled/dev-pipeline-windows/templates/sections/phase-review-full.md +5 -1
- package/bundled/skills/_metadata.json +1 -1
- package/bundled/skills/prizmkit-code-review/SKILL.md +7 -2
- package/bundled/skills/prizmkit-code-review/references/reviewer-agent-prompt.md +6 -2
- package/bundled/skills/prizmkit-test/SKILL.md +107 -13
- package/bundled/skills/prizmkit-test/references/boundary-coverage-protocol.md +220 -0
- package/bundled/skills/prizmkit-test/references/examples.md +82 -9
- package/bundled/skills/prizmkit-test/references/service-boundary-test-catalog.md +225 -0
- package/bundled/skills/prizmkit-test/references/test-generation-steps.md +74 -14
- package/bundled/skills/prizmkit-test/references/test-report-template.md +74 -5
- package/bundled/skills/prizmkit-test/scripts/validate_boundary_report.py +347 -0
- package/bundled/skills-windows/prizmkit-code-review/SKILL.md +7 -2
- package/bundled/skills-windows/prizmkit-code-review/references/reviewer-agent-prompt.md +6 -2
- package/bundled/skills-windows/prizmkit-test/SKILL.md +107 -13
- package/bundled/skills-windows/prizmkit-test/references/boundary-coverage-protocol.md +220 -0
- package/bundled/skills-windows/prizmkit-test/references/examples.md +82 -9
- package/bundled/skills-windows/prizmkit-test/references/service-boundary-test-catalog.md +225 -0
- package/bundled/skills-windows/prizmkit-test/references/test-generation-steps.md +74 -14
- package/bundled/skills-windows/prizmkit-test/references/test-report-template.md +74 -5
- package/bundled/skills-windows/prizmkit-test/scripts/validate_boundary_report.py +347 -0
- package/package.json +1 -1
|
@@ -1,11 +1,11 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: "prizmkit-test"
|
|
3
|
-
description: "Full-stack test generation and orchestration. Detects architecture, discovers/runs existing tests, analyzes coverage gaps, generates missing
|
|
3
|
+
description: "Full-stack test generation and orchestration. Detects architecture, discovers/runs existing tests, analyzes coverage gaps, generates missing unit/integration/E2E tests including business-specific boundary cases, validates every interface with a required boundary coverage matrix, and outputs a unified report. Use after completing development to verify quality before deploy. Trigger on: 'test', 'run tests', 'check quality', 'verify code', 'generate tests', 'test coverage', 'fill test gaps', 'boundary tests', 'edge cases', 'quality check', '测试', '验证', '跑测试', '补测试', '边界测试'. (project)"
|
|
4
4
|
---
|
|
5
5
|
|
|
6
6
|
# PrizmKit Test
|
|
7
7
|
|
|
8
|
-
A comprehensive test generation and orchestration skill. Discovers existing tests, runs them, compares coverage against spec acceptance criteria and module interfaces, generates missing tests at three levels (unit → integration → E2E), and produces a unified report.
|
|
8
|
+
A comprehensive test generation and orchestration skill. Discovers existing tests, runs them, compares coverage against spec acceptance criteria and module interfaces, generates missing tests at three levels (unit → integration → E2E), verifies business-specific boundary coverage for every in-scope interface, and produces a unified report.
|
|
9
9
|
|
|
10
10
|
### When to Use
|
|
11
11
|
- After completing one or more features/refactors/bugfixes
|
|
@@ -34,11 +34,26 @@ Before execution, load context once:
|
|
|
34
34
|
2. **Project config**: Read `.prizmkit/config.json` (tech stack, AI CLI config).
|
|
35
35
|
3. **Dependencies**: Read `package.json` or equivalent to detect test framework and project type.
|
|
36
36
|
|
|
37
|
+
For `scope=this-change`, load only:
|
|
38
|
+
- `.prizmkit/prizm-docs/root.prizm`
|
|
39
|
+
- relevant L1/L2 docs inferred from in-scope changed files
|
|
40
|
+
- `artifact_dir/spec.md`
|
|
41
|
+
- `artifact_dir/plan.md`
|
|
42
|
+
- `artifact_dir/context-snapshot.md` if present
|
|
43
|
+
- `artifact_dir/review-report.md` if present
|
|
44
|
+
- changed source files and directly related existing tests
|
|
45
|
+
|
|
46
|
+
Do not load all modules or all specs merely because they exist.
|
|
47
|
+
|
|
37
48
|
## Input
|
|
38
49
|
|
|
39
50
|
| Parameter | Required | Description |
|
|
40
51
|
|-----------|----------|-------------|
|
|
41
|
-
| `scope` | No |
|
|
52
|
+
| `scope` | No | `full-project`, `module:<name>`, `feature:<slug>`, or `this-change`. In headless mode, defaults to `this-change` only when `artifact_dir` points to `.prizmkit/specs/<FEATURE_SLUG>/`; otherwise defaults to full project. |
|
|
53
|
+
| `artifact_dir` | No | Feature artifact directory. For `scope=this-change`, must be `.prizmkit/specs/<FEATURE_SLUG>/`. Do not infer this-change scope from `.prizmkit/bugfix/` or `.prizmkit/refactor/`. |
|
|
54
|
+
| `diff_base` | No | Git ref used to compute changed files for this feature. |
|
|
55
|
+
| `changed_files` | No | Explicit caller-provided changed file list. Takes precedence over `diff_base`. |
|
|
56
|
+
| `generation_policy` | No | Defaults to `in-scope-only` for `scope=this-change`. Out-of-scope gaps are reported but not generated. |
|
|
42
57
|
|
|
43
58
|
## Execution
|
|
44
59
|
|
|
@@ -72,7 +87,14 @@ Before execution, load context once:
|
|
|
72
87
|
2. **Single module** — pick from L1 doc module names (e.g., "auth", "payment")
|
|
73
88
|
3. **Single feature** — pick from `.prizmkit/specs/###-*/` directories
|
|
74
89
|
|
|
75
|
-
**Headless mode
|
|
90
|
+
**Headless mode**:
|
|
91
|
+
1. If `scope=this-change`, require `artifact_dir=.prizmkit/specs/<FEATURE_SLUG>/` and run in feature-scoped mode.
|
|
92
|
+
2. If `artifact_dir=.prizmkit/specs/<FEATURE_SLUG>/` is provided without `scope`, treat it as `scope=this-change`.
|
|
93
|
+
3. If `scope=full-project` is explicitly provided, run full-project mode even if an artifact directory exists.
|
|
94
|
+
4. If `artifact_dir` points to `.prizmkit/bugfix/` or `.prizmkit/refactor/`, do not infer `scope=this-change`.
|
|
95
|
+
5. If neither `scope` nor feature `artifact_dir` is provided, default to full project.
|
|
96
|
+
|
|
97
|
+
`scope=this-change` is feature-only. It is valid only when `artifact_dir` is under `.prizmkit/specs/<FEATURE_SLUG>/`. Do not automatically apply `scope=this-change` to `.prizmkit/bugfix/<BUG_ID>/` or `.prizmkit/refactor/<REFACTOR_ID>/`. If the caller passes `scope=this-change` with a bugfix/refactor artifact directory, stop and report that this mode is unsupported for that artifact type. Bugfix and refactor workflows keep their existing reproduction-test and behavior-preservation test gates.
|
|
76
98
|
|
|
77
99
|
### Phase 2: Run Existing Tests
|
|
78
100
|
|
|
@@ -83,13 +105,37 @@ Before execution, load context once:
|
|
|
83
105
|
- Which test files exist (for gap analysis)
|
|
84
106
|
- Raw output (saved to report directory)
|
|
85
107
|
|
|
86
|
-
3.
|
|
108
|
+
3. Classify existing test failures before continuing:
|
|
109
|
+
- For `scope=this-change`, failures in changed tests, tests mapped to in-scope source files, or tests directly covering feature acceptance criteria are `In-Scope Failures` unless a captured pre-feature baseline proves they already failed before this feature.
|
|
110
|
+
- For `scope=this-change`, unrelated failures are `Baseline Failures`; record them without failing the scoped verdict.
|
|
111
|
+
- For other scopes, record failures as existing test failures and do not assume they are caused by this test-generation pass.
|
|
112
|
+
|
|
113
|
+
4. If `scope=this-change` has any unexplained `In-Scope Failures`, the final report verdict must be `NEEDS_FIXES` or `BLOCKED`, not `PASS`. Do not hide them as baseline failures.
|
|
87
114
|
|
|
88
115
|
### Phase 3: Coverage Gap Analysis
|
|
89
116
|
|
|
90
117
|
Staleness of `.prizmkit/prizm-docs/` was already checked during Context Loading (see Precondition). Gap analysis proceeds with the available data.
|
|
91
118
|
|
|
92
|
-
|
|
119
|
+
For `scope=this-change`:
|
|
120
|
+
|
|
121
|
+
1. Build the in-scope source set using this priority:
|
|
122
|
+
- `changed_files` parameter
|
|
123
|
+
- `git diff --name-only <diff_base>`
|
|
124
|
+
- files/tasks listed in `artifact_dir/plan.md`
|
|
125
|
+
- File Manifest in `artifact_dir/context-snapshot.md`
|
|
126
|
+
- direct file/module mapping from `artifact_dir/spec.md` acceptance criteria
|
|
127
|
+
2. Build the in-scope test set:
|
|
128
|
+
- tests already changed in this feature
|
|
129
|
+
- tests matching in-scope source files by project naming convention
|
|
130
|
+
- directly affected integration/E2E tests for feature acceptance criteria
|
|
131
|
+
3. Analyze only:
|
|
132
|
+
- changed/new public interfaces
|
|
133
|
+
- feature acceptance criteria from `artifact_dir/spec.md`
|
|
134
|
+
- integration boundaries directly affected by in-scope files
|
|
135
|
+
- UI/E2E flows directly affected by this feature
|
|
136
|
+
4. For unrelated historical gaps, record them under `Out of Scope Gaps`; do not generate tests and do not fail the feature verdict because of them.
|
|
137
|
+
|
|
138
|
+
For other scopes, compare what exists against what should exist, across three levels:
|
|
93
139
|
|
|
94
140
|
**Unit test gaps** — for each module in scope:
|
|
95
141
|
- Read the corresponding L2 `.prizm` doc INTERFACES section to get exported functions/classes. If no L2 doc exists for a module, analyze source files directly to identify exported functions/classes.
|
|
@@ -106,28 +152,76 @@ Compare what exists against what should exist, across three levels:
|
|
|
106
152
|
- Check existing E2E test files against these criteria
|
|
107
153
|
- Flag uncovered criteria
|
|
108
154
|
|
|
109
|
-
### Phase 4:
|
|
155
|
+
### Phase 4: Boundary Coverage Matrix
|
|
156
|
+
|
|
157
|
+
Phase 3 answers "which interfaces are missing tests?" Phase 4 answers "which business boundaries are missing for each interface?" Build this matrix before generating tests so the run does not mistake broad happy-path coverage for real safety coverage.
|
|
158
|
+
|
|
159
|
+
A path, method, function, or class is not considered covered merely because a happy-path test exists; happy path proves the interface works once, while boundary tests prove it behaves safely under realistic invalid, unauthorized, edge, and failure states.
|
|
160
|
+
|
|
161
|
+
Read `${SKILL_DIR}/references/boundary-coverage-protocol.md` for the required matrix format, category definitions, and completion gate. For every in-scope interface and API endpoint, classify applicable categories such as request validation, auth/permission/ownership, domain invariants, collection/pagination, date/time boundaries, state transitions, dependency failures, and response contract checks. In `scope=this-change`, build this matrix only for the in-scope feature interfaces/endpoints; unrelated interfaces belong in `Out of Scope Gaps`.
|
|
162
|
+
|
|
163
|
+
For each category, mark one of:
|
|
164
|
+
- `covered` — an existing test already asserts the boundary behavior
|
|
165
|
+
- `generated` — this run generated and passed the test
|
|
166
|
+
- `needs-review` — a valid generated test found behavior mismatch
|
|
167
|
+
- `skipped` — not applicable, with a concrete reason
|
|
168
|
+
|
|
169
|
+
If an interface has only happy-path coverage, mark it `boundary-missing` and generate applicable boundary tests in Phase 5. Do not mark happy-path-only coverage as `covered`.
|
|
110
170
|
|
|
111
|
-
|
|
171
|
+
### Phase 5: Generate Missing and Boundary Tests
|
|
112
172
|
|
|
113
|
-
|
|
173
|
+
Read `${SKILL_DIR}/references/test-generation-steps.md` for the detailed generation procedure — boundary-matrix-driven workflow, priority order (unit → integration → E2E), failure handling rules, and generation patterns for each test level.
|
|
174
|
+
|
|
175
|
+
When generating unit tests for service-like functions or API/integration tests for endpoints with business-specific behavior, use `${SKILL_DIR}/references/service-boundary-test-catalog.md` via the generation procedure so boundary cases reflect the interface's business responsibility, not only generic null/empty inputs.
|
|
176
|
+
|
|
177
|
+
For `scope=this-change`:
|
|
178
|
+
- Generate or update tests only for the in-scope source/test set.
|
|
179
|
+
- Check for equivalent existing tests before writing new tests.
|
|
180
|
+
- Do not create tests for unrelated modules just because coverage is missing.
|
|
181
|
+
- Do not refactor unrelated tests.
|
|
182
|
+
- Do not modify unrelated production code.
|
|
183
|
+
- Do not expand to full-project coverage unless the user explicitly requested `scope=full-project`.
|
|
184
|
+
- Generated tests must assert real behavior, boundaries, or integration contracts; do not add empty assertions or mock-success-only tests.
|
|
185
|
+
|
|
186
|
+
### Phase 6: Unified Report and Boundary Validation
|
|
114
187
|
|
|
115
188
|
Create `.prizmkit/test/{YYYY_MM_DD_HH_MM_SS}_testresult/` directory. Read `${SKILL_DIR}/references/test-report-template.md` for the full report format and artifact list.
|
|
116
189
|
|
|
190
|
+
For `scope=this-change`, the report must include:
|
|
191
|
+
- `## Scope` with mode, artifact dir, diff base, changed-files source, generation policy, in-scope source files, and in-scope test files
|
|
192
|
+
- `## Existing Tests Run`
|
|
193
|
+
- `## Generated / Updated Tests`
|
|
194
|
+
- `## In-Scope Failures`
|
|
195
|
+
- `## Baseline Failures`
|
|
196
|
+
- `## Out of Scope Gaps`
|
|
197
|
+
- `## Verdict` with `PASS`, `NEEDS_FIXES`, or `BLOCKED`
|
|
198
|
+
|
|
199
|
+
`Out of Scope Gaps` are informational only for `scope=this-change`: they must not trigger generated tests and must not fail the verdict.
|
|
200
|
+
|
|
201
|
+
After writing the report, run the deterministic boundary-report validator when the project has an OpenAPI file or the report contains a Boundary Matrix:
|
|
202
|
+
|
|
203
|
+
```bash
|
|
204
|
+
python3 ${SKILL_DIR}/scripts/validate_boundary_report.py \
|
|
205
|
+
--report .prizmkit/test/{timestamp}_testresult/test-report.md \
|
|
206
|
+
--openapi openapi.yaml
|
|
207
|
+
```
|
|
208
|
+
|
|
209
|
+
If the project has no OpenAPI file, omit `--openapi`. If validation fails, keep the report but do not claim the run is complete; report that tests may pass while boundary coverage remains incomplete.
|
|
210
|
+
|
|
117
211
|
## Output
|
|
118
212
|
|
|
119
213
|
- Test report: `.prizmkit/test/{timestamp}_testresult/test-report.md`
|
|
120
214
|
- Generated test files written to project test directories
|
|
121
|
-
- Existing test output, generated test copies,
|
|
215
|
+
- Existing test output, generated test copies, E2E artifacts, and boundary validation output in the report directory
|
|
122
216
|
|
|
123
217
|
## Recovery
|
|
124
218
|
|
|
125
219
|
If the session is interrupted:
|
|
126
220
|
- Check `.prizmkit/test/` for the most recent report directory — it contains what was completed before interruption
|
|
127
|
-
- Re-run `/prizmkit-test` — it starts fresh, but Phase 2 will skip tests that already pass, and Phase 3 will re-evaluate gaps
|
|
221
|
+
- Re-run `/prizmkit-test` — it starts fresh, but Phase 2 will skip tests that already pass, and Phase 3/4 will re-evaluate gaps and boundary coverage
|
|
128
222
|
|
|
129
223
|
## Examples
|
|
130
224
|
|
|
131
|
-
Read `${SKILL_DIR}/references/examples.md` for worked examples of full-project
|
|
225
|
+
Read `${SKILL_DIR}/references/examples.md` for worked examples of full-project, single-feature, and happy-path-only remediation runs.
|
|
132
226
|
|
|
133
|
-
**HANDOFF:** Independent skill — no handoff. User may proceed to `/prizmkit-committer` if
|
|
227
|
+
**HANDOFF:** Independent skill — no handoff. User may proceed to `/prizmkit-committer` only if tests pass and the boundary completion gate passes. If tests are marked `needs-review` or boundary validation fails, fix issues manually or run another targeted `/prizmkit-test` pass.
|
|
@@ -0,0 +1,220 @@
|
|
|
1
|
+
# Boundary Coverage Protocol
|
|
2
|
+
|
|
3
|
+
Use this protocol during `/prizmkit-test` Phase 4 and update it during Phase 5. The goal is to prevent a common false positive: treating "there is a test for this interface" as equivalent to "this interface is safely covered." A happy path proves that the interface can work once; boundary tests prove that the interface handles realistic invalid inputs, authorization failures, ownership rules, state transitions, and dependency failures.
|
|
4
|
+
|
|
5
|
+
## Required Boundary Matrix
|
|
6
|
+
|
|
7
|
+
Build this matrix before writing tests and update it after generated tests pass or produce `needs-review` findings. In `scope=this-change`, build the matrix only for in-scope feature interfaces/endpoints; unrelated interfaces are recorded under `Out of Scope Gaps` and do not trigger generated tests.
|
|
8
|
+
|
|
9
|
+
| Module | Interface | Service Type | Happy Path | Request Validation | Auth / Permission / Ownership | Domain Invariants | Collection / Pagination | Date / Time | State Transitions | Dependency Failure | Response Contract | N/A Reasons | Final Status |
|
|
10
|
+
|--------|-----------|--------------|------------|--------------------|-------------------------------|-------------------|-------------------------|-------------|-------------------|-------------------|-------------------|-------------|--------------|
|
|
11
|
+
| auth | POST /auth/login | Auth / Session | covered | generated | N/A — public route | generated | N/A — no collection input | N/A — no date input | covered | skipped — password hasher not injectable | covered | ... | boundary-covered |
|
|
12
|
+
|
|
13
|
+
## Status Values
|
|
14
|
+
|
|
15
|
+
Use these values consistently:
|
|
16
|
+
|
|
17
|
+
- `covered` — an existing test already asserts this behavior.
|
|
18
|
+
- `generated` — this run generated a test and the test passed.
|
|
19
|
+
- `needs-review` — a generated test is valid, but production behavior mismatched the expected contract.
|
|
20
|
+
- `unresolved` — the test could not be made runnable after the allowed fix attempts.
|
|
21
|
+
- `skipped` — not applicable or not practical in this run, with a concrete reason in `N/A Reasons`.
|
|
22
|
+
- `boundary-missing` — applicable boundary exists but no test covers it yet.
|
|
23
|
+
|
|
24
|
+
## Completion Gate
|
|
25
|
+
|
|
26
|
+
An in-scope interface is complete only when:
|
|
27
|
+
|
|
28
|
+
1. It has happy-path coverage when a happy path exists.
|
|
29
|
+
2. Every applicable boundary category is `covered`, `generated`, or `needs-review`.
|
|
30
|
+
3. Every non-applicable category has a concrete reason.
|
|
31
|
+
4. No applicable category remains `boundary-missing`.
|
|
32
|
+
5. The final status is one of:
|
|
33
|
+
- `boundary-covered`
|
|
34
|
+
- `generated-needs-review`
|
|
35
|
+
- `skipped-with-reason`
|
|
36
|
+
|
|
37
|
+
If an interface has only happy-path coverage, set `Final Status` to `boundary-missing` and continue generating applicable boundary tests. Do not call the run complete while any in-scope interface is `boundary-missing` or `happy-path-only`.
|
|
38
|
+
|
|
39
|
+
## How to Decide Whether Existing Tests Count
|
|
40
|
+
|
|
41
|
+
An existing test counts for a boundary category only when it:
|
|
42
|
+
|
|
43
|
+
- targets the same public interface or endpoint,
|
|
44
|
+
- creates the boundary input, state, dependency failure, or authorization context,
|
|
45
|
+
- asserts the expected error code, exception, state change, response shape, or side effect,
|
|
46
|
+
- verifies behavior rather than only checking that the call returned a response.
|
|
47
|
+
|
|
48
|
+
Examples:
|
|
49
|
+
|
|
50
|
+
- `POST /users/create` returns HTTP 200 with non-empty data → happy path only.
|
|
51
|
+
- `POST /users/create` with missing email asserts `code=10001` → request validation boundary.
|
|
52
|
+
- `POST /items/delete` with another user's item asserts forbidden/not-found code → ownership boundary.
|
|
53
|
+
- Service create method with mocked repository failure asserts no partial side effect → dependency failure boundary.
|
|
54
|
+
|
|
55
|
+
## Boundary Categories
|
|
56
|
+
|
|
57
|
+
### Request Validation
|
|
58
|
+
|
|
59
|
+
Use when the interface accepts client input or public parameters.
|
|
60
|
+
|
|
61
|
+
Generate applicable cases for:
|
|
62
|
+
- missing required fields,
|
|
63
|
+
- empty string / empty array / empty object,
|
|
64
|
+
- malformed JSON or wrong content type when the framework exposes that behavior,
|
|
65
|
+
- wrong type,
|
|
66
|
+
- unsupported enum,
|
|
67
|
+
- invalid numeric values: zero, negative, min/max, just above max,
|
|
68
|
+
- invalid string format: URL, email, phone, date, timestamp, UUID/ULID, object key.
|
|
69
|
+
|
|
70
|
+
### Auth / Permission / Ownership
|
|
71
|
+
|
|
72
|
+
Use when the interface depends on identity, roles, tenant, ownership, entitlement, or token state.
|
|
73
|
+
|
|
74
|
+
Generate applicable cases for:
|
|
75
|
+
- missing auth,
|
|
76
|
+
- malformed auth header,
|
|
77
|
+
- invalid/expired/revoked token,
|
|
78
|
+
- authenticated user accessing another user's resource,
|
|
79
|
+
- missing role/permission/entitlement,
|
|
80
|
+
- default-deny when ownership cannot be proven.
|
|
81
|
+
|
|
82
|
+
### Domain Invariants
|
|
83
|
+
|
|
84
|
+
Use when business rules constrain combinations of fields or state.
|
|
85
|
+
|
|
86
|
+
Generate applicable cases for:
|
|
87
|
+
- ratios or totals that must sum to a value,
|
|
88
|
+
- start date before end date,
|
|
89
|
+
- calorie/weight/amount ranges,
|
|
90
|
+
- unsupported lifecycle status,
|
|
91
|
+
- duplicate unique identifiers,
|
|
92
|
+
- device binding requirements,
|
|
93
|
+
- token rotation/reuse semantics,
|
|
94
|
+
- image quality / provider classification rules,
|
|
95
|
+
- product-specific thresholds such as 90%-110% calorie hit boundaries.
|
|
96
|
+
|
|
97
|
+
### Collection / Pagination
|
|
98
|
+
|
|
99
|
+
Use when the interface accepts or returns lists, pages, filters, or cursors.
|
|
100
|
+
|
|
101
|
+
Generate applicable cases for:
|
|
102
|
+
- empty result,
|
|
103
|
+
- one item,
|
|
104
|
+
- many items,
|
|
105
|
+
- duplicate IDs,
|
|
106
|
+
- omitted page / page size defaults,
|
|
107
|
+
- page 0, negative page, max page size, max + 1,
|
|
108
|
+
- invalid filters,
|
|
109
|
+
- nullable filters such as true/false/null,
|
|
110
|
+
- stable ordering or total count when specified.
|
|
111
|
+
|
|
112
|
+
### Date / Time
|
|
113
|
+
|
|
114
|
+
Use when the interface parses, stores, filters, expires, schedules, or compares time.
|
|
115
|
+
|
|
116
|
+
Generate applicable cases for:
|
|
117
|
+
- invalid date / timestamp format,
|
|
118
|
+
- timezone assumptions,
|
|
119
|
+
- inclusive vs exclusive date range boundaries,
|
|
120
|
+
- start after end,
|
|
121
|
+
- exact expiry instant and just before/after expiry,
|
|
122
|
+
- today/yesterday/stale streak or schedule semantics,
|
|
123
|
+
- leap day/month end when relevant.
|
|
124
|
+
|
|
125
|
+
### State Transitions
|
|
126
|
+
|
|
127
|
+
Use when the interface creates, updates, deletes, confirms, unbinds, retries, logs out, refreshes, or changes status.
|
|
128
|
+
|
|
129
|
+
Generate applicable cases for:
|
|
130
|
+
- create then read,
|
|
131
|
+
- update sets expected derived fields,
|
|
132
|
+
- delete excludes from later reads,
|
|
133
|
+
- double delete / unbind idempotency when specified,
|
|
134
|
+
- refresh rotates token and rejects old token,
|
|
135
|
+
- batch operation with empty list vs explicit IDs,
|
|
136
|
+
- retry count below/at/above max,
|
|
137
|
+
- no partial writes when one operation fails.
|
|
138
|
+
|
|
139
|
+
### Dependency Failure
|
|
140
|
+
|
|
141
|
+
Use when collaborators are mockable or the test layer can safely simulate failures.
|
|
142
|
+
|
|
143
|
+
Generate applicable cases for:
|
|
144
|
+
- repository/database error,
|
|
145
|
+
- storage/signing provider failure,
|
|
146
|
+
- SMS/push/email provider failure,
|
|
147
|
+
- LLM/API client error,
|
|
148
|
+
- timeout/network failure,
|
|
149
|
+
- malformed provider response,
|
|
150
|
+
- ID/random/clock failure when injectable.
|
|
151
|
+
|
|
152
|
+
If a dependency cannot be mocked in the current test layer, mark it `skipped` with a reason and consider a lower-level unit test where it can be mocked.
|
|
153
|
+
|
|
154
|
+
### Response Contract
|
|
155
|
+
|
|
156
|
+
Use for API endpoints and public adapters.
|
|
157
|
+
|
|
158
|
+
Generate applicable cases for:
|
|
159
|
+
- unified envelope fields (`code`, `message`, `data`) or framework-specific response format,
|
|
160
|
+
- fixed business error codes,
|
|
161
|
+
- response field names and snake_case/camelCase contract,
|
|
162
|
+
- omitted vs empty data shape,
|
|
163
|
+
- generated IDs and derived fields,
|
|
164
|
+
- side-effect visibility in response.
|
|
165
|
+
|
|
166
|
+
## Endpoint and Interface Mapping
|
|
167
|
+
|
|
168
|
+
Use names, routes, methods, schemas, binding tags, repository calls, and service logic to choose categories. When names and behavior disagree, trust behavior.
|
|
169
|
+
|
|
170
|
+
| Pattern | Likely Categories |
|
|
171
|
+
|---------|-------------------|
|
|
172
|
+
| register / signup / create user | request validation, auth/session, domain invariants, dependency failure, response contract |
|
|
173
|
+
| login | request validation, auth/session, rate limit, token response, dependency failure |
|
|
174
|
+
| refresh / logout | token state, rotation/revocation, idempotency, dependency failure |
|
|
175
|
+
| get / detail | auth/ownership, not found, response contract, dependency failure |
|
|
176
|
+
| list / search / trends | auth, filters, pagination/empty result, date range, response contract, dependency failure |
|
|
177
|
+
| create | request validation, domain invariants, state transition, dependency failure, response contract |
|
|
178
|
+
| update / set goal / confirm | request validation, ownership, partial update semantics, domain invariants, state transition |
|
|
179
|
+
| delete / unbind | request validation, ownership, not found, soft delete, double operation behavior |
|
|
180
|
+
| upload / sign url | request validation, file/storage rules, provider failure, response contract |
|
|
181
|
+
| AI / provider / recognize | request validation, date/time, provider error mapping, quality classifications, async job state |
|
|
182
|
+
| stats / analytics | date range, empty data, boundary thresholds, response shape, dependency failure |
|
|
183
|
+
| achievement / streak | empty state, date continuity, threshold boundaries, state persistence |
|
|
184
|
+
|
|
185
|
+
## N/A Reason Rules
|
|
186
|
+
|
|
187
|
+
A skipped category must explain why it does not apply or why it cannot be exercised at this level.
|
|
188
|
+
|
|
189
|
+
Good reasons:
|
|
190
|
+
- `N/A — public route; no authenticated principal expected`
|
|
191
|
+
- `N/A — endpoint has no collection input or paginated response`
|
|
192
|
+
- `N/A — dependency is not injectable at handler level; covered by service unit test`
|
|
193
|
+
- `N/A — function is pure and has no stateful side effects`
|
|
194
|
+
- `N/A — E2E skipped because project has no UI layer`
|
|
195
|
+
|
|
196
|
+
Bad reasons:
|
|
197
|
+
- `N/A`
|
|
198
|
+
- `not tested`
|
|
199
|
+
- `probably fine`
|
|
200
|
+
- `covered by happy path`
|
|
201
|
+
|
|
202
|
+
## Report Requirements
|
|
203
|
+
|
|
204
|
+
The final report must include:
|
|
205
|
+
|
|
206
|
+
1. The Boundary Matrix.
|
|
207
|
+
2. A Completion Gate summary:
|
|
208
|
+
- interfaces total,
|
|
209
|
+
- boundary-covered count,
|
|
210
|
+
- generated-needs-review count,
|
|
211
|
+
- skipped-with-reason count,
|
|
212
|
+
- boundary-missing count,
|
|
213
|
+
- happy-path-only count.
|
|
214
|
+
3. A clear final status:
|
|
215
|
+
- `All tests passing and boundary coverage complete`, or
|
|
216
|
+
- `Tests passing, but boundary coverage incomplete`, or
|
|
217
|
+
- `Tests failing`, or
|
|
218
|
+
- `Needs human review`.
|
|
219
|
+
|
|
220
|
+
If any interface remains `boundary-missing` or `happy-path-only`, the final report must not say "ready for commit" without qualifying that boundary coverage is incomplete.
|
|
@@ -26,17 +26,26 @@ Scope: Full project (all modules, all specs)
|
|
|
26
26
|
| payment | api/checkout | integration | missing |
|
|
27
27
|
| — | Checkout flow | e2e | missing |
|
|
28
28
|
|
|
29
|
-
**Phase 4
|
|
29
|
+
**Phase 4 — Boundary Matrix** (excerpt):
|
|
30
|
+
| Module | Interface | Service Type | Happy Path | Request Validation | Auth / Permission / Ownership | Domain Invariants | Dependency Failure | Final Status |
|
|
31
|
+
|--------|-----------|--------------|------------|--------------------|-------------------------------|-------------------|-------------------|--------------|
|
|
32
|
+
| auth | login() | Auth / Session | covered | boundary-missing | boundary-missing | skipped | boundary-missing | boundary-missing |
|
|
33
|
+
| payment | processPayment() | Payment / Billing | covered | boundary-missing | boundary-missing | boundary-missing | boundary-missing | boundary-missing |
|
|
30
34
|
|
|
31
|
-
**Phase 5
|
|
35
|
+
**Phase 5**: Generated 5 unit boundary tests (3 passing, 2 assertion failures marked `needs-review`), 1 integration boundary test (passing), 1 E2E test (passing).
|
|
36
|
+
|
|
37
|
+
**Phase 6 — Report Summary**:
|
|
32
38
|
```
|
|
33
39
|
Generated: 5 unit (2 needs-review), 1 integration, 1 E2E
|
|
40
|
+
Boundary gate: failed — 2 interfaces need human review
|
|
34
41
|
Report: .prizmkit/test/2026_05_23_14_30_00_testresult/test-report.md
|
|
35
42
|
Needs human review: processPayment (returns 400 for negative amounts, expected 422), refund (missing authorization header causes 500 instead of 401)
|
|
36
43
|
```
|
|
37
44
|
|
|
38
45
|
## Example 2: Single-Feature Targeted Test
|
|
39
46
|
|
|
47
|
+
This is an interactive single-feature selection. It is different from headless `scope=this-change`, which is driven by `.prizmkit/specs/<FEATURE_SLUG>/` and changed files.
|
|
48
|
+
|
|
40
49
|
**User**: `/prizmkit-test`
|
|
41
50
|
|
|
42
51
|
**Phase 0 — Architecture Detection**:
|
|
@@ -49,22 +58,86 @@ Test framework: Vitest
|
|
|
49
58
|
```
|
|
50
59
|
Scope: Single feature — 042-payment-gateway
|
|
51
60
|
Test files in scope: 2
|
|
61
|
+
```
|
|
52
62
|
|
|
53
63
|
**Phase 2 — Existing Tests**: 2 test files, 8 tests, all passing.
|
|
54
64
|
|
|
55
65
|
**Phase 3 — Gap Analysis** (excerpt):
|
|
56
66
|
| Module | Interface | Type | Status |
|
|
57
67
|
|--------|-----------|------|--------|
|
|
58
|
-
| payment | processPayment() | unit | missing |
|
|
59
|
-
| payment | refund() | unit | missing |
|
|
68
|
+
| payment | processPayment() | unit | boundary-missing |
|
|
69
|
+
| payment | refund() | unit | boundary-missing |
|
|
70
|
+
|
|
71
|
+
**Phase 4 — Boundary Matrix**:
|
|
72
|
+
| Module | Interface | Service Type | Happy Path | Request Validation | Auth / Permission / Ownership | Domain Invariants | State Transitions | Dependency Failure | Final Status |
|
|
73
|
+
|--------|-----------|--------------|------------|--------------------|-------------------------------|-------------------|-------------------|-------------------|--------------|
|
|
74
|
+
| payment | processPayment() | Payment / Billing | covered | boundary-missing | boundary-missing | boundary-missing | generated | boundary-missing | boundary-missing |
|
|
75
|
+
| payment | refund() | Payment / Refund | covered | boundary-missing | boundary-missing | boundary-missing | boundary-missing | skipped — provider sandbox unavailable | boundary-missing |
|
|
60
76
|
|
|
61
|
-
**Phase
|
|
77
|
+
**Phase 5**: Generated 8 unit tests covering amount boundaries, currency mismatch, missing entitlement, duplicate idempotency key, partial refund, full refund, over-refund, and repository failure. All passing.
|
|
62
78
|
|
|
63
|
-
**Phase
|
|
79
|
+
**Phase 6 — Report Summary**:
|
|
64
80
|
```
|
|
65
|
-
Generated:
|
|
81
|
+
Generated: 8 unit tests (all passing)
|
|
82
|
+
Boundary gate: passed
|
|
66
83
|
Report: .prizmkit/test/2026_05_23_15_00_00_testresult/test-report.md
|
|
67
|
-
All tests passing. Ready for commit.
|
|
84
|
+
All tests passing and boundary coverage complete. Ready for commit.
|
|
85
|
+
```
|
|
86
|
+
|
|
87
|
+
## Example 3: Existing Happy Path Is Not Enough
|
|
88
|
+
|
|
89
|
+
**User**: `/prizmkit-test user module`
|
|
90
|
+
|
|
91
|
+
**Existing tests found**:
|
|
92
|
+
```
|
|
93
|
+
TestHandler_UpdateProfile_Success
|
|
94
|
+
TestHandler_GetProfile_Success
|
|
95
|
+
```
|
|
96
|
+
|
|
97
|
+
**Boundary Matrix result**:
|
|
98
|
+
| Module | Interface | Service Type | Happy Path | Request Validation | Auth / Permission / Ownership | Domain Invariants | Dependency Failure | Final Status |
|
|
99
|
+
|--------|-----------|--------------|------------|--------------------|-------------------------------|-------------------|-------------------|--------------|
|
|
100
|
+
| user | POST /user/update-profile | Validation / CRUD | covered | boundary-missing | boundary-missing | boundary-missing | boundary-missing | boundary-missing |
|
|
101
|
+
| user | POST /user/get-profile | CRUD / Auth | covered | N/A — empty request | boundary-missing | N/A — no domain input | boundary-missing | boundary-missing |
|
|
102
|
+
|
|
103
|
+
**Correct action**:
|
|
104
|
+
Generate tests for:
|
|
105
|
+
- missing auth,
|
|
106
|
+
- invalid gender,
|
|
107
|
+
- invalid activity level,
|
|
108
|
+
- implausible height/weight when specified by service contract,
|
|
109
|
+
- user not found,
|
|
110
|
+
- repository failure,
|
|
111
|
+
- response envelope shape.
|
|
112
|
+
|
|
113
|
+
**Incorrect action**:
|
|
114
|
+
Do not report `user` as covered just because both endpoints have success tests.
|
|
115
|
+
|
|
116
|
+
|
|
117
|
+
## Example 3: Feature Pipeline this-change Run
|
|
118
|
+
|
|
119
|
+
**Caller**: `/prizmkit-test scope=this-change artifact_dir=.prizmkit/specs/{{FEATURE_SLUG}}/`
|
|
120
|
+
|
|
121
|
+
**Expected behavior**:
|
|
122
|
+
- Reads `.prizmkit/specs/{{FEATURE_SLUG}}/spec.md`, `plan.md`, and optional `context-snapshot.md`.
|
|
123
|
+
- Builds in-scope files from changed files and feature artifacts.
|
|
124
|
+
- Generates or updates only tests for this feature's changed files, public interfaces, and acceptance criteria.
|
|
125
|
+
- Records unrelated historical gaps under `Out of Scope Gaps`.
|
|
126
|
+
- Does not generate tests for unrelated modules.
|
|
127
|
+
- Does not infer this-change mode from `.prizmkit/bugfix/{{BUG_ID}}/` or `.prizmkit/refactor/{{REFACTOR_ID}}/`.
|
|
128
|
+
|
|
129
|
+
**Report excerpt**:
|
|
130
|
+
```markdown
|
|
131
|
+
## Scope
|
|
132
|
+
- Mode: this-change
|
|
133
|
+
- Artifact Dir: .prizmkit/specs/{{FEATURE_SLUG}}/
|
|
134
|
+
- Generation Policy: in-scope-only
|
|
135
|
+
|
|
136
|
+
## Out of Scope Gaps
|
|
137
|
+
- payment: missing legacy tests, but this feature did not change payment code.
|
|
138
|
+
|
|
139
|
+
## Verdict
|
|
140
|
+
PASS
|
|
68
141
|
```
|
|
69
142
|
|
|
70
|
-
**HANDOFF**: Independent skill — no handoff. User may proceed to `/prizmkit-committer` if
|
|
143
|
+
**HANDOFF**: Independent skill — no handoff. User may proceed to `/prizmkit-committer` only if tests pass and the boundary completion gate passes, or fix issues manually if tests are marked `needs-review`.
|