prizmkit 1.1.86 → 1.1.87

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (41) hide show
  1. package/bundled/VERSION.json +3 -3
  2. package/bundled/dev-pipeline/scripts/generate-bootstrap-prompt.py +462 -4
  3. package/bundled/dev-pipeline/templates/bootstrap-prompt.md +1 -1
  4. package/bundled/dev-pipeline/templates/bootstrap-tier1.md +14 -0
  5. package/bundled/dev-pipeline/templates/bootstrap-tier2.md +18 -3
  6. package/bundled/dev-pipeline/templates/bootstrap-tier3.md +18 -3
  7. package/bundled/dev-pipeline/templates/sections/phase-commit-full.md +2 -2
  8. package/bundled/dev-pipeline/templates/sections/phase-commit.md +2 -2
  9. package/bundled/dev-pipeline/templates/sections/phase-prizmkit-test.md +260 -0
  10. package/bundled/dev-pipeline/templates/sections/phase-review-agent.md +5 -1
  11. package/bundled/dev-pipeline/templates/sections/phase-review-full.md +5 -1
  12. package/bundled/dev-pipeline/tests/test_generate_bootstrap_prompt.py +429 -0
  13. package/bundled/dev-pipeline-windows/scripts/generate-bootstrap-prompt.py +462 -4
  14. package/bundled/dev-pipeline-windows/templates/bootstrap-prompt.md +1 -1
  15. package/bundled/dev-pipeline-windows/templates/bootstrap-tier1.md +14 -0
  16. package/bundled/dev-pipeline-windows/templates/bootstrap-tier2.md +18 -3
  17. package/bundled/dev-pipeline-windows/templates/bootstrap-tier3.md +18 -3
  18. package/bundled/dev-pipeline-windows/templates/sections/phase-commit-full.md +2 -2
  19. package/bundled/dev-pipeline-windows/templates/sections/phase-commit.md +2 -2
  20. package/bundled/dev-pipeline-windows/templates/sections/phase-prizmkit-test.md +186 -0
  21. package/bundled/dev-pipeline-windows/templates/sections/phase-review-agent.md +5 -1
  22. package/bundled/dev-pipeline-windows/templates/sections/phase-review-full.md +5 -1
  23. package/bundled/skills/_metadata.json +1 -1
  24. package/bundled/skills/prizmkit-code-review/SKILL.md +7 -2
  25. package/bundled/skills/prizmkit-code-review/references/reviewer-agent-prompt.md +6 -2
  26. package/bundled/skills/prizmkit-test/SKILL.md +106 -14
  27. package/bundled/skills/prizmkit-test/references/boundary-coverage-protocol.md +220 -0
  28. package/bundled/skills/prizmkit-test/references/examples.md +82 -9
  29. package/bundled/skills/prizmkit-test/references/test-generation-steps.md +68 -12
  30. package/bundled/skills/prizmkit-test/references/test-report-template.md +73 -9
  31. package/bundled/skills/prizmkit-test/scripts/validate_boundary_report.py +347 -0
  32. package/bundled/skills-windows/prizmkit-code-review/SKILL.md +7 -2
  33. package/bundled/skills-windows/prizmkit-code-review/references/reviewer-agent-prompt.md +6 -2
  34. package/bundled/skills-windows/prizmkit-test/SKILL.md +106 -14
  35. package/bundled/skills-windows/prizmkit-test/references/boundary-coverage-protocol.md +220 -0
  36. package/bundled/skills-windows/prizmkit-test/references/examples.md +82 -9
  37. package/bundled/skills-windows/prizmkit-test/references/service-boundary-test-catalog.md +4 -4
  38. package/bundled/skills-windows/prizmkit-test/references/test-generation-steps.md +68 -12
  39. package/bundled/skills-windows/prizmkit-test/references/test-report-template.md +73 -9
  40. package/bundled/skills-windows/prizmkit-test/scripts/validate_boundary_report.py +347 -0
  41. package/package.json +1 -1
@@ -0,0 +1,220 @@
1
+ # Boundary Coverage Protocol
2
+
3
+ Use this protocol during `/prizmkit-test` Phase 4 and update it during Phase 5. The goal is to prevent a common false positive: treating "there is a test for this interface" as equivalent to "this interface is safely covered." A happy path proves that the interface can work once; boundary tests prove that the interface handles realistic invalid inputs, authorization failures, ownership rules, state transitions, and dependency failures.
4
+
5
+ ## Required Boundary Matrix
6
+
7
+ Build this matrix before writing tests and update it after generated tests pass or produce `needs-review` findings. In `scope=this-change`, build the matrix only for in-scope feature interfaces/endpoints; unrelated interfaces are recorded under `Out of Scope Gaps` and do not trigger generated tests.
8
+
9
+ | Module | Interface | Service Type | Happy Path | Request Validation | Auth / Permission / Ownership | Domain Invariants | Collection / Pagination | Date / Time | State Transitions | Dependency Failure | Response Contract | N/A Reasons | Final Status |
10
+ |--------|-----------|--------------|------------|--------------------|-------------------------------|-------------------|-------------------------|-------------|-------------------|-------------------|-------------------|-------------|--------------|
11
+ | auth | POST /auth/login | Auth / Session | covered | generated | N/A — public route | generated | N/A — no collection input | N/A — no date input | covered | skipped — password hasher not injectable | covered | ... | boundary-covered |
12
+
13
+ ## Status Values
14
+
15
+ Use these values consistently:
16
+
17
+ - `covered` — an existing test already asserts this behavior.
18
+ - `generated` — this run generated a test and the test passed.
19
+ - `needs-review` — a generated test is valid, but production behavior mismatched the expected contract.
20
+ - `unresolved` — the test could not be made runnable after the allowed fix attempts.
21
+ - `skipped` — not applicable or not practical in this run, with a concrete reason in `N/A Reasons`.
22
+ - `boundary-missing` — applicable boundary exists but no test covers it yet.
23
+
24
+ ## Completion Gate
25
+
26
+ An in-scope interface is complete only when:
27
+
28
+ 1. It has happy-path coverage when a happy path exists.
29
+ 2. Every applicable boundary category is `covered`, `generated`, or `needs-review`.
30
+ 3. Every non-applicable category has a concrete reason.
31
+ 4. No applicable category remains `boundary-missing`.
32
+ 5. The final status is one of:
33
+ - `boundary-covered`
34
+ - `generated-needs-review`
35
+ - `skipped-with-reason`
36
+
37
+ If an interface has only happy-path coverage, set `Final Status` to `boundary-missing` and continue generating applicable boundary tests. Do not call the run complete while any in-scope interface is `boundary-missing` or `happy-path-only`.
38
+
39
+ ## How to Decide Whether Existing Tests Count
40
+
41
+ An existing test counts for a boundary category only when it:
42
+
43
+ - targets the same public interface or endpoint,
44
+ - creates the boundary input, state, dependency failure, or authorization context,
45
+ - asserts the expected error code, exception, state change, response shape, or side effect,
46
+ - verifies behavior rather than only checking that the call returned a response.
47
+
48
+ Examples:
49
+
50
+ - `POST /users/create` returns HTTP 200 with non-empty data → happy path only.
51
+ - `POST /users/create` with missing email asserts `code=10001` → request validation boundary.
52
+ - `POST /items/delete` with another user's item asserts forbidden/not-found code → ownership boundary.
53
+ - Service create method with mocked repository failure asserts no partial side effect → dependency failure boundary.
54
+
55
+ ## Boundary Categories
56
+
57
+ ### Request Validation
58
+
59
+ Use when the interface accepts client input or public parameters.
60
+
61
+ Generate applicable cases for:
62
+ - missing required fields,
63
+ - empty string / empty array / empty object,
64
+ - malformed JSON or wrong content type when the framework exposes that behavior,
65
+ - wrong type,
66
+ - unsupported enum,
67
+ - invalid numeric values: zero, negative, min/max, just above max,
68
+ - invalid string format: URL, email, phone, date, timestamp, UUID/ULID, object key.
69
+
70
+ ### Auth / Permission / Ownership
71
+
72
+ Use when the interface depends on identity, roles, tenant, ownership, entitlement, or token state.
73
+
74
+ Generate applicable cases for:
75
+ - missing auth,
76
+ - malformed auth header,
77
+ - invalid/expired/revoked token,
78
+ - authenticated user accessing another user's resource,
79
+ - missing role/permission/entitlement,
80
+ - default-deny when ownership cannot be proven.
81
+
82
+ ### Domain Invariants
83
+
84
+ Use when business rules constrain combinations of fields or state.
85
+
86
+ Generate applicable cases for:
87
+ - ratios or totals that must sum to a value,
88
+ - start date before end date,
89
+ - calorie/weight/amount ranges,
90
+ - unsupported lifecycle status,
91
+ - duplicate unique identifiers,
92
+ - device binding requirements,
93
+ - token rotation/reuse semantics,
94
+ - image quality / provider classification rules,
95
+ - product-specific thresholds such as 90%-110% calorie hit boundaries.
96
+
97
+ ### Collection / Pagination
98
+
99
+ Use when the interface accepts or returns lists, pages, filters, or cursors.
100
+
101
+ Generate applicable cases for:
102
+ - empty result,
103
+ - one item,
104
+ - many items,
105
+ - duplicate IDs,
106
+ - omitted page / page size defaults,
107
+ - page 0, negative page, max page size, max + 1,
108
+ - invalid filters,
109
+ - nullable filters such as true/false/null,
110
+ - stable ordering or total count when specified.
111
+
112
+ ### Date / Time
113
+
114
+ Use when the interface parses, stores, filters, expires, schedules, or compares time.
115
+
116
+ Generate applicable cases for:
117
+ - invalid date / timestamp format,
118
+ - timezone assumptions,
119
+ - inclusive vs exclusive date range boundaries,
120
+ - start after end,
121
+ - exact expiry instant and just before/after expiry,
122
+ - today/yesterday/stale streak or schedule semantics,
123
+ - leap day/month end when relevant.
124
+
125
+ ### State Transitions
126
+
127
+ Use when the interface creates, updates, deletes, confirms, unbinds, retries, logs out, refreshes, or changes status.
128
+
129
+ Generate applicable cases for:
130
+ - create then read,
131
+ - update sets expected derived fields,
132
+ - delete excludes from later reads,
133
+ - double delete / unbind idempotency when specified,
134
+ - refresh rotates token and rejects old token,
135
+ - batch operation with empty list vs explicit IDs,
136
+ - retry count below/at/above max,
137
+ - no partial writes when one operation fails.
138
+
139
+ ### Dependency Failure
140
+
141
+ Use when collaborators are mockable or the test layer can safely simulate failures.
142
+
143
+ Generate applicable cases for:
144
+ - repository/database error,
145
+ - storage/signing provider failure,
146
+ - SMS/push/email provider failure,
147
+ - LLM/API client error,
148
+ - timeout/network failure,
149
+ - malformed provider response,
150
+ - ID/random/clock failure when injectable.
151
+
152
+ If a dependency cannot be mocked in the current test layer, mark it `skipped` with a reason and consider a lower-level unit test where it can be mocked.
153
+
154
+ ### Response Contract
155
+
156
+ Use for API endpoints and public adapters.
157
+
158
+ Generate applicable cases for:
159
+ - unified envelope fields (`code`, `message`, `data`) or framework-specific response format,
160
+ - fixed business error codes,
161
+ - response field names and snake_case/camelCase contract,
162
+ - omitted vs empty data shape,
163
+ - generated IDs and derived fields,
164
+ - side-effect visibility in response.
165
+
166
+ ## Endpoint and Interface Mapping
167
+
168
+ Use names, routes, methods, schemas, binding tags, repository calls, and service logic to choose categories. When names and behavior disagree, trust behavior.
169
+
170
+ | Pattern | Likely Categories |
171
+ |---------|-------------------|
172
+ | register / signup / create user | request validation, auth/session, domain invariants, dependency failure, response contract |
173
+ | login | request validation, auth/session, rate limit, token response, dependency failure |
174
+ | refresh / logout | token state, rotation/revocation, idempotency, dependency failure |
175
+ | get / detail | auth/ownership, not found, response contract, dependency failure |
176
+ | list / search / trends | auth, filters, pagination/empty result, date range, response contract, dependency failure |
177
+ | create | request validation, domain invariants, state transition, dependency failure, response contract |
178
+ | update / set goal / confirm | request validation, ownership, partial update semantics, domain invariants, state transition |
179
+ | delete / unbind | request validation, ownership, not found, soft delete, double operation behavior |
180
+ | upload / sign url | request validation, file/storage rules, provider failure, response contract |
181
+ | AI / provider / recognize | request validation, date/time, provider error mapping, quality classifications, async job state |
182
+ | stats / analytics | date range, empty data, boundary thresholds, response shape, dependency failure |
183
+ | achievement / streak | empty state, date continuity, threshold boundaries, state persistence |
184
+
185
+ ## N/A Reason Rules
186
+
187
+ A skipped category must explain why it does not apply or why it cannot be exercised at this level.
188
+
189
+ Good reasons:
190
+ - `N/A — public route; no authenticated principal expected`
191
+ - `N/A — endpoint has no collection input or paginated response`
192
+ - `N/A — dependency is not injectable at handler level; covered by service unit test`
193
+ - `N/A — function is pure and has no stateful side effects`
194
+ - `N/A — E2E skipped because project has no UI layer`
195
+
196
+ Bad reasons:
197
+ - `N/A`
198
+ - `not tested`
199
+ - `probably fine`
200
+ - `covered by happy path`
201
+
202
+ ## Report Requirements
203
+
204
+ The final report must include:
205
+
206
+ 1. The Boundary Matrix.
207
+ 2. A Completion Gate summary:
208
+ - interfaces total,
209
+ - boundary-covered count,
210
+ - generated-needs-review count,
211
+ - skipped-with-reason count,
212
+ - boundary-missing count,
213
+ - happy-path-only count.
214
+ 3. A clear final status:
215
+ - `All tests passing and boundary coverage complete`, or
216
+ - `Tests passing, but boundary coverage incomplete`, or
217
+ - `Tests failing`, or
218
+ - `Needs human review`.
219
+
220
+ If any interface remains `boundary-missing` or `happy-path-only`, the final report must not say "ready for commit" without qualifying that boundary coverage is incomplete.
@@ -26,17 +26,26 @@ Scope: Full project (all modules, all specs)
26
26
  | payment | api/checkout | integration | missing |
27
27
  | — | Checkout flow | e2e | missing |
28
28
 
29
- **Phase 4**: Generated 5 unit tests (3 passing, 2 assertion failures marked "needs-review"), 1 integration test (passing), 1 E2E test (passing).
29
+ **Phase 4 Boundary Matrix** (excerpt):
30
+ | Module | Interface | Service Type | Happy Path | Request Validation | Auth / Permission / Ownership | Domain Invariants | Dependency Failure | Final Status |
31
+ |--------|-----------|--------------|------------|--------------------|-------------------------------|-------------------|-------------------|--------------|
32
+ | auth | login() | Auth / Session | covered | boundary-missing | boundary-missing | skipped | boundary-missing | boundary-missing |
33
+ | payment | processPayment() | Payment / Billing | covered | boundary-missing | boundary-missing | boundary-missing | boundary-missing | boundary-missing |
30
34
 
31
- **Phase 5 Report Summary**:
35
+ **Phase 5**: Generated 5 unit boundary tests (3 passing, 2 assertion failures marked `needs-review`), 1 integration boundary test (passing), 1 E2E test (passing).
36
+
37
+ **Phase 6 — Report Summary**:
32
38
  ```
33
39
  Generated: 5 unit (2 needs-review), 1 integration, 1 E2E
40
+ Boundary gate: failed — 2 interfaces need human review
34
41
  Report: .prizmkit/test/2026_05_23_14_30_00_testresult/test-report.md
35
42
  Needs human review: processPayment (returns 400 for negative amounts, expected 422), refund (missing authorization header causes 500 instead of 401)
36
43
  ```
37
44
 
38
45
  ## Example 2: Single-Feature Targeted Test
39
46
 
47
+ This is an interactive single-feature selection. It is different from headless `scope=this-change`, which is driven by `.prizmkit/specs/<FEATURE_SLUG>/` and changed files.
48
+
40
49
  **User**: `/prizmkit-test`
41
50
 
42
51
  **Phase 0 — Architecture Detection**:
@@ -49,22 +58,86 @@ Test framework: Vitest
49
58
  ```
50
59
  Scope: Single feature — 042-payment-gateway
51
60
  Test files in scope: 2
61
+ ```
52
62
 
53
63
  **Phase 2 — Existing Tests**: 2 test files, 8 tests, all passing.
54
64
 
55
65
  **Phase 3 — Gap Analysis** (excerpt):
56
66
  | Module | Interface | Type | Status |
57
67
  |--------|-----------|------|--------|
58
- | payment | processPayment() | unit | missing |
59
- | payment | refund() | unit | missing |
68
+ | payment | processPayment() | unit | boundary-missing |
69
+ | payment | refund() | unit | boundary-missing |
70
+
71
+ **Phase 4 — Boundary Matrix**:
72
+ | Module | Interface | Service Type | Happy Path | Request Validation | Auth / Permission / Ownership | Domain Invariants | State Transitions | Dependency Failure | Final Status |
73
+ |--------|-----------|--------------|------------|--------------------|-------------------------------|-------------------|-------------------|-------------------|--------------|
74
+ | payment | processPayment() | Payment / Billing | covered | boundary-missing | boundary-missing | boundary-missing | generated | boundary-missing | boundary-missing |
75
+ | payment | refund() | Payment / Refund | covered | boundary-missing | boundary-missing | boundary-missing | boundary-missing | skipped — provider sandbox unavailable | boundary-missing |
60
76
 
61
- **Phase 4**: Generated 2 unit tests (both passing). No integration/E2E applicable for this scope.
77
+ **Phase 5**: Generated 8 unit tests covering amount boundaries, currency mismatch, missing entitlement, duplicate idempotency key, partial refund, full refund, over-refund, and repository failure. All passing.
62
78
 
63
- **Phase 5 — Report Summary**:
79
+ **Phase 6 — Report Summary**:
64
80
  ```
65
- Generated: 2 unit tests (all passing)
81
+ Generated: 8 unit tests (all passing)
82
+ Boundary gate: passed
66
83
  Report: .prizmkit/test/2026_05_23_15_00_00_testresult/test-report.md
67
- All tests passing. Ready for commit.
84
+ All tests passing and boundary coverage complete. Ready for commit.
85
+ ```
86
+
87
+ ## Example 3: Existing Happy Path Is Not Enough
88
+
89
+ **User**: `/prizmkit-test user module`
90
+
91
+ **Existing tests found**:
92
+ ```
93
+ TestHandler_UpdateProfile_Success
94
+ TestHandler_GetProfile_Success
95
+ ```
96
+
97
+ **Boundary Matrix result**:
98
+ | Module | Interface | Service Type | Happy Path | Request Validation | Auth / Permission / Ownership | Domain Invariants | Dependency Failure | Final Status |
99
+ |--------|-----------|--------------|------------|--------------------|-------------------------------|-------------------|-------------------|--------------|
100
+ | user | POST /user/update-profile | Validation / CRUD | covered | boundary-missing | boundary-missing | boundary-missing | boundary-missing | boundary-missing |
101
+ | user | POST /user/get-profile | CRUD / Auth | covered | N/A — empty request | boundary-missing | N/A — no domain input | boundary-missing | boundary-missing |
102
+
103
+ **Correct action**:
104
+ Generate tests for:
105
+ - missing auth,
106
+ - invalid gender,
107
+ - invalid activity level,
108
+ - implausible height/weight when specified by service contract,
109
+ - user not found,
110
+ - repository failure,
111
+ - response envelope shape.
112
+
113
+ **Incorrect action**:
114
+ Do not report `user` as covered just because both endpoints have success tests.
115
+
116
+
117
+ ## Example 3: Feature Pipeline this-change Run
118
+
119
+ **Caller**: `/prizmkit-test scope=this-change artifact_dir=.prizmkit/specs/{{FEATURE_SLUG}}/`
120
+
121
+ **Expected behavior**:
122
+ - Reads `.prizmkit/specs/{{FEATURE_SLUG}}/spec.md`, `plan.md`, and optional `context-snapshot.md`.
123
+ - Builds in-scope files from changed files and feature artifacts.
124
+ - Generates or updates only tests for this feature's changed files, public interfaces, and acceptance criteria.
125
+ - Records unrelated historical gaps under `Out of Scope Gaps`.
126
+ - Does not generate tests for unrelated modules.
127
+ - Does not infer this-change mode from `.prizmkit/bugfix/{{BUG_ID}}/` or `.prizmkit/refactor/{{REFACTOR_ID}}/`.
128
+
129
+ **Report excerpt**:
130
+ ```markdown
131
+ ## Scope
132
+ - Mode: this-change
133
+ - Artifact Dir: .prizmkit/specs/{{FEATURE_SLUG}}/
134
+ - Generation Policy: in-scope-only
135
+
136
+ ## Out of Scope Gaps
137
+ - payment: missing legacy tests, but this feature did not change payment code.
138
+
139
+ ## Verdict
140
+ PASS
68
141
  ```
69
142
 
70
- **HANDOFF**: Independent skill — no handoff. User may proceed to `/prizmkit-committer` if all tests pass, or fix issues manually if tests are marked "needs-review".
143
+ **HANDOFF**: Independent skill — no handoff. User may proceed to `/prizmkit-committer` only if tests pass and the boundary completion gate passes, or fix issues manually if tests are marked `needs-review`.
@@ -1,13 +1,13 @@
1
1
  # Service Boundary Test Catalog
2
2
 
3
- Use this catalog during `/prizmkit-test` Phase 4a when generating unit tests for service-like functions or class methods. The goal is to turn "write boundary tests" into service-specific scenarios that match the unit's business responsibility.
3
+ Use this catalog during `/prizmkit-test` Phase 4 and Phase 5 when classifying boundary responsibilities and generating unit tests for service-like functions, integration tests for module boundaries, or API/handler tests for endpoints with business-specific behavior. The goal is to turn "write boundary tests" into service-specific scenarios that match the interface's business responsibility instead of stopping at generic null/empty inputs or happy paths.
4
4
 
5
5
  ## How to Use This Catalog
6
6
 
7
- 1. Classify the tested unit by reading its name, module path, parameters, dependencies, state changes, and related Prizm docs/spec acceptance criteria.
8
- 2. Select every service type that applies. A function can be both a payment service and an idempotency service, or both a repository and pagination service.
7
+ 1. Classify the tested unit, module boundary, or endpoint by reading its name, route, method, module path, parameters, request schema, dependencies, state changes, and related Prizm docs/spec acceptance criteria.
8
+ 2. Select every service type that applies. A function or endpoint can be both a payment service and an idempotency service, or both a repository and pagination service.
9
9
  3. Map candidate boundaries to the actual contract in code/docs. Assert behavior that is specified or clearly implied; if expected behavior is unclear, mark the case as `needs-review` instead of inventing production behavior.
10
- 4. Mock external dependencies for unit tests: databases, network calls, file storage, queues, payment gateways, email providers, clocks, and random ID generators.
10
+ 4. Mock external dependencies for unit tests and handler-level tests when the project already uses fake repositories/providers: databases, network calls, file storage, queues, payment gateways, email providers, clocks, and random ID generators.
11
11
  5. Record non-applicable categories as `N/A` with a short reason in the boundary coverage report so skipped categories are explicit.
12
12
 
13
13
  ## Service Type Detection
@@ -1,6 +1,13 @@
1
- # Test Generation Steps (Phase 4)
1
+ # Test Generation Steps (Phase 5)
2
2
 
3
- Generate tests in priority order: unit → integration → E2E. After each batch, run immediately.
3
+ ## Scope Guard
4
+
5
+ Before generating any test, determine whether the run is `scope=this-change`.
6
+
7
+ For `scope=this-change`, use `generation_policy=in-scope-only` by default. Generate tests only for the in-scope source/test set built from the feature artifact directory and changed files. If a missing test is unrelated to the feature, record it in `Out of Scope Gaps` and do not generate it. Do not infer this-change scope from `.prizmkit/bugfix/` or `.prizmkit/refactor/`.
8
+
9
+
10
+ Generate tests from the Phase 4 Boundary Coverage Matrix in priority order: unit → integration → E2E. After each batch, run immediately.
4
11
 
5
12
  When generated tests fail, distinguish two cases:
6
13
  - **Test bug** (syntax error, wrong import, wrong mock, wrong framework API usage) → fix the test and re-run
@@ -8,9 +15,49 @@ When generated tests fail, distinguish two cases:
8
15
 
9
16
  If a test fails repeatedly after 2 fix attempts, skip it and mark as "unresolved" in the report.
10
17
 
11
- ## 4a. Unit Tests (always applicable)
18
+ ## 5a. Boundary Matrix First
19
+
20
+ Before writing any new test, create or update the per-interface Boundary Test Matrix from `references/boundary-coverage-protocol.md`. This step matters because a test suite with broad happy-path coverage can still miss the conditions that cause production failures: invalid input, unauthorized access, ownership mistakes, state transitions, dependency errors, and product-specific invariants.
21
+
22
+ For each in-scope interface or endpoint:
23
+ - Identify existing happy-path coverage.
24
+ - Identify existing boundary coverage by reading test assertions, not just test names.
25
+ - Classify applicable categories: request validation, auth/permission/ownership, domain invariants, collection/pagination, date/time, state transitions, dependency failure, and response contract.
26
+ - Mark non-applicable categories as `N/A` with a concrete reason.
27
+ - Mark applicable uncovered categories as `boundary-missing`.
28
+
29
+ Do not generate only happy-path tests. If happy path is missing, generate it, then continue with applicable boundary tests in the same batch.
30
+
31
+ ### Existing Test Evaluation
32
+
33
+ An existing test counts as boundary coverage only when it:
34
+ - targets the same interface/endpoint,
35
+ - sends the relevant boundary input or creates the relevant state/dependency failure,
36
+ - asserts the expected business code, exception, response shape, state change, or side effect,
37
+ - verifies behavior beyond "HTTP 200" or "non-empty response".
12
38
 
13
- For each uncovered interface:
39
+ A test that only checks the route returns successfully is happy-path coverage, not boundary coverage.
40
+
41
+ ### API Endpoint Boundary Discovery
42
+
43
+ For API endpoints, derive boundary cases from:
44
+ - OpenAPI request schemas and response schemas,
45
+ - handler binding tags and request parsing behavior,
46
+ - service validation logic,
47
+ - repository/service error handling,
48
+ - Prizm L2 TRAPS and DECISIONS,
49
+ - existing tests and source behavior,
50
+ - product rules in project brief/specs.
51
+
52
+ Map each endpoint to business-specific cases. Examples:
53
+ - `POST /auth/refresh`: missing token, malformed token, reused/rotated token, revoked token, token-version race if testable.
54
+ - `POST /meals/update`: missing ID, missing foods, invalid meal type, invalid timestamp, not found, wrong owner, recomputed totals, confirmed state transition.
55
+ - `POST /stats/trends`: invalid type, invalid period, invalid date range, empty data, each supported trend type, inclusive end date behavior.
56
+ - `POST /upload/sign-url`: unsupported content type, file_count 0/default/max/max+1, scene handling, storage provider failure.
57
+
58
+ ## 5b. Unit Tests (always applicable)
59
+
60
+ For each in-scope uncovered interface:
14
61
  - Read the source file to understand function signature, branch logic, dependencies, state changes, side effects, and failure paths.
15
62
  - Classify the function/class method by service responsibility using names, module path, parameters, dependencies, and related Prizm docs/spec acceptance criteria.
16
63
  - Read `references/service-boundary-test-catalog.md` when the unit is service-like or has business-specific inputs — it maps service responsibilities to boundary cases such as auth/session, payment/wallet, repository, pagination, file storage, notification, queue, cache/idempotency, API client, and date/expiration behavior.
@@ -24,15 +71,16 @@ For each uncovered interface:
24
71
  - Do NOT test framework internals or third-party library behavior
25
72
  - Run tests immediately after generating
26
73
 
27
- ## 4b. Integration Tests
74
+ ## 5c. Integration Tests
28
75
 
29
- For each module with dependencies:
30
- - Generate tests for the module's primary interface exercising its real dependencies
31
- - If API endpoints exist: request/response tests (valid input, invalid input, missing params, auth)
32
- - If database operations exist: CRUD tests using the project's existing test database config
33
- - Run tests immediately after generating
76
+ For each in-scope module with dependencies:
77
+ - Generate tests for the module's primary interface exercising its real dependencies.
78
+ - If API endpoints exist, generate request/response tests for every applicable boundary category in the Boundary Matrix, not only valid input. Cover auth, missing params, invalid formats/enums, ownership/not-found, state transitions, and response envelope or schema.
79
+ - If database operations exist, generate CRUD tests using the project's existing test database config when available. Cover transactions, duplicate keys, soft-delete visibility, rollback, and query boundary behavior when applicable.
80
+ - If a dependency failure is not practical at integration level, mark it `N/A` with the reason and cover it in a unit test where mocks are available.
81
+ - Run tests immediately after generating.
34
82
 
35
- ## 4c. E2E Tests (conditional)
83
+ ## 5d. E2E Tests (conditional)
36
84
 
37
85
  Preconditions (ALL must be met):
38
86
  - Playwright is available (`@playwright/test` or `playwright` in package.json dependencies, or `npx playwright --version` succeeds as fallback)
@@ -44,10 +92,18 @@ Preconditions (ALL must be met):
44
92
  If ALL met:
45
93
  - **Before starting dev server**: detect whether it's already running: (1) check `package.json` scripts for port flags (`--port`, `-p`, `PORT=`), (2) fall back to framework defaults (3000 for React/Next/Express, 5173 for Vite, 8000 for Django, 5000 for Flask), (3) if still unknown, ask the user. Use the detected port in `lsof -i :<port>` to check. If running, tell user: "Dev server appears to be already running on port {N}. Use this running instance for E2E tests?" If user confirms, use existing instance. If user declines, skip E2E.
46
94
  - Start the project dev server (only if not already running). Wait for it to be ready.
47
- - For each uncovered acceptance criterion, generate a Playwright test script
95
+ - For each in-scope uncovered acceptance criterion, generate a Playwright test script.
96
+ - Include boundary user journeys when acceptance criteria mention validation, permissions, empty states, error messages, or lifecycle transitions.
48
97
  - Run generated E2E tests, capture screenshots of failure.
49
98
  - **Only stop dev server if you started it.** Never stop a server that was already running.
50
99
 
51
100
  If NOT met:
52
101
  - Note what was skipped and why in the report
53
102
  - If only the server wasn't startable: still generate E2E script files but mark as "not run — manual execution needed"
103
+
104
+ ## 5e. Boundary Completion Check
105
+
106
+ After generating and running tests:
107
+ - Update the Boundary Matrix statuses based on passing tests and `needs-review` findings.
108
+ - If any interface remains `boundary-missing` or happy-path-only, run another targeted generation pass for those categories before final reporting.
109
+ - Stop only when all in-scope interfaces are `boundary-covered`, `generated-needs-review`, or `skipped-with-reason`, or when remaining gaps are explicitly listed as unresolved.
@@ -1,6 +1,6 @@
1
1
  # Test Report Output Template
2
2
 
3
- Used in Phase 5 of `/prizmkit-test`. Create `.prizmkit/test/{YYYY_MM_DD_HH_MM_SS}_testresult/` directory and write `test-report.md`.
3
+ Used in Phase 6 of `/prizmkit-test`. Create `.prizmkit/test/{YYYY_MM_DD_HH_MM_SS}_testresult/` directory and write `test-report.md`.
4
4
 
5
5
  ## test-report.md Format
6
6
 
@@ -13,7 +13,19 @@ Used in Phase 5 of `/prizmkit-test`. Create `.prizmkit/test/{YYYY_MM_DD_HH_MM_SS
13
13
  - Test framework: {name}
14
14
  - Mode: {interactive / headless}
15
15
 
16
- ## Existing Tests
16
+ ## Scope
17
+ - Mode: {full-project / module:<name> / feature:<slug> / this-change}
18
+ - Artifact Dir: {none / .prizmkit/specs/<FEATURE_SLUG>/}
19
+ - Diff Base: {ref or none}
20
+ - Changed Files Source: {changed_files / diff_base / plan.md / context-snapshot.md / spec.md / not applicable}
21
+ - Generation Policy: {in-scope-only / full-project}
22
+ - In-Scope Source Files:
23
+ - {...}
24
+ - In-Scope Test Files:
25
+ - {...}
26
+
27
+
28
+ ## Existing Tests Run
17
29
  - Total: {N} | Passed: {N} | Failed: {N}
18
30
  {failed test details or "All passing"}
19
31
  {or "No existing tests found — generated first batch"}
@@ -21,21 +33,57 @@ Used in Phase 5 of `/prizmkit-test`. Create `.prizmkit/test/{YYYY_MM_DD_HH_MM_SS
21
33
  ## Coverage Gaps Found
22
34
  | Module | Interface | Type | Status |
23
35
  |--------|-----------|------|--------|
24
- | ... | ... | unit/integration/e2e | covered / generated / needs-review / unresolved / skipped |
36
+ | ... | ... | unit/integration/e2e | covered / generated / needs-review / unresolved / skipped / boundary-missing |
37
+
38
+ ## Boundary Matrix
39
+ | Module | Interface | Service Type | Happy Path | Request Validation | Auth / Permission / Ownership | Domain Invariants | Collection / Pagination | Date / Time | State Transitions | Dependency Failure | Response Contract | N/A Reasons | Final Status |
40
+ |--------|-----------|--------------|------------|--------------------|-------------------------------|-------------------|-------------------------|-------------|-------------------|-------------------|-------------------|-------------|--------------|
41
+ | ... | ... | auth/repository/upload/etc. | covered/generated/skipped | covered/generated/needs-review/skipped/boundary-missing | ... | ... | ... | ... | ... | ... | ... | category — reason | boundary-covered / generated-needs-review / skipped-with-reason / boundary-missing |
42
+
43
+ ## Boundary Completion Gate
44
+ - Interfaces total: {N}
45
+ - Boundary-covered: {N}
46
+ - Generated-needs-review: {N}
47
+ - Skipped-with-reason: {N}
48
+ - Boundary-missing: {N}
49
+ - Happy-path-only: {N}
50
+ - Completion gate passed: {yes / no}
25
51
 
26
- ## Boundary Coverage
27
- | Module | Interface | Service Type | Boundary Categories Covered | N/A Categories |
28
- |--------|-----------|--------------|-----------------------------|----------------|
29
- | ... | ... | auth/payment/repository/etc. | happy path, service-specific, branch paths, side effects, dependency failures | concurrency — no shared mutable state |
52
+ If `Boundary-missing > 0` or `Happy-path-only > 0`, write:
53
+ `Tests may pass, but boundary coverage is incomplete.`
30
54
 
31
- ## Generated Tests
55
+ ## Boundary Coverage Notes
56
+ | Module | Interface | Boundary Categories Covered | Missing Applicable Boundaries | N/A Categories with Reasons |
57
+ |--------|-----------|-----------------------------|-------------------------------|-----------------------------|
58
+ | ... | ... | request validation, ownership, dependency failure | invalid timestamp, repo failure | pagination — endpoint has no list semantics |
59
+
60
+ ## Generated / Updated Tests
32
61
  - Unit: {N} generated, {N} needs-review, {N} unresolved
33
62
  - Integration: {N} generated, {N} needs-review, {N} unresolved
34
63
  - E2E: {N} generated, {N} skipped ({reason})
35
64
 
65
+ ## Boundary Validation
66
+ - Validator command: `{command or "not run — reason"}`
67
+ - Validator result: {passed / failed / skipped}
68
+ - Validator output: {summary or path to raw output}
69
+
70
+ ## In-Scope Failures
71
+ - {failures caused by this-change scope, or "None"}
72
+
73
+ ## Baseline Failures
74
+ - {pre-existing failures, or "None"}
75
+
76
+ ## Out of Scope Gaps
77
+ - {unrelated historical gaps; informational only for scope=this-change}
78
+
79
+ ## Verdict
80
+ PASS | NEEDS_FIXES | BLOCKED
81
+
36
82
  ## Final Status
37
83
  - All tests passing: {yes / no}
38
- - Needs human review: {list of tests marked needs-review}
84
+ - Boundary completion gate passed: {yes / no}
85
+ - Ready for commit: {yes / no — include reason}
86
+ - Needs human review: {list of tests/interfaces marked needs-review}
39
87
  {remaining failures if any}
40
88
  ```
41
89
 
@@ -45,3 +93,19 @@ Also save alongside `test-report.md`:
45
93
  - `existing-test-output.txt` — raw output from Phase 2
46
94
  - `generated-tests/` — copies of all generated test files
47
95
  - `e2e-output/` — E2E logs and failure screenshots (if applicable)
96
+ - `boundary-validation-output.txt` — output from `scripts/validate_boundary_report.py` when run
97
+
98
+ ## Final Status Rules
99
+
100
+ Use these rules so the report does not overstate quality:
101
+
102
+ | Tests Passing | Boundary Gate | Needs Review | Ready for Commit |
103
+ |---------------|---------------|--------------|------------------|
104
+ | yes | yes | no | yes |
105
+ | yes | yes | yes | no — human review required |
106
+ | yes | no | any | no — boundary coverage incomplete |
107
+ | no | any | any | no — tests failing |
108
+
109
+ Do not write "All tests passing. Ready for commit." if any interface is `boundary-missing`, `happy-path-only`, `unresolved`, or `needs-review`.
110
+
111
+ Out-of-scope gaps are informational only. They must not trigger generated tests and must not fail a `scope=this-change` run.