bmad-method-test-architecture-enterprise 1.2.0 → 1.2.2

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -7,14 +7,17 @@ description: Audit test quality using TEA's comprehensive knowledge base and get
7
7
 
8
8
  Use TEA's `test-review` workflow to audit test quality with objective scoring and actionable feedback. TEA reviews tests against its knowledge base of best practices.
9
9
 
10
+ Coverage scoring is intentionally excluded from `test-review`. Use `trace` for requirements coverage analysis and coverage gate decisions.
11
+
10
12
  ## When to Use This
11
13
 
12
14
  - Want to validate test quality objectively
13
- - Need quality metrics for release gates
15
+ - Need quality metrics for test quality gates
14
16
  - Preparing for production deployment
15
17
  - Reviewing team-written tests
16
18
  - Auditing AI-generated tests
17
19
  - Onboarding new team members (show good patterns)
20
+ - Validating migration quality before coverage expansion
18
21
 
19
22
  ## Prerequisites
20
23
 
@@ -311,42 +314,34 @@ test('should show validation error for expired card', async ({ page }) => {});
311
314
 
312
315
  ## Quality Scores by Category
313
316
 
314
- | Category | Score | Target | Status |
315
- | --------------- | ----- | ------ | -------------------- |
316
- | **Determinism** | 26/35 | 30/35 | ⚠️ Needs Improvement |
317
- | **Isolation** | 22/25 | 20/25 | ✅ Good |
318
- | **Assertions** | 18/20 | 16/20 | Good |
319
- | **Structure** | 7/10 | 8/10 | ⚠️ Minor Issues |
320
- | **Performance** | 3/10 | 8/10 | ❌ Critical |
317
+ | Category | Score | Target | Status |
318
+ | ------------------- | ----- | ------ | -------------------- |
319
+ | **Determinism** | 72 | 80 | ⚠️ Needs Improvement |
320
+ | **Isolation** | 88 | 80 | ✅ Good |
321
+ | **Maintainability** | 70 | 80 | ⚠️ Needs Improvement |
322
+ | **Performance** | 60 | 80 | Critical |
321
323
 
322
324
  ### Scoring Breakdown
323
325
 
324
- **Determinism (35 points max):**
325
-
326
- - No hard waits: 0/10 ❌ (found 3 instances)
327
- - No conditionals: 8/10 ⚠️ (found 2 instances)
328
- - No try-catch flow control: 10/10 ✅
329
- - Network-first patterns: 8/15 ⚠️ (some tests missing)
330
-
331
- **Isolation (25 points max):**
326
+ **Determinism (30% weight):**
332
327
 
333
- - Self-cleaning: 20/20
334
- - No global state: 5/5
335
- - Parallel-safe: 0/0 ✅ (not tested)
328
+ - Hard waits and race conditions penalized
329
+ - Unstable control flow patterns penalized
336
330
 
337
- **Assertions (20 points max):**
331
+ **Isolation (30% weight):**
338
332
 
339
- - Explicit in test body: 15/15
340
- - Specific and meaningful: 3/5 ⚠️ (some weak assertions)
333
+ - Shared state and order dependency penalized
334
+ - Cleanup and parallel-safety rewarded
341
335
 
342
- **Structure (10 points max):**
336
+ **Maintainability (25% weight):**
343
337
 
344
- - Test size < 300 lines: 5/5
345
- - Clear names: 2/5 ⚠️ (some vague names)
338
+ - Overly large files and copy-paste patterns penalized
339
+ - Naming clarity and structure rewarded
346
340
 
347
- **Performance (10 points max):**
341
+ **Performance (15% weight):**
348
342
 
349
- - Execution time < 1.5 min: 3/10 ❌ (3 tests exceed limit)
343
+ - Serial bottlenecks and inefficient setup penalized
344
+ - Parallel-friendly structure rewarded
350
345
 
351
346
  ## Files Reviewed
352
347
 
@@ -402,31 +397,30 @@ TEA reviewed against these patterns:
402
397
 
403
398
  ### Scoring Criteria
404
399
 
405
- **Determinism (35 points):**
400
+ **Determinism (30%):**
406
401
  - Tests produce same result every run
407
402
  - No random failures (flakiness)
408
403
  - No environment-dependent behavior
409
404
 
410
- **Isolation (25 points):**
405
+ **Isolation (30%):**
411
406
  - Tests don't depend on each other
412
407
  - Can run in any order
413
408
  - Clean up after themselves
414
409
 
415
- **Assertions (20 points):**
416
- - Verify actual behavior
417
- - Specific and meaningful
418
- - Not abstracted away in helpers
419
-
420
- **Structure (10 points):**
410
+ **Maintainability (25%):**
421
411
  - Readable and maintainable
422
412
  - Appropriate size
423
413
  - Clear naming
424
414
 
425
- **Performance (10 points):**
415
+ **Performance (15%):**
426
416
  - Fast execution
427
417
  - Efficient selectors
428
418
  - No unnecessary waits
429
419
 
420
+ **Coverage:**
421
+ - Not scored in `test-review`
422
+ - Use `trace` for coverage percentage, requirement mapping, and gate decisions
423
+
430
424
  ## What You Get
431
425
 
432
426
  ### Quality Report
@@ -457,9 +451,9 @@ TEA reviewed against these patterns:
457
451
  Make test review part of release checklist:
458
452
 
459
453
  ```markdown
460
- ## Release Checklist
454
+ ## Quality Checklist (Test-Review)
461
455
  - [ ] All tests passing
462
- - [ ] Test review score > 80
456
+ - [ ] Test-review quality score > 80
463
457
  - [ ] Critical issues resolved
464
458
  - [ ] Performance within budget
465
459
  ````
@@ -483,21 +477,23 @@ Use scores as quality gates:
483
477
  # .github/workflows/test.yml
484
478
  - name: Review test quality
485
479
  run: |
486
- # Run test review
487
- # Parse score from report
480
+ # Run test-review quality gate
481
+ # Parse quality score from report
488
482
  if [ $SCORE -lt 80 ]; then
489
- echo "Test quality below threshold"
483
+ echo "Test-review quality gate below threshold"
490
484
  exit 1
491
485
  fi
492
486
  ```
493
487
 
488
+ Coverage gate checks are handled by `trace`, not `test-review`.
489
+
494
490
  ### Review Regularly
495
491
 
496
492
  Schedule periodic reviews:
497
493
 
498
494
  - **Per story:** Optional (spot check new tests)
499
495
  - **Per epic:** Recommended (ensure consistency)
500
- - **Per release:** Recommended for quality gates (required if using formal gate process)
496
+ - **Per release:** Recommended for test quality gates (coverage gates remain in `trace`)
501
497
  - **Quarterly:** Audit entire suite
502
498
 
503
499
  ### Focus Reviews
@@ -619,8 +615,8 @@ Don't try to fix everything at once.
619
615
  ## Related Guides
620
616
 
621
617
  - [How to Run ATDD](/docs/how-to/workflows/run-atdd.md) - Generate tests to review
622
- - [How to Run Automate](/docs/how-to/workflows/run-automate.md) - Expand coverage to review
623
- - [How to Run Trace](/docs/how-to/workflows/run-trace.md) - Coverage complements quality
618
+ - [How to Run Automate](/docs/how-to/workflows/run-automate.md) - Expand and improve tests before review
619
+ - [How to Run Trace](/docs/how-to/workflows/run-trace.md) - Coverage analysis and gate decisions
624
620
 
625
621
  ## Understanding the Concepts
626
622
 
@@ -489,7 +489,7 @@ TEA makes evidence-based gate decision and writes to separate file.
489
489
 
490
490
  | Metric | Threshold | Actual | Status |
491
491
  | ------------------ | --------- | ------ | ------ |
492
- | P0/P1 Coverage | >95% | 100% | ✅ |
492
+ | P0/P1 Coverage | P0=100%, P1>=90% | 100% / 100% | ✅ |
493
493
  | Test Quality Score | >80 | 84 | ✅ |
494
494
  | NFR Status | PASS | PASS | ✅ |
495
495
 
@@ -609,7 +609,7 @@ TEA uses deterministic rules when decision_mode = "deterministic":
609
609
 
610
610
  **Evidence:**
611
611
 
612
- - P0 coverage: 60% (below 95% threshold)
612
+ - P0 coverage: 60% (below required 100%)
613
613
  - Critical security vulnerability (CVE-2024-12345)
614
614
  - Test quality: 55/100
615
615
 
@@ -787,8 +787,16 @@ Use traceability in CI:
787
787
  run: |
788
788
  # Run trace Phase 1
789
789
  # Parse coverage percentages
790
- if [ $P0_COVERAGE -lt 95 ]; then
791
- echo "P0 coverage below 95%"
790
+ if [ $P0_COVERAGE -lt 100 ]; then
791
+ echo "P0 coverage below required 100%"
792
+ exit 1
793
+ fi
794
+ if [ $P1_COVERAGE -lt 80 ]; then
795
+ echo "P1 coverage below minimum 80%"
796
+ exit 1
797
+ fi
798
+ if [ $OVERALL_COVERAGE -lt 80 ]; then
799
+ echo "Overall coverage below minimum 80%"
792
800
  exit 1
793
801
  fi
794
802
  ```
@@ -916,7 +924,7 @@ Result: PARTIAL coverage (3/4 criteria)
916
924
 
917
925
  **Use CONCERNS** ⚠️ if:
918
926
 
919
- - P1 coverage 85-90% (close to threshold)
927
+ - P1 coverage 80-89% (below PASS target, above minimum)
920
928
  - Minor quality issues (score 70-79)
921
929
  - NFRs have mitigation plans
922
930
  - Team agrees risk is acceptable
@@ -924,7 +932,7 @@ Result: PARTIAL coverage (3/4 criteria)
924
932
  **Use FAIL** ❌ if:
925
933
 
926
934
  - P0 coverage <100% (critical path gaps)
927
- - P1 coverage <85%
935
+ - P1 coverage <80%
928
936
  - Critical security/performance issues
929
937
  - No mitigation possible
930
938
 
@@ -232,15 +232,15 @@ Quick reference for all 9 TEA (Test Engineering Architect) workflows. For detail
232
232
  - `test-review.md` with quality score (0-100)
233
233
  - Critical issues with fixes
234
234
  - Recommendations
235
- - Category scores (Determinism, Isolation, Assertions, Structure, Performance)
235
+ - Category scores (Determinism, Isolation, Maintainability, Performance)
236
+ - Coverage guidance is informational only; coverage scoring and gates are handled by `trace`
236
237
 
237
238
  **Scoring Categories:**
238
239
 
239
- - Determinism: 35 points
240
- - Isolation: 25 points
241
- - Assertions: 20 points
242
- - Structure: 10 points
243
- - Performance: 10 points
240
+ - Determinism: 30%
241
+ - Isolation: 30%
242
+ - Maintainability: 25%
243
+ - Performance: 15%
244
244
 
245
245
  **How-To Guide:** [Run Test Review](/docs/how-to/workflows/run-test-review.md)
246
246
 
package/package.json CHANGED
@@ -1,7 +1,7 @@
1
1
  {
2
2
  "$schema": "https://json.schemastore.org/package.json",
3
3
  "name": "bmad-method-test-architecture-enterprise",
4
- "version": "1.2.0",
4
+ "version": "1.2.2",
5
5
  "description": "Master Test Architect for quality strategy, test automation, and release gates",
6
6
  "keywords": [
7
7
  "bmad",
package/release_notes.md CHANGED
@@ -1,10 +1,10 @@
1
- ## 🚀 What's New in v1.2.0
1
+ ## 🚀 What's New in v1.2.2
2
2
 
3
- ### New Features
4
- - feat:enhance api-request with operation-based overload and update documentation
3
+ ### 🐛 Bug Fixes
4
+ - fix: issue 29
5
5
 
6
6
  ### 📦 Other Changes
7
- - Merge pull request #28 from bmad-code-org/feat/knowledge-update-pw-utils-3-14-0
7
+ - Merge pull request #31 from bmad-code-org/fix/issue-29
8
8
 
9
9
 
10
10
  ## 📦 Installation
@@ -14,4 +14,4 @@ npx bmad-method install
14
14
  # Select "Test Architect" from module menu
15
15
  ```
16
16
 
17
- **Full Changelog**: https://github.com/bmad-code-org/bmad-method-test-architecture-enterprise/compare/v1.1.0...v1.2.0
17
+ **Full Changelog**: https://github.com/bmad-code-org/bmad-method-test-architecture-enterprise/compare/v1.2.1...v1.2.2
@@ -1,10 +1,10 @@
1
1
  module,phase,name,code,sequence,workflow-file,command,required,agent,options,description,output-location,outputs,
2
- tea,0-learning,Teach Me Testing,TMT,10,_bmad/tea/workflows/testarch/teach-me-testing/workflow.md,bmad_tea_teach-me-testing,false,tea,Create Mode,"Teach testing fundamentals through 7 sessions (TEA Academy)",test_artifacts,"progress file|session notes|certificate",
3
- tea,3-solutioning,Test Design,TD,10,_bmad/tea/workflows/testarch/test-design/workflow.yaml,bmad_tea_test-design,false,tea,Create Mode,"Risk-based test planning",test_artifacts,"test design document",
4
- tea,3-solutioning,Test Framework,TF,20,_bmad/tea/workflows/testarch/framework/workflow.yaml,bmad_tea_framework,false,tea,Create Mode,"Initialize production-ready test framework",test_artifacts,"framework scaffold",
5
- tea,3-solutioning,CI Setup,CI,30,_bmad/tea/workflows/testarch/ci/workflow.yaml,bmad_tea_ci,false,tea,Create Mode,"Configure CI/CD quality pipeline",test_artifacts,"ci config",
6
- tea,4-implementation,ATDD,AT,10,_bmad/tea/workflows/testarch/atdd/workflow.yaml,bmad_tea_atdd,false,tea,Create Mode,"Generate failing tests (TDD red phase)",test_artifacts,"atdd tests",
7
- tea,4-implementation,Test Automation,TA,20,_bmad/tea/workflows/testarch/automate/workflow.yaml,bmad_tea_automate,false,tea,Create Mode,"Expand test coverage",test_artifacts,"test suite",
8
- tea,4-implementation,Test Review,RV,30,_bmad/tea/workflows/testarch/test-review/workflow.yaml,bmad_tea_test-review,false,tea,Validate Mode,"Quality audit (0-100 scoring)",test_artifacts,"review report",
9
- tea,4-implementation,NFR Assessment,NR,40,_bmad/tea/workflows/testarch/nfr-assess/workflow.yaml,bmad_tea_nfr-assess,false,tea,Create Mode,"Non-functional requirements",test_artifacts,"nfr report",
10
- tea,4-implementation,Traceability,TR,50,_bmad/tea/workflows/testarch/trace/workflow.yaml,bmad_tea_trace,false,tea,Create Mode,"Coverage traceability and gate",test_artifacts,"traceability matrix|gate decision",
2
+ tea,0-learning,Teach Me Testing,TMT,10,_bmad/tea/workflows/testarch/teach-me-testing/workflow.md,bmad-tea-teach-me-testing,false,tea,Create Mode,"Teach testing fundamentals through 7 sessions (TEA Academy)",test_artifacts,"progress file|session notes|certificate",
3
+ tea,3-solutioning,Test Design,TD,10,_bmad/tea/workflows/testarch/test-design/workflow.yaml,bmad-tea-testarch-test-design,false,tea,Create Mode,"Risk-based test planning",test_artifacts,"test design document",
4
+ tea,3-solutioning,Test Framework,TF,20,_bmad/tea/workflows/testarch/framework/workflow.yaml,bmad-tea-testarch-framework,false,tea,Create Mode,"Initialize production-ready test framework",test_artifacts,"framework scaffold",
5
+ tea,3-solutioning,CI Setup,CI,30,_bmad/tea/workflows/testarch/ci/workflow.yaml,bmad-tea-testarch-ci,false,tea,Create Mode,"Configure CI/CD quality pipeline",test_artifacts,"ci config",
6
+ tea,4-implementation,ATDD,AT,10,_bmad/tea/workflows/testarch/atdd/workflow.yaml,bmad-tea-testarch-atdd,false,tea,Create Mode,"Generate failing tests (TDD red phase)",test_artifacts,"atdd tests",
7
+ tea,4-implementation,Test Automation,TA,20,_bmad/tea/workflows/testarch/automate/workflow.yaml,bmad-tea-testarch-automate,false,tea,Create Mode,"Expand test coverage",test_artifacts,"test suite",
8
+ tea,4-implementation,Test Review,RV,30,_bmad/tea/workflows/testarch/test-review/workflow.yaml,bmad-tea-testarch-test-review,false,tea,Validate Mode,"Quality audit (0-100 scoring)",test_artifacts,"review report",
9
+ tea,4-implementation,NFR Assessment,NR,40,_bmad/tea/workflows/testarch/nfr-assess/workflow.yaml,bmad-tea-testarch-nfr,false,tea,Create Mode,"Non-functional requirements",test_artifacts,"nfr report",
10
+ tea,4-implementation,Traceability,TR,50,_bmad/tea/workflows/testarch/trace/workflow.yaml,bmad-tea-testarch-trace,false,tea,Create Mode,"Coverage traceability and gate",test_artifacts,"traceability matrix|gate decision",
@@ -7,6 +7,7 @@ Use this checklist to validate that the test quality review workflow completed s
7
7
  ## Prerequisites
8
8
 
9
9
  Note: `test-review` is optional and only audits existing tests; it does not generate tests.
10
+ Coverage analysis is out of scope for this workflow. Use `trace` for coverage metrics and coverage gate decisions.
10
11
 
11
12
  ### Test File Discovery
12
13
 
@@ -72,6 +73,8 @@ Note: `test-review` is optional and only audits existing tests; it does not gene
72
73
 
73
74
  ### Step 3: Quality Criteria Validation
74
75
 
76
+ Coverage criteria are intentionally excluded from this checklist.
77
+
75
78
  **For Each Enabled Criterion:**
76
79
 
77
80
  #### BDD Format (if `check_given_when_then: true`)
@@ -9,6 +9,8 @@
9
9
 
10
10
  Review test quality using TEA knowledge base and produce a 0–100 quality score with actionable findings.
11
11
 
12
+ Coverage assessment is intentionally out of scope for this workflow. Use `trace` for requirements coverage and coverage gate decisions.
13
+
12
14
  ---
13
15
 
14
16
  ## WORKFLOW ARCHITECTURE
@@ -96,6 +96,8 @@ If available:
96
96
 
97
97
  Summarize what was found.
98
98
 
99
+ Coverage mapping and coverage gates are out of scope in `test-review`. Route those concerns to `trace`.
100
+
99
101
  ---
100
102
 
101
103
  ## 4. Save Progress
@@ -1,6 +1,6 @@
1
1
  ---
2
2
  name: 'step-03-quality-evaluation'
3
- description: 'Orchestrate parallel quality dimension checks (5 subprocesses)'
3
+ description: 'Orchestrate parallel quality dimension checks (4 subprocesses)'
4
4
  nextStepFile: './step-03f-aggregate-scores.md'
5
5
  ---
6
6
 
@@ -8,14 +8,21 @@ nextStepFile: './step-03f-aggregate-scores.md'
8
8
 
9
9
  ## STEP GOAL
10
10
 
11
- Launch 5 parallel subprocesses to evaluate independent quality dimensions simultaneously for maximum performance.
11
+ Launch 4 parallel subprocesses to evaluate test quality dimensions:
12
+
13
+ - Determinism
14
+ - Isolation
15
+ - Maintainability
16
+ - Performance
17
+
18
+ Coverage is intentionally excluded from this workflow and handled by `trace`.
12
19
 
13
20
  ## MANDATORY EXECUTION RULES
14
21
 
15
22
  - 📖 Read the entire step file before acting
16
23
  - ✅ Speak in `{communication_language}`
17
- - ✅ Launch FIVE subprocesses in PARALLEL
18
- - ✅ Wait for ALL subprocesses to complete
24
+ - ✅ Launch four subprocesses in PARALLEL
25
+ - ✅ Wait for all subprocesses to complete
19
26
  - ❌ Do NOT evaluate quality sequentially (use subprocesses)
20
27
  - ❌ Do NOT proceed until all subprocesses finish
21
28
 
@@ -57,33 +64,27 @@ const subprocessContext = {
57
64
 
58
65
  ---
59
66
 
60
- ### 2. Launch 5 Parallel Quality Subprocesses
67
+ ### 2. Launch 4 Parallel Quality Subprocesses
61
68
 
62
- **Subprocess A: Determinism Check**
69
+ **Subprocess A: Determinism**
63
70
 
64
71
  - File: `./step-03a-subprocess-determinism.md`
65
72
  - Output: `/tmp/tea-test-review-determinism-${timestamp}.json`
66
73
  - Status: Running in parallel... ⟳
67
74
 
68
- **Subprocess B: Isolation Check**
75
+ **Subprocess B: Isolation**
69
76
 
70
77
  - File: `./step-03b-subprocess-isolation.md`
71
78
  - Output: `/tmp/tea-test-review-isolation-${timestamp}.json`
72
79
  - Status: Running in parallel... ⟳
73
80
 
74
- **Subprocess C: Maintainability Check**
81
+ **Subprocess C: Maintainability**
75
82
 
76
83
  - File: `./step-03c-subprocess-maintainability.md`
77
84
  - Output: `/tmp/tea-test-review-maintainability-${timestamp}.json`
78
85
  - Status: Running in parallel... ⟳
79
86
 
80
- **Subprocess D: Coverage Check**
81
-
82
- - File: `./step-03d-subprocess-coverage.md`
83
- - Output: `/tmp/tea-test-review-coverage-${timestamp}.json`
84
- - Status: Running in parallel... ⟳
85
-
86
- **Subprocess E: Performance Check**
87
+ **Subprocess D: Performance**
87
88
 
88
89
  - File: `./step-03e-subprocess-performance.md`
89
90
  - Output: `/tmp/tea-test-review-performance-${timestamp}.json`
@@ -94,22 +95,8 @@ const subprocessContext = {
94
95
  ### 3. Wait for All Subprocesses
95
96
 
96
97
  ```
97
- ⏳ Waiting for 5 quality subprocesses to complete...
98
- ├── Subprocess A (Determinism): Running...
99
- ├── Subprocess B (Isolation): Running... ⟳
100
- ├── Subprocess C (Maintainability): Running... ⟳
101
- ├── Subprocess D (Coverage): Running... ⟳
102
- └── Subprocess E (Performance): Running... ⟳
103
-
104
- [... time passes ...]
105
-
106
- ├── Subprocess A (Determinism): Complete ✅
107
- ├── Subprocess B (Isolation): Complete ✅
108
- ├── Subprocess C (Maintainability): Complete ✅
109
- ├── Subprocess D (Coverage): Complete ✅
110
- └── Subprocess E (Performance): Complete ✅
111
-
112
- ✅ All 5 quality subprocesses completed successfully!
98
+ ⏳ Waiting for 4 quality subprocesses to complete...
99
+ All 4 quality subprocesses completed successfully!
113
100
  ```
114
101
 
115
102
  ---
@@ -117,7 +104,7 @@ const subprocessContext = {
117
104
  ### 4. Verify All Outputs Exist
118
105
 
119
106
  ```javascript
120
- const outputs = ['determinism', 'isolation', 'maintainability', 'coverage', 'performance'].map(
107
+ const outputs = ['determinism', 'isolation', 'maintainability', 'performance'].map(
121
108
  (dim) => `/tmp/tea-test-review-${dim}-${timestamp}.json`,
122
109
  );
123
110
 
@@ -134,7 +121,7 @@ outputs.forEach((output) => {
134
121
 
135
122
  ```
136
123
  🚀 Performance Report:
137
- - Execution Mode: PARALLEL (5 subprocesses)
124
+ - Execution Mode: PARALLEL (4 subprocesses)
138
125
  - Total Elapsed: ~max(all subprocesses) minutes
139
126
  - Sequential Would Take: ~sum(all subprocesses) minutes
140
127
  - Performance Gain: ~60-70% faster!
@@ -144,11 +131,13 @@ outputs.forEach((output) => {
144
131
 
145
132
  ### 6. Proceed to Aggregation
146
133
 
134
+ Pass the same `timestamp` value to Step 3F (do not regenerate it). Step 3F must read the exact temp files written in this step.
135
+
147
136
  Load next step: `{nextStepFile}`
148
137
 
149
138
  The aggregation step (3F) will:
150
139
 
151
- - Read all 5 subprocess outputs
140
+ - Read all 4 subprocess outputs
152
141
  - Calculate weighted overall score (0-100)
153
142
  - Aggregate violations by severity
154
143
  - Generate review report with top suggestions
@@ -159,7 +148,7 @@ The aggregation step (3F) will:
159
148
 
160
149
  Proceed to Step 3F when:
161
150
 
162
- - ✅ All 5 subprocesses completed successfully
151
+ - ✅ All 4 subprocesses completed successfully
163
152
  - ✅ All output files exist and are valid JSON
164
153
  - ✅ Performance metrics displayed
165
154
 
@@ -171,7 +160,7 @@ Proceed to Step 3F when:
171
160
 
172
161
  ### ✅ SUCCESS:
173
162
 
174
- - All 5 subprocesses launched and completed
163
+ - All 4 subprocesses launched and completed
175
164
  - Output files generated and valid
176
165
  - Parallel execution achieved ~60% performance gain
177
166
 
@@ -9,7 +9,7 @@ outputFile: '{test_artifacts}/test-review.md'
9
9
 
10
10
  ## STEP GOAL
11
11
 
12
- Read outputs from 5 parallel quality subprocesses, calculate weighted overall score (0-100), and aggregate violations for report generation.
12
+ Read outputs from 4 quality subprocesses, calculate weighted overall score (0-100), and aggregate violations for report generation.
13
13
 
14
14
  ---
15
15
 
@@ -17,7 +17,7 @@ Read outputs from 5 parallel quality subprocesses, calculate weighted overall sc
17
17
 
18
18
  - 📖 Read the entire step file before acting
19
19
  - ✅ Speak in `{communication_language}`
20
- - ✅ Read all 5 subprocess outputs
20
+ - ✅ Read all 4 subprocess outputs
21
21
  - ✅ Calculate weighted overall score
22
22
  - ✅ Aggregate violations by severity
23
23
  - ❌ Do NOT re-evaluate quality (use subprocess outputs)
@@ -37,11 +37,16 @@ Read outputs from 5 parallel quality subprocesses, calculate weighted overall sc
37
37
  ### 1. Read All Subprocess Outputs
38
38
 
39
39
  ```javascript
40
- const dimensions = ['determinism', 'isolation', 'maintainability', 'coverage', 'performance'];
40
+ // Use the SAME timestamp generated in Step 3 (do not regenerate).
41
+ const timestamp = subprocessContext?.timestamp;
42
+ if (!timestamp) {
43
+ throw new Error('Missing timestamp from Step 3 context. Pass Step 3 timestamp into Step 3F.');
44
+ }
45
+ const dimensions = ['determinism', 'isolation', 'maintainability', 'performance'];
41
46
  const results = {};
42
47
 
43
48
  dimensions.forEach((dim) => {
44
- const outputPath = `/tmp/tea-test-review-${dim}-{{timestamp}}.json`;
49
+ const outputPath = `/tmp/tea-test-review-${dim}-${timestamp}.json`;
45
50
  results[dim] = JSON.parse(fs.readFileSync(outputPath, 'utf8'));
46
51
  });
47
52
  ```
@@ -63,11 +68,10 @@ if (!allSucceeded) {
63
68
 
64
69
  ```javascript
65
70
  const weights = {
66
- determinism: 0.25, // 25% - Most critical for reliability
67
- isolation: 0.25, // 25% - Critical for parallel execution
68
- maintainability: 0.2, // 20% - Important for long-term health
69
- coverage: 0.15, // 15% - Important but can be improved iteratively
70
- performance: 0.15, // 15% - Important but less critical than correctness
71
+ determinism: 0.3, // 30% - Reliability and flake prevention
72
+ isolation: 0.3, // 30% - Parallel safety and independence
73
+ maintainability: 0.25, // 25% - Readability and long-term health
74
+ performance: 0.15, // 15% - Speed and execution efficiency
71
75
  };
72
76
  ```
73
77
 
@@ -157,7 +161,6 @@ const reviewSummary = {
157
161
  determinism: results.determinism.score,
158
162
  isolation: results.isolation.score,
159
163
  maintainability: results.maintainability.score,
160
- coverage: results.coverage.score,
161
164
  performance: results.performance.score,
162
165
  },
163
166
 
@@ -165,7 +168,6 @@ const reviewSummary = {
165
168
  determinism: results.determinism.grade,
166
169
  isolation: results.isolation.grade,
167
170
  maintainability: results.maintainability.grade,
168
- coverage: results.coverage.grade,
169
171
  performance: results.performance.grade,
170
172
  },
171
173
 
@@ -177,12 +179,12 @@ const reviewSummary = {
177
179
 
178
180
  top_10_recommendations: prioritizedRecommendations,
179
181
 
180
- subprocess_execution: 'PARALLEL (5 quality dimensions)',
182
+ subprocess_execution: 'PARALLEL (4 quality dimensions)',
181
183
  performance_gain: '~60% faster than sequential',
182
184
  };
183
185
 
184
186
  // Save for Step 4 (report generation)
185
- fs.writeFileSync('/tmp/tea-test-review-summary-{{timestamp}}.json', JSON.stringify(reviewSummary, null, 2), 'utf8');
187
+ fs.writeFileSync(`/tmp/tea-test-review-summary-${timestamp}.json`, JSON.stringify(reviewSummary, null, 2), 'utf8');
186
188
  ```
187
189
 
188
190
  ---
@@ -198,9 +200,10 @@ fs.writeFileSync('/tmp/tea-test-review-summary-{{timestamp}}.json', JSON.stringi
198
200
  - Determinism: {determinism_score}/100 ({determinism_grade})
199
201
  - Isolation: {isolation_score}/100 ({isolation_grade})
200
202
  - Maintainability: {maintainability_score}/100 ({maintainability_grade})
201
- - Coverage: {coverage_score}/100 ({coverage_grade})
202
203
  - Performance: {performance_score}/100 ({performance_grade})
203
204
 
205
+ ℹ️ Coverage is excluded from `test-review` scoring. Use `trace` for coverage analysis and gates.
206
+
204
207
  ⚠️ Violations Found:
205
208
  - HIGH: {high_count} violations
206
209
  - MEDIUM: {medium_count} violations
@@ -260,7 +263,7 @@ Load next step: `{nextStepFile}`
260
263
 
261
264
  ### ✅ SUCCESS:
262
265
 
263
- - All 5 subprocess outputs read and parsed
266
+ - All 4 subprocess outputs read and parsed
264
267
  - Overall score calculated with proper weights
265
268
  - Violations aggregated correctly
266
269
  - Summary complete and saved
@@ -271,4 +274,4 @@ Load next step: `{nextStepFile}`
271
274
  - Score calculation incorrect
272
275
  - Summary missing or incomplete
273
276
 
274
- **Master Rule:** All 5 quality dimensions MUST be aggregated for accurate overall score.
277
+ **Master Rule:** Aggregate determinism, isolation, maintainability, and performance only.
@@ -42,6 +42,7 @@ Use `test-review-template.md` to produce `{outputFile}` including:
42
42
  - Critical findings with fixes
43
43
  - Warnings and recommendations
44
44
  - Context references (story/test-design if available)
45
+ - Coverage boundary note: `test-review` does not score coverage. Direct coverage findings to `trace`.
45
46
 
46
47
  ---
47
48
 
@@ -14,6 +14,7 @@ lastSaved: ''
14
14
  ---
15
15
 
16
16
  Note: This review audits existing tests; it does not generate tests.
17
+ Coverage mapping and coverage gates are out of scope here. Use `trace` for coverage decisions.
17
18
 
18
19
  ## Executive Summary
19
20
 
@@ -216,7 +217,7 @@ Grade: {grade}
216
217
  - **Fixtures Used**: {fixture_count} ({fixture_names})
217
218
  - **Data Factories Used**: {factory_count} ({factory_names})
218
219
 
219
- ### Test Coverage Scope
220
+ ### Test Scope
220
221
 
221
222
  - **Test IDs**: {test_id_list}
222
223
  - **Priority Distribution**:
@@ -241,7 +242,6 @@ Grade: {grade}
241
242
  {If story file found:}
242
243
 
243
244
  - **Story File**: [{story_filename}]({story_path})
244
- - **Acceptance Criteria Mapped**: {ac_mapped}/{ac_total} ({ac_coverage}%)
245
245
 
246
246
  {If test-design found:}
247
247
 
@@ -249,18 +249,6 @@ Grade: {grade}
249
249
  - **Risk Assessment**: {risk_level}
250
250
  - **Priority Framework**: P0-P3 applied
251
251
 
252
- ### Acceptance Criteria Validation
253
-
254
- {If story file available, map tests to ACs:}
255
-
256
- | Acceptance Criterion | Test ID | Status | Notes |
257
- | -------------------- | --------- | -------------------------- | ------- |
258
- | {AC_1} | {test_id} | {✅ Covered \| ❌ Missing} | {notes} |
259
- | {AC_2} | {test_id} | {✅ Covered \| ❌ Missing} | {notes} |
260
- | {AC_3} | {test_id} | {✅ Covered \| ❌ Missing} | {notes} |
261
-
262
- **Coverage**: {covered_count}/{total_count} criteria covered ({coverage_percentage}%)
263
-
264
252
  ---
265
253
 
266
254
  ## Knowledge Base References
@@ -276,7 +264,8 @@ This review consulted the following knowledge base fragments:
276
264
  - **[selective-testing.md](../../../testarch/knowledge/selective-testing.md)** - Duplicate coverage detection
277
265
  - **[ci-burn-in.md](../../../testarch/knowledge/ci-burn-in.md)** - Flakiness detection patterns (10-iteration loop)
278
266
  - **[test-priorities.md](../../../testarch/knowledge/test-priorities.md)** - P0/P1/P2/P3 classification framework
279
- - **[traceability.md](../../../testarch/knowledge/traceability.md)** - Requirements-to-tests mapping
267
+
268
+ For coverage mapping, consult `trace` workflow outputs.
280
269
 
281
270
  See [tea-index.csv](../../../testarch/tea-index.csv) for complete knowledge base.
282
271
 
@@ -95,6 +95,10 @@ This checklist covers **two sequential phases**:
95
95
  - [ ] Criteria with PARTIAL status
96
96
  - [ ] Criteria with UNIT-ONLY status
97
97
  - [ ] Criteria with INTEGRATION-ONLY status
98
+ - [ ] Coverage heuristics gaps identified:
99
+ - [ ] Endpoints referenced in requirements but not covered by API tests
100
+ - [ ] Auth/authz criteria missing denied/invalid path tests
101
+ - [ ] Criteria with happy-path-only coverage (missing error scenarios)
98
102
  - [ ] Gaps prioritized by risk level using test-priorities framework:
99
103
  - [ ] **CRITICAL** - P0 criteria without FULL coverage (BLOCKER)
100
104
  - [ ] **HIGH** - P1 criteria without FULL coverage (PR blocker)
@@ -306,9 +310,10 @@ Knowledge fragments referenced:
306
310
  **P1 Criteria Evaluation:**
307
311
 
308
312
  - [ ] P1 test pass rate evaluated (threshold: min_p1_pass_rate)
309
- - [ ] P1 acceptance criteria coverage evaluated (threshold: 95%)
313
+ - [ ] P1 acceptance criteria coverage evaluated (PASS >=90%, CONCERNS 80-89%, FAIL <80%)
310
314
  - [ ] Overall test pass rate evaluated (threshold: min_overall_pass_rate)
311
- - [ ] Code coverage evaluated (threshold: min_coverage)
315
+ - [ ] Overall requirements coverage evaluated (threshold: >=80%)
316
+ - [ ] Code coverage considered if available (informational unless explicitly required by policy)
312
317
  - [ ] P1 decision recorded: PASS or CONCERNS
313
318
 
314
319
  **P2/P3 Criteria Evaluation:**
@@ -58,7 +58,25 @@ Record test IDs, describe blocks, and priority markers if present.
58
58
 
59
59
  ---
60
60
 
61
- ### 3. Save Progress
61
+ ## 3. Build Coverage Heuristics Inventory
62
+
63
+ Capture explicit coverage signals so Phase 1 can detect common blind spots:
64
+
65
+ - API endpoint coverage
66
+ - Inventory endpoints referenced by requirements/specs and endpoints exercised by API tests
67
+ - Mark endpoints with no direct tests
68
+ - Authentication/authorization coverage
69
+ - Detect tests for login/session/token flows and permission-denied paths
70
+ - Mark auth/authz requirements with missing negative-path tests
71
+ - Error-path coverage
72
+ - Detect validation, timeout, network-failure, and server-error scenarios
73
+ - Mark criteria with happy-path-only tests
74
+
75
+ Record these findings in step output as `coverage_heuristics` for Step 3/4.
76
+
77
+ ---
78
+
79
+ ### 4. Save Progress
62
80
 
63
81
  **Save this step's accumulated work to `{outputFile}`.**
64
82
 
@@ -42,6 +42,10 @@ For each acceptance criterion:
42
42
  - Map to matching tests
43
43
  - Mark coverage status: FULL / PARTIAL / NONE / UNIT-ONLY / INTEGRATION-ONLY
44
44
  - Record test level and priority
45
+ - Record heuristic signals:
46
+ - Endpoint coverage present/missing (for API-impacting criteria)
47
+ - Auth/authz coverage present/missing (positive and negative paths)
48
+ - Error-path coverage present/missing (validation, timeout, network/server failures)
45
49
 
46
50
  ---
47
51
 
@@ -51,6 +55,9 @@ Ensure:
51
55
 
52
56
  - P0/P1 criteria have coverage
53
57
  - No duplicate coverage across levels without justification
58
+ - Criteria are not happy-path-only when requirements imply error handling
59
+ - API criteria are not marked FULL if endpoint-level checks are missing
60
+ - Auth/authz criteria include at least one denied/invalid-path test where applicable
54
61
 
55
62
  ---
56
63
 
@@ -10,7 +10,7 @@ tempOutputFile: '/tmp/tea-trace-coverage-matrix-{{timestamp}}.json'
10
10
 
11
11
  ## STEP GOAL
12
12
 
13
- **Phase 1 Final Step:** Analyze coverage gaps, generate recommendations, and output complete coverage matrix to temp file for Phase 2 (gate decision).
13
+ **Phase 1 Final Step:** Analyze coverage gaps (including endpoint/auth/error-path blind spots), generate recommendations, and output complete coverage matrix to temp file for Phase 2 (gate decision).
14
14
 
15
15
  ---
16
16
 
@@ -60,7 +60,27 @@ const lowGaps = uncoveredRequirements.filter((req) => req.priority === 'P3');
60
60
 
61
61
  ---
62
62
 
63
- ### 2. Generate Recommendations
63
+ ### 2. Coverage Heuristics Checks
64
+
65
+ Use the heuristics inventory from Step 2 and mapped criteria from Step 3 to flag common coverage blind spots:
66
+
67
+ ```javascript
68
+ const endpointCoverageGaps = coverageHeuristics?.endpoints_without_tests || [];
69
+ const authCoverageGaps = coverageHeuristics?.auth_missing_negative_paths || [];
70
+ const errorPathGaps = coverageHeuristics?.criteria_happy_path_only || [];
71
+
72
+ const heuristicGapCounts = {
73
+ endpoints_without_tests: endpointCoverageGaps.length,
74
+ auth_missing_negative_paths: authCoverageGaps.length,
75
+ happy_path_only_criteria: errorPathGaps.length,
76
+ };
77
+ ```
78
+
79
+ Heuristics are advisory but must influence gap severity and recommendations, especially for P0/P1 criteria.
80
+
81
+ ---
82
+
83
+ ### 3. Generate Recommendations
64
84
 
65
85
  **Based on gap analysis:**
66
86
 
@@ -94,6 +114,30 @@ if (partialCoverage.length > 0) {
94
114
  });
95
115
  }
96
116
 
117
+ if (endpointCoverageGaps.length > 0) {
118
+ recommendations.push({
119
+ priority: 'HIGH',
120
+ action: `Add API tests for ${endpointCoverageGaps.length} uncovered endpoint(s)`,
121
+ requirements: endpointCoverageGaps.map((r) => r.id || r.endpoint || 'unknown'),
122
+ });
123
+ }
124
+
125
+ if (authCoverageGaps.length > 0) {
126
+ recommendations.push({
127
+ priority: 'HIGH',
128
+ action: `Add negative-path auth/authz tests for ${authCoverageGaps.length} requirement(s)`,
129
+ requirements: authCoverageGaps.map((r) => r.id || 'unknown'),
130
+ });
131
+ }
132
+
133
+ if (errorPathGaps.length > 0) {
134
+ recommendations.push({
135
+ priority: 'MEDIUM',
136
+ action: `Add error/edge scenario tests for ${errorPathGaps.length} happy-path-only criterion/criteria`,
137
+ requirements: errorPathGaps.map((r) => r.id || 'unknown'),
138
+ });
139
+ }
140
+
97
141
  // Quality issues
98
142
  recommendations.push({
99
143
  priority: 'LOW',
@@ -104,24 +148,35 @@ recommendations.push({
104
148
 
105
149
  ---
106
150
 
107
- ### 3. Calculate Coverage Statistics
151
+ ### 4. Calculate Coverage Statistics
108
152
 
109
153
  ```javascript
110
154
  const totalRequirements = traceabilityMatrix.length;
111
155
  const coveredRequirements = traceabilityMatrix.filter((r) => r.coverage === 'FULL' || r.coverage === 'PARTIAL').length;
112
156
  const fullyCovered = traceabilityMatrix.filter((r) => r.coverage === 'FULL').length;
113
157
 
114
- const coveragePercentage = Math.round((fullyCovered / totalRequirements) * 100);
158
+ const safePct = (covered, total) => (total > 0 ? Math.round((covered / total) * 100) : 100);
159
+ const coveragePercentage = safePct(fullyCovered, totalRequirements);
115
160
 
116
161
  // Priority-specific coverage
117
162
  const p0Total = traceabilityMatrix.filter((r) => r.priority === 'P0').length;
118
163
  const p0Covered = traceabilityMatrix.filter((r) => r.priority === 'P0' && r.coverage === 'FULL').length;
119
- const p0CoveragePercentage = Math.round((p0Covered / p0Total) * 100);
164
+ const p1Total = traceabilityMatrix.filter((r) => r.priority === 'P1').length;
165
+ const p1Covered = traceabilityMatrix.filter((r) => r.priority === 'P1' && r.coverage === 'FULL').length;
166
+ const p2Total = traceabilityMatrix.filter((r) => r.priority === 'P2').length;
167
+ const p2Covered = traceabilityMatrix.filter((r) => r.priority === 'P2' && r.coverage === 'FULL').length;
168
+ const p3Total = traceabilityMatrix.filter((r) => r.priority === 'P3').length;
169
+ const p3Covered = traceabilityMatrix.filter((r) => r.priority === 'P3' && r.coverage === 'FULL').length;
170
+
171
+ const p0CoveragePercentage = safePct(p0Covered, p0Total);
172
+ const p1CoveragePercentage = safePct(p1Covered, p1Total);
173
+ const p2CoveragePercentage = safePct(p2Covered, p2Total);
174
+ const p3CoveragePercentage = safePct(p3Covered, p3Total);
120
175
  ```
121
176
 
122
177
  ---
123
178
 
124
- ### 4. Generate Complete Coverage Matrix
179
+ ### 5. Generate Complete Coverage Matrix
125
180
 
126
181
  **Compile all Phase 1 outputs:**
127
182
 
@@ -141,15 +196,9 @@ const coverageMatrix = {
141
196
 
142
197
  priority_breakdown: {
143
198
  P0: { total: p0Total, covered: p0Covered, percentage: p0CoveragePercentage },
144
- P1: {
145
- /* calculate */
146
- },
147
- P2: {
148
- /* calculate */
149
- },
150
- P3: {
151
- /* calculate */
152
- },
199
+ P1: { total: p1Total, covered: p1Covered, percentage: p1CoveragePercentage },
200
+ P2: { total: p2Total, covered: p2Covered, percentage: p2CoveragePercentage },
201
+ P3: { total: p3Total, covered: p3Covered, percentage: p3CoveragePercentage },
153
202
  },
154
203
  },
155
204
 
@@ -162,13 +211,20 @@ const coverageMatrix = {
162
211
  unit_only_items: unitOnlyCoverage,
163
212
  },
164
213
 
214
+ coverage_heuristics: {
215
+ endpoint_gaps: endpointCoverageGaps,
216
+ auth_negative_path_gaps: authCoverageGaps,
217
+ happy_path_only_gaps: errorPathGaps,
218
+ counts: heuristicGapCounts,
219
+ },
220
+
165
221
  recommendations: recommendations,
166
222
  };
167
223
  ```
168
224
 
169
225
  ---
170
226
 
171
- ### 5. Output Coverage Matrix to Temp File
227
+ ### 6. Output Coverage Matrix to Temp File
172
228
 
173
229
  **Write to temp file for Phase 2:**
174
230
 
@@ -181,7 +237,7 @@ console.log(`✅ Phase 1 Complete: Coverage matrix saved to ${outputPath}`);
181
237
 
182
238
  ---
183
239
 
184
- ### 6. Display Phase 1 Summary
240
+ ### 7. Display Phase 1 Summary
185
241
 
186
242
  ```
187
243
  ✅ Phase 1 Complete: Coverage Matrix Generated
@@ -194,9 +250,9 @@ console.log(`✅ Phase 1 Complete: Coverage matrix saved to ${outputPath}`);
194
250
 
195
251
  🎯 Priority Coverage:
196
252
  - P0: {p0Covered}/{p0Total} ({p0CoveragePercentage}%)
197
- - P1: {p1Coverage}%
198
- - P2: {p2Coverage}%
199
- - P3: {p3Coverage}%
253
+ - P1: {p1Covered}/{p1Total} ({p1CoveragePercentage}%)
254
+ - P2: {p2Covered}/{p2Total} ({p2CoveragePercentage}%)
255
+ - P3: {p3Covered}/{p3Total} ({p3CoveragePercentage}%)
200
256
 
201
257
  ⚠️ Gaps Identified:
202
258
  - Critical (P0): {criticalGaps.length}
@@ -204,6 +260,11 @@ console.log(`✅ Phase 1 Complete: Coverage matrix saved to ${outputPath}`);
204
260
  - Medium (P2): {mediumGaps.length}
205
261
  - Low (P3): {lowGaps.length}
206
262
 
263
+ 🔍 Coverage Heuristics:
264
+ - Endpoints without tests: {endpointCoverageGaps.length}
265
+ - Auth negative-path gaps: {authCoverageGaps.length}
266
+ - Happy-path-only criteria: {errorPathGaps.length}
267
+
207
268
  📝 Recommendations: {recommendations.length}
208
269
 
209
270
  🔄 Phase 2: Gate decision (next step)
@@ -225,7 +286,7 @@ console.log(`✅ Phase 1 Complete: Coverage matrix saved to ${outputPath}`);
225
286
 
226
287
  ---
227
288
 
228
- ### 7. Save Progress
289
+ ### 8. Save Progress
229
290
 
230
291
  **Save this step's accumulated work to `{outputFile}`.**
231
292
 
@@ -64,6 +64,9 @@ if (coverageMatrix.phase !== 'PHASE_1_COMPLETE') {
64
64
  ```javascript
65
65
  const stats = coverageMatrix.coverage_statistics;
66
66
  const p0Coverage = stats.priority_breakdown.P0.percentage;
67
+ const p1Coverage = stats.priority_breakdown.P1.percentage;
68
+ const hasP1Requirements = (stats.priority_breakdown.P1.total || 0) > 0;
69
+ const effectiveP1Coverage = hasP1Requirements ? p1Coverage : 100;
67
70
  const overallCoverage = stats.overall_coverage_percentage;
68
71
  const criticalGaps = coverageMatrix.gap_analysis.critical_gaps.length;
69
72
 
@@ -75,23 +78,34 @@ if (p0Coverage < 100) {
75
78
  gateDecision = 'FAIL';
76
79
  rationale = `P0 coverage is ${p0Coverage}% (required: 100%). ${criticalGaps} critical requirements uncovered.`;
77
80
  }
78
- // Rule 2: Overall coverage >= 90% with P0 at 100% → PASS
79
- else if (overallCoverage >= 90) {
81
+ // Rule 2: Overall coverage must be >= 80%
82
+ else if (overallCoverage < 80) {
83
+ gateDecision = 'FAIL';
84
+ rationale = `Overall coverage is ${overallCoverage}% (minimum: 80%). Significant gaps exist.`;
85
+ }
86
+ // Rule 3: P1 coverage < 80% → FAIL
87
+ else if (effectiveP1Coverage < 80) {
88
+ gateDecision = 'FAIL';
89
+ rationale = hasP1Requirements
90
+ ? `P1 coverage is ${effectiveP1Coverage}% (minimum: 80%). High-priority gaps must be addressed.`
91
+ : `P1 requirements are not present; continuing with remaining gate criteria.`;
92
+ }
93
+ // Rule 4: P1 coverage >= 90% and overall >= 80% with P0 at 100% → PASS
94
+ else if (effectiveP1Coverage >= 90) {
80
95
  gateDecision = 'PASS';
81
- rationale = `P0 coverage is 100% and overall coverage is ${overallCoverage}% (target: 90%).`;
96
+ rationale = hasP1Requirements
97
+ ? `P0 coverage is 100%, P1 coverage is ${effectiveP1Coverage}% (target: 90%), and overall coverage is ${overallCoverage}% (minimum: 80%).`
98
+ : `P0 coverage is 100% and overall coverage is ${overallCoverage}% (minimum: 80%). No P1 requirements detected.`;
82
99
  }
83
- // Rule 3: Overall coverage >= 75% with P0 at 100% → CONCERNS
84
- else if (overallCoverage >= 75) {
100
+ // Rule 5: P1 coverage 80-89% with P0 at 100% and overall >= 80% → CONCERNS
101
+ else if (effectiveP1Coverage >= 80) {
85
102
  gateDecision = 'CONCERNS';
86
- rationale = `P0 coverage is 100% but overall coverage is ${overallCoverage}% (target: 90%). Consider expanding coverage.`;
87
- }
88
- // Rule 4: P0 at 100% but overall < 75% FAIL
89
- else {
90
- gateDecision = 'FAIL';
91
- rationale = `Overall coverage is ${overallCoverage}% (minimum: 75%). Significant gaps exist.`;
103
+ rationale = hasP1Requirements
104
+ ? `P0 coverage is 100% and overall coverage is ${overallCoverage}% (minimum: 80%), but P1 coverage is ${effectiveP1Coverage}% (target: 90%).`
105
+ : `P0 coverage is 100% and overall coverage is ${overallCoverage}% (minimum: 80%), but additional non-P1 gaps need mitigation.`;
92
106
  }
93
107
 
94
- // Rule 5: Manual waiver option
108
+ // Rule 6: Manual waiver option
95
109
  const manualWaiver = false; // Can be set via config or user input
96
110
  if (manualWaiver) {
97
111
  gateDecision = 'WAIVED';
@@ -116,9 +130,14 @@ const gateReport = {
116
130
  p0_coverage_actual: `${p0Coverage}%`,
117
131
  p0_status: p0Coverage === 100 ? 'MET' : 'NOT MET',
118
132
 
119
- overall_coverage_target: '90%',
133
+ p1_coverage_target_pass: '90%',
134
+ p1_coverage_minimum: '80%',
135
+ p1_coverage_actual: `${effectiveP1Coverage}%`,
136
+ p1_status: effectiveP1Coverage >= 90 ? 'MET' : effectiveP1Coverage >= 80 ? 'PARTIAL' : 'NOT MET',
137
+
138
+ overall_coverage_minimum: '80%',
120
139
  overall_coverage_actual: `${overallCoverage}%`,
121
- overall_status: overallCoverage >= 90 ? 'MET' : overallCoverage >= 75 ? 'PARTIAL' : 'NOT MET',
140
+ overall_status: overallCoverage >= 80 ? 'MET' : 'NOT MET',
122
141
  },
123
142
 
124
143
  uncovered_requirements: coverageMatrix.gap_analysis.critical_gaps.concat(coverageMatrix.gap_analysis.high_gaps),
@@ -174,7 +193,8 @@ fs.writeFileSync('{outputFile}', reportContent, 'utf8');
174
193
 
175
194
  📊 Coverage Analysis:
176
195
  - P0 Coverage: {p0Coverage}% (Required: 100%) → {p0_status}
177
- - Overall Coverage: {overallCoverage}% (Target: 90%) → {overall_status}
196
+ - P1 Coverage: {effectiveP1Coverage}% (PASS target: 90%, minimum: 80%) → {p1_status}
197
+ - Overall Coverage: {overallCoverage}% (Minimum: 80%) → {overall_status}
178
198
 
179
199
  ✅ Decision Rationale:
180
200
  {rationale}
@@ -243,4 +263,4 @@ Then append the gate decision summary (from section 5 above) to the end of the e
243
263
  - Gate decision logic incorrect
244
264
  - Report missing or incomplete
245
265
 
246
- **Master Rule:** Gate decision MUST be deterministic based on clear criteria (P0 100%, overall 90/75%).
266
+ **Master Rule:** Gate decision MUST be deterministic based on clear criteria (P0 100%, P1 90/80, overall >=80).
@@ -136,6 +136,31 @@ Note: This workflow does not generate tests. If gaps exist, run `*atdd` or `*aut
136
136
 
137
137
  ---
138
138
 
139
+ ### Coverage Heuristics Findings
140
+
141
+ #### Endpoint Coverage Gaps
142
+
143
+ - Endpoints without direct API tests: {endpoint_gap_count}
144
+ - Examples:
145
+ - {endpoint_gap_1}
146
+ - {endpoint_gap_2}
147
+
148
+ #### Auth/Authz Negative-Path Gaps
149
+
150
+ - Criteria missing denied/invalid-path tests: {auth_negative_gap_count}
151
+ - Examples:
152
+ - {auth_gap_1}
153
+ - {auth_gap_2}
154
+
155
+ #### Happy-Path-Only Criteria
156
+
157
+ - Criteria missing error/edge scenarios: {happy_path_only_gap_count}
158
+ - Examples:
159
+ - {happy_path_gap_1}
160
+ - {happy_path_gap_2}
161
+
162
+ ---
163
+
139
164
  ### Quality Assessment
140
165
 
141
166
  #### Tests with Issues
@@ -1,111 +0,0 @@
1
- ---
2
- name: 'step-03d-subprocess-coverage'
3
- description: 'Subprocess: Check test coverage (completeness, edge cases)'
4
- subprocess: true
5
- outputFile: '/tmp/tea-test-review-coverage-{{timestamp}}.json'
6
- ---
7
-
8
- # Subprocess 3D: Coverage Quality Check
9
-
10
- ## SUBPROCESS CONTEXT
11
-
12
- This is an **isolated subprocess** running in parallel with other quality dimension checks.
13
-
14
- **Your task:** Analyze test files for COVERAGE violations only.
15
-
16
- ---
17
-
18
- ## MANDATORY EXECUTION RULES
19
-
20
- - ✅ Check COVERAGE only (not other quality dimensions)
21
- - ✅ Output structured JSON to temp file
22
- - ❌ Do NOT check determinism, isolation, maintainability, or performance
23
-
24
- ---
25
-
26
- ## SUBPROCESS TASK
27
-
28
- ### 1. Identify Coverage Violations
29
-
30
- **HIGH SEVERITY Violations**:
31
-
32
- - Critical user paths not tested (P0 functionality missing)
33
- - API endpoints without tests
34
- - Error handling not tested (no negative test cases)
35
- - Missing authentication/authorization tests
36
-
37
- **MEDIUM SEVERITY Violations**:
38
-
39
- - Edge cases not covered (boundary values, null/empty inputs)
40
- - Only happy path tested (no error scenarios)
41
- - Missing integration tests (only unit or only E2E)
42
- - Insufficient assertion coverage (tests don't verify important outcomes)
43
-
44
- **LOW SEVERITY Violations**:
45
-
46
- - Could benefit from additional test cases
47
- - Minor edge cases not covered
48
- - Documentation incomplete
49
-
50
- ### 2. Calculate Coverage Score
51
-
52
- ```javascript
53
- const criticalGaps = violations.filter((v) => v.severity === 'HIGH').length;
54
- const score = criticalGaps === 0 ? Math.max(0, 100 - violations.length * 5) : Math.max(0, 50 - criticalGaps * 10); // Heavy penalty for critical gaps
55
- ```
56
-
57
- ---
58
-
59
- ## OUTPUT FORMAT
60
-
61
- ```json
62
- {
63
- "dimension": "coverage",
64
- "score": 70,
65
- "max_score": 100,
66
- "grade": "C",
67
- "violations": [
68
- {
69
- "file": "tests/api/",
70
- "severity": "HIGH",
71
- "category": "missing-endpoint-tests",
72
- "description": "API endpoint /api/users/delete not tested",
73
- "suggestion": "Add tests for user deletion including error scenarios"
74
- },
75
- {
76
- "file": "tests/e2e/checkout.spec.ts",
77
- "line": 25,
78
- "severity": "MEDIUM",
79
- "category": "missing-error-case",
80
- "description": "Only happy path tested - no error handling tests",
81
- "suggestion": "Add tests for payment failure, network errors, validation failures"
82
- }
83
- ],
84
- "passed_checks": 8,
85
- "failed_checks": 4,
86
- "violation_summary": {
87
- "HIGH": 1,
88
- "MEDIUM": 2,
89
- "LOW": 1
90
- },
91
- "coverage_gaps": {
92
- "untested_endpoints": ["/api/users/delete", "/api/orders/cancel"],
93
- "untested_user_paths": ["Password reset flow"],
94
- "missing_error_scenarios": ["Payment failures", "Network timeouts"]
95
- },
96
- "recommendations": [
97
- "Add tests for all CRUD operations (especially DELETE)",
98
- "Test error scenarios for each user path",
99
- "Add integration tests between API and E2E layers"
100
- ],
101
- "summary": "Coverage has critical gaps - 4 violations (1 HIGH critical endpoint missing)"
102
- }
103
- ```
104
-
105
- ---
106
-
107
- ## EXIT CONDITION
108
-
109
- Subprocess completes when JSON output written to temp file.
110
-
111
- **Subprocess terminates here.**