bmad-method-test-architecture-enterprise 1.1.0 → 1.2.1

package/docs/how-to/workflows/run-test-review.md

@@ -7,14 +7,17 @@ description: Audit test quality using TEA's comprehensive knowledge base and get
 
 Use TEA's `test-review` workflow to audit test quality with objective scoring and actionable feedback. TEA reviews tests against its knowledge base of best practices.
 
+Coverage scoring is intentionally excluded from `test-review`. Use `trace` for requirements coverage analysis and coverage gate decisions.
+
 ## When to Use This
 
 - Want to validate test quality objectively
-- Need quality metrics for release gates
+- Need quality metrics for test quality gates
 - Preparing for production deployment
 - Reviewing team-written tests
 - Auditing AI-generated tests
 - Onboarding new team members (show good patterns)
+- Validating migration quality before coverage expansion
 
 ## Prerequisites
 
@@ -311,42 +314,34 @@ test('should show validation error for expired card', async ({ page }) => {});
 
 ## Quality Scores by Category
 
-| Category        | Score | Target | Status               |
-| --------------- | ----- | ------ | -------------------- |
-| **Determinism** | 26/35 | 30/35  | ⚠️ Needs Improvement |
-| **Isolation**   | 22/25 | 20/25  | ✅ Good              |
-| **Assertions**  | 18/20 | 16/20  | ✅ Good              |
-| **Structure**   | 7/10  | 8/10   | ⚠️ Minor Issues      |
-| **Performance** | 3/10  | 8/10   | ❌ Critical          |
+| Category            | Score | Target | Status               |
+| ------------------- | ----- | ------ | -------------------- |
+| **Determinism**     | 72    | 80     | ⚠️ Needs Improvement |
+| **Isolation**       | 88    | 80     | ✅ Good              |
+| **Maintainability** | 70    | 80     | ⚠️ Needs Improvement |
+| **Performance**     | 60    | 80     | ❌ Critical          |
 
 ### Scoring Breakdown
 
-**Determinism (35 points max):**
-
-- No hard waits: 0/10 ❌ (found 3 instances)
-- No conditionals: 8/10 ⚠️ (found 2 instances)
-- No try-catch flow control: 10/10 ✅
-- Network-first patterns: 8/15 ⚠️ (some tests missing)
-
-**Isolation (25 points max):**
+**Determinism (30% weight):**
 
-- Self-cleaning: 20/20 ✅
-- No global state: 5/5 ✅
-- Parallel-safe: 0/0 ✅ (not tested)
+- Hard waits and race conditions penalized
+- Unstable control flow patterns penalized
 
-**Assertions (20 points max):**
+**Isolation (30% weight):**
 
-- Explicit in test body: 15/15 ✅
-- Specific and meaningful: 3/5 ⚠️ (some weak assertions)
+- Shared state and order dependency penalized
+- Cleanup and parallel-safety rewarded
 
-**Structure (10 points max):**
+**Maintainability (25% weight):**
 
-- Test size < 300 lines: 5/5 ✅
-- Clear names: 2/5 ⚠️ (some vague names)
+- Overly large files and copy-paste patterns penalized
+- Naming clarity and structure rewarded
 
-**Performance (10 points max):**
+**Performance (15% weight):**
 
-- Execution time < 1.5 min: 3/10 ❌ (3 tests exceed limit)
+- Serial bottlenecks and inefficient setup penalized
+- Parallel-friendly structure rewarded
 
 ## Files Reviewed
 
@@ -402,31 +397,30 @@ TEA reviewed against these patterns:
 
 ### Scoring Criteria
 
-**Determinism (35 points):**
+**Determinism (30%):**
 - Tests produce same result every run
 - No random failures (flakiness)
 - No environment-dependent behavior
 
-**Isolation (25 points):**
+**Isolation (30%):**
 - Tests don't depend on each other
 - Can run in any order
 - Clean up after themselves
 
-**Assertions (20 points):**
-- Verify actual behavior
-- Specific and meaningful
-- Not abstracted away in helpers
-
-**Structure (10 points):**
+**Maintainability (25%):**
 - Readable and maintainable
 - Appropriate size
 - Clear naming
 
-**Performance (10 points):**
+**Performance (15%):**
 - Fast execution
 - Efficient selectors
 - No unnecessary waits
 
+**Coverage:**
+- Not scored in `test-review`
+- Use `trace` for coverage percentage, requirement mapping, and gate decisions
+
 ## What You Get
 
 ### Quality Report
@@ -457,9 +451,9 @@ TEA reviewed against these patterns:
 Make test review part of release checklist:
 
 ```markdown
-## Release Checklist
+## Quality Checklist (Test-Review)
 - [ ] All tests passing
-- [ ] Test review score > 80
+- [ ] Test-review quality score > 80
 - [ ] Critical issues resolved
 - [ ] Performance within budget
 ````
@@ -483,21 +477,23 @@ Use scores as quality gates:
 # .github/workflows/test.yml
 - name: Review test quality
   run: |
-    # Run test review
-    # Parse score from report
+    # Run test-review quality gate
+    # Parse quality score from report
     if [ $SCORE -lt 80 ]; then
-      echo "Test quality below threshold"
+      echo "Test-review quality gate below threshold"
       exit 1
     fi
 ```
 
+Coverage gate checks are handled by `trace`, not `test-review`.
+
 ### Review Regularly
 
 Schedule periodic reviews:
 
 - **Per story:** Optional (spot check new tests)
 - **Per epic:** Recommended (ensure consistency)
-- **Per release:** Recommended for quality gates (required if using formal gate process)
+- **Per release:** Recommended for test quality gates (coverage gates remain in `trace`)
 - **Quarterly:** Audit entire suite
 
 ### Focus Reviews
@@ -619,8 +615,8 @@ Don't try to fix everything at once.
 ## Related Guides
 
 - [How to Run ATDD](/docs/how-to/workflows/run-atdd.md) - Generate tests to review
-- [How to Run Automate](/docs/how-to/workflows/run-automate.md) - Expand coverage to review
-- [How to Run Trace](/docs/how-to/workflows/run-trace.md) - Coverage complements quality
+- [How to Run Automate](/docs/how-to/workflows/run-automate.md) - Expand and improve tests before review
+- [How to Run Trace](/docs/how-to/workflows/run-trace.md) - Coverage analysis and gate decisions
 
 ## Understanding the Concepts
 

package/docs/how-to/workflows/run-trace.md

@@ -489,7 +489,7 @@ TEA makes evidence-based gate decision and writes to separate file.
 
 | Metric             | Threshold | Actual | Status |
 | ------------------ | --------- | ------ | ------ |
-| P0/P1 Coverage     | >95%      | 100%   | ✅      |
+| P0/P1 Coverage     | P0=100%, P1>=90% | 100% / 100% | ✅      |
 | Test Quality Score | >80       | 84     | ✅      |
 | NFR Status         | PASS      | PASS   | ✅      |
 
@@ -609,7 +609,7 @@ TEA uses deterministic rules when decision_mode = "deterministic":
 
 **Evidence:**
 
-- P0 coverage: 60% (below 95% threshold)
+- P0 coverage: 60% (below required 100%)
 - Critical security vulnerability (CVE-2024-12345)
 - Test quality: 55/100
 
@@ -787,8 +787,16 @@ Use traceability in CI:
   run: |
     # Run trace Phase 1
     # Parse coverage percentages
-    if [ $P0_COVERAGE -lt 95 ]; then
-      echo "P0 coverage below 95%"
+    if [ $P0_COVERAGE -lt 100 ]; then
+      echo "P0 coverage below required 100%"
+      exit 1
+    fi
+    if [ $P1_COVERAGE -lt 80 ]; then
+      echo "P1 coverage below minimum 80%"
+      exit 1
+    fi
+    if [ $OVERALL_COVERAGE -lt 80 ]; then
+      echo "Overall coverage below minimum 80%"
       exit 1
     fi
 ```
@@ -916,7 +924,7 @@ Result: PARTIAL coverage (3/4 criteria)
 
 **Use CONCERNS** ⚠️ if:
 
-- P1 coverage 85-90% (close to threshold)
+- P1 coverage 80-89% (below PASS target, above minimum)
 - Minor quality issues (score 70-79)
 - NFRs have mitigation plans
 - Team agrees risk is acceptable
@@ -924,7 +932,7 @@ Result: PARTIAL coverage (3/4 criteria)
 **Use FAIL** ❌ if:
 
 - P0 coverage <100% (critical path gaps)
-- P1 coverage <85%
+- P1 coverage <80%
 - Critical security/performance issues
 - No mitigation possible
 

package/docs/reference/commands.md

@@ -232,15 +232,15 @@ Quick reference for all 9 TEA (Test Engineering Architect) workflows. For detail
 - `test-review.md` with quality score (0-100)
 - Critical issues with fixes
 - Recommendations
-- Category scores (Determinism, Isolation, Assertions, Structure, Performance)
+- Category scores (Determinism, Isolation, Maintainability, Performance)
+- Coverage guidance is informational only; coverage scoring and gates are handled by `trace`
 
 **Scoring Categories:**
 
-- Determinism: 35 points
-- Isolation: 25 points
-- Assertions: 20 points
-- Structure: 10 points
-- Performance: 10 points
+- Determinism: 30%
+- Isolation: 30%
+- Maintainability: 25%
+- Performance: 15%
 
 **How-To Guide:** [Run Test Review](/docs/how-to/workflows/run-test-review.md)
 

package/package.json

@@ -1,7 +1,7 @@
 {
   "$schema": "https://json.schemastore.org/package.json",
   "name": "bmad-method-test-architecture-enterprise",
-  "version": "1.1.0",
+  "version": "1.2.1",
   "description": "Master Test Architect for quality strategy, test automation, and release gates",
   "keywords": [
     "bmad",

package/release_notes.md

@@ -1,15 +1,11 @@
-## 🚀 What's New in v1.1.0
-
-### ✨ New Features
-- feat: prereq prompts part2
+## 🚀 What's New in v1.2.1
 
 ### 🐛 Bug Fixes
-- fix: issues 22 23
+- fix: test review vs trace for coverage
 
 ### 📦 Other Changes
-- Merge pull request #25 from bmad-code-org/feat/prereq-prompts-part2
-- addressed PR comments
-- Merge pull request #26 from bmad-code-org/fix/issue-22-23
+- improved trace with coverage
+- Merge pull request #30 from bmad-code-org/fix/test-review-vs-trace-for-coverage
 
 
 ## 📦 Installation
@@ -19,4 +15,4 @@ npx bmad-method install
 # Select "Test Architect" from module menu
 ```
 
-**Full Changelog**: https://github.com/bmad-code-org/bmad-method-test-architecture-enterprise/compare/v1.0.1...v1.1.0
+**Full Changelog**: https://github.com/bmad-code-org/bmad-method-test-architecture-enterprise/compare/v1.2.0...v1.2.1

package/src/testarch/knowledge/api-request.md

@@ -371,6 +371,95 @@ test.describe('GraphQL API', () => {
 - Check `body.errors` for GraphQL errors (not status code)
 - Works for queries and mutations
 
+### Example 8: Operation-Based Overload (OpenAPI / Code Generators)
+
+**Context**: When using a code generator (orval, openapi-generator, custom scripts) that produces typed operation definitions from an OpenAPI spec, pass the operation object directly to `apiRequest`. This eliminates manual `method`/`path` extraction and `typeof` assertions while preserving full type inference for request body, response, and query parameters. Available since v3.14.0.
+
+**Implementation**:
+
+```typescript
+// Generated operation definition — structural typing, no import from playwright-utils needed
+// type OperationShape = { path: string; method: 'POST'|'GET'|'PUT'|'DELETE'|'PATCH'|'HEAD'; response: unknown; request: unknown; query?: unknown }
+
+import { test, expect } from '@seontechnologies/playwright-utils/api-request/fixtures';
+
+// --- Basic usage: operation replaces method + path ---
+test('should upsert person via operation overload', async ({ apiRequest }) => {
+  const { status, body } = await apiRequest({
+    operation: upsertPersonv2({ customerId }),
+    headers: getHeaders(customerId),
+    body: personInput, // compile-time typed as Schemas.PersonInput
+  });
+
+  expect(status).toBe(200);
+  expect(body.id).toBeDefined(); // body typed as Schemas.Person
+});
+
+// --- Typed query parameters (replaces string concatenation) ---
+test('should list people with typed query', async ({ apiRequest }) => {
+  const { body } = await apiRequest({
+    operation: getPeoplev2({ customerId }),
+    headers: getHeaders(customerId),
+    query: { page: 0, page_size: 5 }, // typed from operation's query definition
+  });
+
+  expect(body.items).toHaveLength(5);
+});
+
+// --- Params escape hatch (pre-formatted query strings) ---
+test('should fetch billing history with raw params', async ({ apiRequest }) => {
+  const { body } = await apiRequest({
+    operation: getBillingHistoryv2({ customerId }),
+    headers: getHeaders(customerId),
+    params: {
+      'filters[start_date]': getThisMonthTimestamp(),
+      'filters[date_type]': 'MONTH',
+    },
+  });
+
+  expect(body.entries.length).toBeGreaterThan(0);
+});
+
+// --- Works with recurse (polling) ---
+test('should poll until person is reviewed', async ({ apiRequest, recurse }) => {
+  await recurse(
+    async () =>
+      apiRequest({
+        operation: getPersonv2({ customerId, hash }),
+        headers: getHeaders(customerId),
+      }),
+    (res) => {
+      expect(res.status).toBe(200);
+      expect(res.body.status).toBe('REVIEWED');
+    },
+    { timeout: 30000, interval: 1000 },
+  );
+});
+
+// --- Schema validation chains work identically ---
+test('should create movie with schema validation', async ({ apiRequest }) => {
+  const { body } = await apiRequest({
+    operation: createMovieOp,
+    headers: commonHeaders(authToken),
+    body: movie,
+  }).validateSchema(CreateMovieResponseSchema, {
+    shape: { status: 200, data: { name: movie.name } },
+  });
+
+  expect(body.data.id).toBeDefined();
+});
+```
+
+**Key Points**:
+
+- Pass `operation` instead of `method` + `path` — mutually exclusive at compile time
+- Response body, request body, and query types inferred from operation definition
+- Uses structural typing (duck typing) — works with any code generator producing `{ path, method, response, request, query? }`
+- `query` field auto-serializes to bracket notation (`filters[type]=pep`, `ids[0]=10`)
+- `params` escape hatch for pre-formatted strings — wins over `query` on conflict
+- Fully composable with `recurse`, `validateSchema`, and all existing features
+- `response`/`request`/`query` on the operation are type-level only — runtime never reads their values
+
 ## Comparison with Vanilla Playwright
 
 | Vanilla Playwright                             | playwright-utils apiRequest                                                        |
@@ -393,6 +482,7 @@ test.describe('GraphQL API', () => {
 - ✅ Tests requiring retry logic
 - ✅ Background API calls in UI tests
 - ✅ Contract testing support
+- ✅ Type-safe API testing with OpenAPI-generated operations (v3.14.0+)
 
 **Stick with vanilla Playwright for:**
 
@@ -440,3 +530,34 @@ const response: any = await apiRequest({ method: 'GET', path: '/users' });
 const { body } = await apiRequest<User[]>({ method: 'GET', path: '/users' });
 // body is typed as User[]
 ```
+
+**❌ Mixing operation overload with explicit generics:**
+
+```typescript
+// Don't pass a generic when using operation — types are inferred from the operation
+const { body } = await apiRequest<MyType>({
+  operation: getPersonv2({ customerId }),
+  headers: getHeaders(customerId),
+});
+```
+
+**✅ Let the operation infer the types:**
+
+```typescript
+const { body } = await apiRequest({
+  operation: getPersonv2({ customerId }),
+  headers: getHeaders(customerId),
+});
+// body type inferred from operation.response
+```
+
+**❌ Mixing operation with method/path:**
+
+```typescript
+// Compile error — operation and method/path are mutually exclusive
+await apiRequest({
+  operation: getPersonv2({ customerId }),
+  method: 'GET', // Error: method?: never
+  path: '/api/person', // Error: path?: never
+});
+```

package/src/testarch/knowledge/api-testing-patterns.md

@@ -197,6 +197,7 @@ test.describe('Orders API', () => {
 - `validateSchema` throws if response doesn't match
 - Built-in retry for transient failures
 - Type-safe `body` access
+- **Note**: If your project uses code-generated operations from an OpenAPI spec, see [Example 8](#example-8-operation-based-api-testing-openapi--code-generators) for the preferred `operation`-based overload (v3.14.0+)
 
 ### Example 3: Microservice-to-Microservice Testing
 
@@ -713,6 +714,69 @@ test.describe('Authenticated API Tests', () => {
 - Test auth, expired tokens, and RBAC
 - Pure API testing without UI
 
+### Example 8: Operation-Based API Testing (OpenAPI / Code Generators)
+
+**Context**: When your project uses code-generated operation definitions from an OpenAPI spec, leverage the operation-based overload of `apiRequest` (v3.14.0+) instead of manual `method`/`path` extraction. This eliminates `typeof` assertions and provides full type inference for request body, response, and query parameters.
+
+**Implementation**:
+
+```typescript
+// tests/api/operations.spec.ts
+import { test, expect } from '@seontechnologies/playwright-utils/api-request/fixtures';
+
+test.describe('API Tests with Generated Operations', () => {
+  test('should create entity with full type safety', async ({ apiRequest }) => {
+    // Operation object from code generator — contains path, method, and type info
+    const { status, body } = await apiRequest({
+      operation: createEntityOp({ workspaceId }),
+      headers: getHeaders(workspaceId),
+      body: entityInput, // Compile-time typed from operation.request
+    });
+
+    expect(status).toBe(201);
+    expect(body.id).toBeDefined(); // body typed from operation.response
+  });
+
+  test('should list with typed query parameters', async ({ apiRequest }) => {
+    // query field replaces manual string concatenation
+    const { body } = await apiRequest({
+      operation: listEntitiesOp({ workspaceId }),
+      headers: getHeaders(workspaceId),
+      query: { page: 0, page_size: 10, status: 'active' },
+    });
+
+    expect(body.items).toHaveLength(10);
+    expect(body.total).toBeGreaterThan(10);
+  });
+
+  test('should poll async operation until complete', async ({ apiRequest, recurse }) => {
+    const { body: job } = await apiRequest({
+      operation: startJobOp({ workspaceId }),
+      headers: getHeaders(workspaceId),
+      body: { type: 'export' },
+    });
+
+    await recurse(
+      async () =>
+        apiRequest({
+          operation: getJobOp({ workspaceId, jobId: job.id }),
+          headers: getHeaders(workspaceId),
+        }),
+      (res) => res.body.status === 'completed',
+      { timeout: 60000, interval: 2000 },
+    );
+  });
+});
+```
+
+**Key Points**:
+
+- `operation` replaces `method` + `path` — mutually exclusive at compile time
+- Types for body, response, and query all inferred from the operation definition
+- Works with any code generator using structural typing (no imports from playwright-utils needed in generator)
+- Composable with `recurse`, `validateSchema`, and all existing `apiRequest` features
+- Preferred approach over `typeof operation.response` for generated operations
+
 ## API Test Configuration
 
 ### Playwright Config for API-Only Tests

package/src/testarch/knowledge/overview.md

@@ -38,17 +38,17 @@ npm install -D @seontechnologies/playwright-utils
 
 ### Core Testing Utilities
 
-| Utility                    | Purpose                                            | Test Context       |
-| -------------------------- | -------------------------------------------------- | ------------------ |
-| **api-request**            | Typed HTTP client with schema validation and retry | **API/Backend**    |
-| **recurse**                | Polling for async operations, background jobs      | **API/Backend**    |
-| **auth-session**           | Token persistence, multi-user, service-to-service  | **API/Backend/UI** |
-| **log**                    | Playwright report-integrated logging               | **API/Backend/UI** |
-| **file-utils**             | CSV/XLSX/PDF/ZIP reading & validation              | **API/Backend/UI** |
-| **burn-in**                | Smart test selection with git diff                 | **CI/CD**          |
-| **network-recorder**       | HAR record/playback for offline testing            | UI only            |
-| **intercept-network-call** | Network spy/stub with auto JSON parsing            | UI only            |
-| **network-error-monitor**  | Automatic HTTP 4xx/5xx detection                   | UI only            |
+| Utility                    | Purpose                                                                       | Test Context       |
+| -------------------------- | ----------------------------------------------------------------------------- | ------------------ |
+| **api-request**            | Typed HTTP client with schema validation, retry, and operation-based overload | **API/Backend**    |
+| **recurse**                | Polling for async operations, background jobs                                 | **API/Backend**    |
+| **auth-session**           | Token persistence, multi-user, service-to-service                             | **API/Backend/UI** |
+| **log**                    | Playwright report-integrated logging                                          | **API/Backend/UI** |
+| **file-utils**             | CSV/XLSX/PDF/ZIP reading & validation                                         | **API/Backend/UI** |
+| **burn-in**                | Smart test selection with git diff                                            | **CI/CD**          |
+| **network-recorder**       | HAR record/playback for offline testing                                       | UI only            |
+| **intercept-network-call** | Network spy/stub with auto JSON parsing                                       | UI only            |
+| **network-error-monitor**  | Automatic HTTP 4xx/5xx detection                                              | UI only            |
 
 **Note**: 6 of 9 utilities work without a browser. Only 3 are UI-specific (network-recorder, intercept-network-call, network-error-monitor).
 

package/src/testarch/tea-index.csv

@@ -21,7 +21,7 @@ test-healing-patterns,Test Healing Patterns,"Common failure patterns and automat
 selector-resilience,Selector Resilience,"Robust selector strategies and debugging techniques","selectors,locators,debugging,ui",knowledge/selector-resilience.md
 timing-debugging,Timing Debugging,"Race condition identification and deterministic wait fixes","timing,async,debugging",knowledge/timing-debugging.md
 overview,Playwright Utils Overview,"Installation, design principles, fixture patterns for API and UI testing","playwright-utils,fixtures,api,backend,ui",knowledge/overview.md
-api-request,API Request,"Typed HTTP client, schema validation, retry logic for API and service testing","api,backend,service-testing,api-testing,playwright-utils",knowledge/api-request.md
+api-request,API Request,"Typed HTTP client, schema validation, retry logic, operation-based overload for API and service testing","api,backend,service-testing,api-testing,playwright-utils,openapi,codegen,operation",knowledge/api-request.md
 network-recorder,Network Recorder,"HAR record/playback, CRUD detection for offline UI testing","network,playwright-utils,ui,har",knowledge/network-recorder.md
 auth-session,Auth Session,"Token persistence, multi-user, API and browser authentication","auth,playwright-utils,api,backend,jwt,token",knowledge/auth-session.md
 intercept-network-call,Intercept Network Call,"Network spy/stub, JSON parsing for UI tests","network,playwright-utils,ui",knowledge/intercept-network-call.md

package/src/workflows/testarch/test-review/checklist.md

@@ -7,6 +7,7 @@ Use this checklist to validate that the test quality review workflow completed s
 ## Prerequisites
 
 Note: `test-review` is optional and only audits existing tests; it does not generate tests.
+Coverage analysis is out of scope for this workflow. Use `trace` for coverage metrics and coverage gate decisions.
 
 ### Test File Discovery
 
@@ -72,6 +73,8 @@ Note: `test-review` is optional and only audits existing tests; it does not gene
 
 ### Step 3: Quality Criteria Validation
 
+Coverage criteria are intentionally excluded from this checklist.
+
 **For Each Enabled Criterion:**
 
 #### BDD Format (if `check_given_when_then: true`)

package/src/workflows/testarch/test-review/instructions.md

@@ -9,6 +9,8 @@
 
 Review test quality using TEA knowledge base and produce a 0–100 quality score with actionable findings.
 
+Coverage assessment is intentionally out of scope for this workflow. Use `trace` for requirements coverage and coverage gate decisions.
+
 ---
 
 ## WORKFLOW ARCHITECTURE

package/src/workflows/testarch/test-review/steps-c/step-01-load-context.md

@@ -96,6 +96,8 @@ If available:
 
 Summarize what was found.
 
+Coverage mapping and coverage gates are out of scope in `test-review`. Route those concerns to `trace`.
+
 ---
 
 ## 4. Save Progress