ma-agents 2.19.2 → 2.20.0

package/lib/bmad-cache/tea/src/workflows/testarch/bmad-testarch-test-review/steps-c/step-03a-subagent-determinism.md

@@ -0,0 +1,214 @@
+---
+name: 'step-03a-subagent-determinism'
+description: 'Subagent: Check test determinism (no random/time dependencies)'
+subagent: true
+outputFile: '/tmp/tea-test-review-determinism-{{timestamp}}.json'
+---
+
+# Subagent 3A: Determinism Quality Check
+
+## SUBAGENT CONTEXT
+
+This is an **isolated subagent** running in parallel with other quality dimension checks.
+
+**What you have from parent workflow:**
+
+- Test files discovered in Step 2
+- Knowledge fragment: test-quality (determinism criteria)
+- Config: test framework
+
+**Your task:** Analyze test files for DETERMINISM violations only.
+
+---
+
+## MANDATORY EXECUTION RULES
+
+- 📖 Read this entire subagent file before acting
+- ✅ Check DETERMINISM only (not other quality dimensions)
+- ✅ Output structured JSON to temp file
+- ❌ Do NOT check isolation, maintainability, coverage, or performance (other subagents)
+- ❌ Do NOT modify test files (read-only analysis)
+- ❌ Do NOT run tests (just analyze code)
+
+---
+
+## SUBAGENT TASK
+
+### 1. Identify Determinism Violations
+
+**Scan test files for non-deterministic patterns:**
+
+**HIGH SEVERITY Violations**:
+
+- `Math.random()` - Random number generation
+- `Date.now()` or `new Date()` without mocking
+- `setTimeout` / `setInterval` without proper waits
+- External API calls without mocking
+- File system operations on random paths
+- Database queries with non-deterministic ordering
+
+**MEDIUM SEVERITY Violations**:
+
+- `page.waitForTimeout(N)` - Hard waits instead of conditions
+- Flaky selectors (CSS classes that may change)
+- Race conditions (missing proper synchronization)
+- Test order dependencies (test A must run before test B)
+
+**LOW SEVERITY Violations**:
+
+- Missing test isolation (shared state between tests)
+- Console timestamps without fixed timezone
+
+### 2. Analyze Each Test File
+
+For each test file from Step 2:
+
+```javascript
+const violations = [];
+
+// Check for Math.random()
+if (testFileContent.includes('Math.random()')) {
+  violations.push({
+    file: testFile,
+    line: findLineNumber('Math.random()'),
+    severity: 'HIGH',
+    category: 'random-generation',
+    description: 'Test uses Math.random() - non-deterministic',
+    suggestion: 'Use faker.seed(12345) for deterministic random data',
+  });
+}
+
+// Check for Date.now()
+if (testFileContent.includes('Date.now()') || testFileContent.includes('new Date()')) {
+  violations.push({
+    file: testFile,
+    line: findLineNumber('Date.now()'),
+    severity: 'HIGH',
+    category: 'time-dependency',
+    description: 'Test uses Date.now() or new Date() without mocking',
+    suggestion: 'Mock system time with test.useFakeTimers() or use fixed timestamps',
+  });
+}
+
+// Check for hard waits
+if (testFileContent.includes('waitForTimeout')) {
+  violations.push({
+    file: testFile,
+    line: findLineNumber('waitForTimeout'),
+    severity: 'MEDIUM',
+    category: 'hard-wait',
+    description: 'Test uses waitForTimeout - creates flakiness',
+    suggestion: 'Replace with expect(locator).toBeVisible() or waitForResponse',
+  });
+}
+
+// ... check other patterns
+```
+
+### 3. Calculate Determinism Score
+
+**Scoring Logic**:
+
+```javascript
+const totalChecks = testFiles.length * checksPerFile;
+const failedChecks = violations.length;
+const passedChecks = totalChecks - failedChecks;
+
+// Weight violations by severity
+const severityWeights = { HIGH: 10, MEDIUM: 5, LOW: 2 };
+const totalPenalty = violations.reduce((sum, v) => sum + severityWeights[v.severity], 0);
+
+// Score: 100 - (penalty points)
+const score = Math.max(0, 100 - totalPenalty);
+```
+
+---
+
+## OUTPUT FORMAT
+
+Write JSON to temp file: `/tmp/tea-test-review-determinism-{{timestamp}}.json`
+
+```json
+{
+  "dimension": "determinism",
+  "score": 85,
+  "max_score": 100,
+  "grade": "B",
+  "violations": [
+    {
+      "file": "tests/api/user.spec.ts",
+      "line": 42,
+      "severity": "HIGH",
+      "category": "random-generation",
+      "description": "Test uses Math.random() - non-deterministic",
+      "suggestion": "Use faker.seed(12345) for deterministic random data",
+      "code_snippet": "const userId = Math.random() * 1000;"
+    },
+    {
+      "file": "tests/e2e/checkout.spec.ts",
+      "line": 78,
+      "severity": "MEDIUM",
+      "category": "hard-wait",
+      "description": "Test uses waitForTimeout - creates flakiness",
+      "suggestion": "Replace with expect(locator).toBeVisible()",
+      "code_snippet": "await page.waitForTimeout(5000);"
+    }
+  ],
+  "passed_checks": 12,
+  "failed_checks": 3,
+  "total_checks": 15,
+  "violation_summary": {
+    "HIGH": 1,
+    "MEDIUM": 1,
+    "LOW": 1
+  },
+  "recommendations": [
+    "Use faker with fixed seed for all random data",
+    "Replace all waitForTimeout with conditional waits",
+    "Mock Date.now() in tests that use current time"
+  ],
+  "summary": "Tests are mostly deterministic with 3 violations (1 HIGH, 1 MEDIUM, 1 LOW)"
+}
+```
+
+**On Error:**
+
+```json
+{
+  "dimension": "determinism",
+  "success": false,
+  "error": "Error message describing what went wrong"
+}
+```
+
+---
+
+## EXIT CONDITION
+
+Subagent completes when:
+
+- ✅ All test files analyzed for determinism violations
+- ✅ Score calculated (0-100)
+- ✅ Violations categorized by severity
+- ✅ Recommendations generated
+- ✅ JSON output written to temp file
+
+**Subagent terminates here.** Parent workflow will read output and aggregate with other quality dimensions.
+
+---
+
+## 🚨 SUBAGENT SUCCESS METRICS
+
+### ✅ SUCCESS:
+
+- All test files scanned for determinism violations
+- Score calculated with proper severity weighting
+- JSON output valid and complete
+- Only determinism checked (not other dimensions)
+
+### ❌ FAILURE:
+
+- Checked quality dimensions other than determinism
+- Invalid or missing JSON output
+- Score calculation incorrect
+- Modified test files (should be read-only)

package/lib/bmad-cache/tea/src/workflows/testarch/bmad-testarch-test-review/steps-c/step-03b-subagent-isolation.md

@@ -0,0 +1,125 @@
+---
+name: 'step-03b-subagent-isolation'
+description: 'Subagent: Check test isolation (no shared state/dependencies)'
+subagent: true
+outputFile: '/tmp/tea-test-review-isolation-{{timestamp}}.json'
+---
+
+# Subagent 3B: Isolation Quality Check
+
+## SUBAGENT CONTEXT
+
+This is an **isolated subagent** running in parallel with other quality dimension checks.
+
+**Your task:** Analyze test files for ISOLATION violations only.
+
+---
+
+## MANDATORY EXECUTION RULES
+
+- ✅ Check ISOLATION only (not other quality dimensions)
+- ✅ Output structured JSON to temp file
+- ❌ Do NOT check determinism, maintainability, coverage, or performance
+- ❌ Do NOT modify test files (read-only analysis)
+
+---
+
+## SUBAGENT TASK
+
+### 1. Identify Isolation Violations
+
+**Scan test files for isolation issues:**
+
+**HIGH SEVERITY Violations**:
+
+- Global state mutations (global variables modified)
+- Test order dependencies (test B depends on test A running first)
+- Shared database records without cleanup
+- beforeAll/afterAll with side effects leaking to other tests
+
+**MEDIUM SEVERITY Violations**:
+
+- Missing test cleanup (created data not deleted)
+- Shared fixtures that mutate state
+- Tests that assume specific execution order
+- Environment variables modified without restoration
+
+**LOW SEVERITY Violations**:
+
+- Tests sharing test data (but not mutating)
+- Missing test.describe grouping
+- Tests that could be more isolated
+
+### 2. Calculate Isolation Score
+
+```javascript
+const totalChecks = testFiles.length * checksPerFile;
+const failedChecks = violations.length;
+const severityWeights = { HIGH: 10, MEDIUM: 5, LOW: 2 };
+const totalPenalty = violations.reduce((sum, v) => sum + severityWeights[v.severity], 0);
+const score = Math.max(0, 100 - totalPenalty);
+```
+
+---
+
+## OUTPUT FORMAT
+
+```json
+{
+  "dimension": "isolation",
+  "score": 90,
+  "max_score": 100,
+  "grade": "A-",
+  "violations": [
+    {
+      "file": "tests/api/integration.spec.ts",
+      "line": 15,
+      "severity": "HIGH",
+      "category": "test-order-dependency",
+      "description": "Test depends on previous test creating user record",
+      "suggestion": "Each test should create its own test data in beforeEach",
+      "code_snippet": "test('should update user', async () => { /* assumes user exists */ });"
+    }
+  ],
+  "passed_checks": 14,
+  "failed_checks": 1,
+  "total_checks": 15,
+  "violation_summary": {
+    "HIGH": 1,
+    "MEDIUM": 0,
+    "LOW": 0
+  },
+  "recommendations": [
+    "Add beforeEach hooks to create test data",
+    "Add afterEach hooks to cleanup created records",
+    "Use test.describe.configure({ mode: 'parallel' }) to enforce isolation"
+  ],
+  "summary": "Tests are well isolated with 1 HIGH severity violation"
+}
+```
+
+---
+
+## EXIT CONDITION
+
+Subagent completes when:
+
+- ✅ All test files analyzed for isolation violations
+- ✅ Score calculated
+- ✅ JSON output written to temp file
+
+**Subagent terminates here.**
+
+---
+
+## 🚨 SUBAGENT SUCCESS METRICS
+
+### ✅ SUCCESS:
+
+- Only isolation checked (not other dimensions)
+- JSON output valid and complete
+
+### ❌ FAILURE:
+
+- Checked quality dimensions other than isolation
+- Invalid or missing JSON output

package/lib/bmad-cache/tea/src/workflows/testarch/bmad-testarch-test-review/steps-c/step-03c-subagent-maintainability.md

@@ -0,0 +1,102 @@
+---
+name: 'step-03c-subagent-maintainability'
+description: 'Subagent: Check test maintainability (readability, structure, DRY)'
+subagent: true
+outputFile: '/tmp/tea-test-review-maintainability-{{timestamp}}.json'
+---
+
+# Subagent 3C: Maintainability Quality Check
+
+## SUBAGENT CONTEXT
+
+This is an **isolated subagent** running in parallel with other quality dimension checks.
+
+**Your task:** Analyze test files for MAINTAINABILITY violations only.
+
+---
+
+## MANDATORY EXECUTION RULES
+
+- ✅ Check MAINTAINABILITY only (not other quality dimensions)
+- ✅ Output structured JSON to temp file
+- ❌ Do NOT check determinism, isolation, coverage, or performance
+
+---
+
+## SUBAGENT TASK
+
+### 1. Identify Maintainability Violations
+
+**HIGH SEVERITY Violations**:
+
+- Tests >100 lines (too complex)
+- No test.describe grouping
+- Duplicate test logic (copy-paste)
+- Unclear test names (no Given/When/Then structure)
+- Magic numbers/strings without constants
+
+**MEDIUM SEVERITY Violations**:
+
+- Tests missing comments for complex logic
+- Inconsistent naming conventions
+- Excessive nesting (>3 levels)
+- Large setup/teardown blocks
+
+**LOW SEVERITY Violations**:
+
+- Minor code style issues
+- Could benefit from helper functions
+- Inconsistent assertion styles
+
+### 2. Calculate Maintainability Score
+
+```javascript
+const severityWeights = { HIGH: 10, MEDIUM: 5, LOW: 2 };
+const totalPenalty = violations.reduce((sum, v) => sum + severityWeights[v.severity], 0);
+const score = Math.max(0, 100 - totalPenalty);
+```
+
+---
+
+## OUTPUT FORMAT
+
+```json
+{
+  "dimension": "maintainability",
+  "score": 75,
+  "max_score": 100,
+  "grade": "C",
+  "violations": [
+    {
+      "file": "tests/e2e/complex-flow.spec.ts",
+      "line": 1,
+      "severity": "HIGH",
+      "category": "test-too-long",
+      "description": "Test file is 250 lines - too complex to maintain",
+      "suggestion": "Split into multiple smaller test files by feature area",
+      "code_snippet": "test.describe('Complex flow', () => { /* 250 lines */ });"
+    }
+  ],
+  "passed_checks": 10,
+  "failed_checks": 5,
+  "violation_summary": {
+    "HIGH": 2,
+    "MEDIUM": 2,
+    "LOW": 1
+  },
+  "recommendations": [
+    "Split large test files into smaller, focused files (<100 lines each)",
+    "Add test.describe grouping for related tests",
+    "Extract duplicate logic into helper functions"
+  ],
+  "summary": "Tests have maintainability issues - 5 violations (2 HIGH)"
+}
+```
+
+---
+
+## EXIT CONDITION
+
+Subagent completes when JSON output written to temp file.
+
+**Subagent terminates here.**

package/lib/bmad-cache/tea/src/workflows/testarch/bmad-testarch-test-review/steps-c/step-03e-subagent-performance.md

@@ -0,0 +1,117 @@
+---
+name: 'step-03e-subagent-performance'
+description: 'Subagent: Check test performance (speed, efficiency, parallelization)'
+subagent: true
+outputFile: '/tmp/tea-test-review-performance-{{timestamp}}.json'
+---
+
+# Subagent 3E: Performance Quality Check
+
+## SUBAGENT CONTEXT
+
+This is an **isolated subagent** running in parallel with other quality dimension checks.
+
+**Your task:** Analyze test files for PERFORMANCE violations only.
+
+---
+
+## MANDATORY EXECUTION RULES
+
+- ✅ Check PERFORMANCE only (not other quality dimensions)
+- ✅ Output structured JSON to temp file
+- ❌ Do NOT check determinism, isolation, maintainability, or coverage
+
+---
+
+## SUBAGENT TASK
+
+### 1. Identify Performance Violations
+
+**HIGH SEVERITY Violations**:
+
+- Tests not parallelizable (using test.describe.serial unnecessarily)
+- Slow setup/teardown (creating fresh DB for every test)
+- Excessive navigation (reloading pages unnecessarily)
+- No fixture reuse (repeating expensive operations)
+
+**MEDIUM SEVERITY Violations**:
+
+- Hard waits >2 seconds (waitForTimeout(5000))
+- Inefficient selectors (page.$$ instead of locators)
+- Large data sets in tests without pagination
+- Missing performance optimizations
+
+**LOW SEVERITY Violations**:
+
+- Could use parallelization (test.describe.configure({ mode: 'parallel' }))
+- Minor inefficiencies
+- Excessive logging
+
+### 2. Calculate Performance Score
+
+```javascript
+const severityWeights = { HIGH: 10, MEDIUM: 5, LOW: 2 };
+const totalPenalty = violations.reduce((sum, v) => sum + severityWeights[v.severity], 0);
+const score = Math.max(0, 100 - totalPenalty);
+```
+
+---
+
+## OUTPUT FORMAT
+
+```json
+{
+  "dimension": "performance",
+  "score": 80,
+  "max_score": 100,
+  "grade": "B",
+  "violations": [
+    {
+      "file": "tests/e2e/search.spec.ts",
+      "line": 10,
+      "severity": "HIGH",
+      "category": "not-parallelizable",
+      "description": "Tests use test.describe.serial unnecessarily - reduces parallel execution",
+      "suggestion": "Remove .serial unless tests truly share state",
+      "code_snippet": "test.describe.serial('Search tests', () => { ... });"
+    },
+    {
+      "file": "tests/api/bulk-operations.spec.ts",
+      "line": 35,
+      "severity": "MEDIUM",
+      "category": "slow-setup",
+      "description": "Test creates 1000 records in setup - very slow",
+      "suggestion": "Use smaller data sets or fixture factories",
+      "code_snippet": "beforeEach(async () => { for (let i=0; i<1000; i++) { ... } });"
+    }
+  ],
+  "passed_checks": 13,
+  "failed_checks": 2,
+  "violation_summary": {
+    "HIGH": 1,
+    "MEDIUM": 1,
+    "LOW": 0
+  },
+  "performance_metrics": {
+    "parallelizable_tests": 80,
+    "serial_tests": 20,
+    "avg_test_duration_estimate": "~2 seconds",
+    "slow_tests": ["bulk-operations.spec.ts (>30s)"]
+  },
+  "recommendations": [
+    "Enable parallel mode where possible",
+    "Reduce setup data to minimum needed",
+    "Use fixtures to share expensive setup across tests",
+    "Remove unnecessary .serial constraints"
+  ],
+  "summary": "Good performance with 2 violations - 80% tests can run in parallel"
+}
+```
+
+---
+
+## EXIT CONDITION
+
+Subagent completes when JSON output written to temp file.
+
+**Subagent terminates here.**