ma-agents 2.19.2 → 2.20.1

package/lib/bmad-cache/tea/src/workflows/testarch/bmad-testarch-test-review/steps-c/step-03f-aggregate-scores.md

@@ -0,0 +1,277 @@
+---
+name: 'step-03f-aggregate-scores'
+description: 'Aggregate quality dimension scores into overall 0-100 score'
+nextStepFile: './step-04-generate-report.md'
+outputFile: '{test_artifacts}/test-review.md'
+---
+
+# Step 3F: Aggregate Quality Scores
+
+## STEP GOAL
+
+Read outputs from 4 quality subagents, calculate weighted overall score (0-100), and aggregate violations for report generation.
+
+---
+
+## MANDATORY EXECUTION RULES
+
+- 📖 Read the entire step file before acting
+- ✅ Speak in `{communication_language}`
+- ✅ Read all 4 subagent outputs
+- ✅ Calculate weighted overall score
+- ✅ Aggregate violations by severity
+- ❌ Do NOT re-evaluate quality (use subagent outputs)
+
+---
+
+## EXECUTION PROTOCOLS:
+
+- 🎯 Follow the MANDATORY SEQUENCE exactly
+- 💾 Record outputs before proceeding
+- 📖 Load the next step only when instructed
+
+---
+
+## MANDATORY SEQUENCE
+
+### 1. Read All Subagent Outputs
+
+```javascript
+// Use the SAME timestamp generated in Step 3 (do not regenerate).
+const timestamp = subagentContext?.timestamp;
+if (!timestamp) {
+  throw new Error('Missing timestamp from Step 3 context. Pass Step 3 timestamp into Step 3F.');
+}
+const dimensions = ['determinism', 'isolation', 'maintainability', 'performance'];
+const results = {};
+
+dimensions.forEach((dim) => {
+  const outputPath = `/tmp/tea-test-review-${dim}-${timestamp}.json`;
+  results[dim] = JSON.parse(fs.readFileSync(outputPath, 'utf8'));
+});
+```
+
+**Verify all succeeded:**
+
+```javascript
+const allSucceeded = dimensions.every((dim) => results[dim].score !== undefined);
+if (!allSucceeded) {
+  throw new Error('One or more quality subagents failed!');
+}
+```
+
+---
+
+### 2. Calculate Weighted Overall Score
+
+**Dimension Weights** (based on TEA quality priorities):
+
+```javascript
+const weights = {
+  determinism: 0.3, // 30% - Reliability and flake prevention
+  isolation: 0.3, // 30% - Parallel safety and independence
+  maintainability: 0.25, // 25% - Readability and long-term health
+  performance: 0.15, // 15% - Speed and execution efficiency
+};
+```
+
+**Calculate overall score:**
+
+```javascript
+const overallScore = dimensions.reduce((sum, dim) => {
+  return sum + results[dim].score * weights[dim];
+}, 0);
+
+const roundedScore = Math.round(overallScore);
+```
+
+**Determine grade:**
+
+```javascript
+const getGrade = (score) => {
+  if (score >= 90) return 'A';
+  if (score >= 80) return 'B';
+  if (score >= 70) return 'C';
+  if (score >= 60) return 'D';
+  return 'F';
+};
+
+const overallGrade = getGrade(roundedScore);
+```
+
+---
+
+### 3. Aggregate Violations by Severity
+
+**Collect all violations from all dimensions:**
+
+```javascript
+const allViolations = dimensions.flatMap((dim) =>
+  results[dim].violations.map((v) => ({
+    ...v,
+    dimension: dim,
+  })),
+);
+
+// Group by severity
+const highSeverity = allViolations.filter((v) => v.severity === 'HIGH');
+const mediumSeverity = allViolations.filter((v) => v.severity === 'MEDIUM');
+const lowSeverity = allViolations.filter((v) => v.severity === 'LOW');
+
+const violationSummary = {
+  total: allViolations.length,
+  HIGH: highSeverity.length,
+  MEDIUM: mediumSeverity.length,
+  LOW: lowSeverity.length,
+};
+```
+
+---
+
+### 4. Prioritize Recommendations
+
+**Extract recommendations from all dimensions:**
+
+```javascript
+const allRecommendations = dimensions.flatMap((dim) =>
+  results[dim].recommendations.map((rec) => ({
+    dimension: dim,
+    recommendation: rec,
+    impact: results[dim].score < 70 ? 'HIGH' : 'MEDIUM',
+  })),
+);
+
+// Sort by impact (HIGH first)
+const prioritizedRecommendations = allRecommendations.sort((a, b) => (a.impact === 'HIGH' ? -1 : 1)).slice(0, 10); // Top 10 recommendations
+```
+
+---
+
+### 5. Create Review Summary Object
+
+**Aggregate all results:**
+
+```javascript
+const reviewSummary = {
+  overall_score: roundedScore,
+  overall_grade: overallGrade,
+  quality_assessment: getQualityAssessment(roundedScore),
+
+  dimension_scores: {
+    determinism: results.determinism.score,
+    isolation: results.isolation.score,
+    maintainability: results.maintainability.score,
+    performance: results.performance.score,
+  },
+
+  dimension_grades: {
+    determinism: results.determinism.grade,
+    isolation: results.isolation.grade,
+    maintainability: results.maintainability.grade,
+    performance: results.performance.grade,
+  },
+
+  violations_summary: violationSummary,
+
+  all_violations: allViolations,
+
+  high_severity_violations: highSeverity,
+
+  top_10_recommendations: prioritizedRecommendations,
+
+  subagent_execution: 'PARALLEL (4 quality dimensions)',
+  performance_gain: '~60% faster than sequential',
+};
+
+// Save for Step 4 (report generation)
+fs.writeFileSync(`/tmp/tea-test-review-summary-${timestamp}.json`, JSON.stringify(reviewSummary, null, 2), 'utf8');
+```
+
+---
+
+### 6. Display Summary to User
+
+```
+✅ Quality Evaluation Complete (Parallel Execution)
+
+📊 Overall Quality Score: {roundedScore}/100 (Grade: {overallGrade})
+
+📈 Dimension Scores:
+- Determinism:      {determinism_score}/100 ({determinism_grade})
+- Isolation:        {isolation_score}/100 ({isolation_grade})
+- Maintainability:  {maintainability_score}/100 ({maintainability_grade})
+- Performance:      {performance_score}/100 ({performance_grade})
+
+ℹ️ Coverage is excluded from `test-review` scoring. Use `trace` for coverage analysis and gates.
+
+⚠️ Violations Found:
+- HIGH:   {high_count} violations
+- MEDIUM: {medium_count} violations
+- LOW:    {low_count} violations
+- TOTAL:  {total_count} violations
+
+🚀 Performance: Parallel execution ~60% faster than sequential
+
+✅ Ready for report generation (Step 4)
+```
+
+---
+
+---
+
+### 7. Save Progress
+
+**Save this step's accumulated work to `{outputFile}`.**
+
+- **If `{outputFile}` does not exist** (first save), create it using the workflow template (if available) with YAML frontmatter:
+
+  ```yaml
+  ---
+  stepsCompleted: ['step-03f-aggregate-scores']
+  lastStep: 'step-03f-aggregate-scores'
+  lastSaved: '{date}'
+  ---
+  ```
+
+  Then write this step's output below the frontmatter.
+
+- **If `{outputFile}` already exists**, update:
+  - Add `'step-03f-aggregate-scores'` to `stepsCompleted` array (only if not already present)
+  - Set `lastStep: 'step-03f-aggregate-scores'`
+  - Set `lastSaved: '{date}'`
+  - Append this step's output to the appropriate section of the document.
+
+---
+
+## EXIT CONDITION
+
+Proceed to Step 4 when:
+
+- ✅ All subagent outputs read successfully
+- ✅ Overall score calculated
+- ✅ Violations aggregated
+- ✅ Recommendations prioritized
+- ✅ Summary saved to temp file
+- ✅ Output displayed to user
+- ✅ Progress saved to output document
+
+Load next step: `{nextStepFile}`
+
+---
+
+## 🚨 SYSTEM SUCCESS METRICS
+
+### ✅ SUCCESS:
+
+- All 4 subagent outputs read and parsed
+- Overall score calculated with proper weights
+- Violations aggregated correctly
+- Summary complete and saved
+
+### ❌ FAILURE:
+
+- Failed to read one or more subagent outputs
+- Score calculation incorrect
+- Summary missing or incomplete
+
+**Master Rule:** Aggregate determinism, isolation, maintainability, and performance only.

package/lib/bmad-cache/tea/src/workflows/testarch/bmad-testarch-test-review/steps-c/step-04-generate-report.md

@@ -0,0 +1,111 @@
+---
+name: 'step-04-generate-report'
+description: 'Create test-review report and validate'
+outputFile: '{test_artifacts}/test-review.md'
+---
+
+# Step 4: Generate Report & Validate
+
+## STEP GOAL
+
+Produce the test-review report and validate against checklist.
+
+## MANDATORY EXECUTION RULES
+
+- 📖 Read the entire step file before acting
+- ✅ Speak in `{communication_language}`
+
+---
+
+## EXECUTION PROTOCOLS:
+
+- 🎯 Follow the MANDATORY SEQUENCE exactly
+- 💾 Record outputs before proceeding
+- 📖 Load the next step only when instructed
+
+## CONTEXT BOUNDARIES:
+
+- Available context: config, loaded artifacts, and knowledge fragments
+- Focus: this step's goal only
+- Limits: do not execute future steps
+- Dependencies: prior steps' outputs (if any)
+
+## MANDATORY SEQUENCE
+
+**CRITICAL:** Follow this sequence exactly. Do not skip, reorder, or improvise.
+
+## 1. Report Generation
+
+Use `test-review-template.md` to produce `{outputFile}` including:
+
+- Score summary
+- Critical findings with fixes
+- Warnings and recommendations
+- Context references (story/test-design if available)
+- Coverage boundary note: `test-review` does not score coverage. Direct coverage findings to `trace`.
+
+---
+
+## 2. Polish Output
+
+Before finalizing, review the complete output document for quality:
+
+1. **Remove duplication**: Progressive-append workflow may have created repeated sections — consolidate
+2. **Verify consistency**: Ensure terminology, risk scores, and references are consistent throughout
+3. **Check completeness**: All template sections should be populated or explicitly marked N/A
+4. **Format cleanup**: Ensure markdown formatting is clean (tables aligned, headers consistent, no orphaned references)
+
+---
+
+## 3. Validation
+
+Validate against `checklist.md` and fix any gaps.
+
+- [ ] CLI sessions cleaned up (no orphaned browsers)
+- [ ] Temp artifacts stored in `{test_artifacts}/` not random locations
+
+---
+
+## 4. Save Progress
+
+**Save this step's accumulated work to `{outputFile}`.**
+
+- **If `{outputFile}` does not exist** (first save), create it using the workflow template (if available) with YAML frontmatter:
+
+  ```yaml
+  ---
+  stepsCompleted: ['step-04-generate-report']
+  lastStep: 'step-04-generate-report'
+  lastSaved: '{date}'
+  ---
+  ```
+
+  Then write this step's output below the frontmatter.
+
+- **If `{outputFile}` already exists**, update:
+  - Add `'step-04-generate-report'` to `stepsCompleted` array (only if not already present)
+  - Set `lastStep: 'step-04-generate-report'`
+  - Set `lastSaved: '{date}'`
+  - Append this step's output to the appropriate section of the document.
+
+---
+
+## 5. Completion Summary
+
+Report:
+
+- Scope reviewed
+- Overall score
+- Critical blockers
+- Next recommended workflow (e.g., `automate` or `trace`)
+
+## 🚨 SYSTEM SUCCESS/FAILURE METRICS:
+
+### ✅ SUCCESS:
+
+- Step completed in full with required outputs
+
+### ❌ SYSTEM FAILURE:
+
+- Skipped sequence steps or missing outputs
+  **Master Rule:** Skipping steps is FORBIDDEN.

package/lib/bmad-cache/tea/src/workflows/testarch/bmad-testarch-test-review/steps-e/step-01-assess.md

@@ -0,0 +1,65 @@
+---
+name: 'step-01-assess'
+description: 'Load an existing output for editing'
+nextStepFile: './step-02-apply-edit.md'
+---
+
+# Step 1: Assess Edit Target
+
+## STEP GOAL:
+
+Identify which output should be edited and load it.
+
+## MANDATORY EXECUTION RULES (READ FIRST):
+
+### Universal Rules:
+
+- 📖 Read the complete step file before taking any action
+- ✅ Speak in `{communication_language}`
+
+### Role Reinforcement:
+
+- ✅ You are the Master Test Architect
+
+### Step-Specific Rules:
+
+- 🎯 Ask the user which output file to edit
+- 🚫 Do not edit until target is confirmed
+
+## EXECUTION PROTOCOLS:
+
+- 🎯 Follow the MANDATORY SEQUENCE exactly
+
+## CONTEXT BOUNDARIES:
+
+- Available context: existing outputs
+- Focus: select edit target
+- Limits: no edits yet
+
+## MANDATORY SEQUENCE
+
+**CRITICAL:** Follow this sequence exactly.
+
+### 1. Identify Target
+
+Ask the user to provide the output file path or select from known outputs.
+
+### 2. Load Target
+
+Read the provided output file in full.
+
+### 3. Confirm
+
+Confirm the target and proceed to edit.
+
+Load next step: `{nextStepFile}`
+
+## 🚨 SYSTEM SUCCESS/FAILURE METRICS:
+
+### ✅ SUCCESS:
+
+- Target identified and loaded
+
+### ❌ SYSTEM FAILURE:
+
+- Proceeding without a confirmed target

package/lib/bmad-cache/tea/src/workflows/testarch/bmad-testarch-test-review/steps-e/step-02-apply-edit.md

@@ -0,0 +1,60 @@
+---
+name: 'step-02-apply-edit'
+description: 'Apply edits to the selected output'
+---
+
+# Step 2: Apply Edits
+
+## STEP GOAL:
+
+Apply the requested edits to the selected output and confirm changes.
+
+## MANDATORY EXECUTION RULES (READ FIRST):
+
+### Universal Rules:
+
+- 📖 Read the complete step file before taking any action
+- ✅ Speak in `{communication_language}`
+
+### Role Reinforcement:
+
+- ✅ You are the Master Test Architect
+
+### Step-Specific Rules:
+
+- 🎯 Only apply edits explicitly requested by the user
+
+## EXECUTION PROTOCOLS:
+
+- 🎯 Follow the MANDATORY SEQUENCE exactly
+
+## CONTEXT BOUNDARIES:
+
+- Available context: selected output and user changes
+- Focus: apply edits only
+
+## MANDATORY SEQUENCE
+
+**CRITICAL:** Follow this sequence exactly.
+
+### 1. Confirm Requested Changes
+
+Restate what will be changed and confirm.
+
+### 2. Apply Changes
+
+Update the output file accordingly.
+
+### 3. Report
+
+Summarize the edits applied.
+
+## 🚨 SYSTEM SUCCESS/FAILURE METRICS:
+
+### ✅ SUCCESS:
+
+- Changes applied and confirmed
+
+### ❌ SYSTEM FAILURE:
+
+- Unconfirmed edits or missing update

package/lib/bmad-cache/tea/src/workflows/testarch/bmad-testarch-test-review/steps-v/step-01-validate.md

@@ -0,0 +1,67 @@
+---
+name: 'step-01-validate'
+description: 'Validate workflow outputs against checklist'
+outputFile: '{test_artifacts}/test-review-validation-report.md'
+validationChecklist: '../checklist.md'
+---
+
+# Step 1: Validate Outputs
+
+## STEP GOAL:
+
+Validate outputs using the workflow checklist and record findings.
+
+## MANDATORY EXECUTION RULES (READ FIRST):
+
+### Universal Rules:
+
+- 📖 Read the complete step file before taking any action
+- ✅ Speak in `{communication_language}`
+
+### Role Reinforcement:
+
+- ✅ You are the Master Test Architect
+
+### Step-Specific Rules:
+
+- 🎯 Validate against `{validationChecklist}`
+- 🚫 Do not skip checks
+
+## EXECUTION PROTOCOLS:
+
+- 🎯 Follow the MANDATORY SEQUENCE exactly
+- 💾 Write findings to `{outputFile}`
+
+## CONTEXT BOUNDARIES:
+
+- Available context: workflow outputs and checklist
+- Focus: validation only
+- Limits: do not modify outputs in this step
+
+## MANDATORY SEQUENCE
+
+**CRITICAL:** Follow this sequence exactly.
+
+### 1. Load Checklist
+
+Read `{validationChecklist}` and list all criteria.
+
+### 2. Validate Outputs
+
+Evaluate outputs against each checklist item.
+
+### 3. Write Report
+
+Write a validation report to `{outputFile}` with PASS/WARN/FAIL per section.
+
+## 🚨 SYSTEM SUCCESS/FAILURE METRICS:
+
+### ✅ SUCCESS:
+
+- Validation report written
+- All checklist items evaluated
+
+### ❌ SYSTEM FAILURE:
+
+- Skipped checklist items
+- No report produced