ma-agents 2.19.2 → 2.20.0

package/lib/bmad-cache/tea/src/workflows/testarch/bmad-testarch-test-review/steps-c/step-01-load-context.md

@@ -0,0 +1,197 @@
+---
+name: 'step-01-load-context'
+description: 'Load knowledge base, determine scope, and gather context'
+nextStepFile: './step-02-discover-tests.md'
+knowledgeIndex: '{project-root}/_bmad/tea/testarch/tea-index.csv'
+outputFile: '{test_artifacts}/test-review.md'
+---
+
+# Step 1: Load Context & Knowledge Base
+
+## STEP GOAL
+
+Determine review scope, load required knowledge fragments, and gather related artifacts.
+
+## MANDATORY EXECUTION RULES
+
+- 📖 Read the entire step file before acting
+- ✅ Speak in `{communication_language}`
+
+---
+
+## EXECUTION PROTOCOLS:
+
+- 🎯 Follow the MANDATORY SEQUENCE exactly
+- 💾 Record outputs before proceeding
+- 📖 Load the next step only when instructed
+
+## CONTEXT BOUNDARIES:
+
+- Available context: config, loaded artifacts, and knowledge fragments
+- Focus: this step's goal only
+- Limits: do not execute future steps
+- Dependencies: prior steps' outputs (if any)
+
+## MANDATORY SEQUENCE
+
+**CRITICAL:** Follow this sequence exactly. Do not skip, reorder, or improvise.
+
+## 1. Determine Scope and Stack
+
+Use `review_scope`:
+
+- **single**: one file
+- **directory**: all tests in folder
+- **suite**: all tests in repo
+
+If unclear, ask the user.
+
+**Stack Detection** (for context-aware loading):
+
+Read `test_stack_type` from `{config_source}`. If `"auto"` or not configured, infer `{detected_stack}` by scanning `{project-root}`:
+
+- **Frontend indicators**: `playwright.config.*`, `cypress.config.*`, `package.json` with react/vue/angular
+- **Backend indicators**: `pyproject.toml`, `pom.xml`/`build.gradle`, `go.mod`, `*.csproj`, `Gemfile`, `Cargo.toml`
+- **Both present** → `fullstack`; only frontend → `frontend`; only backend → `backend`
+- Explicit `test_stack_type` overrides auto-detection
+
+---
+
+### Tiered Knowledge Loading
+
+Load fragments based on their `tier` classification in `tea-index.csv`:
+
+1. **Core tier** (always load): Foundational fragments required for this workflow
+2. **Extended tier** (load on-demand): Load when deeper analysis is needed or when the user's context requires it
+3. **Specialized tier** (load only when relevant): Load only when the specific use case matches (e.g., contract-testing only for microservices, email-auth only for email flows)
+
+> **Context Efficiency**: Loading only core fragments reduces context usage by 40-50% compared to loading all fragments.
+
+### Playwright Utils Loading Profiles
+
+**If `tea_use_playwright_utils` is enabled**, select the appropriate loading profile:
+
+- **API-only profile** (when `{detected_stack}` is `backend` or no `page.goto`/`page.locator` found in test files):
+  Load: `overview`, `api-request`, `auth-session`, `recurse` (~1,800 lines)
+
+- **Full UI+API profile** (when `{detected_stack}` is `frontend`/`fullstack` or browser tests detected):
+  Load: all Playwright Utils core fragments (~4,500 lines)
+
+**Detection**: Scan `{test_dir}` for files containing `page.goto` or `page.locator`. If none found, use API-only profile.
+
+### Pact.js Utils Loading
+
+**If `tea_use_pactjs_utils` is enabled** (and contract tests detected in review scope):
+
+Load: `pactjs-utils-overview.md`, `pactjs-utils-provider-verifier.md`, `pactjs-utils-request-filter.md` (the 3 most relevant for reviewing provider verification tests)
+
+**If `tea_use_pactjs_utils` is disabled** but contract tests are in review scope:
+
+Load: `contract-testing.md`
+
+### Pact MCP Loading
+
+**If `tea_pact_mcp` is `"mcp"`:**
+
+Load: `pact-mcp.md` — enables agent to use SmartBear MCP "Review Pact Tests" tool for automated best-practice feedback during test review.
+
+## 2. Load Knowledge Base
+
+From `{knowledgeIndex}` load:
+
+Read `{config_source}` and check `tea_use_playwright_utils`, `tea_use_pactjs_utils`, `tea_pact_mcp`, and `tea_browser_automation` to select the correct fragment set.
+
+**Core:**
+
+- `test-quality.md`
+- `data-factories.md`
+- `test-levels-framework.md`
+- `selective-testing.md`
+- `test-healing-patterns.md`
+- `selector-resilience.md`
+- `timing-debugging.md`
+
+**If Playwright Utils enabled:**
+
+- `overview.md`, `api-request.md`, `network-recorder.md`, `auth-session.md`, `intercept-network-call.md`, `recurse.md`, `log.md`, `file-utils.md`, `burn-in.md`, `network-error-monitor.md`, `fixtures-composition.md`
+
+**If disabled:**
+
+- `fixture-architecture.md`
+- `network-first.md`
+- `playwright-config.md`
+- `component-tdd.md`
+- `ci-burn-in.md`
+
+**Playwright CLI (if `tea_browser_automation` is "cli" or "auto"):**
+
+- `playwright-cli.md`
+
+**MCP Patterns (if `tea_browser_automation` is "mcp" or "auto"):**
+
+- (existing MCP-related fragments, if any are added in future)
+
+**Pact.js Utils (if enabled and contract tests in review scope):**
+
+- `pactjs-utils-overview.md`, `pactjs-utils-provider-verifier.md`, `pactjs-utils-request-filter.md`
+
+**Contract Testing (if pactjs-utils disabled but contract tests in review scope):**
+
+- `contract-testing.md`
+
+**Pact MCP (if tea_pact_mcp is "mcp"):**
+
+- `pact-mcp.md`
+
+---
+
+## 3. Gather Context Artifacts
+
+If available:
+
+- Story file (acceptance criteria)
+- Test design doc (priorities)
+- Framework config
+
+Summarize what was found.
+
+Coverage mapping and coverage gates are out of scope in `test-review`. Route those concerns to `trace`.
+
+---
+
+## 4. Save Progress
+
+**Save this step's accumulated work to `{outputFile}`.**
+
+- **If `{outputFile}` does not exist** (first save), create it using the workflow template (if available) with YAML frontmatter:
+
+  ```yaml
+  ---
+  stepsCompleted: ['step-01-load-context']
+  lastStep: 'step-01-load-context'
+  lastSaved: '{date}'
+  ---
+  ```
+
+  Then write this step's output below the frontmatter.
+
+- **If `{outputFile}` already exists**, update:
+  - Add `'step-01-load-context'` to `stepsCompleted` array (only if not already present)
+  - Set `lastStep: 'step-01-load-context'`
+  - Set `lastSaved: '{date}'`
+  - Append this step's output to the appropriate section of the document.
+
+**Update `inputDocuments`**: Set `inputDocuments` in the output template frontmatter to the list of artifact paths loaded in this step (e.g., knowledge fragments, test design documents, configuration files).
+
+Load next step: `{nextStepFile}`
+
+## 🚨 SYSTEM SUCCESS/FAILURE METRICS:
+
+### ✅ SUCCESS:
+
+- Step completed in full with required outputs
+
+### ❌ SYSTEM FAILURE:
+
+- Skipped sequence steps or missing outputs
+  **Master Rule:** Skipping steps is FORBIDDEN.

package/lib/bmad-cache/tea/src/workflows/testarch/bmad-testarch-test-review/steps-c/step-01b-resume.md

@@ -0,0 +1,104 @@
+---
+name: 'step-01b-resume'
+description: 'Resume interrupted workflow from last completed step'
+outputFile: '{test_artifacts}/test-review.md'
+---
+
+# Step 1b: Resume Workflow
+
+## STEP GOAL
+
+Resume an interrupted workflow by loading the existing output document, displaying progress, and routing to the next incomplete step.
+
+## MANDATORY EXECUTION RULES
+
+- Read the entire step file before acting
+- Speak in `{communication_language}`
+
+---
+
+## EXECUTION PROTOCOLS:
+
+- Follow the MANDATORY SEQUENCE exactly
+- Load the next step only when instructed
+
+## CONTEXT BOUNDARIES:
+
+- Available context: Output document with progress frontmatter
+- Focus: Load progress and route to next step
+- Limits: Do not re-execute completed steps
+- Dependencies: Output document must exist from a previous run
+
+## MANDATORY SEQUENCE
+
+**CRITICAL:** Follow this sequence exactly.
+
+### 1. Load Output Document
+
+Read `{outputFile}` and parse YAML frontmatter for:
+
+- `stepsCompleted` -- array of completed step names
+- `lastStep` -- last completed step name
+- `lastSaved` -- timestamp of last save
+
+**If `{outputFile}` does not exist**, display:
+
+"No previous progress found. There is no output document to resume from. Please use **[C] Create** to start a fresh workflow run."
+
+**THEN:** Halt. Do not proceed.
+
+---
+
+### 2. Display Progress Dashboard
+
+Display progress with checkmark/empty indicators:
+
+```
+Test Quality Review - Resume Progress:
+
+1. Load Context (step-01-load-context)              [completed/pending]
+2. Discover Tests (step-02-discover-tests)           [completed/pending]
+3. Quality Evaluation + Aggregate (step-03f-aggregate-scores) [completed/pending]
+4. Generate Report (step-04-generate-report)         [completed/pending]
+
+Last saved: {lastSaved}
+```
+
+---
+
+### 3. Route to Next Step
+
+Based on `lastStep`, load the next incomplete step:
+
+| lastStep                    | Next Step File                    |
+| --------------------------- | --------------------------------- |
+| `step-01-load-context`      | `./step-02-discover-tests.md`     |
+| `step-02-discover-tests`    | `./step-03-quality-evaluation.md` |
+| `step-03f-aggregate-scores` | `./step-04-generate-report.md`    |
+| `step-04-generate-report`   | **Workflow already complete.**    |
+
+**If `lastStep` is the final step** (`step-04-generate-report`), display: "All steps completed. Use **[C] Create** to start fresh, **[V] Validate** to review outputs, or **[E] Edit** to make revisions." Then halt.
+
+**If `lastStep` does not match any value above**, display: "Unknown progress state (`lastStep`: {lastStep}). Please use **[C] Create** to start fresh." Then halt.
+
+**Otherwise**, load the identified step file, read completely, and execute.
+
+The existing content in `{outputFile}` provides context from previously completed steps.
+
+---
+
+## SYSTEM SUCCESS/FAILURE METRICS
+
+### SUCCESS:
+
+- Output document loaded and parsed correctly
+- Progress dashboard displayed accurately
+- Routed to correct next step
+
+### FAILURE:
+
+- Not loading output document
+- Incorrect progress display
+- Routing to wrong step
+
+**Master Rule:** Resume MUST route to the exact next incomplete step. Never re-execute completed steps.

package/lib/bmad-cache/tea/src/workflows/testarch/bmad-testarch-test-review/steps-c/step-02-discover-tests.md

@@ -0,0 +1,113 @@
+---
+name: 'step-02-discover-tests'
+description: 'Find and parse test files'
+nextStepFile: './step-03-quality-evaluation.md'
+outputFile: '{test_artifacts}/test-review.md'
+---
+
+# Step 2: Discover & Parse Tests
+
+## STEP GOAL
+
+Collect test files in scope and parse structure/metadata.
+
+## MANDATORY EXECUTION RULES
+
+- 📖 Read the entire step file before acting
+- ✅ Speak in `{communication_language}`
+
+---
+
+## EXECUTION PROTOCOLS:
+
+- 🎯 Follow the MANDATORY SEQUENCE exactly
+- 💾 Record outputs before proceeding
+- 📖 Load the next step only when instructed
+
+## CONTEXT BOUNDARIES:
+
+- Available context: config, loaded artifacts, and knowledge fragments
+- Focus: this step's goal only
+- Limits: do not execute future steps
+- Dependencies: prior steps' outputs (if any)
+
+## MANDATORY SEQUENCE
+
+**CRITICAL:** Follow this sequence exactly. Do not skip, reorder, or improvise.
+
+## 1. Discover Test Files
+
+- **single**: use provided file path
+- **directory**: glob under `{test_dir}` or selected folder
+- **suite**: glob all tests in repo
+
+Halt if no tests are found.
+
+---
+
+## 2. Parse Metadata (per file)
+
+Collect:
+
+- File size and line count
+- Test framework detected
+- Describe/test block counts
+- Test IDs and priority markers
+- Imports, fixtures, factories, network interception
+- Waits/timeouts and control flow (if/try/catch)
+
+---
+
+## 3. Evidence Collection (if `tea_browser_automation` is `cli` or `auto`)
+
+> **Fallback:** If CLI is not installed, fall back to MCP (if available) or skip evidence collection.
+
+**CLI Evidence Collection:**
+All commands use the same named session to target the correct browser:
+
+1. `playwright-cli -s=tea-review open <target_url>`
+2. `playwright-cli -s=tea-review tracing-start`
+3. Execute the flow under review (using `-s=tea-review` on each command)
+4. `playwright-cli -s=tea-review tracing-stop` → saves trace.zip
+5. `playwright-cli -s=tea-review screenshot --filename={test_artifacts}/review-evidence.png`
+6. `playwright-cli -s=tea-review network` → capture network request log
+7. `playwright-cli -s=tea-review close`
+
+> **Session Hygiene:** Always close sessions using `playwright-cli -s=tea-review close`. Do NOT use `close-all` — it kills every session on the machine and breaks parallel execution.
+
+---
+
+## 4. Save Progress
+
+**Save this step's accumulated work to `{outputFile}`.**
+
+- **If `{outputFile}` does not exist** (first save), create it using the workflow template (if available) with YAML frontmatter:
+
+  ```yaml
+  ---
+  stepsCompleted: ['step-02-discover-tests']
+  lastStep: 'step-02-discover-tests'
+  lastSaved: '{date}'
+  ---
+  ```
+
+  Then write this step's output below the frontmatter.
+
+- **If `{outputFile}` already exists**, update:
+  - Add `'step-02-discover-tests'` to `stepsCompleted` array (only if not already present)
+  - Set `lastStep: 'step-02-discover-tests'`
+  - Set `lastSaved: '{date}'`
+  - Append this step's output to the appropriate section of the document.
+
+Load next step: `{nextStepFile}`
+
+## 🚨 SYSTEM SUCCESS/FAILURE METRICS:
+
+### ✅ SUCCESS:
+
+- Step completed in full with required outputs
+
+### ❌ SYSTEM FAILURE:
+
+- Skipped sequence steps or missing outputs
+  **Master Rule:** Skipping steps is FORBIDDEN.

package/lib/bmad-cache/tea/src/workflows/testarch/bmad-testarch-test-review/steps-c/step-03-quality-evaluation.md

@@ -0,0 +1,274 @@
+---
+name: 'step-03-quality-evaluation'
+description: 'Orchestrate adaptive quality dimension checks (agent-team, subagent, or sequential)'
+nextStepFile: './step-03f-aggregate-scores.md'
+---
+
+# Step 3: Orchestrate Adaptive Quality Evaluation
+
+## STEP GOAL
+
+Select execution mode deterministically, then evaluate quality dimensions using agent-team, subagent, or sequential execution while preserving output contracts:
+
+- Determinism
+- Isolation
+- Maintainability
+- Performance
+
+Coverage is intentionally excluded from this workflow and handled by `trace`.
+
+## MANDATORY EXECUTION RULES
+
+- 📖 Read the entire step file before acting
+- ✅ Speak in `{communication_language}`
+- ✅ Resolve execution mode from config (`tea_execution_mode`, `tea_capability_probe`)
+- ✅ Apply fallback rules deterministically when requested mode is unsupported
+- ✅ Wait for required worker steps to complete
+- ❌ Do NOT skip capability checks when probing is enabled
+- ❌ Do NOT proceed until required worker steps finish
+
+---
+
+## EXECUTION PROTOCOLS:
+
+- 🎯 Follow the MANDATORY SEQUENCE exactly
+- 💾 Wait for subagent outputs
+- 📖 Load the next step only when instructed
+
+## CONTEXT BOUNDARIES:
+
+- Available context: test files from Step 2, knowledge fragments
+- Focus: orchestration only (mode selection + worker dispatch)
+- Limits: do not evaluate quality directly (delegate to worker steps)
+
+---
+
+## MANDATORY SEQUENCE
+
+### 1. Prepare Execution Context
+
+**Generate unique timestamp:**
+
+```javascript
+const timestamp = new Date().toISOString().replace(/[:.]/g, '-');
+```
+
+**Prepare context for all subagents:**
+
+```javascript
+const parseBooleanFlag = (value, defaultValue = true) => {
+  if (typeof value === 'string') {
+    const normalized = value.trim().toLowerCase();
+    if (['false', '0', 'off', 'no'].includes(normalized)) return false;
+    if (['true', '1', 'on', 'yes'].includes(normalized)) return true;
+  }
+  if (value === undefined || value === null) return defaultValue;
+  return Boolean(value);
+};
+
+const subagentContext = {
+  test_files: /* from Step 2 */,
+  knowledge_fragments_loaded: ['test-quality'],
+  config: {
+    execution_mode: config.tea_execution_mode || 'auto',  // "auto" | "subagent" | "agent-team" | "sequential"
+    capability_probe: parseBooleanFlag(config.tea_capability_probe, true),  // supports booleans and "false"/"true" strings
+  },
+  timestamp: timestamp
+};
+```
+
+---
+
+### 2. Resolve Execution Mode with Capability Probe
+
+```javascript
+const normalizeUserExecutionMode = (mode) => {
+  if (typeof mode !== 'string') return null;
+  const normalized = mode.trim().toLowerCase().replace(/[-_]/g, ' ').replace(/\s+/g, ' ');
+
+  if (normalized === 'auto') return 'auto';
+  if (normalized === 'sequential') return 'sequential';
+  if (normalized === 'subagent' || normalized === 'sub agent' || normalized === 'subagents' || normalized === 'sub agents') {
+    return 'subagent';
+  }
+  if (normalized === 'agent team' || normalized === 'agent teams' || normalized === 'agentteam') {
+    return 'agent-team';
+  }
+
+  return null;
+};
+
+const normalizeConfigExecutionMode = (mode) => {
+  if (mode === 'subagent') return 'subagent';
+  if (mode === 'auto' || mode === 'sequential' || mode === 'subagent' || mode === 'agent-team') {
+    return mode;
+  }
+  return null;
+};
+
+// Explicit user instruction in the active run takes priority over config.
+const explicitModeFromUser = normalizeUserExecutionMode(runtime.getExplicitExecutionModeHint?.() || null);
+
+const requestedMode = explicitModeFromUser || normalizeConfigExecutionMode(subagentContext.config.execution_mode) || 'auto';
+const probeEnabled = subagentContext.config.capability_probe;
+
+const supports = {
+  subagent: false,
+  agentTeam: false,
+};
+
+if (probeEnabled) {
+  supports.subagent = runtime.canLaunchSubagents?.() === true;
+  supports.agentTeam = runtime.canLaunchAgentTeams?.() === true;
+}
+
+let resolvedMode = requestedMode;
+
+if (requestedMode === 'auto') {
+  if (supports.agentTeam) resolvedMode = 'agent-team';
+  else if (supports.subagent) resolvedMode = 'subagent';
+  else resolvedMode = 'sequential';
+} else if (probeEnabled && requestedMode === 'agent-team' && !supports.agentTeam) {
+  resolvedMode = supports.subagent ? 'subagent' : 'sequential';
+} else if (probeEnabled && requestedMode === 'subagent' && !supports.subagent) {
+  resolvedMode = 'sequential';
+}
+
+subagentContext.execution = {
+  requestedMode,
+  resolvedMode,
+  probeEnabled,
+  supports,
+};
+```
+
+Resolution precedence:
+
+1. Explicit user request in this run (`agent team` => `agent-team`; `subagent` => `subagent`; `sequential`; `auto`)
+2. `tea_execution_mode` from config
+3. Runtime capability fallback (when probing enabled)
+
+If probing is disabled, honor the requested mode strictly. If that mode cannot be executed at runtime, fail with explicit error instead of silent fallback.
+
+---
+
+### 3. Dispatch 4 Quality Workers
+
+**Subagent A: Determinism**
+
+- File: `./step-03a-subagent-determinism.md`
+- Output: `/tmp/tea-test-review-determinism-${timestamp}.json`
+- Execution:
+  - `agent-team` or `subagent`: launch non-blocking
+  - `sequential`: run blocking and wait
+- Status: Running... ⟳
+
+**Subagent B: Isolation**
+
+- File: `./step-03b-subagent-isolation.md`
+- Output: `/tmp/tea-test-review-isolation-${timestamp}.json`
+- Status: Running... ⟳
+
+**Subagent C: Maintainability**
+
+- File: `./step-03c-subagent-maintainability.md`
+- Output: `/tmp/tea-test-review-maintainability-${timestamp}.json`
+- Status: Running... ⟳
+
+**Subagent D: Performance**
+
+- File: `./step-03e-subagent-performance.md`
+- Output: `/tmp/tea-test-review-performance-${timestamp}.json`
+- Status: Running... ⟳
+
+In `agent-team` and `subagent` modes, runtime decides worker scheduling and concurrency.
+
+---
+
+### 4. Wait for Expected Worker Completion
+
+**If `resolvedMode` is `agent-team` or `subagent`:**
+
+```
+⏳ Waiting for 4 quality subagents to complete...
+✅ All 4 quality subagents completed successfully!
+```
+
+**If `resolvedMode` is `sequential`:**
+
+```
+✅ Sequential mode: each worker already completed during dispatch.
+```
+
+---
+
+### 5. Verify All Outputs Exist
+
+```javascript
+const outputs = ['determinism', 'isolation', 'maintainability', 'performance'].map(
+  (dim) => `/tmp/tea-test-review-${dim}-${timestamp}.json`,
+);
+
+outputs.forEach((output) => {
+  if (!fs.existsSync(output)) {
+    throw new Error(`Subagent output missing: ${output}`);
+  }
+});
+```
+
+---
+
+### 6. Execution Report
+
+```
+🚀 Performance Report:
+- Execution Mode: {resolvedMode}
+- Total Elapsed: ~mode-dependent
+- Parallel Gain: ~60-70% faster when mode is subagent/agent-team
+```
+
+---
+
+### 7. Proceed to Aggregation
+
+Pass the same `timestamp` value to Step 3F (do not regenerate it). Step 3F must read the exact temp files written in this step.
+
+Load next step: `{nextStepFile}`
+
+The aggregation step (3F) will:
+
+- Read all 4 subagent outputs
+- Calculate weighted overall score (0-100)
+- Aggregate violations by severity
+- Generate review report with top suggestions
+
+---
+
+## EXIT CONDITION
+
+Proceed to Step 3F when:
+
+- ✅ All 4 subagents completed successfully
+- ✅ All output files exist and are valid JSON
+- ✅ Execution metrics displayed
+
+**Do NOT proceed if any subagent failed.**
+
+---
+
+## 🚨 SYSTEM SUCCESS METRICS
+
+### ✅ SUCCESS:
+
+- All 4 subagents launched and completed
+- All required worker steps completed
+- Output files generated and valid
+- Fallback behavior respected configuration and capability probe rules
+
+### ❌ FAILURE:
+
+- One or more subagents failed
+- Output files missing or invalid
+- Unsupported requested mode with probing disabled
+
+**Master Rule:** Deterministic mode selection + stable output contract. Use the best supported mode, then aggregate normally.