ma-agents 3.6.2 → 3.8.0

package/lib/bmad-cache/tea/src/workflows/testarch/bmad-testarch-trace/steps-c/step-01-load-context.md

@@ -1,16 +1,16 @@
 ---
 name: 'step-01-load-context'
-description: 'Load requirements, knowledge base, and related artifacts'
-nextStepFile: './step-02-discover-tests.md'
+description: 'Resolve coverage oracle, load knowledge base, and gather related artifacts'
+nextStepFile: '{skill-root}/steps-c/step-02-discover-tests.md'
 knowledgeIndex: './resources/tea-index.csv'
-outputFile: '{test_artifacts}/traceability-report.md'
+outputFile: '{test_artifacts}/traceability-matrix.md'
 ---
 
-# Step 1: Load Context & Knowledge Base
+# Step 1: Resolve Coverage Oracle & Load Knowledge Base
 
 ## STEP GOAL
 
-Gather acceptance criteria, priorities, and supporting artifacts for traceability.
+Resolve the best available coverage oracle, capture confidence and provenance, and gather supporting artifacts for traceability.
 
 ## MANDATORY EXECUTION RULES
 
@@ -27,7 +27,7 @@ Gather acceptance criteria, priorities, and supporting artifacts for traceabilit
 
 ## CONTEXT BOUNDARIES:
 
-- Available context: config, loaded artifacts, and knowledge fragments
+- Available context: config, source tree, loaded artifacts, and knowledge fragments
 - Focus: this step's goal only
 - Limits: do not execute future steps
 - Dependencies: prior steps' outputs (if any)
@@ -36,12 +36,60 @@ Gather acceptance criteria, priorities, and supporting artifacts for traceabilit
 
 **CRITICAL:** Follow this sequence exactly. Do not skip, reorder, or improvise.
 
-## 1. Prerequisites
-
-- Acceptance criteria available (story or provided inline)
-- Tests exist OR gaps explicitly acknowledged
-
-If acceptance criteria are missing, **HALT** and request them.
+## 1. Resolve Coverage Oracle
+
+At least one of the following must be usable:
+
+- Formal requirements (story/epic acceptance criteria, PRD, test design)
+- Contract/spec artifacts (OpenAPI, GraphQL schema, protobuf, etc.)
+- External pointers to a requirements source that can be resolved through installed adapters/MCPs
+- Analyzable source code that supports synthetic journey/requirement inference
+
+Tests exist OR gaps are explicitly acknowledged.
+
+Resolve the oracle in this order:
+
+1. **Formal requirements first**
+   - Story/epic acceptance criteria
+   - PRD / test design / tech spec
+   - Inline requirements provided by the user
+
+2. **Contract/spec artifacts second**
+   - OpenAPI / Swagger
+   - GraphQL schema or SDL
+   - Other machine-readable contract definitions
+
+3. **External pointers third**
+   - Placeholder files that point to external trackers or docs such as Jira, Linear, Confluence, shared docs, or other systems of record
+   - Follow the pointer automatically only when a compatible adapter/plugin/MCP is available in the active runtime
+   - Record `externalPointerStatus` as one of: `not_used`, `resolved`, `skipped`, or `unavailable`
+
+4. **Synthetic oracle last**
+   - If no formal oracle exists and `allow_synthetic_oracle` is enabled, inspect `{source_dir}` to infer a provisional trace target
+   - For UI apps, infer journeys from:
+     - routes/pages/screens/layout entry points
+     - navigation flows and feature entry links
+     - forms, submit actions, create/update/delete paths
+     - auth/session/logout/role-gated flows
+     - loading, empty, validation, error, and permission-denied states
+     - feature flags and major conditional branches
+   - Deduplicate the inferred items into a compact, traceable list (prefer 5-12 items)
+   - Assign stable IDs such as `J-01`, `J-02`, etc.
+   - Assign provisional priorities using `test-priorities-matrix.md`
+     - `P0`: auth, checkout/payment, destructive data changes, revenue-critical, hard blockers to core use
+     - `P1`: primary user journeys and common CRUD paths
+     - `P2`: secondary workflows and edge scenarios
+     - `P3`: low-risk polish or optional flows
+
+Record the resolved oracle metadata in step output/frontmatter using consistent keys:
+
+- `coverageBasis` (`acceptance_criteria` | `synthetic_requirements` | `openapi_endpoints` | `user_journeys`) — the type of oracle selected for coverage tracing
+- `oracleResolutionMode` (`formal_requirements` | `spec_artifact` | `external_pointer` | `synthetic_source`) — how the oracle was discovered/resolved
+- `oracleConfidence` (`high` | `medium` | `low`) — confidence in the resolved oracle as a coverage source
+- `oracleSources` — list of artifact paths, URIs, or references used to resolve the oracle
+- `externalPointerStatus` (`not_used` | `resolved` | `skipped` | `unavailable`) — status of external pointer resolution when pointer files are present
+
+If none of the four oracle types can be resolved, **HALT** and request the smallest missing clarification needed to continue.
 
 ---
 
@@ -64,8 +112,11 @@ If available:
 - Story file and acceptance criteria
 - Test design doc (priorities)
 - Tech spec / PRD
+- OpenAPI or similar contract/spec files
+- Placeholder files that reference external requirements systems
+- Route maps, page/screen registries, and other source files used for synthetic journey inference
 
-Summarize what was found.
+Summarize what was found and explicitly state the resolved oracle, its confidence, and why that oracle was selected.
 
 ---
 
@@ -80,6 +131,11 @@ Summarize what was found.
   stepsCompleted: ['step-01-load-context']
   lastStep: 'step-01-load-context'
   lastSaved: '{date}'
+  coverageBasis: '{resolved coverage_basis}'
+  oracleConfidence: '{resolved oracle_confidence}'
+  oracleResolutionMode: '{resolved oracle_resolution_mode}'
+  oracleSources: ['{resolved oracle source 1}', '{resolved oracle source 2}']
+  externalPointerStatus: '{resolved external_pointer_status}'
   ---
   ```
 
@@ -89,6 +145,11 @@ Summarize what was found.
   - Add `'step-01-load-context'` to `stepsCompleted` array (only if not already present)
   - Set `lastStep: 'step-01-load-context'`
   - Set `lastSaved: '{date}'`
+  - Set `coverageBasis` to the resolved oracle basis
+  - Set `oracleConfidence` to the resolved oracle confidence
+  - Set `oracleResolutionMode` to the resolved oracle resolution mode
+  - Set `oracleSources` to the resolved oracle sources
+  - Set `externalPointerStatus` to the resolved external pointer status
   - Append this step's output to the appropriate section of the document.
 
 Load next step: `{nextStepFile}`

package/lib/bmad-cache/tea/src/workflows/testarch/bmad-testarch-trace/steps-c/step-01b-resume.md

@@ -1,7 +1,7 @@
 ---
 name: 'step-01b-resume'
 description: 'Resume interrupted workflow from last completed step'
-outputFile: '{test_artifacts}/traceability-report.md'
+outputFile: '{test_artifacts}/traceability-matrix.md'
 ---
 
 # Step 1b: Resume Workflow

package/lib/bmad-cache/tea/src/workflows/testarch/bmad-testarch-trace/steps-c/step-02-discover-tests.md

@@ -1,15 +1,15 @@
 ---
 name: 'step-02-discover-tests'
 description: 'Discover and catalog tests by level'
-nextStepFile: './step-03-map-criteria.md'
-outputFile: '{test_artifacts}/traceability-report.md'
+nextStepFile: '{skill-root}/steps-c/step-03-map-criteria.md'
+outputFile: '{test_artifacts}/traceability-matrix.md'
 ---
 
 # Step 2: Discover & Catalog Tests
 
 ## STEP GOAL
 
-Identify tests relevant to the requirements and classify by test level.
+Identify tests relevant to the resolved coverage oracle and classify by test level.
 
 ## MANDATORY EXECUTION RULES
 
@@ -41,8 +41,17 @@ Search `{test_dir}` for:
 
 - Test IDs (e.g., `1.3-E2E-001`)
 - Feature name matches
+- Resolved oracle item IDs/titles
 - Spec patterns (`*.spec.*`, `*.test.*`)
 
+When the oracle is synthetic (`synthetic_requirements` or `user_journeys`), also search for:
+
+- route/path matches
+- page/screen/component names
+- visible UI labels and CTA names
+- form action verbs (create, edit, save, delete, submit, search, checkout, etc.)
+- auth/session/logout flows
+
 ---
 
 ## 2. Categorize by Level
@@ -54,7 +63,11 @@ Classify as:
 - Component
 - Unit
 
-Record test IDs, describe blocks, and priority markers if present.
+Record test IDs, describe blocks, priority markers, and the per-test identity fields needed for machine-readable output:
+
+- Stable identity fields: `id`, `title`, `file`, `line`, `level`
+- Execution state flags: `skipped`, `pending`, `fixme`
+- Skip or blocker reason when it can be discovered from the test source or runtime metadata
 
 ---
 
@@ -72,6 +85,13 @@ Capture explicit coverage signals so Phase 1 can detect common blind spots:
   - Detect validation, timeout, network-failure, and server-error scenarios
   - Mark criteria with happy-path-only tests
 
+- UI journey coverage (when tracing UI/source-derived oracle items)
+  - Inventory routes/screens/journeys referenced by the oracle and journeys exercised by E2E/component tests
+  - Mark journeys with no end-to-end coverage
+- UI state coverage
+  - Detect loading, empty, validation, error, and permission-denied state assertions
+  - Mark journeys that only verify happy-path rendering
+
 Record these findings in step output as `coverage_heuristics` for Step 3/4.
 
 ---

package/lib/bmad-cache/tea/src/workflows/testarch/bmad-testarch-trace/steps-c/step-03-map-criteria.md

@@ -1,15 +1,15 @@
 ---
 name: 'step-03-map-criteria'
-description: 'Map acceptance criteria to tests and build traceability matrix'
-nextStepFile: './step-04-analyze-gaps.md'
-outputFile: '{test_artifacts}/traceability-report.md'
+description: 'Map coverage oracle items to tests and build traceability matrix'
+nextStepFile: '{skill-root}/steps-c/step-04-analyze-gaps.md'
+outputFile: '{test_artifacts}/traceability-matrix.md'
 ---
 
-# Step 3: Map Criteria to Tests
+# Step 3: Map Coverage Oracle to Tests
 
 ## STEP GOAL
 
-Create the traceability matrix linking requirements to tests.
+Create the traceability matrix linking the resolved oracle items to tests.
 
 ## MANDATORY EXECUTION RULES
 
@@ -37,15 +37,18 @@ Create the traceability matrix linking requirements to tests.
 
 ## 1. Build Matrix
 
-For each acceptance criterion:
+For each resolved oracle item (formal requirement, endpoint/spec item, or synthetic journey):
 
 - Map to matching tests
 - Mark coverage status: FULL / PARTIAL / NONE / UNIT-ONLY / INTEGRATION-ONLY
 - Record test level and priority
+- Preserve each mapped test's stable identity fields (`id`, `title`, `file`, `line`, `level`, status flags) so Phase 1 can deduplicate unique tests before JSON export
 - Record heuristic signals:
-  - Endpoint coverage present/missing (for API-impacting criteria)
+  - Endpoint coverage present/missing (for API-impacting items)
   - Auth/authz coverage present/missing (positive and negative paths)
   - Error-path coverage present/missing (validation, timeout, network/server failures)
+  - UI journey E2E coverage present/missing (for source-derived journeys)
+  - UI state coverage present/missing (loading, empty, validation, error, permission-denied)
 
 ---
 
@@ -53,11 +56,12 @@ For each acceptance criterion:
 
 Ensure:
 
-- P0/P1 criteria have coverage
+- P0/P1 items have coverage
 - No duplicate coverage across levels without justification
-- Criteria are not happy-path-only when requirements imply error handling
-- API criteria are not marked FULL if endpoint-level checks are missing
-- Auth/authz criteria include at least one denied/invalid-path test where applicable
+- Items are not happy-path-only when the oracle implies error handling or alternate states
+- API items are not marked FULL if endpoint-level checks are missing
+- Auth/authz items include at least one denied/invalid-path test where applicable
+- Synthetic UI journeys are not marked FULL when no E2E or component test asserts the critical path and key failure states
 
 ---
 

package/lib/bmad-cache/tea/src/workflows/testarch/bmad-testarch-trace/steps-c/step-04-analyze-gaps.md

@@ -1,8 +1,8 @@
 ---
 name: 'step-04-analyze-gaps'
 description: 'Complete Phase 1 with adaptive orchestration (agent-team, subagent, or sequential)'
-nextStepFile: './step-05-gate-decision.md'
-outputFile: '{test_artifacts}/traceability-report.md'
+nextStepFile: '{skill-root}/steps-c/step-05-gate-decision.md'
+outputFile: '{test_artifacts}/traceability-matrix.md'
 tempOutputFile: '/tmp/tea-trace-coverage-matrix-{{timestamp}}.json'
 ---
 
@@ -33,7 +33,7 @@ tempOutputFile: '/tmp/tea-trace-coverage-matrix-{{timestamp}}.json'
 
 ## CONTEXT BOUNDARIES:
 
-- Available context: requirements from Step 1, tests from Step 2, traceability matrix from Step 3
+- Available context: resolved oracle items from Step 1, tests from Step 2, traceability matrix from Step 3
 - Focus: gap analysis and matrix completion
 - Limits: do not make gate decision (Phase 2 responsibility)
 
@@ -145,11 +145,15 @@ Use the heuristics inventory from Step 2 and mapped criteria from Step 3 to flag
 const endpointCoverageGaps = coverageHeuristics?.endpoints_without_tests || [];
 const authCoverageGaps = coverageHeuristics?.auth_missing_negative_paths || [];
 const errorPathGaps = coverageHeuristics?.criteria_happy_path_only || [];
+const uiJourneyGaps = coverageHeuristics?.ui_journeys_without_e2e || [];
+const uiStateGaps = coverageHeuristics?.ui_states_missing_coverage || [];
 
 const heuristicGapCounts = {
   endpoints_without_tests: endpointCoverageGaps.length,
   auth_missing_negative_paths: authCoverageGaps.length,
   happy_path_only_criteria: errorPathGaps.length,
+  ui_journeys_without_e2e: uiJourneyGaps.length,
+  ui_states_missing_coverage: uiStateGaps.length,
 };
 ```
 
@@ -162,6 +166,48 @@ Heuristics are advisory but must influence gap severity and recommendations, esp
 **Based on gap analysis:**
 
 ```javascript
+const progressDoc = fs.existsSync('{outputFile}') ? fs.readFileSync('{outputFile}', 'utf8') : '';
+const progressFrontmatterMatch = progressDoc.match(/^---\n([\s\S]*?)\n---/);
+const progressFrontmatter = progressFrontmatterMatch ? yaml.parse(progressFrontmatterMatch[1]) : {};
+
+const isUnresolved = (value) => typeof value === 'string' && value.startsWith('{') && value.endsWith('}');
+const normalizeResolvedToken = (value) => {
+  if (value === undefined || value === null) return null;
+  const normalized = String(value).trim().toLowerCase();
+  if (!normalized || normalized === 'auto' || isUnresolved(normalized)) return null;
+  return normalized;
+};
+const firstResolvedToken = (...values) => {
+  for (const value of values) {
+    const normalized = normalizeResolvedToken(value);
+    if (normalized) return normalized;
+  }
+  return null;
+};
+
+const oracleResolutionMode =
+  firstResolvedToken(runtime.getOracleResolutionMode?.(), progressFrontmatter.oracleResolutionMode) || 'formal_requirements';
+const resolvedCoverageBasis =
+  firstResolvedToken(runtime.getResolvedCoverageBasis?.(), progressFrontmatter.coverageBasis) ||
+  {
+    formal_requirements: 'acceptance_criteria',
+    spec_artifact: 'openapi_endpoints',
+    external_pointer: 'acceptance_criteria',
+    synthetic_source: 'user_journeys',
+  }[oracleResolutionMode] ||
+  'acceptance_criteria';
+const resolvedOracleConfidence =
+  firstResolvedToken(runtime.getResolvedOracleConfidence?.(), progressFrontmatter.oracleConfidence) ||
+  {
+    formal_requirements: 'high',
+    spec_artifact: 'high',
+    external_pointer: 'medium',
+    synthetic_source: 'medium',
+  }[oracleResolutionMode] ||
+  'medium';
+const oracleSources = runtime.getOracleSources?.() || progressFrontmatter.oracleSources || [];
+const externalPointerStatus =
+  firstResolvedToken(runtime.getExternalPointerStatus?.(), progressFrontmatter.externalPointerStatus) || 'not_used';
 const recommendations = [];
 
 // Critical gaps (P0)
@@ -215,12 +261,36 @@ if (errorPathGaps.length > 0) {
   });
 }
 
+if (uiJourneyGaps.length > 0) {
+  recommendations.push({
+    priority: 'HIGH',
+    action: `Add E2E or component coverage for ${uiJourneyGaps.length} inferred UI journey(s)`,
+    requirements: uiJourneyGaps.map((r) => r.id || r.route || r.journey || 'unknown'),
+  });
+}
+
+if (uiStateGaps.length > 0) {
+  recommendations.push({
+    priority: 'MEDIUM',
+    action: `Add loading/empty/error/permission state coverage for ${uiStateGaps.length} UI journey(s)`,
+    requirements: uiStateGaps.map((r) => r.id || r.route || r.journey || 'unknown'),
+  });
+}
+
 // Quality issues
 recommendations.push({
   priority: 'LOW',
   action: 'Run /bmad:tea:test-review to assess test quality',
   requirements: [],
 });
+
+if (oracleResolutionMode === 'synthetic_source') {
+  recommendations.push({
+    priority: 'MEDIUM',
+    action: 'Promote inferred journeys into formal acceptance criteria when the team confirms they reflect intended behavior',
+    requirements: traceabilityMatrix.map((r) => r.id),
+  });
+}
 ```
 
 ---
@@ -253,6 +323,117 @@ const p3CoveragePercentage = safePct(p3Covered, p3Total);
 
 ---
 
+### 4b. Build Deduplicated Test Inventory and Trace Metadata
+
+Persist the unique discovered tests in Phase 1 so Step 5 does not need to reconstruct counts from per-requirement mappings.
+
+```javascript
+const coverageEligibleStatuses = new Set(['FULL', 'PARTIAL', 'UNIT-ONLY', 'INTEGRATION-ONLY']);
+const byLevel = {
+  e2e: { tests: 0, criteria_covered: 0 },
+  api: { tests: 0, criteria_covered: 0 },
+  component: { tests: 0, criteria_covered: 0 },
+  unit: { tests: 0, criteria_covered: 0 },
+  other: { tests: 0, criteria_covered: 0 }, // captures tests with unrecognized or empty level
+};
+
+const normalizeTestStatus = (test) => {
+  const explicitStatus = String(test.status || '')
+    .trim()
+    .toLowerCase();
+  if (['skipped', 'pending', 'fixme'].includes(explicitStatus)) return explicitStatus;
+  if (test.fixme === true) return 'fixme';
+  if (test.pending === true) return 'pending';
+  if (test.skipped === true) return 'skipped';
+  return 'active';
+};
+
+const uniqueTests = new Map();
+(traceabilityMatrix || []).forEach((req) => {
+  (req.tests || []).forEach((test, index) => {
+    // Do NOT use the per-requirement `index` as a fallback — the same test can appear
+    // at different indices across requirements, producing spurious duplicate entries.
+    // Use only stable, test-intrinsic fields; omit line when unavailable.
+    const stableId =
+      test.id ||
+      [test.file, test.title || test.name, test.line].filter((value) => value !== undefined && value !== null && value !== '').join(':') ||
+      null; // unresolvable — skip rather than manufacture a key
+
+    if (stableId === null || uniqueTests.has(stableId)) return;
+    const status = normalizeTestStatus(test);
+    uniqueTests.set(stableId, {
+      id: stableId,
+      file: test.file || '',
+      line: test.line ?? null,
+      title: test.title || test.name || stableId,
+      level: String(test.level || '')
+        .trim()
+        .toLowerCase(),
+      status: status,
+      skipped: status === 'skipped',
+      fixme: status === 'fixme',
+      pending: status === 'pending',
+      blocker_reason: test.skip_reason || test.blocker_reason || test.fixme_reason || test.pending_reason || '',
+    });
+  });
+});
+
+[...uniqueTests.values()].forEach((test) => {
+  const bucket = byLevel[test.level] ? test.level : 'other';
+  if (bucket === 'other' && test.level) {
+    console.warn(`[trace] unknown test level "${test.level}" for test "${test.id}" — counted in "other"`);
+  }
+  byLevel[bucket].tests += 1;
+});
+
+(traceabilityMatrix || []).forEach((req) => {
+  if (!coverageEligibleStatuses.has(req.coverage)) return;
+  const requirementLevels = new Set(
+    (req.tests || []).map((test) => {
+      const level = String(test.level || '')
+        .trim()
+        .toLowerCase();
+      return byLevel[level] ? level : 'other';
+    }),
+  );
+  requirementLevels.forEach((level) => {
+    byLevel[level].criteria_covered += 1;
+  });
+});
+
+const deduplicatedTests = [...uniqueTests.values()];
+const deduplicatedTestInventory = {
+  summary: {
+    files: [...new Set(deduplicatedTests.map((test) => test.file).filter(Boolean))].length,
+    cases: deduplicatedTests.length,
+    skipped_cases: deduplicatedTests.filter((test) => test.skipped).length,
+    fixme_cases: deduplicatedTests.filter((test) => test.fixme).length,
+    pending_cases: deduplicatedTests.filter((test) => test.pending).length,
+    by_level: byLevel,
+  },
+  tests: deduplicatedTests,
+  blockers: deduplicatedTests
+    .filter((test) => ['skipped', 'pending', 'fixme'].includes(test.status))
+    .map((test) => ({
+      id: test.id,
+      severity: test.status === 'skipped' ? 'high' : 'medium',
+      reason: test.blocker_reason || `Test marked ${test.status} during trace collection`,
+      test_file: test.file,
+      test_title: test.title,
+    })),
+};
+
+const extractedTargetId = runtime.getTraceTargetId?.() || null;
+const extractedTargetLabel = runtime.getTraceTargetLabel?.() || null;
+const traceTarget = {
+  type: '{gate_type}',
+  id: extractedTargetId, // story_id / epic_num / release_version / hotfix identifier from Step 1
+  label: extractedTargetLabel || null,
+};
+```
+
+---
+
 ### 5. Generate Complete Coverage Matrix
 
 **Compile all Phase 1 outputs:**
@@ -261,6 +442,18 @@ const p3CoveragePercentage = safePct(p3Covered, p3Total);
 const coverageMatrix = {
   phase: 'PHASE_1_COMPLETE',
   generated_at: new Date().toISOString(),
+  trace_target: traceTarget,
+  collection_mode: '{collection_mode}',
+  allow_gate: '{allow_gate}',
+  coverage_basis: resolvedCoverageBasis,
+  summary_confidence: resolvedOracleConfidence,
+  oracle: {
+    resolution_mode: oracleResolutionMode,
+    confidence: resolvedOracleConfidence,
+    sources: oracleSources,
+    external_pointer_status: externalPointerStatus,
+    synthetic: oracleResolutionMode === 'synthetic_source',
+  },
 
   requirements: traceabilityMatrix, // Full matrix from Step 3
 
@@ -292,9 +485,13 @@ const coverageMatrix = {
     endpoint_gaps: endpointCoverageGaps,
     auth_negative_path_gaps: authCoverageGaps,
     happy_path_only_gaps: errorPathGaps,
+    ui_journey_gaps: uiJourneyGaps,
+    ui_state_gaps: uiStateGaps,
     counts: heuristicGapCounts,
   },
 
+  test_inventory: deduplicatedTestInventory,
+  blockers: deduplicatedTestInventory.blockers,
   recommendations: recommendations,
 };
 ```
@@ -312,6 +509,16 @@ fs.writeFileSync(outputPath, JSON.stringify(coverageMatrix, null, 2), 'utf8');
 console.log(`✅ Phase 1 Complete: Coverage matrix saved to ${outputPath}`);
 ```
 
+**Record the resolved path in the progress document** so Step 5 can read the exact same file rather than re-evaluating the timestamp expression:
+
+After writing the temp file, update the YAML frontmatter in `{outputFile}` to include:
+
+```yaml
+tempCoverageMatrixPath: '<resolved outputPath>'
+```
+
+Step 5 reads `tempCoverageMatrixPath` from the frontmatter first; falls back to reconstructing `{tempOutputFile}` only when the key is absent.
+
 ---
 
 ### 7. Display Phase 1 Summary