qaa-agent 1.0.0 → 1.2.0

package/workflows/qa-gap.md

@@ -0,0 +1,303 @@
+<purpose>
+Compare a developer repository against a QA repository to identify coverage gaps, broken tests, quality issues, and missing test cases. Scans both repositories, performs gap analysis, and produces a comprehensive GAP_ANALYSIS.md with coverage maps, missing test inventories, broken test diagnostics, and quality assessments. Both --dev-repo and --qa-repo paths are required. Use this workflow when a QA repo already exists but may have incomplete coverage or quality issues.
+</purpose>
+
+<required_reading>
+- `CLAUDE.md` -- QA automation standards, pipeline stages, module boundaries, verification commands, quality gates
+- `agents/qaa-scanner.md` -- Scanner agent definition (scans both dev and QA repos)
+- `agents/qaa-analyzer.md` -- Analyzer agent definition (gap mode produces GAP_ANALYSIS.md)
+- `templates/scan-manifest.md` -- SCAN_MANIFEST.md format contract
+- `templates/gap-analysis.md` -- GAP_ANALYSIS.md format contract
+- `templates/qa-analysis.md` -- QA_ANALYSIS.md format contract
+- `templates/test-inventory.md` -- TEST_INVENTORY.md format contract
+</required_reading>
+
+<process>
+
+<step name="parse_arguments">
+## Step 1: Parse Arguments
+
+Parse `$ARGUMENTS` for both repository paths. Both are required for gap analysis.
+
+**Supported arguments:**
+- `--dev-repo <path>` -- Path to the developer repository (required)
+- `--qa-repo <path>` -- Path to the existing QA test repository (required)
+
+**Parsing logic:**
+
+```bash
+DEV_REPO=""
+QA_REPO=""
+
+# Parse --dev-repo and --qa-repo from $ARGUMENTS
+```
+
+**Validation:**
+
+If `--dev-repo` is missing or empty:
+```
+Error: --dev-repo is required for gap analysis.
+Usage: /qa-gap --dev-repo <path> --qa-repo <path>
+```
+STOP.
+
+If `--qa-repo` is missing or empty:
+```
+Error: --qa-repo is required for gap analysis.
+Usage: /qa-gap --dev-repo <path> --qa-repo <path>
+
+To analyze a dev repo without an existing QA repo, use /qa-analyze instead.
+```
+STOP.
+
+If `--dev-repo` path does not exist:
+```
+Error: Dev repo path does not exist: {path}
+```
+STOP.
+
+If `--qa-repo` path does not exist:
+```
+Error: QA repo path does not exist: {path}
+```
+STOP.
+</step>
+
+<step name="setup_output">
+## Step 2: Setup Output Directory
+
+Create the output directory and print the gap analysis banner.
+
+```bash
+OUTPUT_DIR=".qa-output"
+mkdir -p "${OUTPUT_DIR}"
+```
+
+**Print banner:**
+
+```
+=== QA Gap Analysis ===
+Dev Repo: {DEV_REPO}
+QA Repo:  {QA_REPO}
+Output:   {OUTPUT_DIR}
+========================
+```
+</step>
+
+<step name="scan_both_repos">
+## Step 3: Spawn Scanner Agent on Both Repos
+
+Spawn the scanner agent to scan both the developer and QA repositories, producing a combined SCAN_MANIFEST.md.
+
+```
+Task(
+  prompt="
+    <objective>Scan both developer and QA repositories and produce SCAN_MANIFEST.md with combined file trees</objective>
+    <execution_context>@agents/qaa-scanner.md</execution_context>
+    <files_to_read>
+    - CLAUDE.md
+    </files_to_read>
+    <parameters>
+    dev_repo_path: {DEV_REPO}
+    qa_repo_path: {QA_REPO}
+    output_path: {OUTPUT_DIR}/SCAN_MANIFEST.md
+    </parameters>
+  "
+)
+```
+
+**Handle scanner return:**
+
+- If scanner returns `decision: STOP` -- print the stop reason and STOP.
+- If scanner returns `decision: PROCEED` -- continue.
+- Extract `has_frontend` for summary reporting.
+
+**Verify SCAN_MANIFEST.md exists:**
+
+```bash
+[ -f "${OUTPUT_DIR}/SCAN_MANIFEST.md" ] && echo "FOUND" || echo "MISSING"
+```
+
+If missing, print error and STOP.
+</step>
+
+<step name="run_gap_analysis">
+## Step 4: Spawn Analyzer Agent in Gap Mode
+
+Spawn the analyzer agent in gap analysis mode to compare dev repo testable surfaces against QA repo test coverage.
+
+```
+Task(
+  prompt="
+    <objective>Perform gap analysis comparing dev repo testable surfaces against QA repo test coverage. Produce QA_ANALYSIS.md, TEST_INVENTORY.md, and GAP_ANALYSIS.md.</objective>
+    <execution_context>@agents/qaa-analyzer.md</execution_context>
+    <files_to_read>
+    - {OUTPUT_DIR}/SCAN_MANIFEST.md
+    - CLAUDE.md
+    </files_to_read>
+    <parameters>
+    workflow_option: 2
+    qa_analysis_path: {OUTPUT_DIR}/QA_ANALYSIS.md
+    test_inventory_path: {OUTPUT_DIR}/TEST_INVENTORY.md
+    gap_analysis_path: {OUTPUT_DIR}/GAP_ANALYSIS.md
+    dev_repo_path: {DEV_REPO}
+    qa_repo_path: {QA_REPO}
+    </parameters>
+  "
+)
+```
+
+**Handle analyzer return:**
+
+Extract from the analyzer's return values:
+- `total_test_count` -- total test cases in inventory
+- `pyramid_breakdown` -- unit/integration/api/e2e counts
+- `risk_count` -- high/medium/low risk counts
+
+**Verify artifacts exist:**
+
+```bash
+[ -f "${OUTPUT_DIR}/QA_ANALYSIS.md" ] && echo "FOUND: QA_ANALYSIS.md" || echo "MISSING: QA_ANALYSIS.md"
+[ -f "${OUTPUT_DIR}/TEST_INVENTORY.md" ] && echo "FOUND: TEST_INVENTORY.md" || echo "MISSING: TEST_INVENTORY.md"
+[ -f "${OUTPUT_DIR}/GAP_ANALYSIS.md" ] && echo "FOUND: GAP_ANALYSIS.md" || echo "MISSING: GAP_ANALYSIS.md"
+```
+
+If any artifact is missing, print error with the specific missing file and STOP.
+</step>
+
+<step name="extract_gap_metrics">
+## Step 5: Extract Gap Metrics
+
+Read the GAP_ANALYSIS.md to extract metrics for the summary.
+
+**Metrics to extract:**
+
+1. **Coverage map:**
+   - Total modules in dev repo
+   - Modules with test coverage in QA repo
+   - Modules without any test coverage
+   - Coverage percentage: `(covered_modules / total_modules) * 100`
+
+2. **Missing tests:**
+   - Count of missing test cases by priority (P0, P1, P2)
+   - Count of missing test cases by pyramid level (unit, integration, API, E2E)
+   - Each missing test has an ID following naming convention
+
+3. **Broken tests:**
+   - Count of existing tests that are broken (fail, import errors, outdated selectors)
+   - Each broken test has a failure reason documented
+
+4. **Quality assessment:**
+   - Locator tier distribution (what percentage of locators are Tier 1, 2, 3, 4)
+   - Assertion quality rating (concrete vs vague assertions)
+   - POM compliance (does existing QA repo follow POM rules)
+
+5. **Recommendations priority:**
+   - Fix broken tests first (highest priority)
+   - Add P0 missing tests
+   - Add P1 missing tests
+   - Standardize existing tests to CLAUDE.md standards
+   - Add P2 missing tests (lowest priority)
+</step>
+
+<step name="print_summary">
+## Step 6: Print Gap Analysis Summary
+
+Print a comprehensive human-readable summary.
+
+```
+=== Gap Analysis Complete ===
+
+Coverage Overview:
+  Dev Repo Modules:       {total_modules}
+  Modules with Coverage:  {covered_modules}
+  Modules without Tests:  {uncovered_modules}
+  Coverage Rate:          {coverage_percent}%
+
+Missing Tests:
+  P0 (blocks release):   {p0_missing}
+  P1 (should fix):       {p1_missing}
+  P2 (nice to have):     {p2_missing}
+  --------------------------
+  Total Missing:          {total_missing}
+
+  By Pyramid Level:
+    Unit:        {unit_missing}
+    Integration: {integration_missing}
+    API:         {api_missing}
+    E2E:         {e2e_missing}
+
+Broken Tests:
+  Total Broken: {broken_count}
+
+Quality Assessment:
+  Locator Quality:
+    Tier 1 (data-testid, ARIA): {tier1_percent}%
+    Tier 2 (labels, text):      {tier2_percent}%
+    Tier 3 (alt, title):        {tier3_percent}%
+    Tier 4 (CSS, XPath):        {tier4_percent}%
+  Assertion Quality: {concrete_percent}% concrete assertions
+  POM Compliance:    {pom_compliant}/{pom_total} files compliant
+
+Recommended Action Order:
+  1. Fix {broken_count} broken tests
+  2. Add {p0_missing} P0 missing tests
+  3. Add {p1_missing} P1 missing tests
+  4. Standardize existing tests
+  5. Add {p2_missing} P2 missing tests
+
+Artifacts Produced:
+  - {OUTPUT_DIR}/SCAN_MANIFEST.md
+  - {OUTPUT_DIR}/QA_ANALYSIS.md
+  - {OUTPUT_DIR}/TEST_INVENTORY.md
+  - {OUTPUT_DIR}/GAP_ANALYSIS.md
+
+No test files generated. No git operations performed.
+To generate missing tests, run the full pipeline with:
+  /qa-start --dev-repo {DEV_REPO} --qa-repo {QA_REPO}
+=============================
+```
+</step>
+
+</process>
+
+<output>
+This workflow compares dev and QA repos to identify coverage gaps.
+
+**Artifacts produced:**
+
+| Artifact | When Produced | Description |
+|----------|---------------|-------------|
+| SCAN_MANIFEST.md | Always | Combined scan of both repos with file trees and framework detection |
+| QA_ANALYSIS.md | Always | Architecture overview, risks, test targets, testing pyramid for the dev repo |
+| TEST_INVENTORY.md | Always | Complete test case inventory for what SHOULD exist |
+| GAP_ANALYSIS.md | Always | Coverage map, missing tests, broken tests, quality assessment, recommendations |
+
+**GAP_ANALYSIS.md sections (per templates/gap-analysis.md):**
+
+1. **Coverage Map** -- Every module from SCAN_MANIFEST.md with covered/uncovered status
+2. **Missing Tests** -- Test cases that should exist but do not, with IDs and priorities
+3. **Broken Tests** -- Existing tests that fail, with failure reasons and file paths
+4. **Quality Assessment** -- Locator tier distribution, assertion quality rating, POM compliance
+5. **Recommendations** -- Prioritized action list: fix broken first, then add P0, then P1
+
+**No side effects:**
+- No git branches created
+- No git commits made
+- No PRs created
+- No test files generated
+- No source files modified
+</output>
+
+<error_handling>
+| Error | Cause | Action |
+|-------|-------|--------|
+| --dev-repo missing | Required argument not provided | Print usage with suggestion to use /qa-analyze for dev-only, STOP |
+| --qa-repo missing | Required argument not provided | Print usage with suggestion to use /qa-analyze for dev-only, STOP |
+| Dev repo path does not exist | Invalid path | Print error with path, STOP |
+| QA repo path does not exist | Invalid path | Print error with path, STOP |
+| Scanner returns STOP | No testable surfaces | Print scanner's reason, STOP |
+| SCAN_MANIFEST.md missing | Scanner failed | Print error, STOP |
+| GAP_ANALYSIS.md missing | Analyzer failed | Print error, STOP |
+| Framework detection LOW confidence | Ambiguous stack | Scanner checkpoints for user confirmation |
+</error_handling>