qaa-agent 1.6.2 → 1.7.0

package/workflows/qa-analyze.md

@@ -1,296 +1,296 @@
-<purpose>
-Analysis-only workflow. Scans a developer repository (and optionally a QA repository) to produce comprehensive analysis artifacts -- SCAN_MANIFEST.md, QA_ANALYSIS.md, TEST_INVENTORY.md, and optionally QA_REPO_BLUEPRINT.md or GAP_ANALYSIS.md. No test generation, no git operations, no PR. Use this workflow to understand a codebase's testability before committing to full test generation.
-</purpose>
-
-<required_reading>
-- `CLAUDE.md` -- QA automation standards, pipeline stages, module boundaries, verification commands
-- `agents/qaa-scanner.md` -- Scanner agent definition (produces SCAN_MANIFEST.md)
-- `agents/qaa-analyzer.md` -- Analyzer agent definition (produces QA_ANALYSIS.md, TEST_INVENTORY.md, QA_REPO_BLUEPRINT.md or GAP_ANALYSIS.md)
-- `templates/scan-manifest.md` -- SCAN_MANIFEST.md format contract
-- `templates/qa-analysis.md` -- QA_ANALYSIS.md format contract
-- `templates/test-inventory.md` -- TEST_INVENTORY.md format contract
-- `templates/qa-repo-blueprint.md` -- QA_REPO_BLUEPRINT.md format contract (Option 1 only)
-- `templates/gap-analysis.md` -- GAP_ANALYSIS.md format contract (Option 2/3 only)
-</required_reading>
-
-<process>
-
-<step name="parse_arguments">
-## Step 1: Parse Arguments
-
-Parse `$ARGUMENTS` for repository paths.
-
-**Supported arguments:**
-- `--dev-repo <path>` -- Path to the developer repository to analyze
-- `--qa-repo <path>` -- Path to an existing QA repository (optional)
-- No arguments -- defaults to the current working directory as the dev repo
-
-**Parsing logic:**
-
-```bash
-DEV_REPO=""
-QA_REPO=""
-
-# Parse --dev-repo and --qa-repo from $ARGUMENTS
-# If --dev-repo is provided, use it
-# If --qa-repo is provided, use it
-# If neither is provided, DEV_REPO defaults to current working directory
-```
-
-**Validation:**
-- If `--dev-repo` is provided, verify the path exists and is a directory. If not, print error: `"Error: Dev repo path does not exist: {path}"` and STOP.
-- If `--qa-repo` is provided, verify the path exists and is a directory. If not, print error: `"Error: QA repo path does not exist: {path}"` and STOP.
-- If no arguments provided, set `DEV_REPO` to the current working directory.
-</step>
-
-<step name="determine_mode">
-## Step 2: Determine Analysis Mode
-
-Set the workflow option based on whether a QA repo was provided.
-
-**Mode selection:**
-
-| QA Repo Provided | Mode | Description |
-|-------------------|------|-------------|
-| No | `full` (Option 1) | Dev-only analysis. Produces QA_ANALYSIS.md + TEST_INVENTORY.md + QA_REPO_BLUEPRINT.md |
-| Yes | `gap` (Option 2/3) | Gap analysis. Produces QA_ANALYSIS.md + TEST_INVENTORY.md + GAP_ANALYSIS.md |
-
-```
-MODE="full"
-if QA_REPO is not empty:
-  MODE="gap"
-```
-
-**Set output directory:**
-
-```bash
-OUTPUT_DIR=".qa-output"
-mkdir -p "${OUTPUT_DIR}"
-```
-
-**Print analysis banner:**
-
-```
-=== QA Analysis Workflow ===
-Mode: {MODE} ({description})
-Dev Repo: {DEV_REPO}
-QA Repo: {QA_REPO or 'N/A'}
-Output: {OUTPUT_DIR}
-============================
-```
-</step>
-
-<step name="run_scanner">
-## Step 3: Spawn Scanner Agent
-
-Spawn the scanner agent to produce SCAN_MANIFEST.md.
-
-**For full mode (Option 1):**
-
-```
-Task(
-  prompt="
-    <objective>Scan repository and produce SCAN_MANIFEST.md</objective>
-    <execution_context>@agents/qaa-scanner.md</execution_context>
-    <files_to_read>
-    - CLAUDE.md
-    </files_to_read>
-    <parameters>
-    dev_repo_path: {DEV_REPO}
-    qa_repo_path: null
-    output_path: {OUTPUT_DIR}/SCAN_MANIFEST.md
-    </parameters>
-  "
-)
-```
-
-**For gap mode (Option 2/3):**
-
-```
-Task(
-  prompt="
-    <objective>Scan both developer and QA repositories and produce SCAN_MANIFEST.md</objective>
-    <execution_context>@agents/qaa-scanner.md</execution_context>
-    <files_to_read>
-    - CLAUDE.md
-    </files_to_read>
-    <parameters>
-    dev_repo_path: {DEV_REPO}
-    qa_repo_path: {QA_REPO}
-    output_path: {OUTPUT_DIR}/SCAN_MANIFEST.md
-    </parameters>
-  "
-)
-```
-
-**Handle scanner return:**
-
-- If scanner returns `decision: STOP` -- print the stop reason and STOP the workflow. The repository has no testable surfaces or an uncertain framework.
-- If scanner returns `decision: PROCEED` -- continue to next step.
-- Extract `has_frontend` and `detection_confidence` from scanner return for use in the summary.
-
-**Verify SCAN_MANIFEST.md exists:**
-
-```bash
-[ -f "${OUTPUT_DIR}/SCAN_MANIFEST.md" ] && echo "FOUND" || echo "MISSING"
-```
-
-If missing, print error: `"Error: Scanner did not produce SCAN_MANIFEST.md. Check scanner output for details."` and STOP.
-</step>
-
-<step name="run_analyzer">
-## Step 4: Spawn Analyzer Agent
-
-Spawn the analyzer agent to produce analysis artifacts.
-
-**For full mode (Option 1):**
-
-```
-Task(
-  prompt="
-    <objective>Analyze scanned repository and produce QA_ANALYSIS.md, TEST_INVENTORY.md, and QA_REPO_BLUEPRINT.md</objective>
-    <execution_context>@agents/qaa-analyzer.md</execution_context>
-    <files_to_read>
-    - {OUTPUT_DIR}/SCAN_MANIFEST.md
-    - CLAUDE.md
-    </files_to_read>
-    <parameters>
-    workflow_option: 1
-    qa_analysis_path: {OUTPUT_DIR}/QA_ANALYSIS.md
-    test_inventory_path: {OUTPUT_DIR}/TEST_INVENTORY.md
-    blueprint_path: {OUTPUT_DIR}/QA_REPO_BLUEPRINT.md
-    </parameters>
-  "
-)
-```
-
-**For gap mode (Option 2/3):**
-
-```
-Task(
-  prompt="
-    <objective>Analyze scanned repositories and produce QA_ANALYSIS.md, TEST_INVENTORY.md, and GAP_ANALYSIS.md</objective>
-    <execution_context>@agents/qaa-analyzer.md</execution_context>
-    <files_to_read>
-    - {OUTPUT_DIR}/SCAN_MANIFEST.md
-    - CLAUDE.md
-    </files_to_read>
-    <parameters>
-    workflow_option: 2
-    qa_analysis_path: {OUTPUT_DIR}/QA_ANALYSIS.md
-    test_inventory_path: {OUTPUT_DIR}/TEST_INVENTORY.md
-    gap_analysis_path: {OUTPUT_DIR}/GAP_ANALYSIS.md
-    </parameters>
-  "
-)
-```
-
-**Handle analyzer return:**
-
-- Extract `total_test_count`, `pyramid_breakdown`, and `risk_count` from the analyzer's return values.
-- Verify all expected artifacts exist on disk.
-
-**Verify artifacts exist:**
-
-```bash
-[ -f "${OUTPUT_DIR}/QA_ANALYSIS.md" ] && echo "FOUND: QA_ANALYSIS.md" || echo "MISSING: QA_ANALYSIS.md"
-[ -f "${OUTPUT_DIR}/TEST_INVENTORY.md" ] && echo "FOUND: TEST_INVENTORY.md" || echo "MISSING: TEST_INVENTORY.md"
-
-# For Option 1 only:
-[ -f "${OUTPUT_DIR}/QA_REPO_BLUEPRINT.md" ] && echo "FOUND: QA_REPO_BLUEPRINT.md" || echo "MISSING: QA_REPO_BLUEPRINT.md"
-
-# For Option 2/3 only:
-[ -f "${OUTPUT_DIR}/GAP_ANALYSIS.md" ] && echo "FOUND: GAP_ANALYSIS.md" || echo "MISSING: GAP_ANALYSIS.md"
-```
-
-If any required artifact is missing, print error with the specific missing file and STOP.
-</step>
-
-<step name="print_summary">
-## Step 5: Print Analysis Summary
-
-Print a human-readable summary of the analysis results. No git operations are performed.
-
-**Read QA_ANALYSIS.md to extract summary data:**
-
-- Count risks by severity: HIGH, MEDIUM, LOW
-- Count test cases by pyramid tier from TEST_INVENTORY.md
-- Extract Top 10 unit test targets
-- Extract architecture overview (framework, language, runtime)
-
-**Print summary:**
-
-```
-=== Analysis Complete ===
-
-Architecture:
-  Framework: {framework}
-  Language: {language}
-  Runtime: {runtime}
-  Frontend: {has_frontend}
-  Detection Confidence: {detection_confidence}
-
-Risk Assessment:
-  HIGH: {high_count}
-  MEDIUM: {medium_count}
-  LOW: {low_count}
-
-Test Case Inventory:
-  Unit Tests:        {unit_count} ({unit_percent}%)
-  Integration Tests: {integration_count} ({integration_percent}%)
-  API Tests:         {api_count} ({api_percent}%)
-  E2E Tests:         {e2e_count} ({e2e_percent}%)
-  --------------------------
-  Total:             {total_count}
-
-Priority Distribution:
-  P0 (blocks release): {p0_count}
-  P1 (should fix):     {p1_count}
-  P2 (nice to have):   {p2_count}
-
-Artifacts Produced:
-  - {OUTPUT_DIR}/SCAN_MANIFEST.md
-  - {OUTPUT_DIR}/QA_ANALYSIS.md
-  - {OUTPUT_DIR}/TEST_INVENTORY.md
-  - {OUTPUT_DIR}/QA_REPO_BLUEPRINT.md  (Option 1 only)
-  - {OUTPUT_DIR}/GAP_ANALYSIS.md       (Option 2/3 only)
-
-No git operations performed. No PR created.
-Run the full pipeline with /qa-start to generate tests.
-===========================
-```
-</step>
-
-</process>
-
-<output>
-This workflow produces analysis artifacts only. No test files are generated. No git operations are performed.
-
-**Artifacts produced:**
-
-| Artifact | When Produced | Description |
-|----------|---------------|-------------|
-| SCAN_MANIFEST.md | Always | Repository scan with file tree, framework detection, testable surfaces |
-| QA_ANALYSIS.md | Always | Architecture overview, risks, top 10 targets, API targets, testing pyramid |
-| TEST_INVENTORY.md | Always | Complete test case inventory with IDs, inputs, expected outcomes |
-| QA_REPO_BLUEPRINT.md | Option 1 (no QA repo) | Repository structure, configs, CI/CD strategy for new QA repo |
-| GAP_ANALYSIS.md | Option 2/3 (QA repo exists) | Coverage gaps, missing tests, broken tests, quality assessment |
-
-**No side effects:**
-- No git branches created
-- No git commits made
-- No PRs created
-- No test files generated
-- No source files modified
-</output>
-
-<error_handling>
-| Error | Cause | Action |
-|-------|-------|--------|
-| Dev repo path does not exist | Invalid --dev-repo argument | Print error with path, STOP |
-| QA repo path does not exist | Invalid --qa-repo argument | Print error with path, STOP |
-| Scanner returns STOP | No testable surfaces or uncertain framework | Print scanner's reason, STOP |
-| SCAN_MANIFEST.md missing after scanner | Scanner failed silently | Print error, STOP |
-| Analyzer artifacts missing | Analyzer failed silently | Print error with specific missing file, STOP |
-| Framework detection LOW confidence | Ambiguous tech stack | Scanner checkpoints for user confirmation |
-</error_handling>
+<purpose>
+Analysis-only workflow. Scans a developer repository (and optionally a QA repository) to produce comprehensive analysis artifacts -- SCAN_MANIFEST.md, QA_ANALYSIS.md, TEST_INVENTORY.md, and optionally QA_REPO_BLUEPRINT.md or GAP_ANALYSIS.md. No test generation, no git operations, no PR. Use this workflow to understand a codebase's testability before committing to full test generation.
+</purpose>
+
+<required_reading>
+- `CLAUDE.md` -- QA automation standards, pipeline stages, module boundaries, verification commands
+- `agents/qaa-scanner.md` -- Scanner agent definition (produces SCAN_MANIFEST.md)
+- `agents/qaa-analyzer.md` -- Analyzer agent definition (produces QA_ANALYSIS.md, TEST_INVENTORY.md, QA_REPO_BLUEPRINT.md or GAP_ANALYSIS.md)
+- `templates/scan-manifest.md` -- SCAN_MANIFEST.md format contract
+- `templates/qa-analysis.md` -- QA_ANALYSIS.md format contract
+- `templates/test-inventory.md` -- TEST_INVENTORY.md format contract
+- `templates/qa-repo-blueprint.md` -- QA_REPO_BLUEPRINT.md format contract (Option 1 only)
+- `templates/gap-analysis.md` -- GAP_ANALYSIS.md format contract (Option 2/3 only)
+</required_reading>
+
+<process>
+
+<step name="parse_arguments">
+## Step 1: Parse Arguments
+
+Parse `$ARGUMENTS` for repository paths.
+
+**Supported arguments:**
+- `--dev-repo <path>` -- Path to the developer repository to analyze
+- `--qa-repo <path>` -- Path to an existing QA repository (optional)
+- No arguments -- defaults to the current working directory as the dev repo
+
+**Parsing logic:**
+
+```bash
+DEV_REPO=""
+QA_REPO=""
+
+# Parse --dev-repo and --qa-repo from $ARGUMENTS
+# If --dev-repo is provided, use it
+# If --qa-repo is provided, use it
+# If neither is provided, DEV_REPO defaults to current working directory
+```
+
+**Validation:**
+- If `--dev-repo` is provided, verify the path exists and is a directory. If not, print error: `"Error: Dev repo path does not exist: {path}"` and STOP.
+- If `--qa-repo` is provided, verify the path exists and is a directory. If not, print error: `"Error: QA repo path does not exist: {path}"` and STOP.
+- If no arguments provided, set `DEV_REPO` to the current working directory.
+</step>
+
+<step name="determine_mode">
+## Step 2: Determine Analysis Mode
+
+Set the workflow option based on whether a QA repo was provided.
+
+**Mode selection:**
+
+| QA Repo Provided | Mode | Description |
+|-------------------|------|-------------|
+| No | `full` (Option 1) | Dev-only analysis. Produces QA_ANALYSIS.md + TEST_INVENTORY.md + QA_REPO_BLUEPRINT.md |
+| Yes | `gap` (Option 2/3) | Gap analysis. Produces QA_ANALYSIS.md + TEST_INVENTORY.md + GAP_ANALYSIS.md |
+
+```
+MODE="full"
+if QA_REPO is not empty:
+  MODE="gap"
+```
+
+**Set output directory:**
+
+```bash
+OUTPUT_DIR=".qa-output"
+mkdir -p "${OUTPUT_DIR}"
+```
+
+**Print analysis banner:**
+
+```
+=== QA Analysis Workflow ===
+Mode: {MODE} ({description})
+Dev Repo: {DEV_REPO}
+QA Repo: {QA_REPO or 'N/A'}
+Output: {OUTPUT_DIR}
+============================
+```
+</step>
+
+<step name="run_scanner">
+## Step 3: Spawn Scanner Agent
+
+Spawn the scanner agent to produce SCAN_MANIFEST.md.
+
+**For full mode (Option 1):**
+
+```
+Task(
+  prompt="
+    <objective>Scan repository and produce SCAN_MANIFEST.md</objective>
+    <execution_context>@agents/qaa-scanner.md</execution_context>
+    <files_to_read>
+    - CLAUDE.md
+    </files_to_read>
+    <parameters>
+    dev_repo_path: {DEV_REPO}
+    qa_repo_path: null
+    output_path: {OUTPUT_DIR}/SCAN_MANIFEST.md
+    </parameters>
+  "
+)
+```
+
+**For gap mode (Option 2/3):**
+
+```
+Task(
+  prompt="
+    <objective>Scan both developer and QA repositories and produce SCAN_MANIFEST.md</objective>
+    <execution_context>@agents/qaa-scanner.md</execution_context>
+    <files_to_read>
+    - CLAUDE.md
+    </files_to_read>
+    <parameters>
+    dev_repo_path: {DEV_REPO}
+    qa_repo_path: {QA_REPO}
+    output_path: {OUTPUT_DIR}/SCAN_MANIFEST.md
+    </parameters>
+  "
+)
+```
+
+**Handle scanner return:**
+
+- If scanner returns `decision: STOP` -- print the stop reason and STOP the workflow. The repository has no testable surfaces or an uncertain framework.
+- If scanner returns `decision: PROCEED` -- continue to next step.
+- Extract `has_frontend` and `detection_confidence` from scanner return for use in the summary.
+
+**Verify SCAN_MANIFEST.md exists:**
+
+```bash
+[ -f "${OUTPUT_DIR}/SCAN_MANIFEST.md" ] && echo "FOUND" || echo "MISSING"
+```
+
+If missing, print error: `"Error: Scanner did not produce SCAN_MANIFEST.md. Check scanner output for details."` and STOP.
+</step>
+
+<step name="run_analyzer">
+## Step 4: Spawn Analyzer Agent
+
+Spawn the analyzer agent to produce analysis artifacts.
+
+**For full mode (Option 1):**
+
+```
+Task(
+  prompt="
+    <objective>Analyze scanned repository and produce QA_ANALYSIS.md, TEST_INVENTORY.md, and QA_REPO_BLUEPRINT.md</objective>
+    <execution_context>@agents/qaa-analyzer.md</execution_context>
+    <files_to_read>
+    - {OUTPUT_DIR}/SCAN_MANIFEST.md
+    - CLAUDE.md
+    </files_to_read>
+    <parameters>
+    workflow_option: 1
+    qa_analysis_path: {OUTPUT_DIR}/QA_ANALYSIS.md
+    test_inventory_path: {OUTPUT_DIR}/TEST_INVENTORY.md
+    blueprint_path: {OUTPUT_DIR}/QA_REPO_BLUEPRINT.md
+    </parameters>
+  "
+)
+```
+
+**For gap mode (Option 2/3):**
+
+```
+Task(
+  prompt="
+    <objective>Analyze scanned repositories and produce QA_ANALYSIS.md, TEST_INVENTORY.md, and GAP_ANALYSIS.md</objective>
+    <execution_context>@agents/qaa-analyzer.md</execution_context>
+    <files_to_read>
+    - {OUTPUT_DIR}/SCAN_MANIFEST.md
+    - CLAUDE.md
+    </files_to_read>
+    <parameters>
+    workflow_option: 2
+    qa_analysis_path: {OUTPUT_DIR}/QA_ANALYSIS.md
+    test_inventory_path: {OUTPUT_DIR}/TEST_INVENTORY.md
+    gap_analysis_path: {OUTPUT_DIR}/GAP_ANALYSIS.md
+    </parameters>
+  "
+)
+```
+
+**Handle analyzer return:**
+
+- Extract `total_test_count`, `pyramid_breakdown`, and `risk_count` from the analyzer's return values.
+- Verify all expected artifacts exist on disk.
+
+**Verify artifacts exist:**
+
+```bash
+[ -f "${OUTPUT_DIR}/QA_ANALYSIS.md" ] && echo "FOUND: QA_ANALYSIS.md" || echo "MISSING: QA_ANALYSIS.md"
+[ -f "${OUTPUT_DIR}/TEST_INVENTORY.md" ] && echo "FOUND: TEST_INVENTORY.md" || echo "MISSING: TEST_INVENTORY.md"
+
+# For Option 1 only:
+[ -f "${OUTPUT_DIR}/QA_REPO_BLUEPRINT.md" ] && echo "FOUND: QA_REPO_BLUEPRINT.md" || echo "MISSING: QA_REPO_BLUEPRINT.md"
+
+# For Option 2/3 only:
+[ -f "${OUTPUT_DIR}/GAP_ANALYSIS.md" ] && echo "FOUND: GAP_ANALYSIS.md" || echo "MISSING: GAP_ANALYSIS.md"
+```
+
+If any required artifact is missing, print error with the specific missing file and STOP.
+</step>
+
+<step name="print_summary">
+## Step 5: Print Analysis Summary
+
+Print a human-readable summary of the analysis results. No git operations are performed.
+
+**Read QA_ANALYSIS.md to extract summary data:**
+
+- Count risks by severity: HIGH, MEDIUM, LOW
+- Count test cases by pyramid tier from TEST_INVENTORY.md
+- Extract Top 10 unit test targets
+- Extract architecture overview (framework, language, runtime)
+
+**Print summary:**
+
+```
+=== Analysis Complete ===
+
+Architecture:
+  Framework: {framework}
+  Language: {language}
+  Runtime: {runtime}
+  Frontend: {has_frontend}
+  Detection Confidence: {detection_confidence}
+
+Risk Assessment:
+  HIGH: {high_count}
+  MEDIUM: {medium_count}
+  LOW: {low_count}
+
+Test Case Inventory:
+  Unit Tests:        {unit_count} ({unit_percent}%)
+  Integration Tests: {integration_count} ({integration_percent}%)
+  API Tests:         {api_count} ({api_percent}%)
+  E2E Tests:         {e2e_count} ({e2e_percent}%)
+  --------------------------
+  Total:             {total_count}
+
+Priority Distribution:
+  P0 (blocks release): {p0_count}
+  P1 (should fix):     {p1_count}
+  P2 (nice to have):   {p2_count}
+
+Artifacts Produced:
+  - {OUTPUT_DIR}/SCAN_MANIFEST.md
+  - {OUTPUT_DIR}/QA_ANALYSIS.md
+  - {OUTPUT_DIR}/TEST_INVENTORY.md
+  - {OUTPUT_DIR}/QA_REPO_BLUEPRINT.md  (Option 1 only)
+  - {OUTPUT_DIR}/GAP_ANALYSIS.md       (Option 2/3 only)
+
+No git operations performed. No PR created.
+Run the full pipeline with /qa-start to generate tests.
+===========================
+```
+</step>
+
+</process>
+
+<output>
+This workflow produces analysis artifacts only. No test files are generated. No git operations are performed.
+
+**Artifacts produced:**
+
+| Artifact | When Produced | Description |
+|----------|---------------|-------------|
+| SCAN_MANIFEST.md | Always | Repository scan with file tree, framework detection, testable surfaces |
+| QA_ANALYSIS.md | Always | Architecture overview, risks, top 10 targets, API targets, testing pyramid |
+| TEST_INVENTORY.md | Always | Complete test case inventory with IDs, inputs, expected outcomes |
+| QA_REPO_BLUEPRINT.md | Option 1 (no QA repo) | Repository structure, configs, CI/CD strategy for new QA repo |
+| GAP_ANALYSIS.md | Option 2/3 (QA repo exists) | Coverage gaps, missing tests, broken tests, quality assessment |
+
+**No side effects:**
+- No git branches created
+- No git commits made
+- No PRs created
+- No test files generated
+- No source files modified
+</output>
+
+<error_handling>
+| Error | Cause | Action |
+|-------|-------|--------|
+| Dev repo path does not exist | Invalid --dev-repo argument | Print error with path, STOP |
+| QA repo path does not exist | Invalid --qa-repo argument | Print error with path, STOP |
+| Scanner returns STOP | No testable surfaces or uncertain framework | Print scanner's reason, STOP |
+| SCAN_MANIFEST.md missing after scanner | Scanner failed silently | Print error, STOP |
+| Analyzer artifacts missing | Analyzer failed silently | Print error with specific missing file, STOP |
+| Framework detection LOW confidence | Ambiguous tech stack | Scanner checkpoints for user confirmation |
+</error_handling>