qaa-agent 1.9.2 → 1.9.6

package/commands/qa-research.md

@@ -1,157 +1,212 @@
-# QA Research
-
-Research the testing ecosystem for the current project. Investigates framework capabilities, best practices, selector syntax, and testing patterns using Context7 MCP and official documentation. Produces research files consumed by all downstream QA agents (analyzer, planner, executor, validator).
-
-## Usage
-
-```
-/qa-research [options]
-```
-
-### Options
-
-- `--focus <mode>` — research mode (default: `stack-testing`)
-  - `stack-testing` — full stack analysis: test runner, assertions, mocking, E2E framework, CI/CD patterns
-  - `framework-deep-dive` — deep dive into the detected/specified framework: full API, patterns, pitfalls, selector syntax
-  - `api-testing` — API testing strategy: endpoint patterns, contract testing, auth testing, error testing
-  - `e2e-strategy` — E2E strategy: framework comparison, POM patterns, selector strategies, visual testing
-- `--dev-repo <path>` — path to developer repository (default: current directory)
-
-### Mode Detection
-
-```
-if --focus framework-deep-dive:
-  MODE = "framework-deep-dive"
-elif --focus api-testing:
-  MODE = "api-testing"
-elif --focus e2e-strategy:
-  MODE = "e2e-strategy"
-else:
-  MODE = "stack-testing"
-```
-
-## What It Produces
-
-All output files are written to `.qa-output/research/`.
-
-| Mode | Output |
-|------|--------|
-| stack-testing | `TESTING_STACK.md` (+ `API_TESTING_STRATEGY.md` and `E2E_STRATEGY.md` if project has APIs and frontend) |
-| framework-deep-dive | `FRAMEWORK_CAPABILITIES.md` |
-| api-testing | `API_TESTING_STRATEGY.md` |
-| e2e-strategy | `E2E_STRATEGY.md` |
-
-These files are consumed by downstream agents:
-
-| File | Consumed By |
-|------|-------------|
-| `TESTING_STACK.md` | Analyzer (framework selection), Planner (dependency setup) |
-| `FRAMEWORK_CAPABILITIES.md` | Executor (idiomatic test writing), Validator (pattern checking), E2E Runner (correct syntax) |
-| `API_TESTING_STRATEGY.md` | Planner (API test case design), Executor (implementation patterns) |
-| `E2E_STRATEGY.md` | Planner (E2E scope decisions), Executor (POM and selector patterns) |
-
-## Instructions
-
-### Step 1: Initialize
-
-Parse `$ARGUMENTS` for flags.
-
-```bash
-MODE="stack-testing"
-DEV_REPO=""
-
-if echo "$ARGUMENTS" | grep -qE '\-\-focus'; then
-  MODE=$(echo "$ARGUMENTS" | grep -oE '\-\-focus\s+[^\s]+' | awk '{print $2}')
-fi
-
-if echo "$ARGUMENTS" | grep -qE '\-\-dev-repo'; then
-  DEV_REPO=$(echo "$ARGUMENTS" | grep -oE '\-\-dev-repo\s+[^\s]+' | awk '{print $2}')
-fi
-
-if [ -z "$DEV_REPO" ]; then
-  DEV_REPO=$(pwd)
-fi
-
-mkdir -p .qa-output/research
-```
-
-Print banner:
-```
-=== QA Research ===
-Mode: {MODE}
-Dev Repo: {DEV_REPO}
-Output: .qa-output/research/
-====================
-```
-
-### Step 2: Read Existing Context (if available)
-
-Read these files if they exist — they provide context to the researcher:
-
-- `.qa-output/SCAN_MANIFEST.md` — detected project stack from scanner
-- `CLAUDE.md` — QA standards that the research must align with
-- `~/.claude/qaa/MY_PREFERENCES.md` — user preferences (framework choices, conventions)
-
-### Step 3: Spawn Researcher Agent
-
-Task(
-  prompt="
-    <objective>Research the testing ecosystem for this project. Mode: {MODE}. Use Context7 MCP as the primary source for all framework and library questions. Verify everything against current documentation — do not rely on training data alone.</objective>
-    <execution_context>@agents/qaa-project-researcher.md</execution_context>
-    <files_to_read>
-    - CLAUDE.md
-    - ~/.claude/qaa/MY_PREFERENCES.md (if exists)
-    - .qa-output/SCAN_MANIFEST.md (if exists)
-    </files_to_read>
-    <parameters>
-    mode: {MODE}
-    dev_repo_path: {DEV_REPO}
-    output_dir: .qa-output/research
-    </parameters>
-  "
-)
-
-### Step 4: Present Results
-
-After the researcher completes, summarize what was found:
-
-```
-=== Research Complete ===
-Mode: {MODE}
-Files produced: {list of files in .qa-output/research/}
-Framework detected: {name + version}
-Key findings:
-  - {finding 1}
-  - {finding 2}
-  - {finding 3}
-
-These files will be automatically consumed by downstream agents
-when you run /qa-create-test, /qa-fix, or /qa-start.
-=========================
-```
-
-$ARGUMENTS
-
-## MANDATORY verification — run ALL commands below, no exceptions, no skipping
-
-Before returning control, copy-paste and run this ENTIRE block. Do NOT decide which commands "apply" — run all of them every time. The output confirms what happened; you do not get to assume the answer.
-
-```bash
-echo "=== QA-RESEARCH CHECKLIST START ==="
-echo "1. Research output files:"
-ls .qa-output/research/ 2>/dev/null || echo "NO_RESEARCH_FILES"
-echo "2. TESTING_STACK.md content check:"
-head -20 .qa-output/research/TESTING_STACK.md 2>/dev/null || echo "NO_TESTING_STACK"
-echo "3. FRAMEWORK_CAPABILITIES.md content check:"
-head -20 .qa-output/research/FRAMEWORK_CAPABILITIES.md 2>/dev/null || echo "NO_FRAMEWORK_CAPABILITIES"
-echo "4. Context7 was used (check for confidence levels):"
-grep -c "HIGH\|MEDIUM\|LOW" .qa-output/research/*.md 2>/dev/null || echo "NO_CONFIDENCE_LEVELS"
-echo "5. Version information present:"
-grep -i "version" .qa-output/research/*.md 2>/dev/null | head -5 || echo "NO_VERSION_INFO"
-echo "=== QA-RESEARCH CHECKLIST END ==="
-```
-
-**Rules:**
-- Run the block AS-IS. Do not modify it. Do not split it. Do not skip lines.
-- If any output shows a problem (NO_RESEARCH_FILES after research completed), fix it before returning.
-- Do NOT mark this task as complete until the block has been executed and you have read every line of output.
+# QA Research
+
+Research the testing ecosystem for the current project. Investigates framework capabilities, best practices, selector syntax, and testing patterns using Context7 MCP and official documentation. Produces research files consumed by all downstream QA agents (analyzer, planner, executor, validator).
+
+## ⚠ MANDATORY: How to Execute This Command
+
+Execute these steps IN ORDER. Every step is mandatory. Do NOT improvise,
+reorder, skip, or invent steps. This command is flag-driven — there is no intent
+detector; parse the flags from `$ARGUMENTS` literally.
+
+1. STEP 1 — Initialize (see "## Instructions"): parse `$ARGUMENTS` for `--focus`
+   and `--dev-repo`, resolve MODE (default `stack-testing`) and DEV_REPO
+   (default: current directory), create `.qa-output/research/`, and print the
+   `=== QA Research ===` banner.
+2. STEP 2 — Read existing context if present: `.qa-output/SCAN_MANIFEST.md`,
+   `CLAUDE.md`, `~/.claude/qaa/MY_PREFERENCES.md`.
+3. STEP 3 — Spawn the project-researcher via Task() with its `<critical_directive>`
+   block, passing the resolved MODE and DEV_REPO. Do NOT do the research inline.
+4. STEP 4 — After the researcher returns, print the `=== Research Complete ===`
+   summary.
+5. Run the "## MANDATORY verification" checklist at the bottom of this file
+   before returning control.
+
+### DO NOT
+- Skip the `=== QA Research ===` banner
+- Do the research yourself inline instead of spawning the project-researcher
+- Invent a mode not in the `--focus` list, or reorder the steps
+- Skip the bottom verification checklist
+- Assume dev-repo path or mode from a previous command invocation — each
+  /qa-research call is self-contained
+
+If you find yourself improvising, doing the research inline, or skipping the
+verification checklist — STOP and restart from step 1.
+
+## Usage
+
+```
+/qa-research [options]
+```
+
+### Options
+
+- `--focus <mode>` — research mode (default: `stack-testing`)
+  - `stack-testing` — full stack analysis: test runner, assertions, mocking, E2E framework, CI/CD patterns
+  - `framework-deep-dive` — deep dive into the detected/specified framework: full API, patterns, pitfalls, selector syntax
+  - `api-testing` — API testing strategy: endpoint patterns, contract testing, auth testing, error testing
+  - `e2e-strategy` — E2E strategy: framework comparison, POM patterns, selector strategies, visual testing
+- `--dev-repo <path>` — path to developer repository (default: current directory)
+
+### Mode Detection
+
+```
+if --focus framework-deep-dive:
+  MODE = "framework-deep-dive"
+elif --focus api-testing:
+  MODE = "api-testing"
+elif --focus e2e-strategy:
+  MODE = "e2e-strategy"
+else:
+  MODE = "stack-testing"
+```
+
+## What It Produces
+
+All output files are written to `.qa-output/research/`.
+
+| Mode | Output |
+|------|--------|
+| stack-testing | `TESTING_STACK.md` (+ `API_TESTING_STRATEGY.md` and `E2E_STRATEGY.md` if project has APIs and frontend) |
+| framework-deep-dive | `FRAMEWORK_CAPABILITIES.md` |
+| api-testing | `API_TESTING_STRATEGY.md` |
+| e2e-strategy | `E2E_STRATEGY.md` |
+
+These files are consumed by downstream agents:
+
+| File | Consumed By |
+|------|-------------|
+| `TESTING_STACK.md` | Analyzer (framework selection), Planner (dependency setup) |
+| `FRAMEWORK_CAPABILITIES.md` | Executor (idiomatic test writing), Validator (pattern checking), E2E Runner (correct syntax) |
+| `API_TESTING_STRATEGY.md` | Planner (API test case design), Executor (implementation patterns) |
+| `E2E_STRATEGY.md` | Planner (E2E scope decisions), Executor (POM and selector patterns) |
+
+## Instructions
+
+### Step 1: Initialize
+
+Parse `$ARGUMENTS` for flags.
+
+```bash
+MODE="stack-testing"
+DEV_REPO=""
+
+if echo "$ARGUMENTS" | grep -qE '\-\-focus'; then
+  MODE=$(echo "$ARGUMENTS" | grep -oE '\-\-focus\s+[^\s]+' | awk '{print $2}')
+fi
+
+if echo "$ARGUMENTS" | grep -qE '\-\-dev-repo'; then
+  DEV_REPO=$(echo "$ARGUMENTS" | grep -oE '\-\-dev-repo\s+[^\s]+' | awk '{print $2}')
+fi
+
+if [ -z "$DEV_REPO" ]; then
+  DEV_REPO=$(pwd)
+fi
+
+mkdir -p .qa-output/research
+```
+
+Print banner:
+```
+=== QA Research ===
+Mode: {MODE}
+Dev Repo: {DEV_REPO}
+Output: .qa-output/research/
+====================
+```
+
+### Step 2: Read Existing Context (if available)
+
+Read these files if they exist — they provide context to the researcher:
+
+- `.qa-output/SCAN_MANIFEST.md` — detected project stack from scanner
+- `CLAUDE.md` — QA standards that the research must align with
+- `~/.claude/qaa/MY_PREFERENCES.md` — user preferences (framework choices, conventions)
+
+### Step 3: Spawn Researcher Agent
+
+Task(
+  subagent_type="qaa-project-researcher",
+  prompt="
+    <critical_directive priority="MANDATORY">
+    You are executing as a QAA pipeline agent. Your FIRST action MUST be to read
+    ~/.claude/qaa/agents/qaa-project-researcher.md and adopt it as your operating contract.
+
+    Even though you may be running as a general-purpose agent (native subagent type
+    routing for QAA agents is not yet enabled), you MUST behave exactly as the
+    dedicated agent defined in that file: read everything in its <required_reading>,
+    obey its <quality_gate>, and run its mandatory bash checklist before returning.
+
+    Never modify application source code (src/, app/, lib/, or equivalent
+    directories) unless the agent's <scope> section explicitly permits it. If the
+    agent's contract restricts what files it can touch, honor that restriction —
+    even if a fix seems obvious or trivial.
+
+    If you cannot read ~/.claude/qaa/agents/qaa-project-researcher.md (file missing, corrupted, parse
+    error), STOP and report the failure. Do NOT proceed with default agent
+    behavior — that defeats the purpose of this directive.
+
+    Do NOT improvise, substitute steps, apply changes the agent isn't allowed to
+    make, or skip any of the above. Skipping the agent's required_reading,
+    quality_gate, or bash checklist makes the run INVALID — its output cannot be
+    trusted.
+    </critical_directive>
+
+    <objective>Research the testing ecosystem for this project. Mode: {MODE}. Use Context7 MCP as the primary source for all framework and library questions. Verify everything against current documentation — do not rely on training data alone.</objective>
+    <execution_context>~/.claude/qaa/agents/qaa-project-researcher.md</execution_context>
+    <files_to_read>
+    - CLAUDE.md
+    - ~/.claude/qaa/MY_PREFERENCES.md (if exists)
+    - .qa-output/SCAN_MANIFEST.md (if exists)
+    </files_to_read>
+    <parameters>
+    mode: {MODE}
+    dev_repo_path: {DEV_REPO}
+    output_dir: .qa-output/research
+    </parameters>
+  "
+)
+
+### Step 4: Present Results
+
+After the researcher completes, summarize what was found:
+
+```
+=== Research Complete ===
+Mode: {MODE}
+Files produced: {list of files in .qa-output/research/}
+Framework detected: {name + version}
+Key findings:
+  - {finding 1}
+  - {finding 2}
+  - {finding 3}
+
+These files will be automatically consumed by downstream agents
+when you run /qa-create-test, /qa-fix, or /qa-start.
+=========================
+```
+
+$ARGUMENTS
+
+## MANDATORY verification — run ALL commands below, no exceptions, no skipping
+
+Before returning control, copy-paste and run this ENTIRE block. Do NOT decide which commands "apply" — run all of them every time. The output confirms what happened; you do not get to assume the answer.
+
+```bash
+echo "=== QA-RESEARCH CHECKLIST START ==="
+echo "1. Research output files:"
+ls .qa-output/research/ 2>/dev/null || echo "NO_RESEARCH_FILES"
+echo "2. TESTING_STACK.md content check:"
+head -20 .qa-output/research/TESTING_STACK.md 2>/dev/null || echo "NO_TESTING_STACK"
+echo "3. FRAMEWORK_CAPABILITIES.md content check:"
+head -20 .qa-output/research/FRAMEWORK_CAPABILITIES.md 2>/dev/null || echo "NO_FRAMEWORK_CAPABILITIES"
+echo "4. Context7 was used (check for confidence levels):"
+grep -c "HIGH\|MEDIUM\|LOW" .qa-output/research/*.md 2>/dev/null || echo "NO_CONFIDENCE_LEVELS"
+echo "5. Version information present:"
+grep -i "version" .qa-output/research/*.md 2>/dev/null | head -5 || echo "NO_VERSION_INFO"
+echo "=== QA-RESEARCH CHECKLIST END ==="
+```
+
+**Rules:**
+- Run the block AS-IS. Do not modify it. Do not split it. Do not skip lines.
+- If any output shows a problem (NO_RESEARCH_FILES after research completed), fix it before returning.
+- Do NOT mark this task as complete until the block has been executed and you have read every line of output.

package/commands/qa-start.md

@@ -4,16 +4,72 @@ Run the complete QA automation pipeline. Analyzes a repository, generates a stan
 
 ## Usage
 
-/qa-start [--dev-repo <path>] [--qa-repo <path>] [--auto]
+```
+/qa-start [--dev-repo <path>] [--qa-repo <path>] [--app-url <url>] [--framework <name>] [--auto]
+```
 
-- No arguments: uses current directory as dev repo (Option 1)
-- --dev-repo: explicit path to developer repository
-- --qa-repo: path to existing QA repository (triggers Option 2 or 3)
-- --auto: enable auto-advance mode (no pauses at safe checkpoints)
+- **No arguments**: uses current directory as dev repo (Option 1)
+- **`--dev-repo <path>`**: explicit path to developer repository
+- **`--qa-repo <path>`**: path to existing QA repository (triggers Option 2 or 3)
+- **`--app-url <url>`**: URL of the running application (used by E2E runner, validator Layer 5, testid-injector verification, bug-detective reproduction). Assumes a repo is also provided (this flag is metadata, not a mode switch)
+- **`--framework <name>`**: override framework auto-detection (playwright, cypress, jest, vitest, pytest, selenium, robot-framework)
+- **`--auto`**: enable auto-advance mode (no pauses at safe checkpoints)
+
+## Natural language fallback
+
+This command also accepts natural language. The intent detector will extract URLs, paths (with or without quotes, with or without spaces), framework hints, and ticket URLs from your prompt. If you don't pass a flag explicitly but mention the value in NL, the system will detect it and show you the resolved interpretation in the **INPUT DETECTION** banner before starting the pipeline. Flags always win over NL.
+
+Examples:
+
+```
+# All flags explicit
+/qa-start --dev-repo ./myproject --app-url https://staging.example.com --auto
+
+# Natural language equivalent
+/qa-start use playwright on my project at ./myproject and test against https://staging.example.com
+
+# Mixed (flags win where present, NL fills gaps)
+/qa-start --dev-repo ./myproject test against https://staging.example.com using playwright
+```
+
+
+## ⚠ MANDATORY: How to Execute This Command
+
+When invoked, you **MUST** follow `@workflows/qa-start.md` literally — every step, in order, as written. The workflow file contains an explicit `<critical_directive>` block at the top with mandatory execution rules. Read and obey those rules before doing anything else.
+
+### DO NOT
+
+- Analyze the target repo before running the workflow's Step 1
+- Decide that "the standard pipeline doesn't apply" because the repo is empty / has a URL / uses a different framework
+- Scaffold files directly without going through the pipeline stages
+- Skip the intent detector (Step 1) regardless of what arguments you received
+- Generate any test files (.spec.ts, .robot, .py, etc.) before reaching the `<step name="generate">` stage
+
+### DO
+
+- Run the workflow's Step 1 (intent detector via `node ~/.claude/qaa/bin/lib/intent-detector.cjs --resolve ...`) as your **FIRST** tool call
+- Show the INPUT DETECTION banner before any other action
+- Run the 5 input validations
+- Pause at the confirmation checkpoint if applicable (interactive mode + any value from NL or default)
+- Let each step's verification gates determine success/failure
+- Report errors as pipeline failures, not as opportunities to adapt
+
+If you find yourself improvising or scaffolding files before Step 1 has run, **STOP, return to the workflow, and start over from Step 1**.
+
+
+### Pass user input literally to the intent detector
+
+When the workflow's Step 1 invokes the intent detector with `$ARGUMENTS`, you must substitute that placeholder with the **raw user input as received in the slash command**, NOT with a pre-translated flag-formatted string.
+
+**WRONG:** mentally parsing the user's NL and constructing `--app-url https://x --dev-repo C:\y` before calling the detector.
+
+**RIGHT:** passing the literal text the user typed (e.g., `"from URL https://x to C:\y using playwright"`) and letting the detector handle the parsing.
+
+If you pre-translate, the detector sees only flags and reports every value as `source: flag`, which causes `ALL_FROM_FLAGS=true` and bypasses the confirmation checkpoint — defeating its purpose. Always pass raw input.
 
 ## Instructions
 
-1. Read `CLAUDE.md` -- all QA standards that govern the pipeline.
+1. Read `CLAUDE.md` — all QA standards that govern the pipeline.
 2. Execute the workflow:
 
 Follow the workflow defined in `@workflows/qa-start.md` end-to-end.