qaa-agent 1.8.5 → 1.8.6

package/CHANGELOG.md

@@ -3,6 +3,18 @@
 
 All notable changes to QAA (QA Automation Agent) are documented here.
 
+## [1.8.6] - 2026-04-20
+
+### Added
+
+- **Fix mode analyze-first flow in `/qa-fix`** — fix mode now runs in two phases: Phase 1 analyzes and classifies all failures without touching any files, then presents a Fix Plan to the user. Phase 2 executes auto-fixes only after explicit user confirmation. Users can refine the plan iteratively (add fixes, remove files, change approach) in a loop until satisfied, then approve or cancel. Replaces the previous fully-automatic fix behavior.
+- **Codebase map context for bug-detective** — `/qa-fix` fix mode now passes all 4 codebase map documents (`CODE_PATTERNS.md`, `API_CONTRACTS.md`, `TEST_SURFACE.md`, `TESTABILITY.md`) to the bug-detective agent via `files_to_read`. Previously the bug-detective classified failures without project-specific context, leading to less accurate classifications.
+- **Mandatory bash checklist in `/qa-fix`** — verification block at the end of `qa-fix.md` that forces the agent to run `ls`/`cat`/`grep` commands to confirm artifacts were produced (classification report, locator registry, codebase map, MCP evidence, test files). Matches the existing checklist pattern in `/qa-create-test`.
+
+### Changed
+
+- **Validator fix loops increased from 3 to 5** — `qaa-validator` agent now has up to 5 fix loop iterations (previously 3), matching the E2E runner's loop budget. Updated across all references: locked decision, fix loop logic, checkpoint return, confidence criteria table, and quality gate checks.
+
 ## [1.8.5] - 2026-04-17
 
 ### Added

package/agents/qaa-validator.md

@@ -25,7 +25,7 @@ Read ALL of the following files BEFORE performing any validation. Do NOT skip.
 
 - **templates/validation-report.md** -- Output format contract. Defines the 5 required sections (Summary, File Details, Unresolved Issues, Fix Loop Log, Confidence Level), all field definitions, confidence criteria table (HIGH/MEDIUM/LOW), worked example, and quality gate checklist (7 items). Your VALIDATION_REPORT.md output MUST match this template exactly.
 
-- **.claude/skills/qa-self-validator/SKILL.md** -- Defines the 4 validation layers (Syntax, Structure, Dependencies, Logic), pass criteria per layer, fix loop protocol (max 3 loops), and output format.
+- **.claude/skills/qa-self-validator/SKILL.md** -- Defines the 4 validation layers (Syntax, Structure, Dependencies, Logic), pass criteria per layer, fix loop protocol (max 5 loops), and output format.
 
 - **~/.claude/qaa/MY_PREFERENCES.md** (optional -- read if exists). User's personal QA preferences saved by the qa-learner skill. If a preference conflicts with CLAUDE.md, the preference wins (it is a user override). Check for rules about: assertion style, locator strategy, naming conventions, framework choices.
 
@@ -257,7 +257,7 @@ Attempt to fix issues found during validation layers. This step encodes ALL 8 lo
 
 **Locked Decision 2: Sequential, fail-fast** -- Layers run in order: Layer 1 (Syntax) -> Layer 2 (Structure) -> Layer 3 (Dependencies) -> Layer 4 (Logic). Fix Layer 1 issues before proceeding to check Layer 2. If Layer 1 fails, fix it and re-validate Layer 1 before moving to Layer 2.
 
-**Locked Decision 3: Max 3 loops** -- The fix loop runs at most 3 times. After 3 loops with unresolved issues, STOP and escalate.
+**Locked Decision 3: Max 5 loops** -- The fix loop runs at most 5 times. After 5 loops with unresolved issues, STOP and escalate.
 
 **Locked Decision 4: Generated files only** -- Only fix files listed in the generation plan. Never modify pre-existing test files.
 
@@ -280,7 +280,7 @@ Attempt to fix issues found during validation layers. This step encodes ALL 8 lo
 **Fix loop execution:**
 
 ```
-Loop iteration (max 3):
+Loop iteration (max 5):
   1. Run all 4 validation layers sequentially (fail-fast)
   2. If all layers PASS: exit loop, proceed to produce_report
   3. If any layer FAIL:
@@ -290,20 +290,20 @@ Loop iteration (max 3):
         - If MEDIUM or LOW: record as unresolved, do NOT apply
      b. Log this loop iteration: issues found, fixes applied, verification
      c. Re-validate from the FAILED layer (not from Layer 1 unless Layer 1 failed)
-     d. If this was loop 3: exit loop regardless of results
+     d. If this was loop 5: exit loop regardless of results
 ```
 
-**After 3 loops with unresolved issues:**
+**After 5 loops with unresolved issues:**
 
 STOP and return a checkpoint:
 
 ```
 CHECKPOINT_RETURN:
 completed: "Validated {N} files across 4 layers. Completed {loop_count} fix loops."
-blocking: "Unresolved validation issues after maximum 3 fix loops"
+blocking: "Unresolved validation issues after maximum 5 fix loops"
 details:
   files_validated: {N}
-  loops_completed: 3
+  loops_completed: 5
   issues_found: {total_count}
   issues_fixed: {fixed_count}
   unresolved:
@@ -314,6 +314,9 @@ details:
       why_not_fixed: "{reason auto-fix was not applied}"
 awaiting: "User decides: fix remaining issues manually, accept with warnings, or abort validation"
 ```
+
+**Note:** The validator now has 5 fix loop iterations (up from 3), matching the E2E runner's loop budget. This gives more room to resolve cascading issues where fixing one problem reveals another.
+```
 </step>
 
 <step name="produce_report">
@@ -391,8 +394,8 @@ Include the confidence criteria table:
 | Level | All Layers PASS | Unresolved Issues | Fix Loops Used | Description |
 |-------|----------------|-------------------|----------------|-------------|
 | HIGH | Yes | 0 | 0-1 | All validations pass with minimal or no fixes needed. Code is ready for delivery. |
-| MEDIUM | Yes (after fixes) | 0-2 minor | 2-3 | All layers eventually pass, but required multiple fix rounds. Minor issues may exist. |
-| LOW | No (any FAIL) | Any critical | 3 (max) | At least one layer still fails, or critical issues remain unresolved. Human review required before delivery. |
+| MEDIUM | Yes (after fixes) | 0-2 minor | 2-5 | All layers eventually pass, but required multiple fix rounds. Minor issues may exist. |
+| LOW | No (any FAIL) | Any critical | 5 (max) | At least one layer still fails, or critical issues remain unresolved. Human review required before delivery. |
 
 Followed by the specific confidence statement:
 `**{LEVEL}:** {one-sentence reasoning referencing specific metrics from the summary}`
@@ -477,8 +480,8 @@ Before considering validation complete, verify ALL of the following.
 - [ ] Only generated files were validated (not pre-existing test files) -- verify every file in the report appears in the generation plan file list
 - [ ] Layer 4 cross-checked existing test files for duplicate IDs and overlapping selectors to prevent collisions
 - [ ] Fix confidence correctly classified (HIGH auto-applied, MEDIUM/LOW flagged for review but NOT auto-applied)
-- [ ] Fix loop count did not exceed 3 iterations
-- [ ] If 3 loops exhausted with unresolved issues: CHECKPOINT_RETURN was provided to escalate to user
+- [ ] Fix loop count did not exceed 5 iterations
+- [ ] If 5 loops exhausted with unresolved issues: CHECKPOINT_RETURN was provided to escalate to user
 - [ ] Validator did NOT commit any files (no git add, no git commit, no qaa-tools commit)
 
 If any check fails, fix the issue before finalizing the output. Do not deliver a validation report that fails its own quality gate.

package/commands/qa-fix.md

@@ -325,7 +325,7 @@ The validator runs 4 layers per file:
 - Cross-reference test locators against real DOM elements
 - Flag locators that don't match, auto-fix mismatches
 
-Max 3 fix loop iterations. Produces VALIDATION_REPORT.md.
+Max 5 fix loop iterations. Produces VALIDATION_REPORT.md.
 
 If `--run` flag is also present and E2E test files exist, invoke E2E runner after static validation:
 
@@ -357,20 +357,99 @@ Same as fix mode below but skip Step 4 (auto-fix). Only classify and report.
 
 ### FIX MODE (default)
 
+Fix mode runs in two phases: **analyze first, then fix after user confirmation.**
+
+#### Phase 1: Analyze & Present Plan (always runs)
+
 1. Read `CLAUDE.md` — classification rules, locator tiers, assertion quality.
-2. Invoke bug-detective agent:
+2. Invoke bug-detective agent in **classify-only mode** (no auto-fixes):
 
 Task(
   prompt="
-    <objective>Run tests, classify failures, and auto-fix TEST CODE ERRORS. Use Playwright MCP to reproduce E2E failures in the browser when available — navigate to failing pages, snapshot DOM, reproduce actions, and screenshot failure state for evidence.</objective>
+    <objective>Run tests and classify failures. Do NOT auto-fix anything yet — this is the analysis phase. Use Playwright MCP to reproduce E2E failures in the browser when available — navigate to failing pages, snapshot DOM, reproduce actions, and screenshot failure state for evidence.</objective>
     <execution_context>@agents/qaa-bug-detective.md</execution_context>
     <files_to_read>
     - CLAUDE.md
     - ~/.claude/qaa/MY_PREFERENCES.md (if exists)
     - .qa-output/locators/LOCATOR_REGISTRY.md (if exists)
+    - .qa-output/codebase/CODE_PATTERNS.md (if exists)
+    - .qa-output/codebase/API_CONTRACTS.md (if exists)
+    - .qa-output/codebase/TEST_SURFACE.md (if exists)
+    - .qa-output/codebase/TESTABILITY.md (if exists)
     </files_to_read>
     <parameters>
     user_input: $ARGUMENTS
+    mode: classify-only
+    app_url: {auto-detect from test config baseURL, or ask user}
+    </parameters>
+  "
+)
+
+3. Present the analysis to the user as a **fix plan**:
+
+```
+=== FIX PLAN ===
+
+Tests run: {N}
+Passed: {N}
+Failed: {N}
+
+Failures classified:
+  APPLICATION BUG:    {N} (will NOT be touched)
+  TEST CODE ERROR:    {N} (can auto-fix)
+  ENVIRONMENT ISSUE:  {N} (resolution steps provided)
+  INCONCLUSIVE:       {N} (needs more info)
+
+Proposed auto-fixes (TEST CODE ERRORS only):
+  1. {file}:{line} — {description of fix} [HIGH confidence]
+  2. {file}:{line} — {description of fix} [HIGH confidence]
+  3. {file}:{line} — {description of fix} [MEDIUM — flagged for review]
+
+APPLICATION BUGs found (for developer action):
+  1. {file}:{line} — {description}
+  2. {file}:{line} — {description}
+
+Proceed with auto-fixes? [yes / modify / cancel]
+================
+```
+
+4. **Wait for user confirmation.** Do NOT proceed until the user approves. This is a refinement loop — repeat until the user is satisfied:
+
+   - **"yes"** / **"proceed"** / **"dale"** → continue to Phase 2
+   - **"cancel"** / **"no"** → stop, deliver only the FAILURE_CLASSIFICATION_REPORT.md (same as --classify)
+   - **Any other response** (feedback, modifications, additions) → treat as a refinement request:
+     - Adjust the fix plan based on the user's instructions (add fixes, remove fixes, change approach, add new checks)
+     - Re-present the updated Fix Plan showing what changed
+     - Wait for user confirmation again
+     - Repeat this loop until the user says "yes" or "cancel"
+
+   **Examples of refinement requests:**
+   - "esto está bien pero también quiero que arregles los imports de utils" → add that fix to the plan
+   - "no toques el archivo de login, dejalo como está" → remove that file from the plan
+   - "cambiá el selector por getByRole en vez de getByTestId" → update the proposed fix
+   - "agregá también una validación de que el status sea 200" → add assertion fix to the plan
+
+#### Phase 2: Execute Fixes (only after user confirmation)
+
+5. Invoke bug-detective agent in **fix mode** with the confirmed plan:
+
+Task(
+  prompt="
+    <objective>Auto-fix the confirmed TEST CODE ERRORS from the analysis phase. Use Playwright MCP to reproduce E2E failures in the browser when available — navigate to failing pages, snapshot DOM, reproduce actions, and screenshot failure state for evidence.</objective>
+    <execution_context>@agents/qaa-bug-detective.md</execution_context>
+    <files_to_read>
+    - CLAUDE.md
+    - ~/.claude/qaa/MY_PREFERENCES.md (if exists)
+    - .qa-output/locators/LOCATOR_REGISTRY.md (if exists)
+    - .qa-output/codebase/CODE_PATTERNS.md (if exists)
+    - .qa-output/codebase/API_CONTRACTS.md (if exists)
+    - .qa-output/codebase/TEST_SURFACE.md (if exists)
+    - .qa-output/codebase/TESTABILITY.md (if exists)
+    - .qa-output/FAILURE_CLASSIFICATION_REPORT.md
+    </files_to_read>
+    <parameters>
+    user_input: $ARGUMENTS
+    mode: fix
     app_url: {auto-detect from test config baseURL, or ask user}
     </parameters>
   "
@@ -394,6 +473,35 @@ Task(
 - Element exists, wrong behavior → APPLICATION BUG
 - Page doesn't load → ENVIRONMENT ISSUE
 
-3. Present results. APPLICATION BUGs are reported for developer action, not auto-fixed.
+6. Present results. APPLICATION BUGs are reported for developer action, not auto-fixed.
 
 $ARGUMENTS
+
+## MANDATORY verification — run ALL commands below, no exceptions, no skipping
+
+Before returning control, copy-paste and run this ENTIRE block. Do NOT decide which commands "apply" — run all of them every time. The output confirms what happened; you do not get to assume the answer.
+
+```bash
+echo "=== QA-FIX CHECKLIST START ==="
+echo "1. Failure classification report:"
+ls .qa-output/FAILURE_CLASSIFICATION_REPORT.md 2>/dev/null || echo "NO_CLASSIFICATION_REPORT"
+echo "2. Locator Registry:"
+ls .qa-output/locators/ 2>/dev/null || echo "NO_LOCATORS_FOUND"
+echo "3. MY_PREFERENCES.md:"
+cat ~/.claude/qaa/MY_PREFERENCES.md 2>/dev/null || echo "FILE_NOT_FOUND"
+echo "4. Codebase map (context for bug-detective):"
+ls .qa-output/codebase/ 2>/dev/null || echo "NO_CODEBASE_MAP"
+echo "5. Test files in scope:"
+find tests/ cypress/ __tests__/ e2e/ spec/ -type f -name "*.spec.*" -o -name "*.test.*" -o -name "*.e2e.*" 2>/dev/null | head -20 || echo "NO_TEST_FILES_FOUND"
+echo "6. MCP evidence (if browser was used):"
+ls .qa-output/mcp-evidence/ 2>/dev/null || echo "NO_MCP_EVIDENCE"
+echo "7. Classification categories in report:"
+grep -cE "APPLICATION BUG|TEST CODE ERROR|ENVIRONMENT ISSUE|INCONCLUSIVE" .qa-output/FAILURE_CLASSIFICATION_REPORT.md 2>/dev/null || echo "NO_CLASSIFICATIONS"
+echo "=== QA-FIX CHECKLIST END ==="
+```
+
+**Rules:**
+- Run the block AS-IS. Do not modify it. Do not split it. Do not skip lines.
+- If any output shows a problem (NO_CLASSIFICATION_REPORT after fix mode completed), fix it before returning.
+- If output shows expected "not found" results (e.g., NO_MCP_EVIDENCE when no browser was used), that is fine — the point is you RAN the command instead of assuming the answer.
+- Do NOT mark this task as complete until the block has been executed and you have read every line of output.

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "qaa-agent",
-  "version": "1.8.5",
+  "version": "1.8.6",
   "description": "QA Automation Agent for Claude Code — multi-agent pipeline that analyzes repos, generates tests, validates, and creates PRs",
   "bin": {
     "qaa-agent": "./bin/install.cjs"
@@ -15,10 +15,6 @@
     "pytest",
     "ai-agent"
   ],
-  "repository": {
-    "type": "git",
-    "url": "https://github.com/Backhaus7997/qaa-testing.git"
-  },
   "author": "Backhaus7997",
   "license": "MIT",
   "dependencies": {