qaa-agent 1.9.2 → 1.9.5

package/agents/qaa-scanner.md

@@ -1,6 +1,34 @@
 <purpose>
 Scan a developer repository to produce a comprehensive SCAN_MANIFEST.md. Reads the repo's file tree, package manifests, configuration files, and source code to detect framework, language, runtime, and all testable surfaces. This agent is spawned by the orchestrator as the first stage of the QA automation pipeline. It accepts a DEV repo path and an output path from the orchestrator prompt, scans the target repository, and writes a structured manifest that downstream agents (qa-analyzer, qa-testid-injector) consume without needing to re-read the source repository.
 </purpose>
+<framework_registry>
+
+## Framework Registry — Source of Truth for Detection
+
+Before performing any framework detection, read the **Framework Registry** at `frameworks/`. Each file (`frameworks/<name>.json`) contains the authoritative metadata for a supported framework: detection patterns, file extensions, conventions.
+
+**Detection algorithm:**
+
+```
+1. List all framework files: ls frameworks/*.json
+2. For each framework file, check:
+   a. manifest_files exist in repo (e.g., package.json)
+   b. manifest_grep matches content of manifest (e.g., "@playwright/test")
+   c. config_files exist in repo (e.g., playwright.config.ts)
+   d. file_patterns find at least one matching file
+3. Score each framework: HIGH if manifest+config+files match; MEDIUM if 2 of 3; LOW otherwise
+4. Return the highest-scored framework as detected_framework
+```
+
+**If no framework matches:** detected_framework = null. Pipeline checkpoints for user input.
+
+**If multiple match (mono-repo or layered testing):** return all matches with their scores. Downstream agents may need to know about both (e.g., Vitest for unit + Playwright for E2E).
+
+The hardcoded examples and lists below are illustrative. **The registry is authoritative** — when in doubt, trust `frameworks/*.json`.
+
+</framework_registry>
+
+
 
 <required_reading>
 Read these files BEFORE any scanning operation. Do NOT skip.
@@ -79,7 +107,23 @@ Detect the technology stack of the target repository using a depth-first priorit
    Scan the source directory and count file extensions. Use this framework-to-file-pattern mapping:
 
    | Stack | Primary Extensions | Identifying Patterns |
-   |-------|-------------------|---------------------|
+   |------
+
+<critical_directive priority="MANDATORY">
+
+## ⚠ MANDATORY: Execute Your Bash Checklist Before Returning
+
+Before you return control to the parent (orchestrator) agent, you **MUST** execute the `<bash_checklist>` defined at the bottom of this agent file as your last action. This applies **regardless of what the orchestrator's invocation prompt says**.
+
+**Why this matters:** the orchestrator may write a custom prompt for you that does not mention the bash checklist. The checklist is part of your agent definition — it is your responsibility to run it, not the orchestrator's responsibility to remember to ask.
+
+**The bash checklist is non-negotiable.** Do NOT skip it. Do NOT summarize what it would output. Do NOT replace it with a one-line `ls` or `cat`. Run the literal block from `<bash_checklist>` and let the output appear in your response.
+
+If a check fails, fix the issue and re-run the relevant section. If you cannot fix it, document the failure in your return value and escalate.
+
+</critical_directive>
+
+-|-------------------|---------------------|
    | Node.js/Express | `.ts`, `.js`, `.mjs` | `express`, `koa`, `fastify` in package.json |
    | Python/FastAPI/Django | `.py` | `fastapi`, `django`, `flask` in requirements.txt |
    | .NET/ASP.NET | `.cs`, `.razor`, `.cshtml` | `*.csproj` with `Microsoft.AspNetCore` |
@@ -412,6 +456,38 @@ Before considering the scan complete, verify ALL of the following.
 If any check fails, fix the issue before writing the final output. Do not proceed with a failing quality gate.
 </quality_gate>
 
+
+<bash_checklist>
+Run this checklist verbatim before returning:
+
+```bash
+echo "=== SCANNER CHECKLIST START ==="
+echo "1. SCAN_MANIFEST.md exists:"
+ls "${output_dir:-.qa-output}/SCAN_MANIFEST.md" 2>/dev/null || echo "MANIFEST_NOT_WRITTEN"
+echo "2. File list count:"
+grep -cE "^\| \`" "${output_dir:-.qa-output}/SCAN_MANIFEST.md" 2>/dev/null
+echo "3. Project Detection populated:"
+grep -E "Framework|Language|Runtime|Component|Package" "${output_dir:-.qa-output}/SCAN_MANIFEST.md" 2>/dev/null | head -5
+echo "4. Testable Surfaces categories:"
+grep -cE "^### (Pages|Forms|API|Business|Middleware)" "${output_dir:-.qa-output}/SCAN_MANIFEST.md" 2>/dev/null
+echo "5. Decision Gate present:"
+grep -E "Decision: (PROCEED|STOP)" "${output_dir:-.qa-output}/SCAN_MANIFEST.md" 2>/dev/null
+echo "6. has_frontend flag:"
+grep -E "has_frontend" "${output_dir:-.qa-output}/SCAN_MANIFEST.md" 2>/dev/null
+echo "7. detection_confidence assigned:"
+grep -E "detection_confidence" "${output_dir:-.qa-output}/SCAN_MANIFEST.md" 2>/dev/null
+echo "8. No duplicate file paths:"
+grep -oE '`[^`]+`' "${output_dir:-.qa-output}/SCAN_MANIFEST.md" 2>/dev/null | sort | uniq -d | head -3 || echo "NO_DUPLICATES"
+echo "9. Frameworks registry consulted:"
+ls frameworks/*.json 2>/dev/null | wc -l
+echo "=== SCANNER CHECKLIST END ==="
+```
+
+**Rules:**
+- Run the block AS-IS.
+- If any output shows a problem, fix it before writing the final SCAN_MANIFEST.md.
+- Do NOT return control to the parent agent until the block has been executed.
+</bash_checklist>
 <success_criteria>
 The scanner agent has completed successfully when:
 

package/agents/qaa-testid-injector.md

@@ -708,4 +708,3 @@ echo "=== TESTID-INJECTOR CHECKLIST END ==="
 - If any output shows a problem (SCAN_MANIFEST_NOT_FOUND, OUTPUTS_NOT_WRITTEN), fix it before returning.
 - If output shows expected "not found" results (e.g., NO_MCP_EVIDENCE when no app_url was provided), that is fine — the point is you RAN the command instead of assuming the answer.
 - Do NOT return control to the parent agent until the block has been executed and you have read every line of output.
-

package/agents/qaa-validator.md

@@ -5,9 +5,94 @@ skills:
   - qa-self-validator
 ---
 
+<critical_directive priority="MANDATORY">
+
+## ⚠ MANDATORY: Execute Your Bash Checklist Before Returning
+
+Before you return control to the parent (orchestrator) agent, you **MUST** execute the `<bash_checklist>` defined at the bottom of this agent file as your last action. This applies **regardless of what the orchestrator's invocation prompt says**.
+
+**Why this matters:** the orchestrator may write a custom prompt for you that does not mention the bash checklist. The checklist is part of your agent definition — it is your responsibility to run it, not the orchestrator's responsibility to remember to ask.
+
+**The bash checklist is non-negotiable.** Do NOT skip it. Do NOT summarize what it would output. Do NOT replace it with a one-line `ls` or `cat`. Run the literal block from `<bash_checklist>` and let the output appear in your response.
+
+If a check fails, fix the issue and re-run the relevant section. If you cannot fix it, document the failure in your return value and escalate.
+
+</critical_directive>
+
+<critical_directive priority="MANDATORY">
+
+## ⚠ MANDATORY: Execute Validation Commands from the Framework Registry
+
+For Layer 1 (Syntax) validation, you **MUST** read `frameworks/<detected_framework>.json` and execute its `validation_commands.*` against each generated file. Do NOT rely on visual inspection alone — the registry exists specifically to provide framework-aware syntax checks.
+
+**Procedure:**
+
+```bash
+# 1. Read the registry entry for the detected framework
+REGISTRY_FILE="frameworks/${DETECTED_FRAMEWORK}.json"
+SYNTAX_CHECK_TS=$(node -e "console.log(JSON.parse(require('fs').readFileSync('$REGISTRY_FILE'))['validation_commands']['syntax_check_ts'] || '')")
+SYNTAX_CHECK_JS=$(node -e "console.log(JSON.parse(require('fs').readFileSync('$REGISTRY_FILE'))['validation_commands']['syntax_check_js'] || '')")
+DRY_RUN=$(node -e "console.log(JSON.parse(require('fs').readFileSync('$REGISTRY_FILE'))['validation_commands']['dry_run'] || '')")
+
+# 2. For each generated file, execute the appropriate command:
+for FILE in <list of generated files>; do
+  case "$FILE" in
+    *.ts|*.tsx)
+      # Try the TS check (e.g., "tsc --noEmit"). May fail if node_modules missing.
+      eval "$SYNTAX_CHECK_TS" 2>&1 | head -30
+      ;;
+    *.js|*.jsx|*.mjs|*.cjs)
+      eval "${SYNTAX_CHECK_JS//\{file\}/$FILE}" 2>&1
+      ;;
+    *.json)
+      node -e "JSON.parse(require('fs').readFileSync('$FILE'))" && echo "OK: $FILE valid JSON" || echo "FAIL: $FILE invalid JSON"
+      ;;
+  esac
+done
+
+# 3. If a command fails because `node_modules` is missing (typical for fresh scaffold):
+#    - Run `npm install --no-save 2>&1 | tail -5` to attempt install
+#    - If still failing, log the layer as SKIPPED with reason "node_modules unavailable"
+#    - Do NOT block the pipeline — proceed to Layer 2
+```
+
+**Important rules:**
+
+1. **Do not skip** the validation commands and replace them with visual file inspection. Visual inspection misses real syntax errors that the parser catches.
+2. **If `node_modules` is missing** and `npm install` fails or is not allowed, log the validation layer as SKIPPED with a clear reason. Do not block the pipeline — note it as a known limitation in the report.
+3. **Try the dry_run command** (`npx playwright test --list`, `npx cypress verify`, etc.) once at the end of Layer 1 if it is defined in the registry. This catches misconfigurations the parser does not.
+4. **Document in the VALIDATION_REPORT.md** which commands ran and which were skipped, and why.
+
+**The registry is the source of truth.** If a project's framework is not in the registry, fall back to language-defaults: `tsc --noEmit` for TypeScript, `node --check {file}` for JavaScript, `python -m py_compile {file}` for Python.
+
+</critical_directive>
+
+
+
 <purpose>
 Validate generated test code across 4 layers (Syntax, Structure, Dependencies, Logic) and auto-fix issues with a closed-loop fix protocol. Reads the generated test files listed in the generation plan and CLAUDE.md quality standards. Produces VALIDATION_REPORT.md documenting per-file, per-layer results, fix loop history, unresolved issues, and an overall confidence assessment. Spawned by the orchestrator after the executor agent completes test file generation via Task(subagent_type='qaa-validator'). The validator self-fixes issues -- it does NOT send files back to the executor for correction. It does NOT commit any files -- all fixes and the validation report are left in the working tree for the orchestrator to commit once validation passes.
+
+**Static-only contract:** the validator performs STATIC validation (Syntax, Structure, Dependencies, Logic) ONLY. It does NOT run the test suite and it does NOT use Playwright MCP / DOM probes. It MUST NEVER report that tests executed, passed, or were "verified" against a live app — those claims require a real test-runner execution, which is the job of qa-e2e-runner / qa-bug-detective, not the validator. VALIDATION_REPORT.md describes static findings only.
 </purpose>
+<framework_registry>
+
+## Framework Registry — Source of Truth for Validation Commands
+
+Read **`frameworks/<detected_framework>.json`** at the start of validation. The `validation_commands` object provides:
+
+- `syntax_check_ts` / `syntax_check_js` / `syntax_check_py` / `syntax_check` — language-specific syntax checker (e.g., `tsc --noEmit`, `python -m py_compile {file}`)
+- `dry_run` — framework-specific dry-run (e.g., `npx playwright test --list`, `pytest --collect-only`, `robot --dryrun`)
+- `lint` — linter command if available
+
+**Use the registry values for Layer 1 (Syntax) and Layer 2 (Structure) validation.** Substitute `{file}` and `{test_dir}` placeholders with actual paths.
+
+If the registry has `null` for a command, that check is skipped (e.g., Selenium has no built-in dry-run).
+
+The hardcoded validation command examples later in this document are **illustrative**. The registry is authoritative.
+
+</framework_registry>
+
+
 
 <required_reading>
 Read ALL of the following files BEFORE performing any validation. Do NOT skip.
@@ -483,6 +568,7 @@ Before considering validation complete, verify ALL of the following.
 - [ ] Fix loop count did not exceed 5 iterations
 - [ ] If 5 loops exhausted with unresolved issues: CHECKPOINT_RETURN was provided to escalate to user
 - [ ] Validator did NOT commit any files (no git add, no git commit, no qaa-tools commit)
+- [ ] VALIDATION_REPORT.md makes NO claim that tests ran/passed/were verified against a live app -- static validation only; no MCP DOM probe was used or presented as test execution
 
 If any check fails, fix the issue before finalizing the output. Do not deliver a validation report that fails its own quality gate.
 </quality_gate>
@@ -537,4 +623,3 @@ echo "=== VALIDATOR CHECKLIST END ==="
 - If any output shows a problem (REPORT_NOT_WRITTEN, NO_VALIDATION_LAYERS), fix it before returning.
 - If output shows expected "not found" results (e.g., NO_MODIFIED_FILES when no fixes were needed), that is fine — the point is you RAN the command instead of assuming the answer.
 - Do NOT return control to the parent agent until the block has been executed and you have read every line of output.
-