qaa-agent 1.6.2 → 1.7.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (78) hide show
  1. package/.mcp.json +8 -8
  2. package/CHANGELOG.md +93 -71
  3. package/CLAUDE.md +553 -553
  4. package/agents/qa-pipeline-orchestrator.md +1378 -1378
  5. package/agents/qaa-analyzer.md +539 -524
  6. package/agents/qaa-bug-detective.md +479 -446
  7. package/agents/qaa-codebase-mapper.md +935 -935
  8. package/agents/qaa-discovery.md +384 -0
  9. package/agents/qaa-e2e-runner.md +416 -415
  10. package/agents/qaa-executor.md +651 -651
  11. package/agents/qaa-planner.md +405 -390
  12. package/agents/qaa-project-researcher.md +319 -319
  13. package/agents/qaa-scanner.md +424 -424
  14. package/agents/qaa-testid-injector.md +643 -585
  15. package/agents/qaa-validator.md +490 -452
  16. package/bin/install.cjs +200 -198
  17. package/bin/lib/commands.cjs +709 -709
  18. package/bin/lib/config.cjs +307 -307
  19. package/bin/lib/core.cjs +497 -497
  20. package/bin/lib/frontmatter.cjs +299 -299
  21. package/bin/lib/init.cjs +989 -989
  22. package/bin/lib/milestone.cjs +241 -241
  23. package/bin/lib/model-profiles.cjs +60 -60
  24. package/bin/lib/phase.cjs +911 -911
  25. package/bin/lib/roadmap.cjs +306 -306
  26. package/bin/lib/state.cjs +748 -748
  27. package/bin/lib/template.cjs +222 -222
  28. package/bin/lib/verify.cjs +842 -842
  29. package/bin/qaa-tools.cjs +607 -607
  30. package/commands/qa-audit.md +119 -0
  31. package/commands/qa-create-test.md +288 -0
  32. package/commands/qa-fix.md +147 -0
  33. package/commands/qa-map.md +137 -0
  34. package/{.claude/commands → commands}/qa-pr.md +23 -23
  35. package/{.claude/commands → commands}/qa-start.md +22 -22
  36. package/{.claude/commands → commands}/qa-testid.md +19 -19
  37. package/docs/COMMANDS.md +341 -341
  38. package/docs/DEMO.md +182 -182
  39. package/docs/TESTING.md +156 -156
  40. package/package.json +6 -7
  41. package/{.claude/settings.json → settings.json} +1 -2
  42. package/templates/failure-classification.md +391 -391
  43. package/templates/gap-analysis.md +409 -409
  44. package/templates/pr-template.md +48 -48
  45. package/templates/qa-analysis.md +381 -381
  46. package/templates/qa-audit-report.md +465 -465
  47. package/templates/qa-repo-blueprint.md +636 -636
  48. package/templates/scan-manifest.md +312 -312
  49. package/templates/test-inventory.md +582 -582
  50. package/templates/testid-audit-report.md +354 -354
  51. package/templates/validation-report.md +243 -243
  52. package/workflows/qa-analyze.md +296 -296
  53. package/workflows/qa-from-ticket.md +536 -536
  54. package/workflows/qa-gap.md +309 -303
  55. package/workflows/qa-pr.md +389 -389
  56. package/workflows/qa-start.md +1192 -1168
  57. package/workflows/qa-testid.md +384 -356
  58. package/workflows/qa-validate.md +299 -295
  59. package/.claude/commands/create-test.md +0 -164
  60. package/.claude/commands/qa-audit.md +0 -37
  61. package/.claude/commands/qa-blueprint.md +0 -54
  62. package/.claude/commands/qa-fix.md +0 -36
  63. package/.claude/commands/qa-from-ticket.md +0 -24
  64. package/.claude/commands/qa-gap.md +0 -20
  65. package/.claude/commands/qa-map.md +0 -47
  66. package/.claude/commands/qa-pom.md +0 -36
  67. package/.claude/commands/qa-pyramid.md +0 -37
  68. package/.claude/commands/qa-report.md +0 -38
  69. package/.claude/commands/qa-research.md +0 -33
  70. package/.claude/commands/qa-validate.md +0 -42
  71. package/.claude/commands/update-test.md +0 -58
  72. package/.claude/skills/qa-learner/SKILL.md +0 -150
  73. /package/{.claude/skills → skills}/qa-bug-detective/SKILL.md +0 -0
  74. /package/{.claude/skills → skills}/qa-repo-analyzer/SKILL.md +0 -0
  75. /package/{.claude/skills → skills}/qa-self-validator/SKILL.md +0 -0
  76. /package/{.claude/skills → skills}/qa-template-engine/SKILL.md +0 -0
  77. /package/{.claude/skills → skills}/qa-testid-injector/SKILL.md +0 -0
  78. /package/{.claude/skills → skills}/qa-workflow-documenter/SKILL.md +0 -0
@@ -0,0 +1,119 @@
1
+ # QA Audit & Report
2
+
3
+ Comprehensive quality audit of a test suite with 6-dimension scoring, testing pyramid analysis, and status reporting. Supports three output modes: full audit, pyramid analysis only, or status report adapted to audience.
4
+
5
+ ## Usage
6
+
7
+ ```
8
+ /qa-audit <path-to-tests> [options]
9
+ ```
10
+
11
+ ### Options
12
+
13
+ - `<path-to-tests>` — directory containing test files to audit
14
+ - `--dev-repo <path>` — path to developer repository (for coverage cross-reference)
15
+ - `--app-url <url>` — URL of running application for locator verification via Playwright MCP
16
+ - `--pyramid` — pyramid analysis only: compare actual vs target distribution with action plan
17
+ - `--report [team|management|client]` — generate status report adapted to audience (default: team)
18
+
19
+ ### Mode Detection
20
+
21
+ ```
22
+ if --pyramid:
23
+ MODE = "pyramid" → PYRAMID_ANALYSIS.md
24
+ elif --report:
25
+ MODE = "report" → QA_STATUS_REPORT.md (adapted to audience)
26
+ else:
27
+ MODE = "audit" → QA_AUDIT_REPORT.md (full 6-dimension audit, default)
28
+ ```
29
+
30
+ ## What It Produces
31
+
32
+ | Mode | Artifact | Description |
33
+ |------|----------|-------------|
34
+ | audit | QA_AUDIT_REPORT.md | 6-dimension scoring, critical issues, recommendations with effort estimates |
35
+ | pyramid | PYRAMID_ANALYSIS.md | Current vs target distribution, gap table, prioritized action plan |
36
+ | report | QA_STATUS_REPORT.md | Metrics, pyramid distribution, risk areas, adapted to audience level |
37
+
38
+ ## Instructions
39
+
40
+ ### AUDIT MODE (default)
41
+
42
+ Scores across 6 dimensions: Locator Quality (20%), Assertion Specificity (20%), POM Compliance (15%), Test Coverage (20%), Naming Convention (15%), Test Data Management (10%).
43
+
44
+ 1. Read `CLAUDE.md` — quality gates, locator tiers, assertion rules, POM rules, naming conventions.
45
+ 2. Invoke validator agent in audit mode:
46
+
47
+ Task(
48
+ prompt="
49
+ <objective>Audit test suite quality and produce QA_AUDIT_REPORT.md with 6-dimension scoring. If Playwright MCP is connected and an app URL is available, verify E2E test locators against the live DOM via browser_navigate + browser_snapshot. Flag stale locators (Tier 4 CSS/XPath that could be upgraded to Tier 1 data-testid) and locators that no longer match any DOM element.</objective>
50
+ <execution_context>@agents/qaa-validator.md</execution_context>
51
+ <files_to_read>
52
+ - CLAUDE.md
53
+ </files_to_read>
54
+ <parameters>
55
+ user_input: $ARGUMENTS
56
+ mode: audit
57
+ app_url: {auto-detect from test config baseURL, or ask user}
58
+ </parameters>
59
+ "
60
+ )
61
+
62
+ 3. Present results with overall score and prioritized recommendations.
63
+
64
+ ---
65
+
66
+ ### PYRAMID MODE (`--pyramid`)
67
+
68
+ Analyze test distribution against the ideal testing pyramid from CLAUDE.md (Unit 60-70%, Integration 10-15%, API 20-25%, E2E 3-5%). Compares actual percentages to targets and produces an action plan.
69
+
70
+ 1. Read `CLAUDE.md` — testing pyramid target percentages.
71
+ 2. Invoke analyzer agent for pyramid analysis:
72
+
73
+ Task(
74
+ prompt="
75
+ <objective>Produce PYRAMID_ANALYSIS.md comparing actual test distribution to target pyramid. Count tests by type (unit, integration, API, E2E), calculate percentages, compare to CLAUDE.md targets, identify gaps, and produce a prioritized action plan to reach the recommended distribution. Adjust target percentages based on the actual app architecture.</objective>
76
+ <execution_context>@agents/qaa-analyzer.md</execution_context>
77
+ <files_to_read>
78
+ - CLAUDE.md
79
+ </files_to_read>
80
+ <parameters>
81
+ user_input: $ARGUMENTS
82
+ mode: pyramid-analysis
83
+ </parameters>
84
+ "
85
+ )
86
+
87
+ 3. Present analysis with gap table and action plan.
88
+
89
+ ---
90
+
91
+ ### REPORT MODE (`--report`)
92
+
93
+ Generate a summary report of current QA status. Adapts detail level to audience.
94
+
95
+ **Audience levels:**
96
+ - `team` (default) — file-level details, specific locator/assertion issues, technical recommendations
97
+ - `management` — high-level metrics, risk areas, coverage percentages, trend indicators
98
+ - `client` — coverage summary, confidence level, test pass rates, risk mitigation status
99
+
100
+ 1. Read `CLAUDE.md` — testing pyramid targets, quality gates.
101
+ 2. Invoke analyzer agent for status reporting:
102
+
103
+ Task(
104
+ prompt="
105
+ <objective>Produce QA_STATUS_REPORT.md with current test suite metrics and coverage. Adapt detail level to the specified audience: team (file-level technical details), management (high-level metrics and risks), or client (coverage summary and confidence). Include testing pyramid distribution, pass/fail rates, risk areas, and actionable recommendations appropriate for the audience.</objective>
106
+ <execution_context>@agents/qaa-analyzer.md</execution_context>
107
+ <files_to_read>
108
+ - CLAUDE.md
109
+ </files_to_read>
110
+ <parameters>
111
+ user_input: $ARGUMENTS
112
+ mode: status-report
113
+ </parameters>
114
+ "
115
+ )
116
+
117
+ 3. Present report to user.
118
+
119
+ $ARGUMENTS
@@ -0,0 +1,288 @@
1
+ # QA Create Test
2
+
3
+ Create, update, or generate tests from tickets — all in one command. Supports three modes: generate tests from code analysis, generate tests from a ticket (Jira/Linear/GitHub), or update/improve existing tests. Uses Playwright MCP to extract real locators from the live app when available.
4
+
5
+ ## Usage
6
+
7
+ ```
8
+ /qa-create-test <feature-or-source> [options]
9
+ ```
10
+
11
+ ### Modes (auto-detected from arguments)
12
+
13
+ | Mode | Trigger | Example |
14
+ |------|---------|---------|
15
+ | **From code** | Feature name (no URL, no path to tests) | `/qa-create-test login` |
16
+ | **From ticket** | URL, shorthand (#123), or `--ticket` flag | `/qa-create-test https://github.com/org/repo/issues/42` |
17
+ | **Update existing** | Path to existing test files or `--update` flag | `/qa-create-test --update tests/e2e/` |
18
+ | **POM only** | `--pom-only` flag | `/qa-create-test --pom-only src/pages/` |
19
+
20
+ ### Options
21
+
22
+ - `--dev-repo <path>` — path to developer repository (default: current directory)
23
+ - `--app-url <url>` — URL of running application for E2E execution and locator extraction (auto-detects if not provided)
24
+ - `--skip-run` — skip E2E execution, only generate and statically validate
25
+ - `--ticket <source>` — force ticket mode with: URL, shorthand (#123, org/repo#123), file path, or plain text
26
+ - `--update <path>` — force update mode: audit and improve existing tests at path
27
+ - `--scope fix|improve|add|full` — for update mode only (default: full)
28
+ - `--pom-only [path]` — generate only Page Object Model files (BasePage + feature POMs), no test specs
29
+ - `--framework <name>` — override framework auto-detection (playwright, cypress, selenium) — used with --pom-only
30
+
31
+ ### Mode Detection Logic
32
+
33
+ ```
34
+ if --pom-only:
35
+ MODE = "pom-only"
36
+ elif argument matches URL pattern ...
37
+ if argument matches URL pattern (github.com, atlassian.net, linear.app) OR contains "#" + digits OR --ticket flag:
38
+ MODE = "from-ticket"
39
+ elif --update flag OR argument is path to existing test directory/files:
40
+ MODE = "update"
41
+ else:
42
+ MODE = "from-code"
43
+ ```
44
+
45
+ ## What It Produces
46
+
47
+ ### From Code Mode
48
+ - Test spec files (unit, API, E2E as appropriate)
49
+ - Page Object Model files (for E2E tests)
50
+ - Fixture files (test data)
51
+ - Locator registry entries (`.qa-output/locators/`)
52
+ - E2E_RUN_REPORT.md (if E2E tests ran against live app)
53
+
54
+ ### From Ticket Mode
55
+ - TEST_CASES_FROM_TICKET.md — traceability matrix (AC → test case)
56
+ - GENERATION_PLAN_TICKET.md — synthetic generation plan
57
+ - Test spec files with `traces_to` fields linking back to ticket ACs
58
+ - VALIDATION_REPORT.md
59
+
60
+ ### Update Mode
61
+ - QA_AUDIT_REPORT.md — current quality assessment
62
+ - Improved test files (after user approval)
63
+
64
+ ## Instructions
65
+
66
+ ### Step 1: Detect Mode
67
+
68
+ Parse `$ARGUMENTS` to determine mode using the detection logic above.
69
+
70
+ Print mode banner:
71
+ ```
72
+ === QA Create Test ===
73
+ Mode: {from-code | from-ticket | update}
74
+ Target: {feature name | ticket URL | test path}
75
+ App URL: {url or "auto-detect"}
76
+ ===========================
77
+ ```
78
+
79
+ ---
80
+
81
+ ### FROM CODE MODE
82
+
83
+ 1. Read `CLAUDE.md` — POM rules, locator tiers, assertion rules, naming conventions, quality gates.
84
+ 2. Read existing analysis artifacts if available:
85
+ - `.qa-output/QA_ANALYSIS.md` — architecture context
86
+ - `.qa-output/TEST_INVENTORY.md` — pre-defined test cases for this feature
87
+ 3. **Check for codebase map** (`.qa-output/codebase/`):
88
+ - Look for: `CODE_PATTERNS.md`, `API_CONTRACTS.md`, `TEST_SURFACE.md`, `TESTABILITY.md`
89
+ - If at least 2 of these files exist: read them all for project context (naming conventions, API shapes, testable surfaces).
90
+ - **If NONE of these files exist: STOP and tell the user:**
91
+ ```
92
+ ⚠ No codebase map found (.qa-output/codebase/ is empty or missing).
93
+
94
+ The codebase map provides critical context: naming conventions, API contracts,
95
+ testable surfaces, and project structure. Without it, generated tests will lack
96
+ project-specific context and may not follow your repo's conventions.
97
+
98
+ Run /qa-map first to generate the codebase map, then re-run /qa-create-test.
99
+
100
+ To skip this check and proceed without context: re-run with --skip-map
101
+ ```
102
+ Only proceed without codebase map if the user explicitly passes `--skip-map`.
103
+
104
+ 4. **Check existing locator registry and extract new locators from live app:**
105
+
106
+ a. Read `.qa-output/locators/LOCATOR_REGISTRY.md` if it exists.
107
+
108
+ b. If locators for this feature already exist in the registry AND no `--app-url` was provided: reuse cached locators.
109
+
110
+ c. If locators are missing or `--app-url` was provided: Use Playwright MCP to navigate the app and extract real locators:
111
+ ```
112
+ mcp__playwright__browser_navigate({ url: "{app_url}/{feature_path}" })
113
+ mcp__playwright__browser_snapshot()
114
+ ```
115
+ Extract all data-testid, ARIA roles, labels, placeholders.
116
+ Navigate through multi-page flows if needed.
117
+ Write per-feature locator file to `.qa-output/locators/{feature}.locators.md`.
118
+ Update the registry `.qa-output/locators/LOCATOR_REGISTRY.md`.
119
+
120
+ If no app URL available and no locators in registry, skip — executor proposes locators from source code.
121
+
122
+ 5. Invoke executor agent to generate test files:
123
+
124
+ Task(
125
+ prompt="
126
+ <objective>Generate test files for the specified feature following CLAUDE.md standards, using codebase map for context</objective>
127
+ <execution_context>@agents/qaa-executor.md</execution_context>
128
+ <files_to_read>
129
+ - CLAUDE.md
130
+ - .qa-output/locators/LOCATOR_REGISTRY.md (if exists)
131
+ - .qa-output/locators/{feature}.locators.md (if exists)
132
+ - .qa-output/codebase/CODE_PATTERNS.md (if exists)
133
+ - .qa-output/codebase/API_CONTRACTS.md (if exists)
134
+ - .qa-output/codebase/TEST_SURFACE.md (if exists)
135
+ - .qa-output/codebase/TESTABILITY.md (if exists)
136
+ </files_to_read>
137
+ <parameters>
138
+ user_input: $ARGUMENTS
139
+ mode: feature-test
140
+ codebase_map_dir: .qa-output/codebase
141
+ locator_registry: .qa-output/locators/LOCATOR_REGISTRY.md
142
+ </parameters>
143
+ "
144
+ )
145
+
146
+ 6. If E2E test files were generated AND `--skip-run` was NOT passed, invoke E2E runner:
147
+
148
+ Task(
149
+ prompt="
150
+ <objective>Run generated E2E tests against live application, capture real locators, fix mismatches, loop until pass</objective>
151
+ <execution_context>@agents/qaa-e2e-runner.md</execution_context>
152
+ <files_to_read>
153
+ - CLAUDE.md
154
+ - {generated E2E test files from executor return}
155
+ - {generated POM files from executor return}
156
+ </files_to_read>
157
+ <parameters>
158
+ app_url: {from --app-url flag or auto-detect}
159
+ output_dir: .qa-output
160
+ </parameters>
161
+ "
162
+ )
163
+
164
+ 7. Present results with file counts and suggest `/qa-pr`.
165
+
166
+ ---
167
+
168
+ ### FROM TICKET MODE
169
+
170
+ 1. Read `CLAUDE.md` — all QA standards.
171
+ 2. Execute the ticket workflow end-to-end:
172
+
173
+ Follow the workflow defined in `@workflows/qa-from-ticket.md` end-to-end.
174
+ Preserve all workflow gates (ticket parsing, acceptance criteria extraction, traceability matrix, validation).
175
+
176
+ Key steps in the workflow:
177
+ - Parse ticket source (GitHub URL, Jira URL, Linear URL, file, or plain text)
178
+ - Fetch ticket content (via `gh issue view`, WebFetch, or file read)
179
+ - Extract acceptance criteria, user stories, edge cases
180
+ - Scan dev repo for related source files
181
+ - Extract locators from live app via Playwright MCP (if app URL available)
182
+ - Generate test cases with traceability matrix (every AC maps to ≥1 test case)
183
+ - Spawn executor agent to produce test files
184
+ - Spawn validator agent for 4-layer validation
185
+ - Print summary with AC coverage and traceability
186
+
187
+ **Traceability guarantee:** Every acceptance criterion maps to at least one test case via the `traces_to` field.
188
+
189
+ **Supported ticket sources:**
190
+
191
+ | Format | Example | Detection |
192
+ |--------|---------|-----------|
193
+ | GitHub Issue URL | `https://github.com/org/repo/issues/123` | Contains `github.com` + `/issues/` |
194
+ | GitHub shorthand | `org/repo#123` or `#123` | Contains `#` + digits |
195
+ | Jira URL | `https://company.atlassian.net/browse/PROJ-123` | Contains `.atlassian.net/browse/` |
196
+ | Linear URL | `https://linear.app/team/issue/TEAM-123` | Contains `linear.app` |
197
+ | File path | `./tickets/feature-spec.md` | Path exists on disk |
198
+ | Plain text | `"As a user I want to..."` | None of the above match |
199
+
200
+ ---
201
+
202
+ ### UPDATE MODE
203
+
204
+ 1. Read `CLAUDE.md` — quality gates, locator tiers, assertion rules, POM rules.
205
+ 2. Invoke validator agent in audit mode:
206
+
207
+ Task(
208
+ prompt="
209
+ <objective>Audit existing test quality and produce QA_AUDIT_REPORT.md. If Playwright MCP is connected, verify E2E test locators against the live DOM via browser_navigate + browser_snapshot.</objective>
210
+ <execution_context>@agents/qaa-validator.md</execution_context>
211
+ <files_to_read>
212
+ - CLAUDE.md
213
+ </files_to_read>
214
+ <parameters>
215
+ user_input: $ARGUMENTS
216
+ mode: audit
217
+ app_url: {auto-detect from test config baseURL, or ask user}
218
+ </parameters>
219
+ "
220
+ )
221
+
222
+ 3. Present audit results and wait for user approval.
223
+ 4. Invoke executor agent to apply approved improvements:
224
+
225
+ Task(
226
+ prompt="
227
+ <objective>Apply approved improvements to existing tests without deleting working tests. If Playwright MCP is connected, use browser_navigate + browser_snapshot to extract real locators when upgrading from Tier 4 to Tier 1.</objective>
228
+ <execution_context>@agents/qaa-executor.md</execution_context>
229
+ <files_to_read>
230
+ - CLAUDE.md
231
+ - .qa-output/QA_AUDIT_REPORT.md
232
+ - .qa-output/locators/LOCATOR_REGISTRY.md (if exists)
233
+ </files_to_read>
234
+ <parameters>
235
+ user_input: $ARGUMENTS
236
+ mode: update
237
+ app_url: {auto-detect from test config baseURL, or ask user}
238
+ </parameters>
239
+ "
240
+ )
241
+
242
+ **Update scopes:**
243
+ - `fix` — repair broken tests only
244
+ - `improve` — upgrade locators, assertions, POM structure
245
+ - `add` — add missing test cases without modifying existing
246
+ - `full` — audit everything, then improve with approval (default)
247
+
248
+ **Rule:** NEVER delete or rewrite working tests without user approval. Surgical: add, fix, improve — never replace.
249
+
250
+ ---
251
+
252
+ ### POM ONLY MODE (`--pom-only`)
253
+
254
+ Generate only Page Object Model files — no test specs.
255
+
256
+ 1. Read `CLAUDE.md` — POM rules, locator tier hierarchy, naming conventions.
257
+ 2. Invoke executor agent in POM-only mode:
258
+
259
+ Task(
260
+ prompt="
261
+ <objective>Generate Page Object Models following CLAUDE.md POM rules. If Playwright MCP is connected and an app URL is available, navigate each page first to extract real locators (data-testid, ARIA roles, labels) from the live DOM via browser_navigate + browser_snapshot before generating POMs. This ensures POM locators match the real app instead of guessing from source code.</objective>
262
+ <execution_context>@agents/qaa-executor.md</execution_context>
263
+ <files_to_read>
264
+ - CLAUDE.md
265
+ - .qa-output/locators/LOCATOR_REGISTRY.md (if exists)
266
+ </files_to_read>
267
+ <parameters>
268
+ user_input: $ARGUMENTS
269
+ mode: pom-only
270
+ app_url: {auto-detect from test config baseURL, or ask user}
271
+ </parameters>
272
+ "
273
+ )
274
+
275
+ **Produces:**
276
+ - BasePage file (if not already present)
277
+ - Feature-specific POM files following `[PageName]Page.[ext]` naming convention
278
+ - No test specs, no fixtures
279
+
280
+ **POM rules enforced:**
281
+ - One class per page — no god objects
282
+ - No assertions in page objects — assertions belong ONLY in test specs
283
+ - Locators as readonly properties — Tier 1 preferred (data-testid, ARIA roles)
284
+ - Actions return void or next page — for fluent chaining
285
+ - State queries return data — let the test decide what to assert
286
+ - Every POM extends BasePage
287
+
288
+ $ARGUMENTS
@@ -0,0 +1,147 @@
1
+ # QA Fix & Validate
2
+
3
+ Validate, diagnose, and fix test files — all in one command. Runs 4-layer static validation (syntax, structure, dependencies, logic), classifies failures, and auto-fixes TEST CODE ERRORS. Uses Playwright MCP to reproduce E2E failures and verify locators against the live app when available. Never touches application code.
4
+
5
+ ## Usage
6
+
7
+ ```
8
+ /qa-fix [<test-directory>] [options]
9
+ ```
10
+
11
+ ### Options
12
+
13
+ - `<test-directory>` — path to test files (auto-detects if omitted)
14
+ - `--validate-only` — run 4-layer static validation only, no test execution or classification
15
+ - `--classify` — run tests and classify failures, but do NOT auto-fix
16
+ - `--run --app-url <url>` — also execute E2E tests against live app after static validation
17
+ - `--app-url <url>` — URL of running application (auto-detects if not provided)
18
+ - `[error output]` — paste test runner output directly (skips running tests, classifies the pasted output)
19
+
20
+ ### Mode Detection
21
+
22
+ ```
23
+ if --validate-only:
24
+ MODE = "validate" → 4-layer static validation + VALIDATION_REPORT.md
25
+ elif --classify:
26
+ MODE = "classify" → run tests + classify failures (no auto-fix)
27
+ else:
28
+ MODE = "fix" → run tests + classify + auto-fix TEST CODE ERRORS (default)
29
+ ```
30
+
31
+ ## What It Produces
32
+
33
+ | Mode | Artifacts |
34
+ |------|-----------|
35
+ | validate | VALIDATION_REPORT.md (syntax, structure, dependencies, logic per file) |
36
+ | classify | FAILURE_CLASSIFICATION_REPORT.md (per-failure evidence, no fixes) |
37
+ | fix | FAILURE_CLASSIFICATION_REPORT.md + auto-fixed test files |
38
+
39
+ ## Instructions
40
+
41
+ ### Step 1: Detect Mode and Test Directory
42
+
43
+ Parse `$ARGUMENTS` for mode flags and test directory path.
44
+
45
+ If no test directory provided, auto-detect:
46
+ - Check for `tests/`, `cypress/`, `__tests__/`, `e2e/`, `spec/` directories
47
+ - Check test config files for `testDir` setting
48
+
49
+ Print banner:
50
+ ```
51
+ === QA Fix & Validate ===
52
+ Mode: {validate | classify | fix}
53
+ Test Directory: {path}
54
+ App URL: {url or "auto-detect"}
55
+ ==========================
56
+ ```
57
+
58
+ ---
59
+
60
+ ### VALIDATE MODE (`--validate-only`)
61
+
62
+ 1. Read `CLAUDE.md` — quality gates, locator tiers, assertion rules.
63
+ 2. Execute static validation workflow:
64
+
65
+ Follow the workflow defined in `@workflows/qa-validate.md` end-to-end.
66
+ Preserve all workflow gates (fix loops, layer checks).
67
+
68
+ The validator runs 4 layers per file:
69
+ 1. **Syntax** — code compiles without errors
70
+ 2. **Structure** — naming, folders, POM compliance
71
+ 3. **Dependencies** — all imports resolve
72
+ 4. **Logic** — concrete assertions, Tier 1-2 locators, no assertions in POMs
73
+
74
+ **Optional Layer 5 (if Playwright MCP connected + app URL available):**
75
+ - Navigate to each page referenced in E2E tests via `mcp__playwright__browser_navigate`
76
+ - Take accessibility snapshot via `mcp__playwright__browser_snapshot`
77
+ - Cross-reference test locators against real DOM elements
78
+ - Flag locators that don't match, auto-fix mismatches
79
+
80
+ Max 3 fix loop iterations. Produces VALIDATION_REPORT.md.
81
+
82
+ If `--run` flag is also present and E2E test files exist, invoke E2E runner after static validation:
83
+
84
+ Task(
85
+ prompt="
86
+ <objective>Run E2E tests against live application, capture real locators, fix mismatches, loop until pass</objective>
87
+ <execution_context>@agents/qaa-e2e-runner.md</execution_context>
88
+ <files_to_read>
89
+ - CLAUDE.md
90
+ - {E2E test files from validated directory}
91
+ - {POM files from validated directory}
92
+ </files_to_read>
93
+ <parameters>
94
+ app_url: {from --app-url flag or auto-detect}
95
+ output_dir: .qa-output
96
+ </parameters>
97
+ "
98
+ )
99
+
100
+ ---
101
+
102
+ ### CLASSIFY MODE (`--classify`)
103
+
104
+ Same as fix mode below but skip Step 4 (auto-fix). Only classify and report.
105
+
106
+ ---
107
+
108
+ ### FIX MODE (default)
109
+
110
+ 1. Read `CLAUDE.md` — classification rules, locator tiers, assertion quality.
111
+ 2. Invoke bug-detective agent:
112
+
113
+ Task(
114
+ prompt="
115
+ <objective>Run tests, classify failures, and auto-fix TEST CODE ERRORS. Use Playwright MCP to reproduce E2E failures in the browser when available — navigate to failing pages, snapshot DOM, reproduce actions, and screenshot failure state for evidence.</objective>
116
+ <execution_context>@agents/qaa-bug-detective.md</execution_context>
117
+ <files_to_read>
118
+ - CLAUDE.md
119
+ </files_to_read>
120
+ <parameters>
121
+ user_input: $ARGUMENTS
122
+ app_url: {auto-detect from test config baseURL, or ask user}
123
+ </parameters>
124
+ "
125
+ )
126
+
127
+ **Classification categories:**
128
+ - **APPLICATION BUG** — error in production code → Report only, NEVER auto-fix
129
+ - **TEST CODE ERROR** — error in test code → Auto-fix if HIGH confidence
130
+ - **ENVIRONMENT ISSUE** — missing env, connection refused → Report with resolution steps
131
+ - **INCONCLUSIVE** — ambiguous → Report what's known, ask for more info
132
+
133
+ **Auto-fix rules:**
134
+ - Only TEST CODE ERROR at HIGH confidence
135
+ - Allowed fixes: import paths, selectors, assertion values, config, missing await, fixture paths
136
+ - Every fix verified by re-running the specific test
137
+ - Never modify application code (src/, app/, lib/)
138
+
139
+ **Browser reproduction (when Playwright MCP connected):**
140
+ - Navigate to failing page → snapshot DOM → reproduce action → screenshot
141
+ - Element not in DOM → TEST CODE ERROR (HIGH confidence)
142
+ - Element exists, wrong behavior → APPLICATION BUG
143
+ - Page doesn't load → ENVIRONMENT ISSUE
144
+
145
+ 3. Present results. APPLICATION BUGs are reported for developer action, not auto-fixed.
146
+
147
+ $ARGUMENTS
@@ -0,0 +1,137 @@
1
+ # QA Codebase Map & Analysis
2
+
3
+ Deep-scan a codebase for QA-relevant information, produce a complete analysis, and generate test inventory. Runs codebase mapping (4 parallel agents) followed by full repository analysis. One command to fully understand a codebase before writing tests.
4
+
5
+ ## Usage
6
+
7
+ ```
8
+ /qa-map [options]
9
+ ```
10
+
11
+ ### Options
12
+
13
+ - No arguments — runs full map + analysis on current directory
14
+ - `--focus <area>` — run a single map area only, skip analysis (testability, risk, patterns, existing-tests)
15
+ - `--dev-repo <path>` — explicit path to developer repository
16
+ - `--qa-repo <path>` — path to existing QA repository (produces gap analysis instead of blueprint)
17
+ - `--skip-map` — skip codebase mapping, only run analysis (lighter, faster)
18
+
19
+ ## What It Produces
20
+
21
+ ### Stage 1: Codebase Map (4 parallel agents)
22
+
23
+ | Focus Area | Documents Produced |
24
+ |------------|-------------------|
25
+ | **testability** | TESTABILITY.md + TEST_SURFACE.md — what's testable, entry points, mocking needs |
26
+ | **risk** | RISK_MAP.md + CRITICAL_PATHS.md — business-critical paths, error handling gaps |
27
+ | **patterns** | CODE_PATTERNS.md + API_CONTRACTS.md — naming conventions, API shapes, auth patterns |
28
+ | **existing-tests** | TEST_ASSESSMENT.md + COVERAGE_GAPS.md — existing test quality, what's missing |
29
+
30
+ All documents written to `.qa-output/codebase/`.
31
+
32
+ ### Stage 2: Repository Analysis
33
+
34
+ | Document | Description |
35
+ |----------|-------------|
36
+ | SCAN_MANIFEST.md | File tree, framework detection, testable surfaces |
37
+ | QA_ANALYSIS.md | Architecture overview, risk assessment, top 10 unit targets, testing pyramid |
38
+ | TEST_INVENTORY.md | Every test case with ID, target, inputs, expected outcome, priority |
39
+ | QA_REPO_BLUEPRINT.md | If no QA repo — full repo structure, configs, CI/CD strategy |
40
+ | GAP_ANALYSIS.md | If QA repo provided — coverage map, missing tests, broken tests, quality assessment |
41
+
42
+ ## Instructions
43
+
44
+ 1. Read `CLAUDE.md` — QA standards.
45
+
46
+ 2. Create output directories:
47
+ ```bash
48
+ mkdir -p .qa-output/codebase
49
+ ```
50
+
51
+ 3. **Stage 1: Codebase Mapping**
52
+
53
+ If `--skip-map` was NOT passed and `--focus` was NOT specified, spawn 4 agents in parallel (one per focus area):
54
+
55
+ ```
56
+ Agent(
57
+ prompt="Analyze this codebase for QA purposes. Focus area: testability. Write TESTABILITY.md and TEST_SURFACE.md to .qa-output/codebase/. Follow your agent definition process.",
58
+ subagent_type="general-purpose",
59
+ execution_context="@agents/qaa-codebase-mapper.md"
60
+ )
61
+
62
+ Agent(
63
+ prompt="Analyze this codebase for QA purposes. Focus area: risk. Write RISK_MAP.md and CRITICAL_PATHS.md to .qa-output/codebase/. Follow your agent definition process.",
64
+ subagent_type="general-purpose",
65
+ execution_context="@agents/qaa-codebase-mapper.md"
66
+ )
67
+
68
+ Agent(
69
+ prompt="Analyze this codebase for QA purposes. Focus area: patterns. Write CODE_PATTERNS.md and API_CONTRACTS.md to .qa-output/codebase/. Follow your agent definition process.",
70
+ subagent_type="general-purpose",
71
+ execution_context="@agents/qaa-codebase-mapper.md"
72
+ )
73
+
74
+ Agent(
75
+ prompt="Analyze this codebase for QA purposes. Focus area: existing-tests. Write TEST_ASSESSMENT.md and COVERAGE_GAPS.md to .qa-output/codebase/. Follow your agent definition process.",
76
+ subagent_type="general-purpose",
77
+ execution_context="@agents/qaa-codebase-mapper.md"
78
+ )
79
+ ```
80
+
81
+ If `--focus <area>` was provided, spawn only that one agent and STOP after it completes (skip Stage 2).
82
+
83
+ If `--skip-map` was passed, skip Stage 1 entirely and go to Stage 2.
84
+
85
+ 4. When all map agents complete, print summary of documents produced.
86
+
87
+ 5. **Stage 2: Repository Analysis**
88
+
89
+ Initialize pipeline context:
90
+ ```bash
91
+ node bin/qaa-tools.cjs init qa-start 2>/dev/null || true
92
+ ```
93
+
94
+ Invoke scanner agent:
95
+
96
+ Task(
97
+ prompt="
98
+ <objective>Scan repository and produce SCAN_MANIFEST.md</objective>
99
+ <execution_context>@agents/qaa-scanner.md</execution_context>
100
+ <files_to_read>
101
+ - CLAUDE.md
102
+ </files_to_read>
103
+ <parameters>
104
+ user_input: $ARGUMENTS
105
+ </parameters>
106
+ "
107
+ )
108
+
109
+ Invoke analyzer agent:
110
+
111
+ Task(
112
+ prompt="
113
+ <objective>Analyze repository and produce QA_ANALYSIS.md, TEST_INVENTORY.md, and blueprint or gap analysis. Use codebase map documents from .qa-output/codebase/ if they exist for deeper, more accurate analysis.</objective>
114
+ <execution_context>@agents/qaa-analyzer.md</execution_context>
115
+ <files_to_read>
116
+ - CLAUDE.md
117
+ - .qa-output/SCAN_MANIFEST.md
118
+ - .qa-output/codebase/TESTABILITY.md (if exists)
119
+ - .qa-output/codebase/RISK_MAP.md (if exists)
120
+ - .qa-output/codebase/CODE_PATTERNS.md (if exists)
121
+ - .qa-output/codebase/TEST_ASSESSMENT.md (if exists)
122
+ - .qa-output/codebase/TEST_SURFACE.md (if exists)
123
+ - .qa-output/codebase/CRITICAL_PATHS.md (if exists)
124
+ - .qa-output/codebase/API_CONTRACTS.md (if exists)
125
+ - .qa-output/codebase/COVERAGE_GAPS.md (if exists)
126
+ </files_to_read>
127
+ <parameters>
128
+ user_input: $ARGUMENTS
129
+ </parameters>
130
+ "
131
+ )
132
+
133
+ 6. Print final summary: all documents produced across both stages.
134
+ No git operations. No test generation.
135
+ Suggest `/qa-create-test` to generate tests from the analysis.
136
+
137
+ $ARGUMENTS