qaa-agent 1.5.0 → 1.6.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -31,7 +31,90 @@ Generate production-ready test files with POM pattern for a specific feature or
31
31
  - `TESTABILITY.md` -- pure functions vs stateful code, mock boundaries
32
32
  If codebase map does not exist, run `/qa-map` first for best results, or proceed without it.
33
33
 
34
- 4. Invoke executor agent to generate test files:
34
+ 4. **Check existing locator registry and extract new locators from live app:**
35
+
36
+ a. **Check the locator registry first.** Read `.qa-output/locators/LOCATOR_REGISTRY.md` if it exists. This is the accumulated registry of all locators previously extracted from the live app. Check if locators for this feature's pages already exist.
37
+
38
+ b. **If locators for this feature's pages are already in the registry AND no `--app-url` was provided:** Skip browser extraction -- reuse existing locators. Print: `"Reusing cached locators for {feature} from registry."`
39
+
40
+ c. **If locators are missing or `--app-url` was provided:** Use the Playwright MCP to navigate the app and extract real locators BEFORE generating tests.
41
+
42
+ **Browser extraction process:**
43
+
44
+ 1. Navigate to the feature's relevant pages:
45
+ ```
46
+ mcp__playwright__browser_navigate({ url: "{app_url}/{feature_path}" })
47
+ ```
48
+
49
+ 2. Take an accessibility snapshot of each page to discover all interactive elements:
50
+ ```
51
+ mcp__playwright__browser_snapshot()
52
+ ```
53
+
54
+ 3. Extract real locators from the snapshot -- collect:
55
+ - All `data-testid` attributes present on the page
56
+ - ARIA roles with accessible names (buttons, inputs, links, etc.)
57
+ - Form labels and placeholders
58
+ - Navigation structure and page layout
59
+
60
+ 4. If the feature has multiple pages/views (e.g., login -> dashboard), navigate through the flow:
61
+ ```
62
+ mcp__playwright__browser_fill_form({ ... })
63
+ mcp__playwright__browser_click({ element: "..." })
64
+ mcp__playwright__browser_snapshot() // capture next page
65
+ ```
66
+
67
+ 5. Write per-feature locator file to `.qa-output/locators/{feature}.locators.md`:
68
+ ```markdown
69
+ # Locators -- {feature}
70
+
71
+ Extracted: {date}
72
+ App URL: {app_url}
73
+
74
+ ## Page: {page_name} ({url})
75
+
76
+ | Element | Locator Type | Locator Value | Tier |
77
+ |---------|-------------|---------------|------|
78
+ | Email input | data-testid | login-email-input | 1 |
79
+ | Password input | data-testid | login-password-input | 1 |
80
+ | Submit button | role + name | button "Log in" | 1 |
81
+ | Remember me | label | "Remember me" | 2 |
82
+
83
+ ## Page: {next_page} ({url})
84
+ ...
85
+ ```
86
+
87
+ 6. Update the registry `.qa-output/locators/LOCATOR_REGISTRY.md` -- merge new locators into the central index:
88
+ ```markdown
89
+ # Locator Registry
90
+
91
+ Last updated: {date}
92
+ Total pages: {N}
93
+ Total locators: {N}
94
+
95
+ ## Index
96
+
97
+ | Feature | File | Pages | Locators | Extracted |
98
+ |---------|------|-------|----------|-----------|
99
+ | login | login.locators.md | 2 | 14 | 2026-03-25 |
100
+ | checkout | checkout.locators.md | 3 | 22 | 2026-03-25 |
101
+ | dashboard | dashboard.locators.md | 1 | 8 | 2026-03-25 |
102
+
103
+ ## All Locators by Page
104
+
105
+ ### /login
106
+ | Element | Locator Type | Locator Value | Tier | Source |
107
+ |---------|-------------|---------------|------|--------|
108
+ | Email input | data-testid | login-email-input | 1 | login.locators.md |
109
+ | ... | ... | ... | ... | ... |
110
+
111
+ ### /dashboard
112
+ ...
113
+ ```
114
+
115
+ If no app URL is available and no locators exist in the registry for this feature, skip this step -- the executor will propose locators based on source code analysis and CLAUDE.md conventions.
116
+
117
+ 5. Invoke executor agent to generate test files:
35
118
 
36
119
  Task(
37
120
  prompt="
@@ -39,6 +122,8 @@ Task(
39
122
  <execution_context>@agents/qaa-executor.md</execution_context>
40
123
  <files_to_read>
41
124
  - CLAUDE.md
125
+ - .qa-output/locators/LOCATOR_REGISTRY.md (if exists -- accumulated real locators)
126
+ - .qa-output/locators/{feature}.locators.md (if exists -- feature-specific locators)
42
127
  - .qa-output/codebase/CODE_PATTERNS.md (if exists)
43
128
  - .qa-output/codebase/API_CONTRACTS.md (if exists)
44
129
  - .qa-output/codebase/TEST_SURFACE.md (if exists)
@@ -48,11 +133,12 @@ Task(
48
133
  user_input: $ARGUMENTS
49
134
  mode: feature-test
50
135
  codebase_map_dir: .qa-output/codebase
136
+ locator_registry: .qa-output/locators/LOCATOR_REGISTRY.md
51
137
  </parameters>
52
138
  "
53
139
  )
54
140
 
55
- 5. If E2E test files were generated AND `--skip-run` was NOT passed, invoke the E2E runner to execute tests against the live app:
141
+ 6. If E2E test files were generated AND `--skip-run` was NOT passed, invoke the E2E runner to execute tests against the live app:
56
142
 
57
143
  Task(
58
144
  prompt="
@@ -70,7 +156,7 @@ Task(
70
156
  "
71
157
  )
72
158
 
73
- 6. Present results:
159
+ 7. Present results:
74
160
  - List generated files with type counts (unit, API, E2E, POM, fixture)
75
161
  - If E2E runner executed: show pass/fail counts, locator fixes applied, app bugs found
76
162
  - Suggest `/qa-pr` to package as a pull request
@@ -1,16 +1,17 @@
1
1
  # Create Tests from Ticket
2
2
 
3
- Generate test cases and executable test files from a ticket (Jira, Linear, GitHub Issue, or plain text user story). Combines the ticket's acceptance criteria with actual source code analysis to produce targeted, concrete tests.
3
+ Generate test cases and executable test files from a ticket (Jira, Linear, GitHub Issue, or plain text user story). Combines the ticket's acceptance criteria with actual source code analysis to produce targeted, concrete tests. If an app URL is available, navigates the live app with Playwright MCP to extract real locators before generating tests.
4
4
 
5
5
  ## Usage
6
6
 
7
- /qa-from-ticket <ticket-source> [--dev-repo <path>]
7
+ /qa-from-ticket <ticket-source> [--dev-repo <path>] [--app-url <url>]
8
8
 
9
9
  - ticket-source: one of:
10
10
  - URL to GitHub/Jira/Linear issue
11
11
  - Plain text user story or acceptance criteria
12
12
  - File path to a .md or .txt with ticket details
13
13
  - --dev-repo: path to developer repository (default: current directory)
14
+ - --app-url: URL of running application for browser-based locator extraction (auto-detects if not provided)
14
15
 
15
16
  ## Instructions
16
17
 
@@ -10,7 +10,8 @@
10
10
  "Agent",
11
11
  "WebFetch",
12
12
  "WebSearch",
13
- "NotebookEdit"
13
+ "NotebookEdit",
14
+ "mcp__playwright__*"
14
15
  ]
15
16
  },
16
17
  "env": {
package/.mcp.json ADDED
@@ -0,0 +1,8 @@
1
+ {
2
+ "mcpServers": {
3
+ "playwright": {
4
+ "command": "npx",
5
+ "args": ["@playwright/mcp@latest"]
6
+ }
7
+ }
8
+ }
package/CHANGELOG.md ADDED
@@ -0,0 +1,71 @@
1
+
2
+ # Changelog
3
+
4
+ All notable changes to QAA (QA Automation Agent) are documented here.
5
+
6
+ ## [1.6.0] - 2026-03-25
7
+
8
+ ### Added
9
+ - Playwright MCP server bundled in agent package (`.mcp.json`) -- starts automatically when opening project in Claude Code
10
+ - Persistent locator registry at `.qa-output/locators/` -- accumulates real locators across features over time
11
+ - Per-feature files: `{feature}.locators.md` -- extracted locators for each feature tested
12
+ - Central index: `LOCATOR_REGISTRY.md` -- all locators by page, searchable by any command
13
+ - Browser-based locator extraction step in `/create-test` and `/qa-from-ticket` -- navigates live app with Playwright MCP and captures real data-testid, ARIA roles, and labels before generating tests
14
+ - Registry cache: if locators for a feature already exist in the registry, browser extraction is skipped (reuses cached locators)
15
+ - `--app-url` flag added to `/qa-from-ticket`
16
+ - CHANGELOG.md
17
+
18
+ ### Changed
19
+ - `qaa-executor` now reads locator registry (when available) to use real locators in POMs instead of proposing them
20
+ - `/create-test` flow: checks registry first, then extracts via browser if needed, BEFORE test generation
21
+ - `/qa-from-ticket` workflow: locator extraction step added after source scan, before test case generation
22
+
23
+ ### Removed
24
+ - `/qa-analyze` command (deprecated since v1.4.0, fully replaced by `/qa-map`)
25
+
26
+ ## [1.5.0] - 2026-03-24
27
+
28
+ ### Added
29
+ - Stable release
30
+
31
+ ## [1.4.0]
32
+
33
+ ### Changed
34
+ - Merged `/qa-analyze` into `/qa-map` -- single command for codebase scanning and analysis
35
+ - Consolidated pipeline flow
36
+
37
+ ### Deprecated
38
+ - `/qa-analyze` command (use `/qa-map` instead)
39
+
40
+ ## [1.3.0]
41
+
42
+ ### Added
43
+ - `qa-learner` skill -- persistent preferences from user corrections
44
+ - Preferences saved to `~/.claude/qaa/MY_PREFERENCES.md`
45
+ - Trigger detection for English and Spanish frustration signals
46
+
47
+ ## [1.2.0]
48
+
49
+ ### Added
50
+ - `qaa-codebase-mapper` agent -- 4 parallel focus areas (testability, risk, patterns, existing tests)
51
+ - `qaa-project-researcher` agent -- researches best testing stack and practices
52
+ - 8 codebase map documents produced by mapper
53
+
54
+ ## [1.1.0]
55
+
56
+ ### Added
57
+ - Workflow definitions for all pipeline stages
58
+ - Interactive installer (`npx qaa-agent`)
59
+ - `qaa init` command for per-project initialization
60
+ - npm package distribution
61
+
62
+ ## [1.0.0]
63
+
64
+ ### Added
65
+ - Full QA automation pipeline -- 11 agents, 17 commands, 10 templates, 7 workflows
66
+ - 3 workflow options (dev-only, immature QA, mature QA)
67
+ - 4-layer test validation (syntax, structure, dependencies, logic)
68
+ - Page Object Model generation with CLAUDE.md standards
69
+ - Test ID injection for frontend components
70
+ - Bug detective failure classification
71
+ - Draft PR delivery with branch naming convention
@@ -34,6 +34,17 @@ Read ALL of the following files BEFORE producing any output. The executor's code
34
34
 
35
35
  - **~/.claude/qaa/MY_PREFERENCES.md** (optional -- read if exists). User's personal QA preferences saved by the qa-learner skill. If a preference conflicts with CLAUDE.md, the preference wins (it is a user override). Check for rules about: framework choices, locator strategy, assertion style, naming conventions, language preferences.
36
36
 
37
+ - **Locator Registry** (optional -- read if it exists):
38
+ - **`.qa-output/locators/LOCATOR_REGISTRY.md`** -- Central index of all locators extracted from the live app across all features. Contains locators per page with element name, locator type, value, and tier.
39
+ - **`.qa-output/locators/{feature}.locators.md`** -- Per-feature locator files with detailed page-by-page locator tables.
40
+
41
+ When locator registry files exist:
42
+ - Use the exact `data-testid` values, ARIA roles, and labels from the registry in POM locator properties
43
+ - Do NOT propose or guess locator values -- use what was captured from the rendered page
44
+ - If an element appears in the registry, its locator is authoritative (Tier 1)
45
+ - If an element needed by a test case is NOT in the registry, fall back to CLAUDE.md locator tier hierarchy as usual
46
+ - Check the feature-specific file first (`{feature}.locators.md`), then fall back to `LOCATOR_REGISTRY.md`
47
+
37
48
  - **Codebase map documents** (optional -- read if they exist in `{codebase_map_dir}/` or `.qa-output/codebase/`):
38
49
  - **CODE_PATTERNS.md** -- Naming conventions, import patterns, code style used in the project. Use to generate tests that feel native to the codebase (matching variable naming, import style, file organization).
39
50
  - **API_CONTRACTS.md** -- Exact request/response shapes, auth patterns, error response formats. Use for API test assertions with real payload shapes and correct auth headers.
@@ -84,7 +95,15 @@ Read all input artifacts and build the execution context.
84
95
  - Extract expected outcome rules
85
96
  - These patterns guide the code generation in step 4
86
97
 
87
- 6. **Read codebase map documents** (if they exist -- check `{codebase_map_dir}/` or `.qa-output/codebase/`):
98
+ 6. **Read Locator Registry** (if it exists):
99
+ - Check for `.qa-output/locators/LOCATOR_REGISTRY.md` (central index)
100
+ - Check for `.qa-output/locators/{feature}.locators.md` (feature-specific, more detailed)
101
+ - Extract all locators per page: element name, locator type, locator value, tier
102
+ - Index by page name for quick lookup during POM generation
103
+ - When generating POM locator properties, use the exact values from the registry instead of proposing values
104
+ - If no locator registry exists, proceed normally -- propose locators based on CLAUDE.md conventions and source code analysis
105
+
106
+ 7. **Read codebase map documents** (if they exist -- check `{codebase_map_dir}/` or `.qa-output/codebase/`):
88
107
  - **CODE_PATTERNS.md** -- Extract naming conventions (variable casing, import style, file organization). Match generated test code to the project's native style.
89
108
  - **API_CONTRACTS.md** -- Extract exact request/response shapes with field types, auth header patterns, error response formats. Use for concrete API test payloads and response assertions.
90
109
  - **TEST_SURFACE.md** -- Extract function signatures with parameter types and return types. Use to write accurate import statements, mock setup, and assertion values.
package/bin/install.cjs CHANGED
@@ -143,6 +143,14 @@ async function main() {
143
143
  copyFile(path.join(ROOT, 'CLAUDE.md'), path.join(qaaDir, 'CLAUDE.md'));
144
144
  ok('Installed QA standards (CLAUDE.md)');
145
145
 
146
+ // Install .mcp.json (Playwright MCP server config)
147
+ const mcpSrc = path.join(ROOT, '.mcp.json');
148
+ if (fs.existsSync(mcpSrc)) {
149
+ const mcpDest = path.join(qaaDir, '.mcp.json');
150
+ copyFile(mcpSrc, mcpDest);
151
+ ok('Installed Playwright MCP server config (.mcp.json)');
152
+ }
153
+
146
154
  // Write version
147
155
  fs.writeFileSync(path.join(qaaDir, 'VERSION'), VERSION);
148
156
  ok(`Wrote VERSION (${VERSION})`);
@@ -175,7 +183,7 @@ async function main() {
175
183
  console.log(' Open Claude Code in any project and run:');
176
184
  console.log('');
177
185
  console.log(' \x1b[1m/qa-start\x1b[0m Full QA pipeline (multi-agent)');
178
- console.log(' \x1b[1m/qa-analyze\x1b[0m Analysis only');
186
+ console.log(' \x1b[1m/qa-map\x1b[0m Codebase map + analysis');
179
187
  console.log(' \x1b[1m/create-test\x1b[0m Tests for a feature');
180
188
  console.log(' \x1b[1m/qa-from-ticket\x1b[0m Tests from a Jira/Linear ticket');
181
189
  console.log(' \x1b[1m/qa-validate\x1b[0m Validate existing tests');
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "qaa-agent",
3
- "version": "1.5.0",
3
+ "version": "1.6.1",
4
4
  "description": "QA Automation Agent for Claude Code — multi-agent pipeline that analyzes repos, generates tests, validates, and creates PRs",
5
5
  "bin": {
6
6
  "qaa-agent": "./bin/install.cjs"
@@ -21,6 +21,9 @@
21
21
  },
22
22
  "author": "Backhaus7997",
23
23
  "license": "MIT",
24
+ "dependencies": {
25
+ "@playwright/mcp": "latest"
26
+ },
24
27
  "files": [
25
28
  "bin/",
26
29
  "agents/",
@@ -30,7 +33,9 @@
30
33
  ".claude/commands/",
31
34
  ".claude/skills/",
32
35
  ".claude/settings.json",
36
+ ".mcp.json",
33
37
  "CLAUDE.md",
38
+ "CHANGELOG.md",
34
39
  "README.md"
35
40
  ]
36
41
  }
@@ -215,8 +215,69 @@ Generating test cases based on ticket content only (no source-level analysis).
215
215
  ```
216
216
  </step>
217
217
 
218
+ <step name="extract_locators_from_app">
219
+ ## Step 5: Check Locator Registry and Extract from Live App (Optional)
220
+
221
+ Check the locator registry for existing locators, and if needed, use Playwright MCP to extract new ones from the live app.
222
+
223
+ **Step 5a: Check existing registry**
224
+
225
+ Read `.qa-output/locators/LOCATOR_REGISTRY.md` if it exists. Check if locators for pages related to this ticket's feature already exist. If they do and no `--app-url` was provided, reuse them and skip browser extraction.
226
+
227
+ **Step 5b: When to extract from browser**
228
+ - Locators for this feature's pages are NOT in the registry, OR
229
+ - An `--app-url` argument was explicitly provided (forces re-extraction)
230
+
231
+ **When to skip entirely:**
232
+ - No app URL available, no dev server detected, AND no registry exists
233
+ - The ticket describes only backend/API functionality with no UI
234
+
235
+ **Extraction process:**
236
+
237
+ 1. Identify relevant pages from the ticket's acceptance criteria and affected components (from Step 3 and Step 4).
238
+
239
+ 2. For each relevant page, navigate and capture:
240
+ ```
241
+ mcp__playwright__browser_navigate({ url: "{app_url}/{page_path}" })
242
+ mcp__playwright__browser_snapshot()
243
+ ```
244
+
245
+ 3. If the ticket describes a user flow (e.g., "user fills form and submits"), walk through the flow:
246
+ ```
247
+ mcp__playwright__browser_fill_form({ ... })
248
+ mcp__playwright__browser_click({ element: "Submit button" })
249
+ mcp__playwright__browser_snapshot() // capture resulting page
250
+ ```
251
+
252
+ 4. From each snapshot, extract:
253
+ - All `data-testid` attributes
254
+ - ARIA roles with accessible names
255
+ - Form labels and placeholders
256
+ - Page structure and navigation elements
257
+
258
+ 5. Write per-feature locator file to `.qa-output/locators/{feature}.locators.md`:
259
+ ```markdown
260
+ # Locators -- {feature}
261
+
262
+ Extracted: {date}
263
+ App URL: {app_url}
264
+
265
+ ## Page: {page_name} ({url})
266
+
267
+ | Element | Locator Type | Locator Value | Tier |
268
+ |---------|-------------|---------------|------|
269
+ | ... | data-testid | ... | 1 |
270
+ | ... | role + name | ... | 1 |
271
+ | ... | label | ... | 2 |
272
+ ```
273
+
274
+ 6. Update the registry `.qa-output/locators/LOCATOR_REGISTRY.md` -- merge new locators into the central index without overwriting locators from other features.
275
+
276
+ If this step is skipped entirely, the executor will propose locators based on source code analysis and CLAUDE.md conventions.
277
+ </step>
278
+
218
279
  <step name="generate_test_cases">
219
- ## Step 5: Generate Test Cases with Traceability Matrix
280
+ ## Step 6: Generate Test Cases with Traceability Matrix
220
281
 
221
282
  Map each acceptance criterion to one or more test cases, following CLAUDE.md test spec rules.
222
283
 
@@ -312,7 +373,7 @@ Write to `{OUTPUT_DIR}/TEST_CASES_FROM_TICKET.md`.
312
373
  </step>
313
374
 
314
375
  <step name="generate_test_files">
315
- ## Step 6: Spawn Executor Agent
376
+ ## Step 7: Spawn Executor Agent
316
377
 
317
378
  Build a synthetic generation plan from the test cases and spawn the executor to write test files.
318
379
 
@@ -361,6 +422,7 @@ Task(
361
422
  <files_to_read>
362
423
  - {OUTPUT_DIR}/GENERATION_PLAN_TICKET.md
363
424
  - {OUTPUT_DIR}/TEST_CASES_FROM_TICKET.md
425
+ - {OUTPUT_DIR}/locators/LOCATOR_REGISTRY.md (if exists -- accumulated real locators)
364
426
  - CLAUDE.md
365
427
  </files_to_read>
366
428
  <parameters>
@@ -376,7 +438,7 @@ Extract: `files_created`, `total_files`, `commit_count`, `test_case_count`.
376
438
  </step>
377
439
 
378
440
  <step name="validate_generated_tests">
379
- ## Step 7: Spawn Validator Agent
441
+ ## Step 8: Spawn Validator Agent
380
442
 
381
443
  Validate the generated test files against CLAUDE.md standards.
382
444
 
@@ -402,7 +464,7 @@ Extract: `overall_status`, `confidence`, `issues_found`, `issues_fixed`, `unreso
402
464
  </step>
403
465
 
404
466
  <step name="print_summary">
405
- ## Step 8: Print Summary
467
+ ## Step 9: Print Summary
406
468
 
407
469
  Print a comprehensive summary showing traceability from ticket to tests.
408
470
 
@@ -1,19 +0,0 @@
1
- # QA Repository Analysis (Deprecated)
2
-
3
- This command has been merged into `/qa-map`, which runs codebase mapping + analysis in one step.
4
-
5
- ## Usage
6
-
7
- Use `/qa-map` instead:
8
-
9
- ```
10
- /qa-map [--dev-repo <path>] [--qa-repo <path>]
11
- ```
12
-
13
- ## Instructions
14
-
15
- Tell the user: "/qa-analyze is now part of /qa-map. Running /qa-map for you."
16
-
17
- Then execute `/qa-map` with the same arguments the user provided.
18
-
19
- $ARGUMENTS
package/README.md DELETED
@@ -1,431 +0,0 @@
1
- # QAA -- QA Automation Agent
2
-
3
- Multi-agent QA automation system for Claude Code. Point it at any repository -- it analyzes the codebase, generates a standards-compliant test suite, validates everything, and delivers the result as a draft PR. Runs locally via Claude Code.
4
-
5
- No manual test writing. No guessing what to cover. One command, full pipeline.
6
-
7
- ## Quick Start
8
-
9
- Prerequisites: Node.js 18+, Claude Code (Pro or Max plan), gh CLI (authenticated), Git.
10
-
11
- ```bash
12
- # 1. Clone this repo alongside your project
13
- git clone <this-repo-url> qa-agent
14
-
15
- # 2. Open Claude Code in the qa-agent directory
16
-
17
- # 3. Run the full pipeline against your dev repo
18
- /qa-start --dev-repo /path/to/your-project
19
-
20
- # 4. Wait for the pipeline to complete, then review the draft PR
21
- ```
22
-
23
- For an existing QA repository:
24
-
25
- ```bash
26
- /qa-start --dev-repo /path/to/dev-repo --qa-repo /path/to/qa-repo
27
- ```
28
-
29
- For fully unattended execution (auto-approve safe checkpoints):
30
-
31
- ```bash
32
- /qa-start --dev-repo /path/to/your-project --auto
33
- ```
34
-
35
- ## Prerequisites
36
-
37
- Every tool below must be installed and working before running the pipeline.
38
-
39
- - **Node.js 18+** -- Runtime for CLI tooling. The pipeline uses Node for configuration management and artifact validation.
40
- - **Claude Code** (Anthropic) -- The AI coding assistant that executes the agents. You must have a **Pro or Max plan** for access to the Opus model, which all agents require.
41
- - **gh CLI** -- GitHub's official command-line tool for creating pull requests. Install from https://cli.github.com and authenticate before first use.
42
- - **Git** -- Version control. The target repository must have a remote origin configured for PR delivery.
43
-
44
- ### Verifying Prerequisites
45
-
46
- Run each command and confirm the expected output:
47
-
48
- ```bash
49
- node --version # Must show v18.x.x or higher
50
- claude --version # Must show Claude Code version
51
- gh auth status # Must show "Logged in to github.com"
52
- git --version # Must show git version 2.x+
53
- ```
54
-
55
- If any command fails, install the missing tool before proceeding.
56
-
57
- ## Installation
58
-
59
- 1. Clone or copy this repository into a local directory:
60
- ```bash
61
- git clone <this-repo-url> qa-agent
62
- cd qa-agent
63
- ```
64
-
65
- 2. Verify the setup is healthy:
66
- ```bash
67
- node bin/qaa-tools.cjs validate health
68
- ```
69
-
70
- 3. Open Claude Code in this directory. The `.claude/commands/` directory provides all slash commands automatically -- no additional setup needed.
71
-
72
- ## Configuration
73
-
74
- The pipeline behavior is controlled by `.planning/config.json`. Default values work for most projects.
75
-
76
- | Setting | Options | Default | Description |
77
- |---------|---------|---------|-------------|
78
- | `mode` | `quality`, `balanced`, `budget` | `quality` | Controls which AI models agents use |
79
- | `granularity` | `coarse`, `standard`, `fine` | `standard` | Detail level of analysis and generation |
80
- | `parallelization` | `true`, `false` | `true` | Enable wave-based parallel agent execution |
81
- | `workflow.auto_advance` | `true`, `false` | `false` | Auto-approve safe checkpoints without pausing |
82
-
83
- Set values via CLI:
84
-
85
- ```bash
86
- node bin/qaa-tools.cjs config set mode balanced
87
- node bin/qaa-tools.cjs config set workflow.auto_advance true
88
- ```
89
-
90
- Or edit `.planning/config.json` directly.
91
-
92
- ## Commands
93
-
94
- All commands are available as slash commands in Claude Code. They are organized into three tiers by frequency of use.
95
-
96
- ### /qa-start -- Full Pipeline (Tier 1: Daily Use)
97
-
98
- The primary command. Runs the entire QA automation pipeline from scan to PR delivery.
99
-
100
- ```
101
- /qa-start [--dev-repo <path>] [--qa-repo <path>] [--auto]
102
- ```
103
-
104
- **Arguments:**
105
- - No arguments: uses current directory as the dev repo (Option 1: dev-only)
106
- - `--dev-repo`: explicit path to the developer repository
107
- - `--qa-repo`: path to an existing QA repository (triggers Option 2 or 3 based on maturity score)
108
- - `--auto`: enable auto-advance mode (skips safe checkpoint pauses)
109
-
110
- **What happens:**
111
- 1. Scans the repository -- detects framework, language, testable surfaces
112
- 2. Analyzes architecture -- produces risk assessment, test inventory, blueprint
113
- 3. Injects test IDs (if frontend components detected)
114
- 4. Plans test generation -- groups test cases by feature domain
115
- 5. Generates test files -- unit, API, integration, E2E with Page Object Models
116
- 6. Validates generated tests -- 4-layer validation with auto-fix (up to 3 loops)
117
- 7. Classifies any remaining failures (if present)
118
- 8. Delivers everything as a draft PR on a `qa/auto-{project}-{date}` branch
119
-
120
- **What it produces:**
121
- - SCAN_MANIFEST.md, QA_ANALYSIS.md, TEST_INVENTORY.md, QA_REPO_BLUEPRINT.md
122
- - Generated test files, POMs, fixtures, and config files
123
- - VALIDATION_REPORT.md with confidence level (HIGH/MEDIUM/LOW)
124
- - A draft pull request with full analysis summary
125
-
126
- ### Analysis Commands (Tier 2: Common Use)
127
-
128
- #### /qa-analyze -- Repository Analysis
129
-
130
- Scan and analyze a repository without generating tests. Produces assessment documents only.
131
-
132
- ```
133
- /qa-analyze [--dev-repo <path>] [--qa-repo <path>]
134
- ```
135
-
136
- Produces: SCAN_MANIFEST.md, QA_ANALYSIS.md, TEST_INVENTORY.md, and either QA_REPO_BLUEPRINT.md (no QA repo) or GAP_ANALYSIS.md (QA repo provided).
137
-
138
- #### /qa-validate -- Test Validation
139
-
140
- Validate existing test files against QA standards. Runs 4-layer checks (syntax, structure, dependencies, logic) and classifies failures.
141
-
142
- ```
143
- /qa-validate <path-to-tests> [--framework <name>]
144
- ```
145
-
146
- Produces: VALIDATION_REPORT.md. If failures are found, also produces FAILURE_CLASSIFICATION_REPORT.md.
147
-
148
- #### /qa-testid -- Test ID Injection
149
-
150
- Scan frontend source code, audit missing `data-testid` attributes, and inject them using the project naming convention. Creates a separate branch for changes.
151
-
152
- ```
153
- /qa-testid <path-to-frontend-source>
154
- ```
155
-
156
- Produces: TESTID_AUDIT_REPORT.md and modified source files with `data-testid` attributes.
157
-
158
- ### Specialized Commands (Tier 3)
159
-
160
- | Command | Purpose | Usage |
161
- |---------|---------|-------|
162
- | `/qa-fix` | Diagnose and fix broken test files | `/qa-fix <path-to-tests> [error output]` |
163
- | `/qa-pom` | Generate Page Object Model files | `/qa-pom <path-to-pages> [--framework <name>]` |
164
- | `/qa-audit` | Full 6-dimension quality audit of a test suite | `/qa-audit <path-to-tests> [--dev-repo <path>]` |
165
- | `/qa-gap` | Gap analysis between dev and QA repos | `/qa-gap --dev-repo <path> --qa-repo <path>` |
166
- | `/qa-blueprint` | Generate QA repository structure blueprint | `/qa-blueprint [--dev-repo <path>]` |
167
- | `/qa-report` | Generate QA status report (team/management/client) | `/qa-report <path-to-tests> [--audience <level>]` |
168
- | `/qa-pyramid` | Analyze test distribution vs. ideal pyramid | `/qa-pyramid <path-to-tests> [--dev-repo <path>]` |
169
- | `/create-test` | Generate tests for a specific feature | `/create-test <feature-name> [--dev-repo <path>] [--app-url <url>]` |
170
- | `/update-test` | Improve existing tests without rewriting them | `/update-test <path-to-tests> [--scope <type>]` |
171
- | `/qa-map` | Deep-scan codebase for testability, risk, patterns | `/qa-map [--focus <area>]` |
172
- | `/qa-research` | Research best testing stack and practices | `/qa-research [--focus <area>]` |
173
- | `/qa-from-ticket` | Generate tests from a Jira/Linear/GitHub ticket | `/qa-from-ticket <source>` |
174
- | `/qa-pr` | Create draft PR from QA artifacts | `/qa-pr [--ticket <id>] [--title <desc>]` |
175
-
176
- ## Workflow Options
177
-
178
- The pipeline automatically selects the right workflow based on the repositories you provide.
179
-
180
- ### Option 1: Dev-Only Repository
181
-
182
- **When to use:** The project has no existing QA repository. You are starting QA from scratch.
183
-
184
- **Trigger:** Run `/qa-start --dev-repo <path>` with no `--qa-repo` argument.
185
-
186
- **What happens:** Full pipeline -- scan, analyze, plan, generate, validate, deliver. Produces a complete test suite with POMs, fixtures, config files, and a QA repository blueprint. The draft PR contains everything needed to bootstrap a QA repo.
187
-
188
- ### Option 2: Dev + Immature QA Repository
189
-
190
- **When to use:** A QA repo exists but has low coverage, inconsistent patterns, or broken tests. Maturity score below 70%.
191
-
192
- **Trigger:** Run `/qa-start --dev-repo <path> --qa-repo <path>` where the QA repo scores below the maturity threshold.
193
-
194
- **What happens:** Scans both repos, runs gap analysis, fixes broken tests, adds missing coverage, standardizes existing tests to match CLAUDE.md conventions. Produces a PR that improves the existing test suite rather than replacing it.
195
-
196
- ### Option 3: Dev + Mature QA Repository
197
-
198
- **When to use:** A solid QA repo already exists with good coverage and patterns. Maturity score 70% or above.
199
-
200
- **Trigger:** Run `/qa-start --dev-repo <path> --qa-repo <path>` where the QA repo scores at or above the maturity threshold.
201
-
202
- **What happens:** Scans both repos, identifies only thin coverage areas, adds surgical test additions without touching existing working tests. Produces a minimal PR with targeted additions.
203
-
204
- ## Example Output
205
-
206
- The following shows a typical `/qa-start` run against a Next.js e-commerce project:
207
-
208
- ```
209
- > /qa-start --dev-repo ./shopflow --auto
210
-
211
- +------------------------------------------+
212
- | QA Automation Pipeline |
213
- | Option: 1 (Dev-only) |
214
- | Target: shopflow |
215
- +------------------------------------------+
216
-
217
- +------------------------------------------+
218
- | STAGE 1: Scan |
219
- | Status: Running... |
220
- +------------------------------------------+
221
- Scanner complete. 847 files scanned, 32 testable surfaces identified.
222
- Output: .qa-output/SCAN_MANIFEST.md
223
-
224
- +------------------------------------------+
225
- | STAGE 2: Analyze |
226
- | Status: Running... |
227
- +------------------------------------------+
228
- Architecture: Next.js 14, TypeScript, Prisma ORM, REST API
229
- Risk areas: authentication (HIGH), payment processing (HIGH), cart logic (MEDIUM)
230
- Output: .qa-output/QA_ANALYSIS.md, .qa-output/TEST_INVENTORY.md, .qa-output/QA_REPO_BLUEPRINT.md
231
-
232
- +------------------------------------------+
233
- | STAGE 3: Test ID Injection |
234
- | Status: Running... |
235
- +------------------------------------------+
236
- Frontend detected. Auditing data-testid coverage...
237
- Coverage: 12% (18 of 147 interactive elements have data-testid)
238
- Injected 94 data-testid attributes across 23 components.
239
- Output: .qa-output/TESTID_AUDIT_REPORT.md
240
-
241
- +------------------------------------------+
242
- | STAGE 4: Plan |
243
- | Status: Running... |
244
- +------------------------------------------+
245
- Grouped 42 test cases into 6 feature domains: auth, products, cart, checkout, orders, admin.
246
-
247
- +------------------------------------------+
248
- | STAGE 5: Generate |
249
- | Status: Running... |
250
- +------------------------------------------+
251
- Generated 38 test files: 24 unit, 8 API, 4 integration, 2 E2E
252
- Created 6 Page Object Models, 4 fixture files, 2 config files.
253
-
254
- +------------------------------------------+
255
- | STAGE 6: Validate |
256
- | Status: Running... |
257
- +------------------------------------------+
258
- Validation loop 1: 3 issues found, 3 auto-fixed.
259
- Validation loop 2: all files PASS.
260
- Confidence: HIGH
261
-
262
- +------------------------------------------+
263
- | STAGE 7: Deliver |
264
- | Status: Running... |
265
- +------------------------------------------+
266
- Branch created: qa/auto-shopflow-2026-03-19
267
- PR created: https://github.com/client/shopflow/pull/42
268
-
269
- +------------------------------------------+
270
- | PIPELINE COMPLETE |
271
- | Tests: 24 unit, 8 API, 4 integration, |
272
- | 2 E2E (38 total) |
273
- | Validation: PASS (HIGH confidence) |
274
- | PR: https://github.com/client/shopflow |
275
- | /pull/42 |
276
- +------------------------------------------+
277
- ```
278
-
279
- ## Troubleshooting
280
-
281
- ### "gh: not authenticated"
282
-
283
- The gh CLI needs to be authenticated before the pipeline can create PRs. Run:
284
-
285
- ```bash
286
- gh auth login
287
- ```
288
-
289
- Select GitHub.com, HTTPS protocol, and authenticate via browser. After login, verify with `gh auth status`.
290
-
291
- ### "No git remote found"
292
-
293
- The target repository must have a remote origin configured for the deliver stage to push and create a PR. Add one:
294
-
295
- ```bash
296
- cd /path/to/target-repo
297
- git remote add origin https://github.com/org/repo.git
298
- ```
299
-
300
- If you only want local output without a PR, the pipeline will fall back gracefully -- all artifacts are still written to `.qa-output/`.
301
-
302
- ### "A branch named 'qa/auto-...' already exists"
303
-
304
- A pipeline was previously run on the same day against the same project. The system automatically appends a numeric suffix (`-2`, `-3`, etc.) to avoid collisions. If you want to clean up old branches:
305
-
306
- ```bash
307
- git branch -D qa/auto-shopflow-2026-03-19
308
- ```
309
-
310
- ### Pipeline stalls at a checkpoint
311
-
312
- Some pipeline stages have verification checkpoints that pause for your input. Type your response in the Claude Code terminal to continue. To skip safe checkpoints automatically, use the `--auto` flag:
313
-
314
- ```
315
- /qa-start --dev-repo <path> --auto
316
- ```
317
-
318
- Or enable auto-advance globally:
319
-
320
- ```bash
321
- node bin/qaa-tools.cjs config set workflow.auto_advance true
322
- ```
323
-
324
- ### Tests fail validation after 3 fix loops
325
-
326
- The validator attempted 3 automatic fix cycles but could not resolve all issues. Review the details in `.qa-output/VALIDATION_REPORT.md` to understand what failed and why. Fix the remaining issues manually, then re-validate:
327
-
328
- ```
329
- /qa-validate <path-to-test-files>
330
- ```
331
-
332
- ### Claude Code says "model not available"
333
-
334
- You need a Pro or Max plan for Opus model access. Check your plan at https://console.anthropic.com. The pipeline requires Opus for all agent operations.
335
-
336
- ## Project Structure
337
-
338
- ```
339
- qa-agent-gsd/
340
- agents/ -- Agent workflow definitions
341
- qa-pipeline-orchestrator.md -- Main pipeline controller (3 options)
342
- qaa-scanner.md -- Repository scanner agent
343
- qaa-codebase-mapper.md -- Codebase deep-scan agent (4 parallel focus areas)
344
- qaa-analyzer.md -- Architecture analyzer agent
345
- qaa-planner.md -- Test generation planner agent
346
- qaa-executor.md -- Test file generator agent
347
- qaa-validator.md -- Test validation agent
348
- qaa-e2e-runner.md -- E2E test execution agent (Playwright browser)
349
- qaa-testid-injector.md -- Test ID injection agent
350
- qaa-bug-detective.md -- Failure classification agent
351
- qaa-project-researcher.md -- Testing stack research agent
352
- bin/ -- CLI tooling
353
- qaa-tools.cjs -- Main CLI entry point
354
- lib/ -- CLI module library
355
- templates/ -- Output artifact templates (9 templates + PR template)
356
- scan-manifest.md -- Scan output template
357
- qa-analysis.md -- Analysis output template
358
- test-inventory.md -- Test inventory template
359
- qa-repo-blueprint.md -- Repository blueprint template
360
- gap-analysis.md -- Gap analysis template
361
- validation-report.md -- Validation report template
362
- failure-classification.md -- Failure classification template
363
- testid-audit-report.md -- Test ID audit template
364
- qa-audit-report.md -- Quality audit template
365
- pr-template.md -- Pull request body template
366
- .claude/commands/ -- Slash commands (17 commands, auto-detected by Claude Code)
367
- qa-start.md -- Tier 1: full pipeline
368
- qa-analyze.md -- Tier 2: analysis only
369
- qa-validate.md -- Tier 2: test validation
370
- qa-testid.md -- Tier 2: test ID injection
371
- qa-fix.md -- Tier 3: fix broken tests
372
- qa-pom.md -- Tier 3: generate POMs
373
- qa-audit.md -- Tier 3: quality audit
374
- qa-gap.md -- Tier 3: gap analysis
375
- qa-blueprint.md -- Tier 3: repo blueprint
376
- qa-report.md -- Tier 3: status report
377
- qa-pyramid.md -- Tier 3: pyramid analysis
378
- create-test.md -- Tier 3: create tests for a feature
379
- update-test.md -- Tier 3: improve existing tests
380
- qa-map.md -- Tier 3: deep-scan codebase
381
- qa-research.md -- Tier 3: research testing stack
382
- qa-from-ticket.md -- Tier 3: tests from ticket
383
- qa-pr.md -- Tier 3: create draft PR
384
- CLAUDE.md -- QA standards, agent coordination, quality gates
385
- .planning/ -- Planning artifacts and project state
386
- config.json -- Pipeline configuration
387
- .qa-output/ -- Generated artifacts (created during pipeline run)
388
- ```
389
-
390
- ## Pipeline Stages
391
-
392
- The full pipeline follows this sequence:
393
-
394
- ```
395
- scan -> codebase-map -> analyze -> [testid-inject if frontend] -> plan -> generate -> validate -> [e2e-runner if E2E tests] -> [bug-detective if failures] -> deliver
396
- ```
397
-
398
- | Stage | Agent | Input | Output |
399
- |-------|-------|-------|--------|
400
- | Scan | qa-scanner | Repository source files | SCAN_MANIFEST.md |
401
- | Codebase Map | qa-codebase-mapper (x4 parallel) | SCAN_MANIFEST.md, source files | 8 codebase documents (testability, risk, patterns, existing tests) |
402
- | Analyze | qa-analyzer | SCAN_MANIFEST.md, codebase map | QA_ANALYSIS.md, TEST_INVENTORY.md, blueprint or gap analysis |
403
- | Test ID Inject | qa-testid-injector | Frontend source files | TESTID_AUDIT_REPORT.md, modified source files |
404
- | Plan | qa-planner | TEST_INVENTORY.md, QA_ANALYSIS.md, codebase map | Generation plan (internal) |
405
- | Generate | qa-executor | Generation plan, codebase map | Test files, POMs, fixtures, configs |
406
- | Validate | qa-validator | Generated test files | VALIDATION_REPORT.md |
407
- | E2E Runner | qa-e2e-runner | E2E test files, live app | E2E_RUN_REPORT.md, fixed locators |
408
- | Bug Detective | qa-bug-detective | Test execution results | FAILURE_CLASSIFICATION_REPORT.md |
409
- | Deliver | orchestrator | All artifacts | Git branch + draft PR |
410
-
411
- Each stage produces artifacts consumed by the next. The pipeline will not advance to the next stage until the current stage's artifacts pass verification.
412
-
413
- ## Output Artifacts
414
-
415
- All artifacts are written to the `.qa-output/` directory during a pipeline run:
416
-
417
- | Artifact | Description |
418
- |----------|-------------|
419
- | SCAN_MANIFEST.md | File tree, framework detection, testable surfaces, file priority |
420
- | QA_ANALYSIS.md | Architecture overview, risk assessment, top 10 unit targets, testing pyramid |
421
- | TEST_INVENTORY.md | Every test case with ID, target, inputs, expected outcome, priority |
422
- | QA_REPO_BLUEPRINT.md | Recommended QA repo structure, configs, CI/CD, definition of done |
423
- | GAP_ANALYSIS.md | Coverage gaps between dev and QA repos (Option 2/3 only) |
424
- | VALIDATION_REPORT.md | 4-layer validation results per file, confidence level, fix loop log |
425
- | FAILURE_CLASSIFICATION_REPORT.md | Failure classification: APP BUG, TEST ERROR, ENV ISSUE, INCONCLUSIVE |
426
- | TESTID_AUDIT_REPORT.md | data-testid coverage score, proposed values, decision gate |
427
- | QA_AUDIT_REPORT.md | 6-dimension quality score with weighted calculation |
428
-
429
- ---
430
-
431
- Powered by Claude Code.