qaa-agent 1.9.1 → 1.9.5
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +43 -22
- package/CLAUDE.md +170 -9
- package/README.md +384 -357
- package/VERSION +1 -0
- package/agents/qa-pipeline-orchestrator.md +336 -22
- package/agents/qaa-analyzer.md +0 -1
- package/agents/qaa-bug-detective.md +163 -4
- package/agents/qaa-codebase-mapper.md +50 -1
- package/agents/qaa-discovery.md +421 -384
- package/agents/qaa-e2e-runner.md +163 -1
- package/agents/qaa-executor.md +142 -1
- package/agents/qaa-planner.md +14 -1
- package/agents/qaa-project-researcher.md +194 -0
- package/agents/qaa-scanner.md +77 -1
- package/agents/qaa-testid-injector.md +0 -1
- package/agents/qaa-validator.md +86 -1
- package/bin/install.cjs +375 -253
- package/bin/lib/context7-cache.cjs +299 -0
- package/bin/lib/intent-detector.cjs +488 -0
- package/commands/qa-audit.md +255 -126
- package/commands/qa-create-test.md +666 -365
- package/commands/qa-fix.md +684 -513
- package/commands/qa-map.md +283 -139
- package/commands/qa-pr.md +63 -0
- package/commands/qa-research.md +181 -157
- package/commands/qa-start.md +62 -6
- package/commands/qa-test-report.md +219 -219
- package/frameworks/cypress.json +54 -0
- package/frameworks/jest.json +59 -0
- package/frameworks/playwright.json +58 -0
- package/frameworks/pytest.json +59 -0
- package/frameworks/robot-framework.json +54 -0
- package/frameworks/selenium.json +65 -0
- package/frameworks/vitest.json +57 -0
- package/package.json +7 -3
- package/workflows/qa-analyze.md +100 -4
- package/workflows/qa-from-ticket.md +50 -2
- package/workflows/qa-gap.md +50 -2
- package/workflows/qa-start.md +819 -33
- package/workflows/qa-testid.md +50 -2
- package/workflows/qa-validate.md +50 -2
package/README.md
CHANGED
|
@@ -1,357 +1,384 @@
|
|
|
1
|
-
# QAA - QA Automation Agent
|
|
2
|
-
|
|
3
|
-
[](https://www.npmjs.com/package/qaa-agent)
|
|
4
|
-
[](https://opensource.org/licenses/MIT)
|
|
5
|
-
|
|
6
|
-
Multi-agent QA pipeline for [Claude Code](https://docs.anthropic.com/en/docs/claude-code). Analyzes any codebase, generates a complete test suite following industry standards, validates everything, and delivers the result as a draft pull request.
|
|
7
|
-
|
|
8
|
-
```
|
|
9
|
-
scan →
|
|
10
|
-
```
|
|
11
|
-
|
|
12
|
-
---
|
|
13
|
-
|
|
14
|
-
## The Problem
|
|
15
|
-
|
|
16
|
-
- **Starting from zero is painful** — a new project with no tests means weeks of setup
|
|
17
|
-
- **Coverage gaps are invisible** — without analysis, teams don't know what's missing until production breaks
|
|
18
|
-
- **Standards drift** — different team members write tests differently: inconsistent locators, vague assertions, mixed naming
|
|
19
|
-
- **QA is always behind dev** — features ship faster than tests get written
|
|
20
|
-
|
|
21
|
-
## The Solution
|
|
22
|
-
|
|
23
|
-
QAA runs a pipeline of 12 specialized AI agents, each responsible for one stage:
|
|
24
|
-
|
|
25
|
-
| Stage | What happens | Output |
|
|
26
|
-
|-------|-------------|--------|
|
|
27
|
-
| **Scan** | Detects framework, language, testable surfaces | `SCAN_MANIFEST.md` |
|
|
28
|
-
| **Research** | Investigates testing ecosystem via Context7 MCP and official docs | `TESTING_STACK.md`, `FRAMEWORK_CAPABILITIES.md` |
|
|
29
|
-
| **Map** | Deep-scans codebase with 4 parallel agents (testability, risk, patterns, existing tests) | 8 codebase documents |
|
|
30
|
-
| **Analyze** | Produces risk assessment, test inventory, testing pyramid | `QA_ANALYSIS.md`, `TEST_INVENTORY.md` |
|
|
31
|
-
| **Plan** | Groups test cases by feature, assigns to files, resolves dependencies | `GENERATION_PLAN.md` |
|
|
32
|
-
| **Generate** | Writes test files, POMs, fixtures, configs following project standards | Test suite on disk |
|
|
33
|
-
| **Validate** | 4-layer validation (syntax, structure, dependencies, logic) with auto-fix | `VALIDATION_REPORT.md` |
|
|
34
|
-
| **Deliver** | Creates branch, commits per stage, opens draft PR | Pull request URL |
|
|
35
|
-
|
|
36
|
-
---
|
|
37
|
-
|
|
38
|
-
## Install
|
|
39
|
-
|
|
40
|
-
```bash
|
|
41
|
-
npx qaa-agent
|
|
42
|
-
```
|
|
43
|
-
|
|
44
|
-
The interactive installer:
|
|
45
|
-
|
|
46
|
-
1. Copies agents, commands, skills, templates, and workflows into your runtime directory
|
|
47
|
-
2. Registers **two MCP servers** in your user-scope config (`~/.claude.json`) so they're available in **all projects**:
|
|
48
|
-
- [Playwright MCP](https://www.npmjs.com/package/@playwright/mcp) — live browser control for E2E tests and locator extraction
|
|
49
|
-
- [Context7 MCP](https://www.npmjs.com/package/@upstash/context7-mcp) — up-to-date library documentation on demand
|
|
50
|
-
3. Merges required permissions into `settings.json`
|
|
51
|
-
|
|
52
|
-
**Supported runtimes:** Claude Code, OpenCode
|
|
53
|
-
|
|
54
|
-
**Install scope:** Global (`~/.claude/`, available in all projects) or Local (`./.claude/`, this project only)
|
|
55
|
-
|
|
56
|
-
### Requirements
|
|
57
|
-
|
|
58
|
-
- [Node.js](https://nodejs.org/) 18+
|
|
59
|
-
- [Claude Code](https://docs.anthropic.com/en/docs/claude-code) installed
|
|
60
|
-
|
|
61
|
-
### Bundled MCP servers
|
|
62
|
-
|
|
63
|
-
Both MCP servers are **registered automatically** in `~/.claude.json` when you run `npx qaa-agent`. No manual setup required — once installed, they're available in every Claude Code project on your machine.
|
|
64
|
-
|
|
65
|
-
#### Playwright MCP — live browser control
|
|
66
|
-
|
|
67
|
-
Uses [`@playwright/mcp`](https://www.npmjs.com/package/@playwright/mcp) to:
|
|
68
|
-
|
|
69
|
-
- Open a real browser and navigate your running app
|
|
70
|
-
- Extract actual locators (`data-testid`, ARIA roles, labels) from live pages
|
|
71
|
-
- Run E2E tests, capture failures, and auto-fix locator mismatches
|
|
72
|
-
- Build a persistent **Locator Registry** (`.qa-output/locators/`) that caches real locators across features
|
|
73
|
-
|
|
74
|
-
#### Context7 MCP — up-to-date library docs
|
|
75
|
-
|
|
76
|
-
Uses [`@upstash/context7-mcp`](https://www.npmjs.com/package/@upstash/context7-mcp) to:
|
|
77
|
-
|
|
78
|
-
- Fetch the latest documentation for Playwright, Cypress, Jest, Vitest, pytest, and any other library the agent is working with
|
|
79
|
-
- Keep generated tests aligned with current framework APIs instead of outdated training data
|
|
80
|
-
- Free tier: ~60 requests/hour, ~3,300 tokens/query
|
|
81
|
-
|
|
82
|
-
#### Verifying the MCPs are connected
|
|
83
|
-
|
|
84
|
-
Open Claude Code in any project and type `/mcp`. You should see both `playwright` and `context7` listed as connected.
|
|
85
|
-
|
|
86
|
-
#### Manual config (fallback)
|
|
87
|
-
|
|
88
|
-
If for any reason the automatic registration fails, you can add the servers manually to `~/.claude.json`:
|
|
89
|
-
|
|
90
|
-
```json
|
|
91
|
-
{
|
|
92
|
-
"mcpServers": {
|
|
93
|
-
"playwright": {
|
|
94
|
-
"command": "npx",
|
|
95
|
-
"args": ["@playwright/mcp@latest"]
|
|
96
|
-
},
|
|
97
|
-
"context7": {
|
|
98
|
-
"command": "npx",
|
|
99
|
-
"args": ["-y", "@upstash/context7-mcp@latest"]
|
|
100
|
-
}
|
|
101
|
-
}
|
|
102
|
-
}
|
|
103
|
-
```
|
|
104
|
-
|
|
105
|
-
---
|
|
106
|
-
|
|
107
|
-
## Quick Start
|
|
108
|
-
|
|
109
|
-
### New project, no tests
|
|
110
|
-
|
|
111
|
-
```
|
|
112
|
-
/qa-start --dev-repo ./myproject --auto
|
|
113
|
-
```
|
|
114
|
-
|
|
115
|
-
Runs the full pipeline end-to-end: scan, map, analyze, plan, generate, validate, and deliver as a draft PR.
|
|
116
|
-
|
|
117
|
-
### Mature project, new feature
|
|
118
|
-
|
|
119
|
-
```
|
|
120
|
-
/qa-map # build the "brain" (once)
|
|
121
|
-
/qa-create-test "password reset" # generate tests using codebase knowledge
|
|
122
|
-
/qa-pr --ticket PROJ-123 "password reset tests" # ship as draft PR
|
|
123
|
-
```
|
|
124
|
-
|
|
125
|
-
### From a Jira ticket
|
|
126
|
-
|
|
127
|
-
```
|
|
128
|
-
/qa-from-ticket https://company.atlassian.net/browse/PROJ-456
|
|
129
|
-
/qa-pr --ticket PROJ-456 "login flow tests"
|
|
130
|
-
```
|
|
131
|
-
|
|
132
|
-
###
|
|
133
|
-
|
|
134
|
-
```
|
|
135
|
-
/qa-
|
|
136
|
-
|
|
137
|
-
|
|
138
|
-
|
|
139
|
-
|
|
140
|
-
|
|
141
|
-
|
|
142
|
-
|
|
143
|
-
|
|
144
|
-
|
|
145
|
-
|
|
146
|
-
|
|
147
|
-
|
|
148
|
-
|
|
149
|
-
|
|
150
|
-
|
|
151
|
-
|
|
152
|
-
|
|
153
|
-
|
|
154
|
-
|
|
155
|
-
|
|
156
|
-
|
|
157
|
-
|
|
158
|
-
|
|
159
|
-
|
|
160
|
-
|
|
161
|
-
|
|
162
|
-
|
|
163
|
-
|
|
164
|
-
|
|
165
|
-
|
|
166
|
-
|
|
167
|
-
|
|
|
168
|
-
|
|
169
|
-
|
|
170
|
-
|
|
171
|
-
|
|
172
|
-
|
|
173
|
-
|
|
174
|
-
|
|
175
|
-
|
|
176
|
-
|
|
177
|
-
|
|
178
|
-
|
|
179
|
-
|
|
180
|
-
|
|
181
|
-
|
|
182
|
-
|
|
183
|
-
|
|
184
|
-
|
|
185
|
-
|
|
186
|
-
|
|
187
|
-
|
|
188
|
-
|
|
189
|
-
|
|
190
|
-
|
|
191
|
-
|
|
192
|
-
|
|
193
|
-
|
|
194
|
-
|
|
195
|
-
---
|
|
196
|
-
|
|
197
|
-
##
|
|
198
|
-
|
|
199
|
-
|
|
200
|
-
|
|
201
|
-
|
|
202
|
-
|
|
203
|
-
|
|
204
|
-
|
|
205
|
-
|
|
206
|
-
|
|
207
|
-
|
|
208
|
-
|
|
209
|
-
|
|
210
|
-
|
|
211
|
-
|
|
212
|
-
|
|
213
|
-
|
|
214
|
-
|
|
215
|
-
|
|
216
|
-
|
|
217
|
-
|
|
218
|
-
|
|
219
|
-
|
|
220
|
-
|
|
221
|
-
|
|
222
|
-
|
|
223
|
-
|
|
224
|
-
|
|
225
|
-
|
|
226
|
-
|
|
227
|
-
|
|
228
|
-
|
|
229
|
-
|
|
230
|
-
|
|
231
|
-
|
|
232
|
-
|
|
233
|
-
|
|
234
|
-
|
|
235
|
-
|
|
236
|
-
|
|
237
|
-
|
|
238
|
-
|
|
239
|
-
|
|
240
|
-
|
|
241
|
-
|
|
242
|
-
|
|
243
|
-
|
|
244
|
-
|
|
245
|
-
|
|
246
|
-
|
|
247
|
-
|
|
248
|
-
|
|
249
|
-
|
|
250
|
-
|
|
251
|
-
|
|
252
|
-
|
|
253
|
-
|
|
254
|
-
|
|
255
|
-
|
|
256
|
-
|
|
257
|
-
|
|
258
|
-
|
|
259
|
-
|
|
260
|
-
|
|
261
|
-
|
|
262
|
-
|
|
263
|
-
|
|
264
|
-
|
|
265
|
-
|
|
266
|
-
|
|
267
|
-
|
|
268
|
-
|
|
269
|
-
|
|
270
|
-
|
|
271
|
-
|
|
272
|
-
|
|
273
|
-
|
|
274
|
-
|
|
275
|
-
|
|
276
|
-
|
|
277
|
-
|
|
278
|
-
|
|
279
|
-
|
|
280
|
-
|
|
281
|
-
|
|
282
|
-
|
|
283
|
-
|
|
284
|
-
|
|
285
|
-
|
|
286
|
-
|
|
287
|
-
|
|
288
|
-
|
|
289
|
-
**
|
|
290
|
-
|
|
291
|
-
**
|
|
292
|
-
|
|
293
|
-
|
|
294
|
-
|
|
295
|
-
|
|
296
|
-
|
|
297
|
-
|
|
298
|
-
|
|
299
|
-
|
|
300
|
-
|
|
301
|
-
|
|
302
|
-
|
|
303
|
-
|
|
304
|
-
|
|
305
|
-
|
|
306
|
-
|
|
307
|
-
|
|
308
|
-
|
|
309
|
-
|
|
310
|
-
|
|
311
|
-
|
|
312
|
-
|
|
313
|
-
|
|
314
|
-
|
|
315
|
-
|
|
316
|
-
|
|
317
|
-
|
|
318
|
-
|
|
319
|
-
|
|
320
|
-
|
|
321
|
-
|
|
322
|
-
|
|
323
|
-
|
|
324
|
-
|
|
325
|
-
|
|
326
|
-
|
|
327
|
-
|
|
328
|
-
|
|
329
|
-
|
|
330
|
-
|
|
331
|
-
|
|
332
|
-
|
|
333
|
-
|
|
334
|
-
|
|
335
|
-
|
|
336
|
-
|
|
337
|
-
|
|
338
|
-
|
|
339
|
-
|
|
340
|
-
|
|
341
|
-
|
|
342
|
-
|
|
343
|
-
|
|
344
|
-
|
|
345
|
-
|
|
346
|
-
|
|
347
|
-
|
|
348
|
-
|
|
349
|
-
|
|
350
|
-
|
|
351
|
-
|
|
352
|
-
|
|
353
|
-
|
|
354
|
-
|
|
355
|
-
|
|
356
|
-
|
|
357
|
-
|
|
1
|
+
# QAA - QA Automation Agent
|
|
2
|
+
|
|
3
|
+
[](https://www.npmjs.com/package/qaa-agent)
|
|
4
|
+
[](https://opensource.org/licenses/MIT)
|
|
5
|
+
|
|
6
|
+
Multi-agent QA pipeline for [Claude Code](https://docs.anthropic.com/en/docs/claude-code). Analyzes any codebase, generates a complete test suite following industry standards, validates everything, and delivers the result as a draft pull request.
|
|
7
|
+
|
|
8
|
+
```
|
|
9
|
+
scan → map → research → analyze → plan → generate → validate → deliver
|
|
10
|
+
```
|
|
11
|
+
|
|
12
|
+
---
|
|
13
|
+
|
|
14
|
+
## The Problem
|
|
15
|
+
|
|
16
|
+
- **Starting from zero is painful** — a new project with no tests means weeks of setup
|
|
17
|
+
- **Coverage gaps are invisible** — without analysis, teams don't know what's missing until production breaks
|
|
18
|
+
- **Standards drift** — different team members write tests differently: inconsistent locators, vague assertions, mixed naming
|
|
19
|
+
- **QA is always behind dev** — features ship faster than tests get written
|
|
20
|
+
|
|
21
|
+
## The Solution
|
|
22
|
+
|
|
23
|
+
QAA runs a pipeline of 12 specialized AI agents, each responsible for one stage:
|
|
24
|
+
|
|
25
|
+
| Stage | What happens | Output |
|
|
26
|
+
|-------|-------------|--------|
|
|
27
|
+
| **Scan** | Detects framework, language, testable surfaces | `SCAN_MANIFEST.md` |
|
|
28
|
+
| **Research** | Investigates testing ecosystem via Context7 MCP and official docs | `TESTING_STACK.md`, `FRAMEWORK_CAPABILITIES.md` |
|
|
29
|
+
| **Map** | Deep-scans codebase with 4 parallel agents (testability, risk, patterns, existing tests) | 8 codebase documents |
|
|
30
|
+
| **Analyze** | Produces risk assessment, test inventory, testing pyramid | `QA_ANALYSIS.md`, `TEST_INVENTORY.md` |
|
|
31
|
+
| **Plan** | Groups test cases by feature, assigns to files, resolves dependencies | `GENERATION_PLAN.md` |
|
|
32
|
+
| **Generate** | Writes test files, POMs, fixtures, configs following project standards | Test suite on disk |
|
|
33
|
+
| **Validate** | 4-layer validation (syntax, structure, dependencies, logic) with auto-fix | `VALIDATION_REPORT.md` |
|
|
34
|
+
| **Deliver** | Creates branch, commits per stage, opens draft PR | Pull request URL |
|
|
35
|
+
|
|
36
|
+
---
|
|
37
|
+
|
|
38
|
+
## Install
|
|
39
|
+
|
|
40
|
+
```bash
|
|
41
|
+
npx qaa-agent
|
|
42
|
+
```
|
|
43
|
+
|
|
44
|
+
The interactive installer:
|
|
45
|
+
|
|
46
|
+
1. Copies agents, commands, skills, templates, and workflows into your runtime directory
|
|
47
|
+
2. Registers **two MCP servers** in your user-scope config (`~/.claude.json`) so they're available in **all projects**:
|
|
48
|
+
- [Playwright MCP](https://www.npmjs.com/package/@playwright/mcp) — live browser control for E2E tests and locator extraction
|
|
49
|
+
- [Context7 MCP](https://www.npmjs.com/package/@upstash/context7-mcp) — up-to-date library documentation on demand
|
|
50
|
+
3. Merges required permissions into `settings.json`
|
|
51
|
+
|
|
52
|
+
**Supported runtimes:** Claude Code, OpenCode
|
|
53
|
+
|
|
54
|
+
**Install scope:** Global (`~/.claude/`, available in all projects) or Local (`./.claude/`, this project only)
|
|
55
|
+
|
|
56
|
+
### Requirements
|
|
57
|
+
|
|
58
|
+
- [Node.js](https://nodejs.org/) 18+
|
|
59
|
+
- [Claude Code](https://docs.anthropic.com/en/docs/claude-code) installed
|
|
60
|
+
|
|
61
|
+
### Bundled MCP servers
|
|
62
|
+
|
|
63
|
+
Both MCP servers are **registered automatically** in `~/.claude.json` when you run `npx qaa-agent`. No manual setup required — once installed, they're available in every Claude Code project on your machine.
|
|
64
|
+
|
|
65
|
+
#### Playwright MCP — live browser control
|
|
66
|
+
|
|
67
|
+
Uses [`@playwright/mcp`](https://www.npmjs.com/package/@playwright/mcp) to:
|
|
68
|
+
|
|
69
|
+
- Open a real browser and navigate your running app
|
|
70
|
+
- Extract actual locators (`data-testid`, ARIA roles, labels) from live pages
|
|
71
|
+
- Run E2E tests, capture failures, and auto-fix locator mismatches
|
|
72
|
+
- Build a persistent **Locator Registry** (`.qa-output/locators/`) that caches real locators across features
|
|
73
|
+
|
|
74
|
+
#### Context7 MCP — up-to-date library docs
|
|
75
|
+
|
|
76
|
+
Uses [`@upstash/context7-mcp`](https://www.npmjs.com/package/@upstash/context7-mcp) to:
|
|
77
|
+
|
|
78
|
+
- Fetch the latest documentation for Playwright, Cypress, Jest, Vitest, pytest, and any other library the agent is working with
|
|
79
|
+
- Keep generated tests aligned with current framework APIs instead of outdated training data
|
|
80
|
+
- Free tier: ~60 requests/hour, ~3,300 tokens/query
|
|
81
|
+
|
|
82
|
+
#### Verifying the MCPs are connected
|
|
83
|
+
|
|
84
|
+
Open Claude Code in any project and type `/mcp`. You should see both `playwright` and `context7` listed as connected.
|
|
85
|
+
|
|
86
|
+
#### Manual config (fallback)
|
|
87
|
+
|
|
88
|
+
If for any reason the automatic registration fails, you can add the servers manually to `~/.claude.json`:
|
|
89
|
+
|
|
90
|
+
```json
|
|
91
|
+
{
|
|
92
|
+
"mcpServers": {
|
|
93
|
+
"playwright": {
|
|
94
|
+
"command": "npx",
|
|
95
|
+
"args": ["@playwright/mcp@latest"]
|
|
96
|
+
},
|
|
97
|
+
"context7": {
|
|
98
|
+
"command": "npx",
|
|
99
|
+
"args": ["-y", "@upstash/context7-mcp@latest"]
|
|
100
|
+
}
|
|
101
|
+
}
|
|
102
|
+
}
|
|
103
|
+
```
|
|
104
|
+
|
|
105
|
+
---
|
|
106
|
+
|
|
107
|
+
## Quick Start
|
|
108
|
+
|
|
109
|
+
### New project, no tests
|
|
110
|
+
|
|
111
|
+
```
|
|
112
|
+
/qa-start --dev-repo ./myproject --auto
|
|
113
|
+
```
|
|
114
|
+
|
|
115
|
+
Runs the full pipeline end-to-end: scan, map, analyze, plan, generate, validate, and deliver as a draft PR.
|
|
116
|
+
|
|
117
|
+
### Mature project, new feature
|
|
118
|
+
|
|
119
|
+
```
|
|
120
|
+
/qa-map # build the "brain" (once)
|
|
121
|
+
/qa-create-test "password reset" # generate tests using codebase knowledge
|
|
122
|
+
/qa-pr --ticket PROJ-123 "password reset tests" # ship as draft PR
|
|
123
|
+
```
|
|
124
|
+
|
|
125
|
+
### From a Jira ticket
|
|
126
|
+
|
|
127
|
+
```
|
|
128
|
+
/qa-from-ticket https://company.atlassian.net/browse/PROJ-456
|
|
129
|
+
/qa-pr --ticket PROJ-456 "login flow tests"
|
|
130
|
+
```
|
|
131
|
+
|
|
132
|
+
### Run pipeline against a live app
|
|
133
|
+
|
|
134
|
+
```
|
|
135
|
+
/qa-start --dev-repo ./myproject --app-url https://staging.example.com --auto
|
|
136
|
+
```
|
|
137
|
+
|
|
138
|
+
The `--app-url` flag tells the pipeline that the app is running at that URL. The E2E runner, validator (Layer 5), and testid-injector use it to verify against the real DOM.
|
|
139
|
+
|
|
140
|
+
### Generate tests directly from a URL (single-shot)
|
|
141
|
+
|
|
142
|
+
```
|
|
143
|
+
/qa-create-test https://practicetestautomation.com/practice-test-login/
|
|
144
|
+
```
|
|
145
|
+
|
|
146
|
+
The new `from-aut` mode in `/qa-create-test` navigates the URL with Playwright MCP, discovers interactive elements, and generates tests directly without running the full pipeline. Faster than `/qa-start --app-url` when you only need tests for one specific page.
|
|
147
|
+
|
|
148
|
+
### Natural language input (any command)
|
|
149
|
+
|
|
150
|
+
```
|
|
151
|
+
/qa-start use playwright on my project at C:\Users\me\My Project test against https://staging.example.com
|
|
152
|
+
```
|
|
153
|
+
|
|
154
|
+
The intent detector extracts URLs, paths (with spaces), and framework hints from natural language. Flags always win over NL. The INPUT DETECTION banner shows the resolved values before starting.
|
|
155
|
+
|
|
156
|
+
### Fix broken tests after a deploy
|
|
157
|
+
|
|
158
|
+
```
|
|
159
|
+
/qa-fix ./tests/e2e/checkout*
|
|
160
|
+
/qa-pr --ticket PROJ-789 "fix checkout tests"
|
|
161
|
+
```
|
|
162
|
+
|
|
163
|
+
---
|
|
164
|
+
|
|
165
|
+
## Commands
|
|
166
|
+
|
|
167
|
+
| Command | Purpose |
|
|
168
|
+
|---------|---------|
|
|
169
|
+
| `/qa-start` | Full pipeline end-to-end (scan through PR) |
|
|
170
|
+
| `/qa-research` | Research testing ecosystem via Context7 MCP |
|
|
171
|
+
| `/qa-map` | Deep codebase analysis with 4 parallel agents |
|
|
172
|
+
| `/qa-create-test <feature>` | Generate tests for a specific feature |
|
|
173
|
+
| `/qa-fix [path]` | Diagnose and fix broken tests |
|
|
174
|
+
| `/qa-audit [path]` | 6-dimension quality audit with scoring |
|
|
175
|
+
| `/qa-pr` | Create a draft pull request from QA artifacts |
|
|
176
|
+
| `/qa-testid [path]` | Inject `data-testid` attributes into components |
|
|
177
|
+
|
|
178
|
+
### Additional Commands
|
|
179
|
+
|
|
180
|
+
| Command | Purpose |
|
|
181
|
+
|---------|---------|
|
|
182
|
+
| `/qa-from-ticket <url>` | Generate tests from a Jira/Linear/GitHub Issue |
|
|
183
|
+
| `/qa-analyze` | Analyze a repo without generating tests |
|
|
184
|
+
| `/qa-validate [path]` | Validate test files against standards |
|
|
185
|
+
| `/qa-gap` | Find coverage gaps between dev and QA repos |
|
|
186
|
+
| `/qa-report` | Generate a QA status report |
|
|
187
|
+
| `/qa-audit` | Full quality audit with weighted scoring |
|
|
188
|
+
| `/qa-blueprint` | Generate QA repo structure from scratch |
|
|
189
|
+
| `/qa-research` | Research best testing stack for a project |
|
|
190
|
+
| `/qa-pom` | Generate Page Object Models |
|
|
191
|
+
| `/update-test` | Improve existing tests incrementally |
|
|
192
|
+
|
|
193
|
+
Run any command in Claude Code to see full usage and available flags.
|
|
194
|
+
|
|
195
|
+
---
|
|
196
|
+
|
|
197
|
+
## Three Workflows
|
|
198
|
+
|
|
199
|
+
QAA adapts to the project's QA maturity:
|
|
200
|
+
|
|
201
|
+
**Option 1: No QA repo yet** — Full pipeline from scratch. Produces a complete test suite, repo blueprint, and draft PR.
|
|
202
|
+
|
|
203
|
+
```
|
|
204
|
+
/qa-start --dev-repo ./myproject
|
|
205
|
+
```
|
|
206
|
+
|
|
207
|
+
**Option 2: Immature QA repo** — Scans both repos, fixes broken tests, fills coverage gaps, standardizes existing tests.
|
|
208
|
+
|
|
209
|
+
```
|
|
210
|
+
/qa-start --dev-repo ./myproject --qa-repo ./tests
|
|
211
|
+
```
|
|
212
|
+
|
|
213
|
+
**Option 3: Mature QA repo** — Surgical additions only. Finds thin coverage areas and adds targeted tests without touching working code.
|
|
214
|
+
|
|
215
|
+
```
|
|
216
|
+
/qa-start --dev-repo ./myproject --qa-repo ./tests
|
|
217
|
+
```
|
|
218
|
+
|
|
219
|
+
---
|
|
220
|
+
|
|
221
|
+
## The "Brain" — Codebase Map
|
|
222
|
+
|
|
223
|
+
Before generating anything, QAA maps the codebase with 4 parallel agents producing 8 documents:
|
|
224
|
+
|
|
225
|
+
| Focus | Documents |
|
|
226
|
+
|-------|-----------|
|
|
227
|
+
| **Testability** | `TESTABILITY.md`, `TEST_SURFACE.md` — what's testable, entry points, mock boundaries |
|
|
228
|
+
| **Risk** | `RISK_MAP.md`, `CRITICAL_PATHS.md` — business-critical paths, security-sensitive areas |
|
|
229
|
+
| **Patterns** | `CODE_PATTERNS.md`, `API_CONTRACTS.md` — naming conventions, API shapes, import style |
|
|
230
|
+
| **Existing tests** | `TEST_ASSESSMENT.md`, `COVERAGE_GAPS.md` — current quality, frameworks, gaps |
|
|
231
|
+
|
|
232
|
+
Every downstream agent reads these documents. The result: generated tests feel native to the codebase, not generic boilerplate.
|
|
233
|
+
|
|
234
|
+
---
|
|
235
|
+
|
|
236
|
+
## Standards Enforced
|
|
237
|
+
|
|
238
|
+
Every generated artifact follows strict rules:
|
|
239
|
+
|
|
240
|
+
### Testing Pyramid
|
|
241
|
+
|
|
242
|
+
```
|
|
243
|
+
/ E2E \ 3-5% (critical path smoke only)
|
|
244
|
+
/ API \ 20-25% (endpoints + contracts)
|
|
245
|
+
/ Integration\ 10-15% (component interactions)
|
|
246
|
+
/ Unit \ 60-70% (business logic, pure functions)
|
|
247
|
+
```
|
|
248
|
+
|
|
249
|
+
### Locator Hierarchy
|
|
250
|
+
|
|
251
|
+
1. **Tier 1 (Best):** `data-testid`, ARIA roles with accessible names
|
|
252
|
+
2. **Tier 2 (Good):** Form labels, placeholders, visible text
|
|
253
|
+
3. **Tier 3 (Acceptable):** Alt text, title attributes
|
|
254
|
+
4. **Tier 4 (Last Resort):** CSS selectors, XPath — always with a `// TODO` comment
|
|
255
|
+
|
|
256
|
+
### Page Object Model
|
|
257
|
+
|
|
258
|
+
- One class per page, no god objects
|
|
259
|
+
- No assertions in POMs — assertions belong in test specs
|
|
260
|
+
- Locators as readonly properties
|
|
261
|
+
- Every POM extends a shared `BasePage`
|
|
262
|
+
|
|
263
|
+
### Assertion Quality
|
|
264
|
+
|
|
265
|
+
```typescript
|
|
266
|
+
// Good — concrete values
|
|
267
|
+
expect(response.status).toBe(200);
|
|
268
|
+
expect(data.name).toBe('Test User');
|
|
269
|
+
|
|
270
|
+
// Bad — never do this
|
|
271
|
+
expect(response.status).toBeTruthy();
|
|
272
|
+
expect(data).toBeDefined();
|
|
273
|
+
```
|
|
274
|
+
|
|
275
|
+
### Test Case IDs
|
|
276
|
+
|
|
277
|
+
Every test case has a unique ID following the pattern:
|
|
278
|
+
- `UT-MODULE-001` — unit tests
|
|
279
|
+
- `INT-MODULE-001` — integration tests
|
|
280
|
+
- `API-RESOURCE-001` — API tests
|
|
281
|
+
- `E2E-FLOW-001` — E2E tests
|
|
282
|
+
|
|
283
|
+
---
|
|
284
|
+
|
|
285
|
+
## Validation
|
|
286
|
+
|
|
287
|
+
Generated tests pass through a 4-layer validation with auto-fix (up to 3 loops):
|
|
288
|
+
|
|
289
|
+
1. **Syntax** — does it parse? Are imports correct?
|
|
290
|
+
2. **Structure** — POM rules, file organization, naming conventions
|
|
291
|
+
3. **Dependencies** — all imports resolve, mocks set up correctly
|
|
292
|
+
4. **Logic** — assertions are concrete, locators follow tier hierarchy
|
|
293
|
+
|
|
294
|
+
If issues remain, the **Bug Detective** classifies each failure:
|
|
295
|
+
|
|
296
|
+
| Classification | Action |
|
|
297
|
+
|----------------|--------|
|
|
298
|
+
| `APPLICATION BUG` | Flagged for developer — not auto-fixed |
|
|
299
|
+
| `TEST CODE ERROR` | Auto-fixed at HIGH confidence |
|
|
300
|
+
| `ENVIRONMENT ISSUE` | Documented with setup instructions |
|
|
301
|
+
| `INCONCLUSIVE` | Flagged with evidence for manual review |
|
|
302
|
+
|
|
303
|
+
---
|
|
304
|
+
|
|
305
|
+
## Framework Support
|
|
306
|
+
|
|
307
|
+
QAA auto-detects the project's existing stack and matches it:
|
|
308
|
+
|
|
309
|
+
**Languages:** JavaScript/TypeScript, Python, Java, .NET/C#, Go, Ruby, PHP, Rust
|
|
310
|
+
|
|
311
|
+
**Test Frameworks:** Playwright, Cypress, Jest, Vitest, pytest, Selenium, Robot Framework, and more
|
|
312
|
+
|
|
313
|
+
**Build Tools:** Vite, Next.js, Nuxt, Angular, Vue, Webpack, SvelteKit
|
|
314
|
+
|
|
315
|
+
**Git Platforms:** GitHub, Azure DevOps, GitLab
|
|
316
|
+
|
|
317
|
+
---
|
|
318
|
+
|
|
319
|
+
## Learning System
|
|
320
|
+
|
|
321
|
+
QAA remembers your preferences across sessions. When you correct it — "use Playwright, not Cypress" or "our branches start with `feature/`" — it saves the rule permanently to `MY_PREFERENCES.md`. Every agent reads your preferences before generating output.
|
|
322
|
+
|
|
323
|
+
Your team's conventions always win over defaults.
|
|
324
|
+
|
|
325
|
+
---
|
|
326
|
+
|
|
327
|
+
## Architecture
|
|
328
|
+
|
|
329
|
+
```
|
|
330
|
+
qaa-agent/
|
|
331
|
+
agents/ # 12 specialized QA agents
|
|
332
|
+
commands/ # 9 slash commands (user-facing entry points)
|
|
333
|
+
skills/ # 6 reusable skills
|
|
334
|
+
templates/ # 10 artifact templates (output format contracts)
|
|
335
|
+
workflows/ # 7 workflow orchestration specs
|
|
336
|
+
frameworks/ # 7 framework registry entries (one JSON per framework)
|
|
337
|
+
bin/ # Installer + intent-detector + context7-cache helpers
|
|
338
|
+
bin/lib/ # intent-detector.cjs, context7-cache.cjs (shared modules)
|
|
339
|
+
docs/ # User documentation
|
|
340
|
+
CLAUDE.md # QA standards (read by every agent)
|
|
341
|
+
.mcp.json # Playwright + Context7 MCP server config
|
|
342
|
+
settings.json # Claude Code permissions
|
|
343
|
+
VERSION # Current version (mirrors package.json version)
|
|
344
|
+
```
|
|
345
|
+
|
|
346
|
+
### Agents
|
|
347
|
+
|
|
348
|
+
| Agent | Responsibility |
|
|
349
|
+
|-------|---------------|
|
|
350
|
+
| `qa-scanner` | Framework detection, file tree scanning |
|
|
351
|
+
| `qa-codebase-mapper` | 4-parallel-agent deep analysis |
|
|
352
|
+
| `qa-analyzer` | Risk assessment, test inventory, pyramid |
|
|
353
|
+
| `qa-planner` | Test case grouping, file assignment |
|
|
354
|
+
| `qa-executor` | Test file, POM, fixture generation |
|
|
355
|
+
| `qa-validator` | 4-layer validation with auto-fix |
|
|
356
|
+
| `qa-e2e-runner` | Browser-based test execution via Playwright MCP |
|
|
357
|
+
| `qa-bug-detective` | Failure classification with evidence |
|
|
358
|
+
| `qa-testid-injector` | `data-testid` attribute injection |
|
|
359
|
+
| `qa-project-researcher` | Testing stack research |
|
|
360
|
+
| `qa-discovery` | Project discovery |
|
|
361
|
+
| `qa-pipeline-orchestrator` | Pipeline coordination |
|
|
362
|
+
|
|
363
|
+
---
|
|
364
|
+
|
|
365
|
+
## Git Workflow
|
|
366
|
+
|
|
367
|
+
QAA follows strict git conventions:
|
|
368
|
+
|
|
369
|
+
- **Branch:** `qa/auto-{project}-{date}` (e.g., `qa/auto-shopflow-2026-03-18`)
|
|
370
|
+
- **Commits:** One per agent stage — `qa(scanner): produce SCAN_MANIFEST.md for shopflow`
|
|
371
|
+
- **PR:** Draft PR with analysis summary, test counts, coverage metrics, validation status
|
|
372
|
+
|
|
373
|
+
---
|
|
374
|
+
|
|
375
|
+
## Documentation
|
|
376
|
+
|
|
377
|
+
All documentation is included in the installed package under `docs/`, `templates/`, and `CLAUDE.md`.
|
|
378
|
+
|
|
379
|
+
---
|
|
380
|
+
|
|
381
|
+
## License
|
|
382
|
+
|
|
383
|
+
MIT
|
|
384
|
+
|