qaa-agent 1.9.2 → 1.9.5
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +206 -179
- package/CLAUDE.md +718 -557
- package/README.md +384 -357
- package/VERSION +1 -1
- package/agents/qa-pipeline-orchestrator.md +1739 -1425
- package/agents/qaa-analyzer.md +0 -1
- package/agents/qaa-bug-detective.md +790 -655
- package/agents/qaa-codebase-mapper.md +50 -1
- package/agents/qaa-discovery.md +421 -384
- package/agents/qaa-e2e-runner.md +715 -577
- package/agents/qaa-executor.md +947 -830
- package/agents/qaa-planner.md +14 -1
- package/agents/qaa-project-researcher.md +533 -400
- package/agents/qaa-scanner.md +77 -1
- package/agents/qaa-testid-injector.md +0 -1
- package/agents/qaa-validator.md +86 -1
- package/bin/install.cjs +375 -253
- package/bin/lib/context7-cache.cjs +299 -0
- package/bin/lib/intent-detector.cjs +488 -0
- package/commands/qa-audit.md +255 -126
- package/commands/qa-create-test.md +666 -365
- package/commands/qa-fix.md +684 -513
- package/commands/qa-map.md +283 -139
- package/commands/qa-pr.md +63 -0
- package/commands/qa-research.md +181 -157
- package/commands/qa-start.md +62 -6
- package/commands/qa-test-report.md +219 -219
- package/frameworks/cypress.json +54 -0
- package/frameworks/jest.json +59 -0
- package/frameworks/playwright.json +58 -0
- package/frameworks/pytest.json +59 -0
- package/frameworks/robot-framework.json +54 -0
- package/frameworks/selenium.json +65 -0
- package/frameworks/vitest.json +57 -0
- package/package.json +5 -2
- package/workflows/qa-analyze.md +100 -4
- package/workflows/qa-from-ticket.md +50 -2
- package/workflows/qa-gap.md +50 -2
- package/workflows/qa-start.md +2048 -1405
- package/workflows/qa-testid.md +50 -2
- package/workflows/qa-validate.md +50 -2
|
@@ -1,365 +1,666 @@
|
|
|
1
|
-
# QA Create Test
|
|
2
|
-
|
|
3
|
-
Create, update, or generate tests from tickets — all in one command. Supports
|
|
4
|
-
|
|
5
|
-
|
|
6
|
-
|
|
7
|
-
|
|
8
|
-
|
|
9
|
-
|
|
10
|
-
|
|
11
|
-
|
|
12
|
-
|
|
13
|
-
|
|
14
|
-
|
|
15
|
-
|
|
16
|
-
|
|
17
|
-
|
|
18
|
-
|
|
19
|
-
|
|
20
|
-
|
|
21
|
-
|
|
22
|
-
|
|
23
|
-
|
|
24
|
-
|
|
25
|
-
|
|
26
|
-
-
|
|
27
|
-
|
|
28
|
-
|
|
29
|
-
|
|
30
|
-
|
|
31
|
-
|
|
32
|
-
-
|
|
33
|
-
|
|
34
|
-
|
|
35
|
-
|
|
36
|
-
|
|
37
|
-
|
|
38
|
-
|
|
39
|
-
|
|
40
|
-
|
|
41
|
-
|
|
42
|
-
|
|
43
|
-
|
|
44
|
-
|
|
45
|
-
|
|
46
|
-
|
|
47
|
-
|
|
48
|
-
|
|
49
|
-
|
|
50
|
-
|
|
51
|
-
|
|
52
|
-
|
|
53
|
-
###
|
|
54
|
-
|
|
55
|
-
|
|
56
|
-
-
|
|
57
|
-
-
|
|
58
|
-
|
|
59
|
-
|
|
60
|
-
|
|
61
|
-
|
|
62
|
-
|
|
63
|
-
|
|
64
|
-
-
|
|
65
|
-
|
|
66
|
-
|
|
67
|
-
|
|
68
|
-
|
|
69
|
-
|
|
70
|
-
|
|
71
|
-
|
|
72
|
-
|
|
73
|
-
|
|
74
|
-
|
|
75
|
-
|
|
76
|
-
|
|
77
|
-
|
|
78
|
-
|
|
79
|
-
|
|
80
|
-
|
|
81
|
-
|
|
82
|
-
|
|
83
|
-
|
|
84
|
-
|
|
85
|
-
|
|
86
|
-
|
|
87
|
-
|
|
88
|
-
|
|
89
|
-
|
|
90
|
-
|
|
91
|
-
|
|
92
|
-
|
|
93
|
-
|
|
94
|
-
|
|
95
|
-
|
|
96
|
-
|
|
97
|
-
|
|
98
|
-
|
|
99
|
-
|
|
100
|
-
|
|
101
|
-
|
|
102
|
-
|
|
103
|
-
|
|
104
|
-
|
|
105
|
-
|
|
106
|
-
|
|
107
|
-
|
|
108
|
-
|
|
109
|
-
|
|
110
|
-
|
|
111
|
-
|
|
112
|
-
|
|
113
|
-
|
|
114
|
-
|
|
115
|
-
|
|
116
|
-
|
|
117
|
-
|
|
118
|
-
|
|
119
|
-
|
|
120
|
-
|
|
121
|
-
|
|
122
|
-
|
|
123
|
-
|
|
124
|
-
|
|
125
|
-
|
|
126
|
-
|
|
127
|
-
|
|
128
|
-
|
|
129
|
-
|
|
130
|
-
|
|
131
|
-
|
|
132
|
-
|
|
133
|
-
|
|
134
|
-
|
|
135
|
-
|
|
136
|
-
|
|
137
|
-
|
|
138
|
-
|
|
139
|
-
|
|
140
|
-
|
|
141
|
-
|
|
142
|
-
|
|
143
|
-
|
|
144
|
-
|
|
145
|
-
|
|
146
|
-
|
|
147
|
-
|
|
148
|
-
|
|
149
|
-
|
|
150
|
-
|
|
151
|
-
|
|
152
|
-
|
|
153
|
-
|
|
154
|
-
|
|
155
|
-
|
|
156
|
-
|
|
157
|
-
|
|
158
|
-
|
|
159
|
-
|
|
160
|
-
|
|
161
|
-
|
|
162
|
-
|
|
163
|
-
)
|
|
164
|
-
|
|
165
|
-
|
|
166
|
-
|
|
167
|
-
|
|
168
|
-
|
|
169
|
-
|
|
170
|
-
|
|
171
|
-
|
|
172
|
-
|
|
173
|
-
|
|
174
|
-
|
|
175
|
-
|
|
176
|
-
|
|
177
|
-
|
|
178
|
-
|
|
179
|
-
|
|
180
|
-
|
|
181
|
-
|
|
182
|
-
|
|
183
|
-
|
|
184
|
-
|
|
185
|
-
|
|
186
|
-
|
|
187
|
-
|
|
188
|
-
|
|
189
|
-
|
|
190
|
-
|
|
191
|
-
|
|
192
|
-
|
|
193
|
-
|
|
194
|
-
2
|
|
195
|
-
|
|
196
|
-
|
|
197
|
-
|
|
198
|
-
|
|
199
|
-
|
|
200
|
-
|
|
201
|
-
-
|
|
202
|
-
|
|
203
|
-
-
|
|
204
|
-
|
|
205
|
-
|
|
206
|
-
|
|
207
|
-
|
|
208
|
-
|
|
209
|
-
|
|
210
|
-
|
|
211
|
-
|
|
212
|
-
|
|
213
|
-
|
|
214
|
-
|
|
215
|
-
|
|
216
|
-
|
|
217
|
-
|
|
218
|
-
|
|
219
|
-
|
|
220
|
-
|
|
221
|
-
|
|
222
|
-
|
|
223
|
-
|
|
224
|
-
|
|
225
|
-
|
|
226
|
-
|
|
227
|
-
|
|
228
|
-
|
|
229
|
-
|
|
230
|
-
|
|
231
|
-
|
|
232
|
-
|
|
233
|
-
|
|
234
|
-
|
|
235
|
-
|
|
236
|
-
|
|
237
|
-
|
|
238
|
-
|
|
239
|
-
|
|
240
|
-
|
|
241
|
-
|
|
242
|
-
|
|
243
|
-
|
|
244
|
-
|
|
245
|
-
|
|
246
|
-
|
|
247
|
-
|
|
248
|
-
|
|
249
|
-
|
|
250
|
-
|
|
251
|
-
|
|
252
|
-
|
|
253
|
-
|
|
254
|
-
|
|
255
|
-
|
|
256
|
-
|
|
257
|
-
|
|
258
|
-
|
|
259
|
-
|
|
260
|
-
|
|
261
|
-
|
|
262
|
-
|
|
263
|
-
-
|
|
264
|
-
-
|
|
265
|
-
|
|
266
|
-
|
|
267
|
-
|
|
268
|
-
|
|
269
|
-
|
|
270
|
-
</
|
|
271
|
-
|
|
272
|
-
|
|
273
|
-
|
|
274
|
-
|
|
275
|
-
|
|
276
|
-
|
|
277
|
-
|
|
278
|
-
|
|
279
|
-
|
|
280
|
-
|
|
281
|
-
|
|
282
|
-
|
|
283
|
-
|
|
284
|
-
|
|
285
|
-
|
|
286
|
-
|
|
287
|
-
|
|
288
|
-
|
|
289
|
-
|
|
290
|
-
|
|
291
|
-
|
|
292
|
-
|
|
293
|
-
|
|
294
|
-
|
|
295
|
-
|
|
296
|
-
|
|
297
|
-
|
|
298
|
-
|
|
299
|
-
|
|
300
|
-
|
|
301
|
-
|
|
302
|
-
|
|
303
|
-
|
|
304
|
-
|
|
305
|
-
|
|
306
|
-
|
|
307
|
-
|
|
308
|
-
|
|
309
|
-
|
|
310
|
-
|
|
311
|
-
|
|
312
|
-
|
|
313
|
-
|
|
314
|
-
|
|
315
|
-
|
|
316
|
-
|
|
317
|
-
-
|
|
318
|
-
|
|
319
|
-
|
|
320
|
-
|
|
321
|
-
|
|
322
|
-
|
|
323
|
-
|
|
324
|
-
|
|
325
|
-
|
|
326
|
-
|
|
327
|
-
|
|
328
|
-
|
|
329
|
-
|
|
330
|
-
|
|
331
|
-
|
|
332
|
-
|
|
333
|
-
|
|
334
|
-
**
|
|
335
|
-
|
|
336
|
-
|
|
337
|
-
|
|
338
|
-
|
|
339
|
-
|
|
340
|
-
|
|
341
|
-
|
|
342
|
-
|
|
343
|
-
|
|
344
|
-
|
|
345
|
-
|
|
346
|
-
|
|
347
|
-
|
|
348
|
-
|
|
349
|
-
|
|
350
|
-
|
|
351
|
-
|
|
352
|
-
|
|
353
|
-
|
|
354
|
-
|
|
355
|
-
|
|
356
|
-
|
|
357
|
-
|
|
358
|
-
|
|
359
|
-
|
|
360
|
-
|
|
361
|
-
|
|
362
|
-
|
|
363
|
-
|
|
364
|
-
|
|
365
|
-
|
|
1
|
+
# QA Create Test
|
|
2
|
+
|
|
3
|
+
Create, update, or generate tests from tickets — all in one command. Supports six modes: generate tests from code analysis, generate tests from a live app URL (from-aut), generate tests from a ticket (Jira/Linear/GitHub), create Test Cases in Azure DevOps from a work item, update/improve existing tests, or generate POM files only. Uses Playwright MCP to extract real locators from the live app when available.
|
|
4
|
+
|
|
5
|
+
|
|
6
|
+
## ⚠ MANDATORY: How to Execute This Command
|
|
7
|
+
|
|
8
|
+
Execute these steps IN ORDER. Every step is mandatory. Do NOT improvise,
|
|
9
|
+
reorder, skip, or invent steps.
|
|
10
|
+
|
|
11
|
+
1. FIRST tool call: run the intent detector with `$ARGUMENTS` passed LITERALLY
|
|
12
|
+
(the actual command lives in the "## Intent Detection" section below).
|
|
13
|
+
2. Print the INPUT DETECTION banner (exact output from intent-detector.cjs).
|
|
14
|
+
3. Determine `ALL_FROM_FLAGS`. If interactive (`IS_AUTO=false`) AND any value
|
|
15
|
+
came from NL/default (`ALL_FROM_FLAGS=false`), PAUSE for yes/no confirmation.
|
|
16
|
+
Do NOT proceed without it.
|
|
17
|
+
```bash
|
|
18
|
+
ALL_FROM_FLAGS=$(echo "$RESOLVED" | node -e "const j=JSON.parse(require('fs').readFileSync(0,'utf8')); const sources=Object.values(j.resolved).map(v=>v.source); console.log(sources.length>0 && sources.every(s=>s==='flag') ? 'true' : 'false')")
|
|
19
|
+
```
|
|
20
|
+
Pause prompt (when applicable):
|
|
21
|
+
```
|
|
22
|
+
The values above were resolved from natural language and/or defaults.
|
|
23
|
+
Do you want to continue with these inputs? (yes/no)
|
|
24
|
+
```
|
|
25
|
+
4. Determine MODE using the "Mode Detection Logic" section below
|
|
26
|
+
(pom-only → ado → from-ticket → from-aut → update → from-code). Use the
|
|
27
|
+
detector's `intent.suggested_mode` as the heuristic — do NOT decide the mode
|
|
28
|
+
by your own NL parsing.
|
|
29
|
+
5. Print the mode banner (Step 1 of Instructions) and execute that mode's flow
|
|
30
|
+
IN ORDER, including its own approval gates (e.g. update mode presents the
|
|
31
|
+
audit and waits for approval). Modes that generate files spawn the
|
|
32
|
+
qaa-executor (or mode-specific) sub-agent — do NOT hand-write test files
|
|
33
|
+
inline instead of the sub-agent.
|
|
34
|
+
|
|
35
|
+
### DO NOT
|
|
36
|
+
- Skip the INPUT DETECTION banner or the mode banner
|
|
37
|
+
- Run AskUserQuestion or any tool before the intent detector
|
|
38
|
+
- Use AskUserQuestion in place of the yes/no confirmation pause — use the exact text prompt above
|
|
39
|
+
- Decide the mode by parsing NL in your own head — use the detector + Mode Detection Logic
|
|
40
|
+
- Generate or overwrite any test files before the inputs are confirmed (step 3)
|
|
41
|
+
- Hand-write test files inline instead of spawning the sub-agent the mode specifies
|
|
42
|
+
- Delete or rewrite working tests without user approval (update mode is surgical: add / fix / improve, never replace)
|
|
43
|
+
- Assume context, app URL, framework, ticket, or any value from previous command
|
|
44
|
+
invocations (/qa-start, prior /qa-create-test calls, etc.). Each invocation is
|
|
45
|
+
self-contained — the detector returns defaults precisely because the user did
|
|
46
|
+
not pass them.
|
|
47
|
+
- Invent steps not in this command, or reorder the ones that are
|
|
48
|
+
|
|
49
|
+
If you find yourself improvising, deciding the mode before the detector runs,
|
|
50
|
+
generating files before confirmation, or asking questions out of order — STOP
|
|
51
|
+
and restart from step 1.
|
|
52
|
+
|
|
53
|
+
### Pass user input literally to the intent detector
|
|
54
|
+
|
|
55
|
+
When you run the intent detector with `$ARGUMENTS`, substitute that placeholder
|
|
56
|
+
with the RAW user input as received, NOT a pre-translated flag string. If you
|
|
57
|
+
pre-translate, the detector reports every value as `source: flag`, sets
|
|
58
|
+
`ALL_FROM_FLAGS=true`, and bypasses the confirmation checkpoint.
|
|
59
|
+
|
|
60
|
+
|
|
61
|
+
## Usage
|
|
62
|
+
|
|
63
|
+
```
|
|
64
|
+
/qa-create-test <feature-or-source> [options]
|
|
65
|
+
```
|
|
66
|
+
|
|
67
|
+
### Modes (auto-detected from arguments)
|
|
68
|
+
|
|
69
|
+
| Mode | Trigger | Example |
|
|
70
|
+
|------|---------|---------|
|
|
71
|
+
| **From code** | Feature name (no URL, no path to tests) | `/qa-create-test login` |
|
|
72
|
+
| **From ticket** | URL, shorthand (#123), or `--ticket` flag | `/qa-create-test https://github.com/org/repo/issues/42` |
|
|
73
|
+
| **From AUT** | Non-ticket URL or `--app-url` (explore live app) | `/qa-create-test signup on https://staging.example.com` |
|
|
74
|
+
| **Azure DevOps** | `--ado` flag with work item ID or ADO URL | `/qa-create-test --ado 85508` |
|
|
75
|
+
| **Update existing** | Path to existing test files or `--update` flag | `/qa-create-test --update tests/e2e/` |
|
|
76
|
+
| **POM only** | `--pom-only` flag | `/qa-create-test --pom-only src/pages/` |
|
|
77
|
+
|
|
78
|
+
### Options
|
|
79
|
+
|
|
80
|
+
- `--dev-repo <path>` — path to developer repository (default: current directory)
|
|
81
|
+
- `--app-url <url>` — URL of running application for E2E execution and locator extraction (auto-detects if not provided)
|
|
82
|
+
- `--skip-run` — skip E2E execution, only generate and statically validate
|
|
83
|
+
- `--ticket <source>` — force ticket mode with: URL, shorthand (#123, org/repo#123), file path, or plain text
|
|
84
|
+
- `--update <path>` — force update mode: audit and improve existing tests at path
|
|
85
|
+
- `--scope fix|improve|add|full` — for update mode only (default: full)
|
|
86
|
+
- `--ado <work-item-id>` — Azure DevOps mode: read a work item and create Test Cases in ADO (accepts ID or full ADO URL)
|
|
87
|
+
- `--area-path <path>` — (ADO mode) override area path for created test cases (default: source work item's area path)
|
|
88
|
+
- `--iteration-path <path>` — (ADO mode) override iteration path for created test cases (default: source work item's iteration path)
|
|
89
|
+
- `--skip-dedup` — (ADO mode) skip deduplication check against existing linked test cases
|
|
90
|
+
- `--pom-only [path]` — generate only Page Object Model files (BasePage + feature POMs), no test specs
|
|
91
|
+
- `--framework <name>` — override framework auto-detection (playwright, cypress, selenium) — used with --pom-only
|
|
92
|
+
|
|
93
|
+
### Mode Detection Logic
|
|
94
|
+
|
|
95
|
+
```
|
|
96
|
+
if --pom-only:
|
|
97
|
+
MODE = "pom-only"
|
|
98
|
+
elif --ado flag OR argument matches ADO URL (dev.azure.com, *.visualstudio.com):
|
|
99
|
+
MODE = "ado"
|
|
100
|
+
elif argument matches ticket URL pattern (github.com, atlassian.net, linear.app) OR contains "#" + digits OR --ticket flag:
|
|
101
|
+
MODE = "from-ticket"
|
|
102
|
+
elif argument matches URL pattern (https?://) AND NOT a ticket platform:
|
|
103
|
+
MODE = "from-aut" # NEW: lightweight URL → tests, single-shot
|
|
104
|
+
elif --update flag OR argument is path to existing test directory/files:
|
|
105
|
+
MODE = "update"
|
|
106
|
+
else:
|
|
107
|
+
MODE = "from-code"
|
|
108
|
+
```
|
|
109
|
+
|
|
110
|
+
**Mode detection uses `bin/lib/intent-detector.cjs`** for robust URL/path/framework parsing. The detector already classifies URLs as ticket vs AUT, so the workflow can branch cleanly between `from-ticket` and `from-aut` modes.
|
|
111
|
+
|
|
112
|
+
## What It Produces
|
|
113
|
+
|
|
114
|
+
### From Code Mode
|
|
115
|
+
- Test spec files (unit, API, E2E as appropriate)
|
|
116
|
+
- Page Object Model files (for E2E tests)
|
|
117
|
+
- Fixture files (test data)
|
|
118
|
+
- Locator registry entries (`.qa-output/locators/`)
|
|
119
|
+
- E2E_RUN_REPORT.md (if E2E tests ran against live app)
|
|
120
|
+
|
|
121
|
+
### From Ticket Mode
|
|
122
|
+
- TEST_CASES_FROM_TICKET.md — traceability matrix (AC → test case)
|
|
123
|
+
- GENERATION_PLAN_TICKET.md — synthetic generation plan
|
|
124
|
+
- Test spec files with `traces_to` fields linking back to ticket ACs
|
|
125
|
+
- VALIDATION_REPORT.md
|
|
126
|
+
|
|
127
|
+
### Azure DevOps Mode
|
|
128
|
+
- Test Cases created directly in Azure DevOps (via `testplan_create_test_case`)
|
|
129
|
+
- Test Cases linked to source work item via *Tested By* relationship
|
|
130
|
+
- Tags applied: `Smoke`, `Regression`, `Critical`, `AutomationCandidate`, `NeedsReview`
|
|
131
|
+
- `ai-tasks/ticket-{id}/test-cases.md` — structured report
|
|
132
|
+
- Report attached to work item (if `ADO_MCP_AUTH_TOKEN` is set) or written to `Custom.QATestCasesReport` field (fallback)
|
|
133
|
+
|
|
134
|
+
### From AUT Mode
|
|
135
|
+
|
|
136
|
+
- Test spec files generated directly from URL exploration (no full pipeline)
|
|
137
|
+
- Page Object Model files for the discovered pages
|
|
138
|
+
- Locator registry entries from real DOM (via Playwright MCP)
|
|
139
|
+
- VALIDATION_REPORT.md
|
|
140
|
+
- No PR created — local delivery only
|
|
141
|
+
|
|
142
|
+
The agent navigates the URL with Playwright MCP, discovers interactive elements, and generates tests targeting them. Skips codebase-map check, scan, and analyze stages. **Faster than `/qa-start --app-url`** when you only need tests for one specific URL.
|
|
143
|
+
|
|
144
|
+
### Update Mode
|
|
145
|
+
- QA_AUDIT_REPORT.md — current quality assessment
|
|
146
|
+
- Improved test files (after user approval)
|
|
147
|
+
|
|
148
|
+
|
|
149
|
+
## Intent Detection
|
|
150
|
+
|
|
151
|
+
Before processing arguments, run the intent detector to support natural-language input alongside flags. Flags always win over NL.
|
|
152
|
+
|
|
153
|
+
```bash
|
|
154
|
+
RESOLVED=$(node ~/.claude/qaa/bin/lib/intent-detector.cjs --resolve "$ARGUMENTS" --aliases '{"app_url":"app-url","framework":"framework","ticket":"ticket","ado":"ado","update":"update","pom_only":"pom-only","skip_map":"skip-map","skip_run":"skip-run"}')
|
|
155
|
+
|
|
156
|
+
# Print INPUT DETECTION banner (always shown, even if defaults are used)
|
|
157
|
+
echo "$RESOLVED" | node -e "const j=JSON.parse(require('fs').readFileSync(0,'utf8')); console.log(j.banner)"
|
|
158
|
+
|
|
159
|
+
# Extract resolved values
|
|
160
|
+
APP_URL=$(echo "$RESOLVED" | node -e "const j=JSON.parse(require('fs').readFileSync(0,'utf8')); console.log(j.resolved.app_url?.value || '')")
|
|
161
|
+
FRAMEWORK=$(echo "$RESOLVED" | node -e "const j=JSON.parse(require('fs').readFileSync(0,'utf8')); console.log(j.resolved.framework?.value || '')")
|
|
162
|
+
TICKET=$(echo "$RESOLVED" | node -e "const j=JSON.parse(require('fs').readFileSync(0,'utf8')); console.log(j.resolved.ticket?.value || '')")
|
|
163
|
+
SUGGESTED_MODE=$(echo "$RESOLVED" | node -e "const j=JSON.parse(require('fs').readFileSync(0,'utf8')); console.log(j.intent.suggested_mode || '')")
|
|
164
|
+
```
|
|
165
|
+
|
|
166
|
+
The detector resolves the mode automatically when possible (from-ticket, from-aut, ado, etc.) and surfaces it in `SUGGESTED_MODE`. The mode detection logic in Step 1 below uses these resolved values.
|
|
167
|
+
|
|
168
|
+
## Instructions
|
|
169
|
+
|
|
170
|
+
### Step 1: Detect Mode
|
|
171
|
+
|
|
172
|
+
Parse `$ARGUMENTS` to determine mode using the detection logic above.
|
|
173
|
+
|
|
174
|
+
Print mode banner:
|
|
175
|
+
```
|
|
176
|
+
=== QA Create Test ===
|
|
177
|
+
Mode: {from-code | from-aut | from-ticket | ado | update | pom-only}
|
|
178
|
+
Target: {feature name | ticket URL | ADO work item ID | test path}
|
|
179
|
+
App URL: {url or "auto-detect"}
|
|
180
|
+
===========================
|
|
181
|
+
```
|
|
182
|
+
|
|
183
|
+
---
|
|
184
|
+
|
|
185
|
+
### FROM CODE MODE
|
|
186
|
+
|
|
187
|
+
1. Read `CLAUDE.md` — POM rules, locator tiers, assertion rules, naming conventions, quality gates.
|
|
188
|
+
1b. Read `~/.claude/qaa/MY_PREFERENCES.md` if it exists — user's personal QA preferences override CLAUDE.md defaults when there is a conflict.
|
|
189
|
+
2. Read existing analysis artifacts if available:
|
|
190
|
+
- `.qa-output/QA_ANALYSIS.md` — architecture context
|
|
191
|
+
- `.qa-output/TEST_INVENTORY.md` — pre-defined test cases for this feature
|
|
192
|
+
3. **Check for codebase map** (`.qa-output/codebase/`):
|
|
193
|
+
- Look for: `CODE_PATTERNS.md`, `API_CONTRACTS.md`, `TEST_SURFACE.md`, `TESTABILITY.md`
|
|
194
|
+
- If at least 2 of these files exist: read them all for project context (naming conventions, API shapes, testable surfaces).
|
|
195
|
+
- **If NONE of these files exist: STOP and tell the user:**
|
|
196
|
+
```
|
|
197
|
+
⚠ No codebase map found (.qa-output/codebase/ is empty or missing).
|
|
198
|
+
|
|
199
|
+
The codebase map provides critical context: naming conventions, API contracts,
|
|
200
|
+
testable surfaces, and project structure. Without it, generated tests will lack
|
|
201
|
+
project-specific context and may not follow your repo's conventions.
|
|
202
|
+
|
|
203
|
+
Run /qa-map first to generate the codebase map, then re-run /qa-create-test.
|
|
204
|
+
|
|
205
|
+
To skip this check and proceed without context: re-run with --skip-map
|
|
206
|
+
```
|
|
207
|
+
Only proceed without codebase map if the user explicitly passes `--skip-map`.
|
|
208
|
+
|
|
209
|
+
4. **Check existing locator registry and extract new locators from live app:**
|
|
210
|
+
|
|
211
|
+
a. Read `.qa-output/locators/LOCATOR_REGISTRY.md` if it exists.
|
|
212
|
+
|
|
213
|
+
b. If locators for this feature already exist in the registry AND no `--app-url` was provided: reuse cached locators.
|
|
214
|
+
|
|
215
|
+
c. If locators are missing or `--app-url` was provided: Use Playwright MCP to navigate the app and extract real locators:
|
|
216
|
+
```
|
|
217
|
+
mcp__playwright__browser_navigate({ url: "{app_url}/{feature_path}" })
|
|
218
|
+
mcp__playwright__browser_snapshot()
|
|
219
|
+
```
|
|
220
|
+
Extract all data-testid, ARIA roles, labels, placeholders.
|
|
221
|
+
Navigate through multi-page flows if needed.
|
|
222
|
+
Write per-feature locator file to `.qa-output/locators/{feature}.locators.md`.
|
|
223
|
+
Update the registry `.qa-output/locators/LOCATOR_REGISTRY.md`.
|
|
224
|
+
|
|
225
|
+
If no app URL available and no locators in registry, skip — executor proposes locators from source code.
|
|
226
|
+
|
|
227
|
+
5. Invoke executor agent to generate test files:
|
|
228
|
+
|
|
229
|
+
Task(
|
|
230
|
+
prompt="
|
|
231
|
+
<critical_directive priority="MANDATORY">
|
|
232
|
+
You are executing as a QAA pipeline agent. Your FIRST action MUST be to read
|
|
233
|
+
~/.claude/qaa/agents/qaa-executor.md and adopt it as your operating contract.
|
|
234
|
+
|
|
235
|
+
Even though you may be running as a general-purpose agent (native subagent type
|
|
236
|
+
routing for QAA agents is not yet enabled), you MUST behave exactly as the
|
|
237
|
+
dedicated agent defined in that file: read everything in its <required_reading>,
|
|
238
|
+
obey its <quality_gate>, and run its mandatory bash checklist before returning.
|
|
239
|
+
|
|
240
|
+
Never modify application source code (src/, app/, lib/, or equivalent
|
|
241
|
+
directories) unless the agent's <scope> section explicitly permits it. If the
|
|
242
|
+
agent's contract restricts what files it can touch, honor that restriction —
|
|
243
|
+
even if a fix seems obvious or trivial.
|
|
244
|
+
|
|
245
|
+
If you cannot read ~/.claude/qaa/agents/qaa-executor.md (file missing, corrupted, parse
|
|
246
|
+
error), STOP and report the failure. Do NOT proceed with default agent
|
|
247
|
+
behavior — that defeats the purpose of this directive.
|
|
248
|
+
|
|
249
|
+
Do NOT improvise, substitute steps, apply changes the agent isn't allowed to
|
|
250
|
+
make, or skip any of the above. Skipping the agent's required_reading,
|
|
251
|
+
quality_gate, or bash checklist makes the run INVALID — its output cannot be
|
|
252
|
+
trusted.
|
|
253
|
+
</critical_directive>
|
|
254
|
+
|
|
255
|
+
<objective>Generate test files for the specified feature following CLAUDE.md standards, using codebase map and research documents for context. Query Context7 MCP to verify framework syntax before generating code.</objective>
|
|
256
|
+
<execution_context>~/.claude/qaa/agents/qaa-executor.md</execution_context>
|
|
257
|
+
<files_to_read>
|
|
258
|
+
- CLAUDE.md
|
|
259
|
+
- ~/.claude/qaa/MY_PREFERENCES.md (if exists)
|
|
260
|
+
- .qa-output/locators/LOCATOR_REGISTRY.md (if exists)
|
|
261
|
+
- .qa-output/locators/{feature}.locators.md (if exists)
|
|
262
|
+
- .qa-output/codebase/CODE_PATTERNS.md (if exists)
|
|
263
|
+
- .qa-output/codebase/API_CONTRACTS.md (if exists)
|
|
264
|
+
- .qa-output/codebase/TEST_SURFACE.md (if exists)
|
|
265
|
+
- .qa-output/codebase/TESTABILITY.md (if exists)
|
|
266
|
+
- .qa-output/research/FRAMEWORK_CAPABILITIES.md (if exists)
|
|
267
|
+
- .qa-output/research/E2E_STRATEGY.md (if exists)
|
|
268
|
+
- .qa-output/research/API_TESTING_STRATEGY.md (if exists)
|
|
269
|
+
- .qa-output/research/TESTING_STACK.md (if exists)
|
|
270
|
+
</files_to_read>
|
|
271
|
+
<parameters>
|
|
272
|
+
user_input: $ARGUMENTS
|
|
273
|
+
mode: feature-test
|
|
274
|
+
codebase_map_dir: .qa-output/codebase
|
|
275
|
+
locator_registry: .qa-output/locators/LOCATOR_REGISTRY.md
|
|
276
|
+
</parameters>
|
|
277
|
+
"
|
|
278
|
+
)
|
|
279
|
+
|
|
280
|
+
6. If E2E test files were generated AND `--skip-run` was NOT passed, invoke E2E runner:
|
|
281
|
+
|
|
282
|
+
Task(
|
|
283
|
+
prompt="
|
|
284
|
+
<critical_directive priority="MANDATORY">
|
|
285
|
+
You are executing as a QAA pipeline agent. Your FIRST action MUST be to read
|
|
286
|
+
~/.claude/qaa/agents/qaa-e2e-runner.md and adopt it as your operating contract.
|
|
287
|
+
|
|
288
|
+
Even though you may be running as a general-purpose agent (native subagent type
|
|
289
|
+
routing for QAA agents is not yet enabled), you MUST behave exactly as the
|
|
290
|
+
dedicated agent defined in that file: read everything in its <required_reading>,
|
|
291
|
+
obey its <quality_gate>, and run its mandatory bash checklist before returning.
|
|
292
|
+
|
|
293
|
+
Never modify application source code (src/, app/, lib/, or equivalent
|
|
294
|
+
directories) unless the agent's <scope> section explicitly permits it. If the
|
|
295
|
+
agent's contract restricts what files it can touch, honor that restriction —
|
|
296
|
+
even if a fix seems obvious or trivial.
|
|
297
|
+
|
|
298
|
+
If you cannot read ~/.claude/qaa/agents/qaa-e2e-runner.md (file missing, corrupted, parse
|
|
299
|
+
error), STOP and report the failure. Do NOT proceed with default agent
|
|
300
|
+
behavior — that defeats the purpose of this directive.
|
|
301
|
+
|
|
302
|
+
Do NOT improvise, substitute steps, apply changes the agent isn't allowed to
|
|
303
|
+
make, or skip any of the above. Skipping the agent's required_reading,
|
|
304
|
+
quality_gate, or bash checklist makes the run INVALID — its output cannot be
|
|
305
|
+
trusted.
|
|
306
|
+
</critical_directive>
|
|
307
|
+
|
|
308
|
+
<objective>Run generated E2E tests against live application, capture real locators, fix mismatches, loop until pass. Query Context7 MCP to verify framework selector syntax before fixing locators.</objective>
|
|
309
|
+
<execution_context>~/.claude/qaa/agents/qaa-e2e-runner.md</execution_context>
|
|
310
|
+
<files_to_read>
|
|
311
|
+
- CLAUDE.md
|
|
312
|
+
- ~/.claude/qaa/MY_PREFERENCES.md (if exists)
|
|
313
|
+
- .qa-output/locators/LOCATOR_REGISTRY.md (if exists)
|
|
314
|
+
- .qa-output/research/FRAMEWORK_CAPABILITIES.md (if exists)
|
|
315
|
+
- .qa-output/research/E2E_STRATEGY.md (if exists)
|
|
316
|
+
- {generated E2E test files from executor return}
|
|
317
|
+
- {generated POM files from executor return}
|
|
318
|
+
</files_to_read>
|
|
319
|
+
<parameters>
|
|
320
|
+
app_url: {from --app-url flag or auto-detect}
|
|
321
|
+
output_dir: .qa-output
|
|
322
|
+
</parameters>
|
|
323
|
+
"
|
|
324
|
+
)
|
|
325
|
+
|
|
326
|
+
7. Present results with file counts and suggest `/qa-pr`.
|
|
327
|
+
|
|
328
|
+
---
|
|
329
|
+
|
|
330
|
+
### FROM AUT MODE
|
|
331
|
+
|
|
332
|
+
Generate tests directly from a running application URL — no ticket, no spec, no full pipeline. Triggered when an AUT URL (non-ticket) is resolved (or `--app-url` in NL) and no other mode applies. Faster than `/qa-start --app-url` when you only need tests for one specific flow. **No PR is created — local delivery only.**
|
|
333
|
+
|
|
334
|
+
**Production-URL safety (read first):** if the AUT is a live/production site, exploration AND generated tests MUST be read-only. NEVER submit forms, create accounts/records, post data, or trigger destructive mutations against production. For flows that open a modal/form, assert on its presence/contents then CANCEL — never submit. Reserve any state-changing path for validation-failure assertions only. Never hardcode credentials.
|
|
335
|
+
|
|
336
|
+
1. Read `CLAUDE.md` (POM rules, locator tiers, assertion rules, naming, quality gates) and `~/.claude/qaa/MY_PREFERENCES.md` if it exists.
|
|
337
|
+
|
|
338
|
+
2. **Match existing conventions.** If a test repo already exists (cypress/, tests/, page-objects/, etc.), read 2-3 existing specs + the POM base + the selectors/fixtures registry so generated tests match the project's structure, naming, and locator style. Do NOT invent a parallel structure.
|
|
339
|
+
|
|
340
|
+
3. **Explore the app with Playwright MCP.** For each route/flow in the request:
|
|
341
|
+
```
|
|
342
|
+
mcp__playwright__browser_navigate({ url: "{app_url}{route}" })
|
|
343
|
+
mcp__playwright__browser_snapshot()
|
|
344
|
+
```
|
|
345
|
+
Discover interactive elements and real locators (data-testid, ARIA roles, labels, placeholders) and the ACTUAL behavior. Reconcile the request against reality: if the live DOM contradicts the user's description (e.g. a button isn't rendered for anonymous users), record the discrepancy — do NOT generate a test that asserts behavior the app does not exhibit. Persist verified locators to `.qa-output/locators/{feature}.locators.md` and update `.qa-output/locators/LOCATOR_REGISTRY.md`.
|
|
346
|
+
|
|
347
|
+
4. **Auth handling.** If a flow requires authentication: if `TEST_USER_*` creds are configured (cypress.env.json / env vars), use the project's login command (e.g. `cy.loginAs()`); otherwise generate the authenticated specs but guard them to skip cleanly (e.g. `cy.skipIfNoTestAccount()`) and tell the user they need creds to run them.
|
|
348
|
+
|
|
349
|
+
5. **CHECKPOINT — present the test plan and WAIT for approval.** Before writing ANY file, present the inferred plan and pause:
|
|
350
|
+
```
|
|
351
|
+
=== FROM-AUT TEST PLAN ===
|
|
352
|
+
App: {app_url}
|
|
353
|
+
Flow: {what was requested}
|
|
354
|
+
Framework: {detected}
|
|
355
|
+
Proposed test cases:
|
|
356
|
+
1. [ID] [priority] {title} — {concrete assertion(s)} (locators: tier {N})
|
|
357
|
+
2. ...
|
|
358
|
+
Discrepancies vs request: {list, or "none"}
|
|
359
|
+
Files to create: {spec paths, POM paths, fixtures}
|
|
360
|
+
Auth: {creds available → live | no creds → generated-but-skipped}
|
|
361
|
+
==========================
|
|
362
|
+
Proceed with generation? (yes / modify / cancel)
|
|
363
|
+
```
|
|
364
|
+
Do NOT generate files until the user approves. On "modify" → adjust and re-present. On "cancel" → stop and deliver only the plan. (This gate is mandatory — from-aut writes specs against a live app; never generate a batch without approval.)
|
|
365
|
+
|
|
366
|
+
6. After approval, invoke the executor to generate specs + POMs + fixtures:
|
|
367
|
+
|
|
368
|
+
Task(
|
|
369
|
+
prompt="
|
|
370
|
+
<critical_directive priority="MANDATORY">
|
|
371
|
+
You are executing as a QAA pipeline agent. Your FIRST action MUST be to read
|
|
372
|
+
~/.claude/qaa/agents/qaa-executor.md and adopt it as your operating contract.
|
|
373
|
+
|
|
374
|
+
Even though you may be running as a general-purpose agent (native subagent type
|
|
375
|
+
routing for QAA agents is not yet enabled), you MUST behave exactly as the
|
|
376
|
+
dedicated agent defined in that file: read everything in its <required_reading>,
|
|
377
|
+
obey its <quality_gate>, and run its mandatory bash checklist before returning.
|
|
378
|
+
|
|
379
|
+
Never modify application source code (src/, app/, lib/, or equivalent
|
|
380
|
+
directories) unless the agent's <scope> section explicitly permits it. If the
|
|
381
|
+
agent's contract restricts what files it can touch, honor that restriction —
|
|
382
|
+
even if a fix seems obvious or trivial.
|
|
383
|
+
|
|
384
|
+
If you cannot read ~/.claude/qaa/agents/qaa-executor.md (file missing, corrupted, parse
|
|
385
|
+
error), STOP and report the failure. Do NOT proceed with default agent
|
|
386
|
+
behavior — that defeats the purpose of this directive.
|
|
387
|
+
|
|
388
|
+
Do NOT improvise, substitute steps, apply changes the agent isn't allowed to
|
|
389
|
+
make, or skip any of the above. Skipping the agent's required_reading,
|
|
390
|
+
quality_gate, or bash checklist makes the run INVALID — its output cannot be
|
|
391
|
+
trusted.
|
|
392
|
+
</critical_directive>
|
|
393
|
+
|
|
394
|
+
<objective>Generate the approved from-aut test cases following CLAUDE.md standards and the existing repo conventions, using the verified locators from the registry. Read-only against production — never generate a spec that submits/creates data. Query Context7 MCP to verify framework syntax before generating code.</objective>
|
|
395
|
+
<execution_context>~/.claude/qaa/agents/qaa-executor.md</execution_context>
|
|
396
|
+
<files_to_read>
|
|
397
|
+
- CLAUDE.md
|
|
398
|
+
- ~/.claude/qaa/MY_PREFERENCES.md (if exists)
|
|
399
|
+
- .qa-output/locators/LOCATOR_REGISTRY.md
|
|
400
|
+
- .qa-output/locators/{feature}.locators.md
|
|
401
|
+
- .qa-output/research/FRAMEWORK_CAPABILITIES.md (if exists)
|
|
402
|
+
- {2-3 existing spec files + the POM base, to match conventions}
|
|
403
|
+
</files_to_read>
|
|
404
|
+
<parameters>
|
|
405
|
+
user_input: $ARGUMENTS
|
|
406
|
+
mode: from-aut
|
|
407
|
+
app_url: {resolved app_url}
|
|
408
|
+
approved_plan: {the test cases the user approved in step 5}
|
|
409
|
+
locator_registry: .qa-output/locators/LOCATOR_REGISTRY.md
|
|
410
|
+
</parameters>
|
|
411
|
+
"
|
|
412
|
+
)
|
|
413
|
+
|
|
414
|
+
7. If E2E specs were generated AND `--skip-run` was NOT passed AND an app URL is available, invoke the e2e-runner (same Task() spawn as FROM CODE MODE step 6). Honor the production-safety rule — read-only flows only.
|
|
415
|
+
|
|
416
|
+
8. Present results (file counts, locator tiers, any discrepancies surfaced in step 3), then suggest `/qa-pr`. No PR is created in this mode — local delivery only.
|
|
417
|
+
|
|
418
|
+
---
|
|
419
|
+
|
|
420
|
+
### FROM TICKET MODE
|
|
421
|
+
|
|
422
|
+
1. Read `CLAUDE.md` — all QA standards.
|
|
423
|
+
2. Execute the ticket workflow end-to-end:
|
|
424
|
+
|
|
425
|
+
Follow the workflow defined in `@workflows/qa-from-ticket.md` end-to-end.
|
|
426
|
+
Preserve all workflow gates (ticket parsing, acceptance criteria extraction, traceability matrix, validation).
|
|
427
|
+
|
|
428
|
+
Key steps in the workflow:
|
|
429
|
+
- Parse ticket source (GitHub URL, Jira URL, Linear URL, file, or plain text)
|
|
430
|
+
- Fetch ticket content (via `gh issue view`, WebFetch, or file read)
|
|
431
|
+
- Extract acceptance criteria, user stories, edge cases
|
|
432
|
+
- Scan dev repo for related source files
|
|
433
|
+
- Extract locators from live app via Playwright MCP (if app URL available)
|
|
434
|
+
- Generate test cases with traceability matrix (every AC maps to ≥1 test case)
|
|
435
|
+
- Spawn executor agent to produce test files
|
|
436
|
+
- Spawn validator agent for 4-layer validation
|
|
437
|
+
- Print summary with AC coverage and traceability
|
|
438
|
+
|
|
439
|
+
**Traceability guarantee:** Every acceptance criterion maps to at least one test case via the `traces_to` field.
|
|
440
|
+
|
|
441
|
+
**Supported ticket sources:**
|
|
442
|
+
|
|
443
|
+
| Format | Example | Detection |
|
|
444
|
+
|--------|---------|-----------|
|
|
445
|
+
| GitHub Issue URL | `https://github.com/org/repo/issues/123` | Contains `github.com` + `/issues/` |
|
|
446
|
+
| GitHub shorthand | `org/repo#123` or `#123` | Contains `#` + digits |
|
|
447
|
+
| Jira URL | `https://company.atlassian.net/browse/PROJ-123` | Contains `.atlassian.net/browse/` |
|
|
448
|
+
| Linear URL | `https://linear.app/team/issue/TEAM-123` | Contains `linear.app` |
|
|
449
|
+
| File path | `./tickets/feature-spec.md` | Path exists on disk |
|
|
450
|
+
| Plain text | `"As a user I want to..."` | None of the above match |
|
|
451
|
+
|
|
452
|
+
---
|
|
453
|
+
|
|
454
|
+
### ADO MODE (Azure DevOps)
|
|
455
|
+
|
|
456
|
+
Create Test Cases directly in Azure DevOps from a work item. Reads the work item content (repro steps, acceptance criteria, comments, attachments), designs test cases with boundary detection and deduplication, and creates them in ADO with full traceability.
|
|
457
|
+
|
|
458
|
+
**Prerequisites:** ADO MCP server must be connected (provides `wit_get_work_item`, `testplan_create_test_case`, etc.).
|
|
459
|
+
|
|
460
|
+
Execute the full ADO workflow defined in `@commands/qa-create-test-ado.md`:
|
|
461
|
+
|
|
462
|
+
1. **Phase 1** — Read pipeline context: CLAUDE.md, MY_PREFERENCES.md, codebase map, locator registry
|
|
463
|
+
2. **Phase 2** — Retrieve work item with relations, comments, and attachments
|
|
464
|
+
3. **Phase 2b** — Deduplication check against existing linked test cases (skip with `--skip-dedup`)
|
|
465
|
+
4. **Phase 3** — Extract test source content based on work item type (Bug → Repro Steps, User Story → Acceptance Criteria)
|
|
466
|
+
5. **Phase 4** — Design test cases with boundary value detection, tagging rules, confidence scoring, and preconditions
|
|
467
|
+
6. **Phase 5** — Create test cases in ADO via `testplan_create_test_case`, link via *Tested By*, set tags
|
|
468
|
+
7. **Phase 6** — Generate structured report to `ai-tasks/ticket-{id}/test-cases.md`
|
|
469
|
+
8. **Phase 7** — Attach report to source work item
|
|
470
|
+
|
|
471
|
+
**Key features:**
|
|
472
|
+
- Boundary value triplets: detects `max`, `min`, `limit`, `threshold` keywords with numeric values → generates N-1, N, N+1 test cases
|
|
473
|
+
- Deduplication: checks existing linked TCs before creating, prevents duplicates
|
|
474
|
+
- Confidence scoring: `Specified` (explicit source) vs `Draft` (inferred, tagged `NeedsReview`)
|
|
475
|
+
- Cross-references codebase map for project-specific context when available
|
|
476
|
+
- Supports `--area-path` and `--iteration-path` overrides
|
|
477
|
+
|
|
478
|
+
For the complete step-by-step process, see `@commands/qa-create-test-ado.md`.
|
|
479
|
+
|
|
480
|
+
---
|
|
481
|
+
|
|
482
|
+
### UPDATE MODE
|
|
483
|
+
|
|
484
|
+
1. Read `CLAUDE.md` — quality gates, locator tiers, assertion rules, POM rules.
|
|
485
|
+
2. Invoke validator agent in audit mode:
|
|
486
|
+
|
|
487
|
+
Task(
|
|
488
|
+
prompt="
|
|
489
|
+
<critical_directive priority="MANDATORY">
|
|
490
|
+
You are executing as a QAA pipeline agent. Your FIRST action MUST be to read
|
|
491
|
+
~/.claude/qaa/agents/qaa-validator.md and adopt it as your operating contract.
|
|
492
|
+
|
|
493
|
+
Even though you may be running as a general-purpose agent (native subagent type
|
|
494
|
+
routing for QAA agents is not yet enabled), you MUST behave exactly as the
|
|
495
|
+
dedicated agent defined in that file: read everything in its <required_reading>,
|
|
496
|
+
obey its <quality_gate>, and run its mandatory bash checklist before returning.
|
|
497
|
+
|
|
498
|
+
Never modify application source code (src/, app/, lib/, or equivalent
|
|
499
|
+
directories) unless the agent's <scope> section explicitly permits it. If the
|
|
500
|
+
agent's contract restricts what files it can touch, honor that restriction —
|
|
501
|
+
even if a fix seems obvious or trivial.
|
|
502
|
+
|
|
503
|
+
If you cannot read ~/.claude/qaa/agents/qaa-validator.md (file missing, corrupted, parse
|
|
504
|
+
error), STOP and report the failure. Do NOT proceed with default agent
|
|
505
|
+
behavior — that defeats the purpose of this directive.
|
|
506
|
+
|
|
507
|
+
Do NOT improvise, substitute steps, apply changes the agent isn't allowed to
|
|
508
|
+
make, or skip any of the above. Skipping the agent's required_reading,
|
|
509
|
+
quality_gate, or bash checklist makes the run INVALID — its output cannot be
|
|
510
|
+
trusted.
|
|
511
|
+
</critical_directive>
|
|
512
|
+
|
|
513
|
+
<objective>Audit existing test quality and produce QA_AUDIT_REPORT.md. If Playwright MCP is connected, verify E2E test locators against the live DOM via browser_navigate + browser_snapshot.</objective>
|
|
514
|
+
<execution_context>~/.claude/qaa/agents/qaa-validator.md</execution_context>
|
|
515
|
+
<files_to_read>
|
|
516
|
+
- CLAUDE.md
|
|
517
|
+
- ~/.claude/qaa/MY_PREFERENCES.md (if exists)
|
|
518
|
+
</files_to_read>
|
|
519
|
+
<parameters>
|
|
520
|
+
user_input: $ARGUMENTS
|
|
521
|
+
mode: audit
|
|
522
|
+
app_url: {auto-detect from test config baseURL, or ask user}
|
|
523
|
+
</parameters>
|
|
524
|
+
"
|
|
525
|
+
)
|
|
526
|
+
|
|
527
|
+
3. Present audit results and wait for user approval.
|
|
528
|
+
4. Invoke executor agent to apply approved improvements:
|
|
529
|
+
|
|
530
|
+
Task(
|
|
531
|
+
prompt="
|
|
532
|
+
<critical_directive priority="MANDATORY">
|
|
533
|
+
You are executing as a QAA pipeline agent. Your FIRST action MUST be to read
|
|
534
|
+
~/.claude/qaa/agents/qaa-executor.md and adopt it as your operating contract.
|
|
535
|
+
|
|
536
|
+
Even though you may be running as a general-purpose agent (native subagent type
|
|
537
|
+
routing for QAA agents is not yet enabled), you MUST behave exactly as the
|
|
538
|
+
dedicated agent defined in that file: read everything in its <required_reading>,
|
|
539
|
+
obey its <quality_gate>, and run its mandatory bash checklist before returning.
|
|
540
|
+
|
|
541
|
+
Never modify application source code (src/, app/, lib/, or equivalent
|
|
542
|
+
directories) unless the agent's <scope> section explicitly permits it. If the
|
|
543
|
+
agent's contract restricts what files it can touch, honor that restriction —
|
|
544
|
+
even if a fix seems obvious or trivial.
|
|
545
|
+
|
|
546
|
+
If you cannot read ~/.claude/qaa/agents/qaa-executor.md (file missing, corrupted, parse
|
|
547
|
+
error), STOP and report the failure. Do NOT proceed with default agent
|
|
548
|
+
behavior — that defeats the purpose of this directive.
|
|
549
|
+
|
|
550
|
+
Do NOT improvise, substitute steps, apply changes the agent isn't allowed to
|
|
551
|
+
make, or skip any of the above. Skipping the agent's required_reading,
|
|
552
|
+
quality_gate, or bash checklist makes the run INVALID — its output cannot be
|
|
553
|
+
trusted.
|
|
554
|
+
</critical_directive>
|
|
555
|
+
|
|
556
|
+
<objective>Apply approved improvements to existing tests without deleting working tests. If Playwright MCP is connected, use browser_navigate + browser_snapshot to extract real locators when upgrading from Tier 4 to Tier 1.</objective>
|
|
557
|
+
<execution_context>~/.claude/qaa/agents/qaa-executor.md</execution_context>
|
|
558
|
+
<files_to_read>
|
|
559
|
+
- CLAUDE.md
|
|
560
|
+
- ~/.claude/qaa/MY_PREFERENCES.md (if exists)
|
|
561
|
+
- .qa-output/QA_AUDIT_REPORT.md
|
|
562
|
+
- .qa-output/locators/LOCATOR_REGISTRY.md (if exists)
|
|
563
|
+
</files_to_read>
|
|
564
|
+
<parameters>
|
|
565
|
+
user_input: $ARGUMENTS
|
|
566
|
+
mode: update
|
|
567
|
+
app_url: {auto-detect from test config baseURL, or ask user}
|
|
568
|
+
</parameters>
|
|
569
|
+
"
|
|
570
|
+
)
|
|
571
|
+
|
|
572
|
+
**Update scopes:**
|
|
573
|
+
- `fix` — repair broken tests only
|
|
574
|
+
- `improve` — upgrade locators, assertions, POM structure
|
|
575
|
+
- `add` — add missing test cases without modifying existing
|
|
576
|
+
- `full` — audit everything, then improve with approval (default)
|
|
577
|
+
|
|
578
|
+
**Rule:** NEVER delete or rewrite working tests without user approval. Surgical: add, fix, improve — never replace.
|
|
579
|
+
|
|
580
|
+
---
|
|
581
|
+
|
|
582
|
+
### POM ONLY MODE (`--pom-only`)
|
|
583
|
+
|
|
584
|
+
Generate only Page Object Model files — no test specs.
|
|
585
|
+
|
|
586
|
+
1. Read `CLAUDE.md` — POM rules, locator tier hierarchy, naming conventions.
|
|
587
|
+
2. Invoke executor agent in POM-only mode:
|
|
588
|
+
|
|
589
|
+
Task(
|
|
590
|
+
prompt="
|
|
591
|
+
<critical_directive priority="MANDATORY">
|
|
592
|
+
You are executing as a QAA pipeline agent. Your FIRST action MUST be to read
|
|
593
|
+
~/.claude/qaa/agents/qaa-executor.md and adopt it as your operating contract.
|
|
594
|
+
|
|
595
|
+
Even though you may be running as a general-purpose agent (native subagent type
|
|
596
|
+
routing for QAA agents is not yet enabled), you MUST behave exactly as the
|
|
597
|
+
dedicated agent defined in that file: read everything in its <required_reading>,
|
|
598
|
+
obey its <quality_gate>, and run its mandatory bash checklist before returning.
|
|
599
|
+
|
|
600
|
+
Never modify application source code (src/, app/, lib/, or equivalent
|
|
601
|
+
directories) unless the agent's <scope> section explicitly permits it. If the
|
|
602
|
+
agent's contract restricts what files it can touch, honor that restriction —
|
|
603
|
+
even if a fix seems obvious or trivial.
|
|
604
|
+
|
|
605
|
+
If you cannot read ~/.claude/qaa/agents/qaa-executor.md (file missing, corrupted, parse
|
|
606
|
+
error), STOP and report the failure. Do NOT proceed with default agent
|
|
607
|
+
behavior — that defeats the purpose of this directive.
|
|
608
|
+
|
|
609
|
+
Do NOT improvise, substitute steps, apply changes the agent isn't allowed to
|
|
610
|
+
make, or skip any of the above. Skipping the agent's required_reading,
|
|
611
|
+
quality_gate, or bash checklist makes the run INVALID — its output cannot be
|
|
612
|
+
trusted.
|
|
613
|
+
</critical_directive>
|
|
614
|
+
|
|
615
|
+
<objective>Generate Page Object Models following CLAUDE.md POM rules. If Playwright MCP is connected and an app URL is available, navigate each page first to extract real locators (data-testid, ARIA roles, labels) from the live DOM via browser_navigate + browser_snapshot before generating POMs. This ensures POM locators match the real app instead of guessing from source code.</objective>
|
|
616
|
+
<execution_context>~/.claude/qaa/agents/qaa-executor.md</execution_context>
|
|
617
|
+
<files_to_read>
|
|
618
|
+
- CLAUDE.md
|
|
619
|
+
- ~/.claude/qaa/MY_PREFERENCES.md (if exists)
|
|
620
|
+
- .qa-output/locators/LOCATOR_REGISTRY.md (if exists)
|
|
621
|
+
</files_to_read>
|
|
622
|
+
<parameters>
|
|
623
|
+
user_input: $ARGUMENTS
|
|
624
|
+
mode: pom-only
|
|
625
|
+
app_url: {auto-detect from test config baseURL, or ask user}
|
|
626
|
+
</parameters>
|
|
627
|
+
"
|
|
628
|
+
)
|
|
629
|
+
|
|
630
|
+
**Produces:**
|
|
631
|
+
- BasePage file (if not already present)
|
|
632
|
+
- Feature-specific POM files following `[PageName]Page.[ext]` naming convention
|
|
633
|
+
- No test specs, no fixtures
|
|
634
|
+
|
|
635
|
+
**POM rules enforced:**
|
|
636
|
+
- One class per page — no god objects
|
|
637
|
+
- No assertions in page objects — assertions belong ONLY in test specs
|
|
638
|
+
- Locators as readonly properties — Tier 1 preferred (data-testid, ARIA roles)
|
|
639
|
+
- Actions return void or next page — for fluent chaining
|
|
640
|
+
- State queries return data — let the test decide what to assert
|
|
641
|
+
- Every POM extends BasePage
|
|
642
|
+
|
|
643
|
+
$ARGUMENTS
|
|
644
|
+
|
|
645
|
+
## MANDATORY verification — run ALL commands below, no exceptions, no skipping
|
|
646
|
+
|
|
647
|
+
Before returning control, copy-paste and run this ENTIRE block. Do NOT decide which commands "apply" — run all of them every time. The output confirms what happened; you do not get to assume the answer.
|
|
648
|
+
|
|
649
|
+
```bash
|
|
650
|
+
echo "=== CHECKLIST START ==="
|
|
651
|
+
echo "1. Locator Registry:"
|
|
652
|
+
ls .qa-output/locators/ 2>/dev/null || echo "NO_LOCATORS_FOUND"
|
|
653
|
+
echo "2. MY_PREFERENCES.md:"
|
|
654
|
+
cat ~/.claude/qaa/MY_PREFERENCES.md 2>/dev/null || echo "FILE_NOT_FOUND"
|
|
655
|
+
echo "3. Generated test files:"
|
|
656
|
+
find tests/ pages/ fixtures/ -type f 2>/dev/null | head -20 || echo "NO_TEST_FILES_FOUND"
|
|
657
|
+
echo "4. MCP evidence (if browser was used):"
|
|
658
|
+
ls .qa-output/mcp-evidence/ 2>/dev/null || echo "NO_MCP_EVIDENCE"
|
|
659
|
+
echo "=== CHECKLIST END ==="
|
|
660
|
+
```
|
|
661
|
+
|
|
662
|
+
**Rules:**
|
|
663
|
+
- Run the block AS-IS. Do not modify it. Do not split it. Do not skip lines.
|
|
664
|
+
- If any output shows a problem (NO_LOCATORS when MCP was used, NO_TEST_FILES after generation), fix it before returning.
|
|
665
|
+
- If output shows expected "not found" results (e.g., NO_MCP_EVIDENCE when no browser was used), that is fine — the point is you RAN the command instead of assuming the answer.
|
|
666
|
+
- Do NOT mark this task as complete until the block has been executed and you have read every line of output.
|