qaa-agent 1.9.2 → 1.9.5
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +206 -179
- package/CLAUDE.md +718 -557
- package/README.md +384 -357
- package/VERSION +1 -1
- package/agents/qa-pipeline-orchestrator.md +1739 -1425
- package/agents/qaa-analyzer.md +0 -1
- package/agents/qaa-bug-detective.md +790 -655
- package/agents/qaa-codebase-mapper.md +50 -1
- package/agents/qaa-discovery.md +421 -384
- package/agents/qaa-e2e-runner.md +715 -577
- package/agents/qaa-executor.md +947 -830
- package/agents/qaa-planner.md +14 -1
- package/agents/qaa-project-researcher.md +533 -400
- package/agents/qaa-scanner.md +77 -1
- package/agents/qaa-testid-injector.md +0 -1
- package/agents/qaa-validator.md +86 -1
- package/bin/install.cjs +375 -253
- package/bin/lib/context7-cache.cjs +299 -0
- package/bin/lib/intent-detector.cjs +488 -0
- package/commands/qa-audit.md +255 -126
- package/commands/qa-create-test.md +666 -365
- package/commands/qa-fix.md +684 -513
- package/commands/qa-map.md +283 -139
- package/commands/qa-pr.md +63 -0
- package/commands/qa-research.md +181 -157
- package/commands/qa-start.md +62 -6
- package/commands/qa-test-report.md +219 -219
- package/frameworks/cypress.json +54 -0
- package/frameworks/jest.json +59 -0
- package/frameworks/playwright.json +58 -0
- package/frameworks/pytest.json +59 -0
- package/frameworks/robot-framework.json +54 -0
- package/frameworks/selenium.json +65 -0
- package/frameworks/vitest.json +57 -0
- package/package.json +5 -2
- package/workflows/qa-analyze.md +100 -4
- package/workflows/qa-from-ticket.md +50 -2
- package/workflows/qa-gap.md +50 -2
- package/workflows/qa-start.md +2048 -1405
- package/workflows/qa-testid.md +50 -2
- package/workflows/qa-validate.md +50 -2
package/commands/qa-fix.md
CHANGED
|
@@ -1,513 +1,684 @@
|
|
|
1
|
-
# QA Fix & Validate
|
|
2
|
-
|
|
3
|
-
Validate, diagnose, and fix test files — all in one command. Runs 4-layer static validation (syntax, structure, dependencies, logic), classifies failures, and auto-fixes TEST CODE ERRORS. Uses Playwright MCP to reproduce E2E failures and verify locators against the live app when available. Never touches application code.
|
|
4
|
-
|
|
5
|
-
|
|
6
|
-
|
|
7
|
-
|
|
8
|
-
|
|
9
|
-
|
|
10
|
-
|
|
11
|
-
|
|
12
|
-
|
|
13
|
-
|
|
14
|
-
|
|
15
|
-
|
|
16
|
-
|
|
17
|
-
|
|
18
|
-
|
|
19
|
-
|
|
20
|
-
|
|
21
|
-
|
|
22
|
-
|
|
23
|
-
|
|
24
|
-
|
|
25
|
-
|
|
26
|
-
|
|
27
|
-
|
|
28
|
-
|
|
29
|
-
|
|
30
|
-
|
|
31
|
-
|
|
32
|
-
|
|
33
|
-
|
|
34
|
-
|
|
35
|
-
|
|
36
|
-
|
|
37
|
-
|
|
38
|
-
|
|
39
|
-
|
|
40
|
-
|
|
41
|
-
|
|
42
|
-
|
|
43
|
-
|
|
44
|
-
|
|
45
|
-
|
|
46
|
-
|
|
47
|
-
|
|
48
|
-
|
|
49
|
-
|
|
50
|
-
|
|
51
|
-
|
|
52
|
-
|
|
53
|
-
|
|
54
|
-
|
|
55
|
-
|
|
56
|
-
|
|
57
|
-
|
|
58
|
-
|
|
59
|
-
|
|
60
|
-
|
|
61
|
-
|
|
62
|
-
|
|
63
|
-
|
|
64
|
-
|
|
65
|
-
|
|
66
|
-
|
|
67
|
-
|
|
68
|
-
|
|
69
|
-
|
|
70
|
-
|
|
71
|
-
|
|
72
|
-
|
|
73
|
-
|
|
74
|
-
|
|
75
|
-
|
|
76
|
-
|
|
77
|
-
|
|
78
|
-
|
|
79
|
-
|
|
80
|
-
|
|
81
|
-
|
|
82
|
-
|
|
83
|
-
|
|
84
|
-
|
|
85
|
-
|
|
86
|
-
|
|
87
|
-
|
|
88
|
-
|
|
89
|
-
|
|
90
|
-
|
|
91
|
-
|
|
92
|
-
|
|
93
|
-
|
|
94
|
-
|
|
95
|
-
|
|
96
|
-
|
|
|
97
|
-
|
|
98
|
-
|
|
|
99
|
-
|
|
100
|
-
|
|
101
|
-
|
|
102
|
-
|
|
103
|
-
|
|
104
|
-
|
|
105
|
-
|
|
106
|
-
|
|
107
|
-
|
|
108
|
-
|
|
109
|
-
|
|
110
|
-
|
|
111
|
-
#
|
|
112
|
-
|
|
113
|
-
|
|
114
|
-
|
|
115
|
-
|
|
116
|
-
|
|
117
|
-
|
|
118
|
-
|
|
119
|
-
|
|
120
|
-
|
|
121
|
-
|
|
122
|
-
|
|
123
|
-
|
|
124
|
-
|
|
125
|
-
|
|
126
|
-
-
|
|
127
|
-
|
|
128
|
-
|
|
129
|
-
|
|
130
|
-
|
|
131
|
-
|
|
132
|
-
|
|
133
|
-
|
|
134
|
-
|
|
135
|
-
|
|
136
|
-
|
|
137
|
-
|
|
138
|
-
|
|
139
|
-
|
|
140
|
-
|
|
141
|
-
|
|
142
|
-
|
|
143
|
-
|
|
144
|
-
|
|
145
|
-
|
|
146
|
-
|
|
147
|
-
|
|
148
|
-
|
|
149
|
-
|
|
150
|
-
|
|
151
|
-
|
|
152
|
-
|
|
153
|
-
|
|
154
|
-
|
|
155
|
-
|
|
156
|
-
|
|
157
|
-
|
|
158
|
-
|
|
159
|
-
|
|
160
|
-
|
|
161
|
-
**
|
|
162
|
-
**
|
|
163
|
-
|
|
164
|
-
|
|
165
|
-
|
|
166
|
-
|
|
167
|
-
|
|
168
|
-
|
|
169
|
-
|
|
170
|
-
|
|
171
|
-
|
|
172
|
-
|
|
173
|
-
|
|
174
|
-
|
|
175
|
-
|
|
176
|
-
|
|
177
|
-
|
|
178
|
-
**
|
|
179
|
-
|
|
180
|
-
|
|
181
|
-
|
|
182
|
-
|
|
183
|
-
|
|
184
|
-
|
|
185
|
-
|
|
186
|
-
|
|
187
|
-
|
|
188
|
-
|
|
189
|
-
|
|
190
|
-
|
|
191
|
-
|
|
192
|
-
|
|
193
|
-
|
|
194
|
-
|
|
195
|
-
|
|
196
|
-
-
|
|
197
|
-
-
|
|
198
|
-
|
|
199
|
-
|
|
200
|
-
|
|
201
|
-
|
|
202
|
-
|
|
203
|
-
|
|
204
|
-
**
|
|
205
|
-
|
|
206
|
-
|
|
207
|
-
|
|
208
|
-
|
|
209
|
-
|
|
210
|
-
|
|
211
|
-
|
|
212
|
-
|
|
213
|
-
|
|
214
|
-
|
|
215
|
-
|
|
216
|
-
|
|
217
|
-
|
|
218
|
-
|
|
219
|
-
|
|
220
|
-
|
|
221
|
-
|
|
222
|
-
|
|
223
|
-
|
|
224
|
-
|
|
225
|
-
|
|
226
|
-
|
|
227
|
-
|
|
228
|
-
|
|
229
|
-
|
|
230
|
-
|
|
231
|
-
|
|
232
|
-
|
|
233
|
-
|
|
234
|
-
|
|
235
|
-
|
|
236
|
-
|
|
|
237
|
-
|
|
238
|
-
|
|
239
|
-
|
|
240
|
-
|
|
241
|
-
|
|
242
|
-
|
|
243
|
-
|
|
244
|
-
|
|
245
|
-
|
|
246
|
-
|
|
247
|
-
|
|
248
|
-
|
|
249
|
-
|
|
250
|
-
|
|
251
|
-
|
|
252
|
-
|
|
253
|
-
|
|
254
|
-
|
|
255
|
-
|
|
256
|
-
|
|
257
|
-
|
|
258
|
-
|
|
259
|
-
|
|
260
|
-
|
|
261
|
-
|
|
262
|
-
|
|
263
|
-
|
|
264
|
-
|
|
265
|
-
|
|
266
|
-
|
|
267
|
-
|
|
268
|
-
|
|
269
|
-
|
|
270
|
-
|
|
271
|
-
|
|
272
|
-
-
|
|
273
|
-
-
|
|
274
|
-
-
|
|
275
|
-
|
|
276
|
-
|
|
277
|
-
|
|
278
|
-
|
|
279
|
-
|
|
280
|
-
|
|
281
|
-
|
|
282
|
-
|
|
283
|
-
|
|
284
|
-
|
|
285
|
-
|
|
286
|
-
|
|
287
|
-
|
|
288
|
-
|
|
289
|
-
|
|
290
|
-
|
|
291
|
-
|
|
292
|
-
|
|
293
|
-
|
|
294
|
-
|
|
295
|
-
|
|
296
|
-
|
|
297
|
-
|
|
298
|
-
|
|
299
|
-
|
|
300
|
-
|
|
301
|
-
|
|
302
|
-
|
|
303
|
-
-
|
|
304
|
-
-
|
|
305
|
-
|
|
306
|
-
|
|
307
|
-
|
|
308
|
-
|
|
309
|
-
|
|
310
|
-
|
|
311
|
-
|
|
312
|
-
|
|
313
|
-
|
|
314
|
-
|
|
315
|
-
|
|
316
|
-
|
|
317
|
-
|
|
318
|
-
|
|
319
|
-
|
|
320
|
-
|
|
321
|
-
|
|
322
|
-
**
|
|
323
|
-
-
|
|
324
|
-
-
|
|
325
|
-
|
|
326
|
-
|
|
327
|
-
|
|
328
|
-
|
|
329
|
-
|
|
330
|
-
|
|
331
|
-
|
|
332
|
-
|
|
333
|
-
|
|
334
|
-
|
|
335
|
-
|
|
336
|
-
|
|
337
|
-
|
|
338
|
-
|
|
339
|
-
|
|
340
|
-
|
|
341
|
-
|
|
342
|
-
|
|
343
|
-
|
|
344
|
-
|
|
345
|
-
|
|
346
|
-
|
|
347
|
-
|
|
348
|
-
|
|
349
|
-
|
|
350
|
-
|
|
351
|
-
|
|
352
|
-
|
|
353
|
-
|
|
354
|
-
|
|
355
|
-
|
|
356
|
-
|
|
357
|
-
|
|
358
|
-
|
|
359
|
-
|
|
360
|
-
|
|
361
|
-
|
|
362
|
-
|
|
363
|
-
|
|
364
|
-
|
|
365
|
-
|
|
366
|
-
|
|
367
|
-
|
|
368
|
-
|
|
369
|
-
|
|
370
|
-
|
|
371
|
-
|
|
372
|
-
|
|
373
|
-
|
|
374
|
-
|
|
375
|
-
|
|
376
|
-
|
|
377
|
-
|
|
378
|
-
|
|
379
|
-
|
|
380
|
-
|
|
381
|
-
|
|
382
|
-
|
|
383
|
-
|
|
384
|
-
|
|
385
|
-
|
|
386
|
-
|
|
387
|
-
|
|
388
|
-
|
|
389
|
-
|
|
390
|
-
|
|
391
|
-
|
|
392
|
-
|
|
393
|
-
|
|
394
|
-
|
|
395
|
-
|
|
396
|
-
|
|
397
|
-
|
|
398
|
-
|
|
399
|
-
|
|
400
|
-
|
|
401
|
-
|
|
402
|
-
|
|
403
|
-
|
|
404
|
-
|
|
405
|
-
|
|
406
|
-
|
|
407
|
-
|
|
408
|
-
|
|
409
|
-
|
|
410
|
-
|
|
411
|
-
|
|
412
|
-
|
|
413
|
-
|
|
414
|
-
|
|
415
|
-
|
|
416
|
-
|
|
417
|
-
|
|
418
|
-
|
|
419
|
-
|
|
420
|
-
|
|
421
|
-
|
|
422
|
-
|
|
423
|
-
|
|
424
|
-
|
|
425
|
-
|
|
426
|
-
|
|
427
|
-
|
|
428
|
-
|
|
429
|
-
|
|
430
|
-
|
|
431
|
-
|
|
432
|
-
|
|
433
|
-
|
|
434
|
-
|
|
435
|
-
|
|
436
|
-
|
|
437
|
-
|
|
438
|
-
|
|
439
|
-
|
|
440
|
-
|
|
441
|
-
|
|
442
|
-
|
|
443
|
-
<
|
|
444
|
-
|
|
445
|
-
|
|
446
|
-
|
|
447
|
-
|
|
448
|
-
|
|
449
|
-
|
|
450
|
-
|
|
451
|
-
|
|
452
|
-
|
|
453
|
-
|
|
454
|
-
|
|
455
|
-
</
|
|
456
|
-
|
|
457
|
-
|
|
458
|
-
|
|
459
|
-
|
|
460
|
-
|
|
461
|
-
|
|
462
|
-
)
|
|
463
|
-
|
|
464
|
-
|
|
465
|
-
-
|
|
466
|
-
-
|
|
467
|
-
|
|
468
|
-
|
|
469
|
-
|
|
470
|
-
|
|
471
|
-
|
|
472
|
-
|
|
473
|
-
|
|
474
|
-
|
|
475
|
-
|
|
476
|
-
|
|
477
|
-
|
|
478
|
-
|
|
479
|
-
|
|
480
|
-
|
|
481
|
-
|
|
482
|
-
|
|
483
|
-
|
|
484
|
-
|
|
485
|
-
|
|
486
|
-
|
|
487
|
-
|
|
488
|
-
|
|
489
|
-
|
|
490
|
-
|
|
491
|
-
|
|
492
|
-
|
|
493
|
-
|
|
494
|
-
|
|
495
|
-
|
|
496
|
-
|
|
497
|
-
|
|
498
|
-
|
|
499
|
-
|
|
500
|
-
|
|
501
|
-
|
|
502
|
-
|
|
503
|
-
|
|
504
|
-
|
|
505
|
-
|
|
506
|
-
|
|
507
|
-
|
|
508
|
-
|
|
509
|
-
|
|
510
|
-
|
|
511
|
-
|
|
512
|
-
|
|
513
|
-
|
|
1
|
+
# QA Fix & Validate
|
|
2
|
+
|
|
3
|
+
Validate, diagnose, and fix test files — all in one command. Runs 4-layer static validation (syntax, structure, dependencies, logic), classifies failures, and auto-fixes TEST CODE ERRORS. Uses Playwright MCP to reproduce E2E failures and verify locators against the live app when available. Never touches application code.
|
|
4
|
+
|
|
5
|
+
|
|
6
|
+
## ⚠ MANDATORY: How to Execute This Command
|
|
7
|
+
|
|
8
|
+
Execute these steps IN ORDER. Every step is mandatory. Do NOT improvise,
|
|
9
|
+
reorder, skip, or invent steps.
|
|
10
|
+
|
|
11
|
+
1. FIRST tool call: run the intent detector with `$ARGUMENTS` passed LITERALLY
|
|
12
|
+
(the actual command lives in the "## Intent Detection" section below).
|
|
13
|
+
2. Print the INPUT DETECTION banner (exact output from intent-detector.cjs).
|
|
14
|
+
3. Determine `ALL_FROM_FLAGS`. If interactive (`IS_AUTO=false`) AND any value
|
|
15
|
+
came from NL/default (`ALL_FROM_FLAGS=false`), PAUSE for yes/no confirmation.
|
|
16
|
+
Do NOT proceed without it.
|
|
17
|
+
```bash
|
|
18
|
+
ALL_FROM_FLAGS=$(echo "$RESOLVED" | node -e "const j=JSON.parse(require('fs').readFileSync(0,'utf8')); const sources=Object.values(j.resolved).map(v=>v.source); console.log(sources.length>0 && sources.every(s=>s==='flag') ? 'true' : 'false')")
|
|
19
|
+
```
|
|
20
|
+
Pause prompt (when applicable):
|
|
21
|
+
```
|
|
22
|
+
The values above were resolved from natural language and/or defaults.
|
|
23
|
+
Do you want to continue with these inputs? (yes/no)
|
|
24
|
+
```
|
|
25
|
+
4. Determine MODE from the resolved values (not from your own NL parsing).
|
|
26
|
+
5. Print the "=== QA Fix & Validate ===" banner (Step 1 of Instructions).
|
|
27
|
+
6. Execute the mode's flow IN ORDER. For FIX MODE specifically:
|
|
28
|
+
Phase 1 includes invoking the bug-detective via Task() in classify-only
|
|
29
|
+
mode. Phase 2 includes invoking the bug-detective via Task() in fix mode.
|
|
30
|
+
Do NOT apply fixes inline with Edit/Write — always go through the sub-agent.
|
|
31
|
+
Phase 1 (analyze → present FIX PLAN → WAIT for user confirmation) MUST
|
|
32
|
+
complete before Phase 2 (execute fixes).
|
|
33
|
+
|
|
34
|
+
### DO NOT
|
|
35
|
+
- Skip the INPUT DETECTION banner or the "=== QA Fix ===" banner
|
|
36
|
+
- Run AskUserQuestion or any tool before the intent detector
|
|
37
|
+
- Present an AskUserQuestion in place of the FIX PLAN confirmation checkpoint
|
|
38
|
+
- Auto-fix anything before the user approves the FIX PLAN (Phase 1 → Phase 2 gate)
|
|
39
|
+
- Apply fixes inline with Edit/Write instead of spawning the bug-detective sub-agent
|
|
40
|
+
- Invent steps not in this command, or reorder the ones that are
|
|
41
|
+
- Assume context, app URL, framework, or any value from previous command
|
|
42
|
+
invocations (/qa-start, prior /qa-fix calls, etc.). Each /qa-fix invocation
|
|
43
|
+
is self-contained — the detector returns defaults precisely because the user
|
|
44
|
+
did not pass them.
|
|
45
|
+
- Modify the FIX PLAN scope after user approval without re-confirming. If the
|
|
46
|
+
analysis reveals the approved fix is insufficient or incorrect, STOP, present
|
|
47
|
+
the updated plan to the user, and wait for re-confirmation before applying.
|
|
48
|
+
|
|
49
|
+
If you find yourself improvising, presenting results before running the
|
|
50
|
+
analysis phase, applying fixes inline, or asking questions out of order —
|
|
51
|
+
STOP and restart from step 1.
|
|
52
|
+
|
|
53
|
+
### Pass user input literally to the intent detector
|
|
54
|
+
|
|
55
|
+
When you run the intent detector with `$ARGUMENTS`, substitute that placeholder
|
|
56
|
+
with the RAW user input as received, NOT a pre-translated flag string. If you
|
|
57
|
+
pre-translate, the detector reports every value as `source: flag`, sets
|
|
58
|
+
`ALL_FROM_FLAGS=true`, and bypasses the confirmation checkpoint.
|
|
59
|
+
|
|
60
|
+
|
|
61
|
+
## Usage
|
|
62
|
+
|
|
63
|
+
```
|
|
64
|
+
/qa-fix [<test-files-or-directory>] [options]
|
|
65
|
+
```
|
|
66
|
+
|
|
67
|
+
### Options
|
|
68
|
+
|
|
69
|
+
- `<test-files-or-directory>` — one or more test file paths or a directory (auto-detects if omitted)
|
|
70
|
+
- `--check` — **final check mode**: full quality verification against company preferences, codebase conventions, and execution
|
|
71
|
+
- `--validate-only` — run 4-layer static validation only, no test execution or classification
|
|
72
|
+
- `--classify` — run tests and classify failures, but do NOT auto-fix
|
|
73
|
+
- `--run --app-url <url>` — also execute E2E tests against live app after static validation
|
|
74
|
+
- `--app-url <url>` — URL of running application (auto-detects if not provided)
|
|
75
|
+
- `[error output]` — paste test runner output directly (skips running tests, classifies the pasted output)
|
|
76
|
+
|
|
77
|
+
### Mode Detection
|
|
78
|
+
|
|
79
|
+
```
|
|
80
|
+
if --check:
|
|
81
|
+
MODE = "check" → full quality check + execution + QA_CHECK_REPORT.md
|
|
82
|
+
elif --validate-only:
|
|
83
|
+
MODE = "validate" → 4-layer static validation + VALIDATION_REPORT.md
|
|
84
|
+
elif --classify:
|
|
85
|
+
MODE = "classify" → run tests + classify failures (no auto-fix)
|
|
86
|
+
else:
|
|
87
|
+
MODE = "fix" → run tests + classify + auto-fix TEST CODE ERRORS (default)
|
|
88
|
+
```
|
|
89
|
+
|
|
90
|
+
## What It Produces
|
|
91
|
+
|
|
92
|
+
| Mode | Artifacts |
|
|
93
|
+
|------|-----------|
|
|
94
|
+
| check | QA_CHECK_REPORT.md (full quality verification with pass/fail per test file) |
|
|
95
|
+
| check --ticket | QA_CHECK_REPORT.md + UAT_VERIFICATION.md (step-by-step screenshots vs ticket acceptance criteria) |
|
|
96
|
+
| validate | VALIDATION_REPORT.md (syntax, structure, dependencies, logic per file) |
|
|
97
|
+
| classify | FAILURE_CLASSIFICATION_REPORT.md (per-failure evidence, no fixes) |
|
|
98
|
+
| fix | FAILURE_CLASSIFICATION_REPORT.md + auto-fixed test files |
|
|
99
|
+
|
|
100
|
+
|
|
101
|
+
## Intent Detection
|
|
102
|
+
|
|
103
|
+
Before processing arguments, run the intent detector to support natural-language input alongside flags. Flags always win over NL.
|
|
104
|
+
|
|
105
|
+
```bash
|
|
106
|
+
RESOLVED=$(node ~/.claude/qaa/bin/lib/intent-detector.cjs --resolve "$ARGUMENTS" --aliases '{"app_url":"app-url","ticket":"ticket","check":"check","validate_only":"validate-only","classify":"classify","run":"run"}')
|
|
107
|
+
|
|
108
|
+
# Print INPUT DETECTION banner (always shown, even if defaults are used)
|
|
109
|
+
echo "$RESOLVED" | node -e "const j=JSON.parse(require('fs').readFileSync(0,'utf8')); console.log(j.banner)"
|
|
110
|
+
|
|
111
|
+
# Extract resolved values for use below
|
|
112
|
+
APP_URL=$(echo "$RESOLVED" | node -e "const j=JSON.parse(require('fs').readFileSync(0,'utf8')); console.log(j.resolved.app_url?.value || '')")
|
|
113
|
+
TICKET=$(echo "$RESOLVED" | node -e "const j=JSON.parse(require('fs').readFileSync(0,'utf8')); console.log(j.resolved.ticket?.value || '')")
|
|
114
|
+
```
|
|
115
|
+
|
|
116
|
+
If you see values you didn't intend in the banner above (interactive mode), you can press Ctrl-C and re-run with explicit flags.
|
|
117
|
+
|
|
118
|
+
## Instructions
|
|
119
|
+
|
|
120
|
+
### Step 1: Detect Mode and Test Directory
|
|
121
|
+
|
|
122
|
+
Parse `$ARGUMENTS` for mode flags and test directory path.
|
|
123
|
+
|
|
124
|
+
If no test directory provided, auto-detect:
|
|
125
|
+
- Check for `tests/`, `cypress/`, `__tests__/`, `e2e/`, `spec/` directories
|
|
126
|
+
- Check test config files for `testDir` setting
|
|
127
|
+
|
|
128
|
+
Print banner:
|
|
129
|
+
```
|
|
130
|
+
=== QA Fix & Validate ===
|
|
131
|
+
Mode: {validate | classify | fix}
|
|
132
|
+
Test Directory: {path}
|
|
133
|
+
App URL: {url or "auto-detect"}
|
|
134
|
+
==========================
|
|
135
|
+
```
|
|
136
|
+
|
|
137
|
+
---
|
|
138
|
+
|
|
139
|
+
### CHECK MODE (`--check`) — Final Quality Verification
|
|
140
|
+
|
|
141
|
+
Full quality check for specific test files. Reads ALL context sources, verifies every aspect of the tests, runs them, and produces a pass/fail report. Use this as a final gate before delivering tests.
|
|
142
|
+
|
|
143
|
+
**Accepts specific files:**
|
|
144
|
+
```
|
|
145
|
+
/qa-fix --check tests/e2e/login.e2e.spec.ts tests/e2e/checkout.e2e.spec.ts
|
|
146
|
+
/qa-fix --check tests/unit/auth.unit.spec.ts
|
|
147
|
+
/qa-fix --check tests/e2e/ --app-url http://localhost:3000
|
|
148
|
+
```
|
|
149
|
+
|
|
150
|
+
**Step 1: Read ALL context sources**
|
|
151
|
+
|
|
152
|
+
Read every available context source — this is not optional, all must be read:
|
|
153
|
+
|
|
154
|
+
1. **CLAUDE.md** — QA standards, POM rules, locator tiers, assertion rules, naming conventions, quality gates
|
|
155
|
+
2. **~/.claude/qaa/MY_PREFERENCES.md** — company/user preferences that OVERRIDE CLAUDE.md rules
|
|
156
|
+
3. **Codebase map** (`.qa-output/codebase/`):
|
|
157
|
+
- `CODE_PATTERNS.md` — naming conventions, import style, file organization (are tests matching the project's style?)
|
|
158
|
+
- `API_CONTRACTS.md` — real API shapes (are API test payloads correct?)
|
|
159
|
+
- `TEST_SURFACE.md` — function signatures (are test targets real?)
|
|
160
|
+
- `TESTABILITY.md` — mock boundaries (are mocks set up correctly?)
|
|
161
|
+
4. **Locator Registry** (`.qa-output/locators/`) — real locators from the app (are POM locators accurate?)
|
|
162
|
+
5. **Existing test patterns** — read 2-3 existing test files in the same repo to understand current conventions (describe block style, import patterns, assertion patterns, fixture usage)
|
|
163
|
+
|
|
164
|
+
If codebase map is missing, STOP and tell the user to run `/qa-map` first.
|
|
165
|
+
|
|
166
|
+
**Step 2: Verify each test file across 7 dimensions**
|
|
167
|
+
|
|
168
|
+
For EACH selected test file, check:
|
|
169
|
+
|
|
170
|
+
| # | Dimension | What to check | Source |
|
|
171
|
+
|---|-----------|---------------|--------|
|
|
172
|
+
| 1 | **Naming** | File name, test IDs, describe/it names follow conventions | CLAUDE.md + CODE_PATTERNS.md + MY_PREFERENCES.md |
|
|
173
|
+
| 2 | **Structure** | Correct directory, imports resolve, follows repo patterns | CODE_PATTERNS.md + existing tests in repo |
|
|
174
|
+
| 3 | **Locators** | POM locators match registry, Tier 1 preferred, no stale selectors | LOCATOR_REGISTRY.md |
|
|
175
|
+
| 4 | **Assertions** | Concrete values (no toBeTruthy alone), match API contracts | CLAUDE.md + API_CONTRACTS.md |
|
|
176
|
+
| 5 | **POM compliance** | No assertions in POMs, locators as properties, extends BasePage | CLAUDE.md |
|
|
177
|
+
| 6 | **Code quality** | No redundant code, no dead code, no hardcoded credentials, no copy-paste | Code review |
|
|
178
|
+
| 7 | **Company conventions** | Matches all rules in MY_PREFERENCES.md | MY_PREFERENCES.md |
|
|
179
|
+
|
|
180
|
+
**Step 3: Run the tests**
|
|
181
|
+
|
|
182
|
+
Execute the selected test files:
|
|
183
|
+
|
|
184
|
+
```bash
|
|
185
|
+
# Detect test runner from project config
|
|
186
|
+
npx playwright test {files} --reporter=json 2>&1 # if Playwright
|
|
187
|
+
npx cypress run --spec {files} 2>&1 # if Cypress
|
|
188
|
+
npx jest {files} --json 2>&1 # if Jest
|
|
189
|
+
npx vitest run {files} --reporter=json 2>&1 # if Vitest
|
|
190
|
+
```
|
|
191
|
+
|
|
192
|
+
If E2E tests and app URL available, you MAY also probe the DOM with Playwright MCP as supplementary evidence — but this is NOT test execution:
|
|
193
|
+
|
|
194
|
+
`[DOM_PROBE — NOT_TEST_EXECUTION]` — MCP DOM inspection only. It MAY confirm an element is present in the DOM; it MUST NOT be reported as a passing/verified test. The real test result comes from the runner above (npx playwright / cypress / jest / vitest). If the runner could NOT execute (binary missing, config error, host won't boot, ECONNREFUSED), HARD STOP: report ENVIRONMENT ISSUE, set `runner_executed: false`, and do NOT substitute the MCP probe for the run.
|
|
195
|
+
|
|
196
|
+
- Navigate to each page referenced in the tests
|
|
197
|
+
- `browser_snapshot()` to confirm elements exist in the DOM (probe, not a pass)
|
|
198
|
+
- Cross-reference locators against real page
|
|
199
|
+
|
|
200
|
+
**Step 4: Fix issues found**
|
|
201
|
+
|
|
202
|
+
For each issue found:
|
|
203
|
+
- **AUTO-FIX** (HIGH confidence): naming, imports, locator mismatches, missing await, Tier 4→Tier 1 upgrade when registry has the value
|
|
204
|
+
- **FLAG for review** (MEDIUM/LOW): logic changes, assertion value changes, structural refactors
|
|
205
|
+
- Re-run tests after fixes (max 5 loops)
|
|
206
|
+
|
|
207
|
+
**Step 5: Produce QA_CHECK_REPORT.md**
|
|
208
|
+
|
|
209
|
+
```markdown
|
|
210
|
+
# QA Check Report
|
|
211
|
+
|
|
212
|
+
## Summary
|
|
213
|
+
|
|
214
|
+
| Metric | Value |
|
|
215
|
+
|--------|-------|
|
|
216
|
+
| Files checked | {N} |
|
|
217
|
+
| Dimensions checked | 7 |
|
|
218
|
+
| Issues found | {N} |
|
|
219
|
+
| Auto-fixed | {N} |
|
|
220
|
+
| Flagged for review | {N} |
|
|
221
|
+
| Tests passed | {N}/{total} |
|
|
222
|
+
| Overall | PASS / PASS WITH WARNINGS / FAIL |
|
|
223
|
+
|
|
224
|
+
## Per-File Results
|
|
225
|
+
|
|
226
|
+
### {file_path}
|
|
227
|
+
|
|
228
|
+
| Dimension | Status | Details |
|
|
229
|
+
|-----------|--------|---------|
|
|
230
|
+
| Naming | PASS/FAIL | {specific details} |
|
|
231
|
+
| Structure | PASS/FAIL | {specific details} |
|
|
232
|
+
| Locators | PASS/FAIL | {specific details} |
|
|
233
|
+
| Assertions | PASS/FAIL | {specific details} |
|
|
234
|
+
| POM compliance | PASS/FAIL | {specific details} |
|
|
235
|
+
| Code quality | PASS/FAIL | {specific details} |
|
|
236
|
+
| Company conventions | PASS/FAIL | {specific details} |
|
|
237
|
+
|
|
238
|
+
**Test execution:** PASS / FAIL ({error if failed})
|
|
239
|
+
**Fixes applied:** {list of auto-fixes}
|
|
240
|
+
**Flagged for review:** {list of items needing human review}
|
|
241
|
+
|
|
242
|
+
[... repeat per file ...]
|
|
243
|
+
|
|
244
|
+
## Flagged Items (Needs Human Review)
|
|
245
|
+
|
|
246
|
+
| File | Dimension | Issue | Suggested Fix |
|
|
247
|
+
|------|-----------|-------|---------------|
|
|
248
|
+
| ... | ... | ... | ... |
|
|
249
|
+
```
|
|
250
|
+
|
|
251
|
+
Write to `.qa-output/QA_CHECK_REPORT.md`.
|
|
252
|
+
|
|
253
|
+
Present results to user with clear PASS/FAIL per file and overall status.
|
|
254
|
+
|
|
255
|
+
**Step 6 (optional): Ticket Verification (`--ticket <source>`)**
|
|
256
|
+
|
|
257
|
+
If `--ticket` flag is provided, perform UAT verification — walk through the test flow step-by-step in the browser, take screenshots at each step, and compare against the ticket's acceptance criteria.
|
|
258
|
+
|
|
259
|
+
**Usage:**
|
|
260
|
+
```
|
|
261
|
+
/qa-fix --check --ticket #123 tests/e2e/login.e2e.spec.ts --app-url http://localhost:3000
|
|
262
|
+
/qa-fix --check --ticket https://company.atlassian.net/browse/PROJ-456 tests/e2e/checkout.e2e.spec.ts
|
|
263
|
+
/qa-fix --check --ticket "User logs in, sees dashboard with welcome message, clicks profile" tests/e2e/login.e2e.spec.ts
|
|
264
|
+
```
|
|
265
|
+
|
|
266
|
+
**Requires:** `--app-url` or auto-detected running app. Cannot do ticket verification without a live app.
|
|
267
|
+
|
|
268
|
+
**Step 6a: Fetch and parse the ticket**
|
|
269
|
+
|
|
270
|
+
Same ticket parsing as `/qa-create-test` from-ticket mode:
|
|
271
|
+
- GitHub Issue: `gh issue view` → extract title, body, ACs
|
|
272
|
+
- Jira/Linear URL: `WebFetch` → extract content
|
|
273
|
+
- Plain text: use directly as acceptance criteria
|
|
274
|
+
- File path: read file content
|
|
275
|
+
|
|
276
|
+
Extract:
|
|
277
|
+
- Acceptance criteria (AC-1, AC-2, ...)
|
|
278
|
+
- Expected user flow (step-by-step)
|
|
279
|
+
- Expected outcomes per step
|
|
280
|
+
|
|
281
|
+
**Step 6b: Walk through the flow with Playwright MCP**
|
|
282
|
+
|
|
283
|
+
For each E2E test file being checked, replay the user journey manually in the browser step-by-step:
|
|
284
|
+
|
|
285
|
+
```
|
|
286
|
+
For each step in the ticket's user flow:
|
|
287
|
+
|
|
288
|
+
1. Execute the action described in the step:
|
|
289
|
+
- Navigate: mcp__playwright__browser_navigate({ url: "{page}" })
|
|
290
|
+
- Fill form: mcp__playwright__browser_fill_form({ ... })
|
|
291
|
+
- Click: mcp__playwright__browser_click({ element: "..." })
|
|
292
|
+
- Wait: mcp__playwright__browser_wait_for({ text: "..." })
|
|
293
|
+
|
|
294
|
+
2. Take screenshot AFTER the action:
|
|
295
|
+
mcp__playwright__browser_take_screenshot()
|
|
296
|
+
→ Save to .qa-output/uat-screenshots/{test-name}-step-{N}.png
|
|
297
|
+
|
|
298
|
+
3. Take accessibility snapshot to read page state:
|
|
299
|
+
mcp__playwright__browser_snapshot()
|
|
300
|
+
|
|
301
|
+
4. Record what the page shows:
|
|
302
|
+
- URL after action
|
|
303
|
+
- Visible text/headings
|
|
304
|
+
- Form state
|
|
305
|
+
- Error messages (if any)
|
|
306
|
+
- Elements visible/hidden
|
|
307
|
+
```
|
|
308
|
+
|
|
309
|
+
**Step 6c: Compare actual vs ticket**
|
|
310
|
+
|
|
311
|
+
For each acceptance criterion from the ticket:
|
|
312
|
+
|
|
313
|
+
| AC | Expected (from ticket) | Actual (from browser) | Screenshot | Verdict |
|
|
314
|
+
|----|----------------------|---------------------|------------|---------|
|
|
315
|
+
| AC-1 | User sees login form | Login form visible with email/password fields | step-1.png | MATCH |
|
|
316
|
+
| AC-2 | After login, redirect to dashboard | Redirected to /dashboard, "Welcome" visible | step-3.png | MATCH |
|
|
317
|
+
| AC-3 | Error message for wrong password | "Invalid credentials" alert shown | step-5.png | MATCH |
|
|
318
|
+
| AC-4 | Remember me keeps session | Session persists after browser close | step-7.png | MISMATCH — session expired |
|
|
319
|
+
|
|
320
|
+
Verdicts:
|
|
321
|
+
- **MATCH** — actual behavior matches what the ticket describes
|
|
322
|
+
- **MISMATCH** — actual behavior differs from ticket (could be app bug OR test not covering this AC)
|
|
323
|
+
- **NOT TESTED** — ticket has an AC but no test step covers it
|
|
324
|
+
- **EXTRA** — test covers something not in the ticket (informational, not a failure)
|
|
325
|
+
|
|
326
|
+
**Step 6d: Produce UAT_VERIFICATION.md**
|
|
327
|
+
|
|
328
|
+
```markdown
|
|
329
|
+
# UAT Verification Report
|
|
330
|
+
|
|
331
|
+
## Ticket Info
|
|
332
|
+
|
|
333
|
+
| Field | Value |
|
|
334
|
+
|-------|-------|
|
|
335
|
+
| Source | {ticket URL or text} |
|
|
336
|
+
| Title | {ticket title} |
|
|
337
|
+
| ACs extracted | {count} |
|
|
338
|
+
| Test files verified | {count} |
|
|
339
|
+
|
|
340
|
+
## Step-by-Step Walkthrough
|
|
341
|
+
|
|
342
|
+
### Step 1: {action description}
|
|
343
|
+
- **Action:** Navigate to /login
|
|
344
|
+
- **Screenshot:** [step-1.png](.qa-output/uat-screenshots/{test}-step-1.png)
|
|
345
|
+
- **Page state:** Login form visible, email and password fields empty, "Log in" button enabled
|
|
346
|
+
- **Matches AC:** AC-1 ✓
|
|
347
|
+
|
|
348
|
+
### Step 2: {action description}
|
|
349
|
+
- **Action:** Fill email "test@example.com", password "SecureP@ss123!"
|
|
350
|
+
- **Screenshot:** [step-2.png]
|
|
351
|
+
- **Page state:** Fields filled, button still enabled
|
|
352
|
+
- **Matches AC:** (intermediate step, no AC)
|
|
353
|
+
|
|
354
|
+
[... repeat per step ...]
|
|
355
|
+
|
|
356
|
+
## AC Coverage Matrix
|
|
357
|
+
|
|
358
|
+
| AC | Description | Tested | Verdict | Evidence |
|
|
359
|
+
|----|-------------|--------|---------|----------|
|
|
360
|
+
| AC-1 | Login form visible | Yes | MATCH | step-1.png |
|
|
361
|
+
| AC-2 | Redirect to dashboard | Yes | MATCH | step-3.png |
|
|
362
|
+
| AC-3 | Error on wrong password | Yes | MATCH | step-5.png |
|
|
363
|
+
| AC-4 | Remember me session | No | NOT TESTED | — |
|
|
364
|
+
|
|
365
|
+
## Summary
|
|
366
|
+
|
|
367
|
+
| Metric | Value |
|
|
368
|
+
|--------|-------|
|
|
369
|
+
| ACs from ticket | {N} |
|
|
370
|
+
| ACs matched | {N} |
|
|
371
|
+
| ACs mismatched | {N} |
|
|
372
|
+
| ACs not tested | {N} |
|
|
373
|
+
| Screenshots captured | {N} |
|
|
374
|
+
| Overall | PASS / PARTIAL / FAIL |
|
|
375
|
+
```
|
|
376
|
+
|
|
377
|
+
Write to `.qa-output/UAT_VERIFICATION.md`.
|
|
378
|
+
|
|
379
|
+
If any AC is MISMATCH or NOT TESTED, present to user with recommendation:
|
|
380
|
+
- MISMATCH → "AC-4 says X but the app does Y — either the app has a bug or the test needs updating"
|
|
381
|
+
- NOT TESTED → "AC-4 is not covered by any test step — consider adding a test case"
|
|
382
|
+
|
|
383
|
+
---
|
|
384
|
+
|
|
385
|
+
### VALIDATE MODE (`--validate-only`)
|
|
386
|
+
|
|
387
|
+
1. Read `CLAUDE.md` — quality gates, locator tiers, assertion rules.
|
|
388
|
+
2. Execute static validation workflow:
|
|
389
|
+
|
|
390
|
+
Follow the workflow defined in `@workflows/qa-validate.md` end-to-end.
|
|
391
|
+
Preserve all workflow gates (fix loops, layer checks).
|
|
392
|
+
|
|
393
|
+
The validator runs 4 layers per file:
|
|
394
|
+
1. **Syntax** — code compiles without errors
|
|
395
|
+
2. **Structure** — naming, folders, POM compliance
|
|
396
|
+
3. **Dependencies** — all imports resolve
|
|
397
|
+
4. **Logic** — concrete assertions, Tier 1-2 locators, no assertions in POMs
|
|
398
|
+
|
|
399
|
+
**Optional Layer 5 — labeled DOM-probe locator cross-check (if Playwright MCP connected + app URL available):**
|
|
400
|
+
|
|
401
|
+
This layer is a STATIC locator cross-check via DOM probe. It is NOT test execution and never reports that tests passed:
|
|
402
|
+
|
|
403
|
+
```
|
|
404
|
+
[DOM_PROBE — NOT_TEST_EXECUTION]
|
|
405
|
+
MCP DOM probe — NOT test execution.
|
|
406
|
+
|
|
407
|
+
This observation comes from mcp__playwright__browser_* (live DOM inspection),
|
|
408
|
+
NOT from the test runner (cypress run / npx playwright test / jest / vitest).
|
|
409
|
+
|
|
410
|
+
It MAY assert ONLY: that the element was present in the DOM at the moment of
|
|
411
|
+
the probe.
|
|
412
|
+
|
|
413
|
+
It MUST NOT assert: that any test passes, that a proposed fix works, or that
|
|
414
|
+
the element is visible/actionable under the framework's own rules (Cypress
|
|
415
|
+
retry-ability, Playwright actionability checks, etc.).
|
|
416
|
+
|
|
417
|
+
Any classification resting SOLELY on this probe MUST be marked LOW confidence.
|
|
418
|
+
HIGH confidence requires a real test-runner execution with captured stdout/
|
|
419
|
+
stderr and exit code.
|
|
420
|
+
```
|
|
421
|
+
|
|
422
|
+
- Navigate to each page referenced in E2E tests via `mcp__playwright__browser_navigate`
|
|
423
|
+
- Take accessibility snapshot via `mcp__playwright__browser_snapshot`
|
|
424
|
+
- Cross-reference test locators against real DOM elements
|
|
425
|
+
- Flag locators that don't match, auto-fix mismatches
|
|
426
|
+
|
|
427
|
+
Max 5 fix loop iterations. Produces VALIDATION_REPORT.md.
|
|
428
|
+
|
|
429
|
+
If `--run` flag is also present and E2E test files exist, invoke E2E runner after static validation:
|
|
430
|
+
|
|
431
|
+
Task(
|
|
432
|
+
prompt="
|
|
433
|
+
<critical_directive priority="MANDATORY">
|
|
434
|
+
You are executing as a QAA pipeline agent. Your FIRST action MUST be to read
|
|
435
|
+
~/.claude/qaa/agents/qaa-e2e-runner.md and adopt it as your operating contract.
|
|
436
|
+
|
|
437
|
+
Even though you may be running as a general-purpose agent (native subagent type
|
|
438
|
+
routing for QAA agents is not yet enabled), you MUST behave exactly as the
|
|
439
|
+
dedicated agent defined in that file: read everything in its <required_reading>,
|
|
440
|
+
obey its <quality_gate>, and run its mandatory bash checklist before returning.
|
|
441
|
+
|
|
442
|
+
Never modify application source code (src/, app/, lib/, or equivalent
|
|
443
|
+
directories) unless the agent's <scope> section explicitly permits it. If the
|
|
444
|
+
agent's contract restricts what files it can touch, honor that restriction —
|
|
445
|
+
even if a fix seems obvious or trivial.
|
|
446
|
+
|
|
447
|
+
If you cannot read ~/.claude/qaa/agents/qaa-e2e-runner.md (file missing, corrupted, parse
|
|
448
|
+
error), STOP and report the failure. Do NOT proceed with default agent
|
|
449
|
+
behavior — that defeats the purpose of this directive.
|
|
450
|
+
|
|
451
|
+
Do NOT improvise, substitute steps, apply changes the agent isn't allowed to
|
|
452
|
+
make, or skip any of the above. Skipping the agent's required_reading,
|
|
453
|
+
quality_gate, or bash checklist makes the run INVALID — its output cannot be
|
|
454
|
+
trusted.
|
|
455
|
+
</critical_directive>
|
|
456
|
+
|
|
457
|
+
<objective>Run E2E tests against live application, capture real locators, fix mismatches, loop until pass. Query Context7 MCP to verify framework selector syntax before fixing locators.</objective>
|
|
458
|
+
<execution_context>~/.claude/qaa/agents/qaa-e2e-runner.md</execution_context>
|
|
459
|
+
<files_to_read>
|
|
460
|
+
- CLAUDE.md
|
|
461
|
+
- ~/.claude/qaa/MY_PREFERENCES.md (if exists)
|
|
462
|
+
- .qa-output/locators/LOCATOR_REGISTRY.md (if exists)
|
|
463
|
+
- .qa-output/research/FRAMEWORK_CAPABILITIES.md (if exists)
|
|
464
|
+
- .qa-output/research/E2E_STRATEGY.md (if exists)
|
|
465
|
+
- {E2E test files from validated directory}
|
|
466
|
+
- {POM files from validated directory}
|
|
467
|
+
</files_to_read>
|
|
468
|
+
<parameters>
|
|
469
|
+
app_url: {from --app-url flag or auto-detect}
|
|
470
|
+
output_dir: .qa-output
|
|
471
|
+
</parameters>
|
|
472
|
+
"
|
|
473
|
+
)
|
|
474
|
+
|
|
475
|
+
---
|
|
476
|
+
|
|
477
|
+
### CLASSIFY MODE (`--classify`)
|
|
478
|
+
|
|
479
|
+
Same as fix mode below but skip Step 4 (auto-fix). Only classify and report.
|
|
480
|
+
|
|
481
|
+
---
|
|
482
|
+
|
|
483
|
+
### FIX MODE (default)
|
|
484
|
+
|
|
485
|
+
Fix mode runs in two phases: **analyze first, then fix after user confirmation.**
|
|
486
|
+
|
|
487
|
+
#### Phase 1: Analyze & Present Plan (always runs)
|
|
488
|
+
|
|
489
|
+
1. Read `CLAUDE.md` — classification rules, locator tiers, assertion quality.
|
|
490
|
+
2. Invoke bug-detective agent in **classify-only mode** (no auto-fixes):
|
|
491
|
+
|
|
492
|
+
Task(
|
|
493
|
+
prompt="
|
|
494
|
+
<critical_directive priority="MANDATORY">
|
|
495
|
+
You are executing as a QAA pipeline agent. Your FIRST action MUST be to read
|
|
496
|
+
~/.claude/qaa/agents/qaa-bug-detective.md and adopt it as your operating contract.
|
|
497
|
+
|
|
498
|
+
Even though you may be running as a general-purpose agent (native subagent type
|
|
499
|
+
routing for QAA agents is not yet enabled), you MUST behave exactly as the
|
|
500
|
+
dedicated agent defined in that file: read everything in its <required_reading>,
|
|
501
|
+
obey its <quality_gate>, and run its mandatory bash checklist before returning.
|
|
502
|
+
|
|
503
|
+
Never modify application source code (src/, app/, lib/, or equivalent
|
|
504
|
+
directories) unless the agent's <scope> section explicitly permits it. If the
|
|
505
|
+
agent's contract restricts what files it can touch, honor that restriction —
|
|
506
|
+
even if a fix seems obvious or trivial.
|
|
507
|
+
|
|
508
|
+
If you cannot read ~/.claude/qaa/agents/qaa-bug-detective.md (file missing, corrupted, parse
|
|
509
|
+
error), STOP and report the failure. Do NOT proceed with default agent
|
|
510
|
+
behavior — that defeats the purpose of this directive.
|
|
511
|
+
|
|
512
|
+
Do NOT improvise, substitute steps, apply changes the agent isn't allowed to
|
|
513
|
+
make, or skip any of the above. Skipping the agent's required_reading,
|
|
514
|
+
quality_gate, or bash checklist makes the run INVALID — its output cannot be
|
|
515
|
+
trusted.
|
|
516
|
+
</critical_directive>
|
|
517
|
+
|
|
518
|
+
<objective>Run tests and classify failures. Do NOT auto-fix anything yet — this is the analysis phase. Use Playwright MCP to reproduce E2E failures in the browser when available — navigate to failing pages, snapshot DOM, reproduce actions, and screenshot failure state for evidence. Query Context7 MCP to verify framework syntax when diagnosing failures.</objective>
|
|
519
|
+
<execution_context>~/.claude/qaa/agents/qaa-bug-detective.md</execution_context>
|
|
520
|
+
<files_to_read>
|
|
521
|
+
- CLAUDE.md
|
|
522
|
+
- ~/.claude/qaa/MY_PREFERENCES.md (if exists)
|
|
523
|
+
- .qa-output/locators/LOCATOR_REGISTRY.md (if exists)
|
|
524
|
+
- .qa-output/codebase/CODE_PATTERNS.md (if exists)
|
|
525
|
+
- .qa-output/codebase/API_CONTRACTS.md (if exists)
|
|
526
|
+
- .qa-output/codebase/TEST_SURFACE.md (if exists)
|
|
527
|
+
- .qa-output/codebase/TESTABILITY.md (if exists)
|
|
528
|
+
- .qa-output/research/FRAMEWORK_CAPABILITIES.md (if exists)
|
|
529
|
+
- .qa-output/research/TESTING_STACK.md (if exists)
|
|
530
|
+
</files_to_read>
|
|
531
|
+
<parameters>
|
|
532
|
+
user_input: $ARGUMENTS
|
|
533
|
+
mode: classify-only
|
|
534
|
+
app_url: {auto-detect from test config baseURL, or ask user}
|
|
535
|
+
</parameters>
|
|
536
|
+
"
|
|
537
|
+
)
|
|
538
|
+
|
|
539
|
+
3. Present the analysis to the user as a **fix plan**:
|
|
540
|
+
|
|
541
|
+
```
|
|
542
|
+
=== FIX PLAN ===
|
|
543
|
+
|
|
544
|
+
Tests run: {N}
|
|
545
|
+
Passed: {N}
|
|
546
|
+
Failed: {N}
|
|
547
|
+
|
|
548
|
+
Failures classified:
|
|
549
|
+
APPLICATION BUG: {N} (will NOT be touched)
|
|
550
|
+
TEST CODE ERROR: {N} (can auto-fix)
|
|
551
|
+
ENVIRONMENT ISSUE: {N} (resolution steps provided)
|
|
552
|
+
INCONCLUSIVE: {N} (needs more info)
|
|
553
|
+
|
|
554
|
+
Proposed auto-fixes (TEST CODE ERRORS only):
|
|
555
|
+
1. {file}:{line} — {description of fix} [HIGH confidence]
|
|
556
|
+
2. {file}:{line} — {description of fix} [HIGH confidence]
|
|
557
|
+
3. {file}:{line} — {description of fix} [MEDIUM — flagged for review]
|
|
558
|
+
|
|
559
|
+
APPLICATION BUGs found (for developer action):
|
|
560
|
+
1. {file}:{line} — {description}
|
|
561
|
+
2. {file}:{line} — {description}
|
|
562
|
+
|
|
563
|
+
Proceed with auto-fixes? [yes / modify / cancel]
|
|
564
|
+
================
|
|
565
|
+
```
|
|
566
|
+
|
|
567
|
+
4. **Wait for user confirmation.** Do NOT proceed until the user approves. This is a refinement loop — repeat until the user is satisfied:
|
|
568
|
+
|
|
569
|
+
- **"yes"** / **"proceed"** / **"dale"** → continue to Phase 2
|
|
570
|
+
- **"cancel"** / **"no"** → stop, deliver only the FAILURE_CLASSIFICATION_REPORT.md (same as --classify)
|
|
571
|
+
- **Any other response** (feedback, modifications, additions) → treat as a refinement request:
|
|
572
|
+
- Adjust the fix plan based on the user's instructions (add fixes, remove fixes, change approach, add new checks)
|
|
573
|
+
- Re-present the updated Fix Plan showing what changed
|
|
574
|
+
- Wait for user confirmation again
|
|
575
|
+
- Repeat this loop until the user says "yes" or "cancel"
|
|
576
|
+
|
|
577
|
+
**Examples of refinement requests:**
|
|
578
|
+
- "esto está bien pero también quiero que arregles los imports de utils" → add that fix to the plan
|
|
579
|
+
- "no toques el archivo de login, dejalo como está" → remove that file from the plan
|
|
580
|
+
- "cambiá el selector por getByRole en vez de getByTestId" → update the proposed fix
|
|
581
|
+
- "agregá también una validación de que el status sea 200" → add assertion fix to the plan
|
|
582
|
+
|
|
583
|
+
#### Phase 2: Execute Fixes (only after user confirmation)
|
|
584
|
+
|
|
585
|
+
5. Invoke bug-detective agent in **fix mode** with the confirmed plan:
|
|
586
|
+
|
|
587
|
+
Task(
|
|
588
|
+
prompt="
|
|
589
|
+
<critical_directive priority="MANDATORY">
|
|
590
|
+
You are executing as a QAA pipeline agent. Your FIRST action MUST be to read
|
|
591
|
+
~/.claude/qaa/agents/qaa-bug-detective.md and adopt it as your operating contract.
|
|
592
|
+
|
|
593
|
+
Even though you may be running as a general-purpose agent (native subagent type
|
|
594
|
+
routing for QAA agents is not yet enabled), you MUST behave exactly as the
|
|
595
|
+
dedicated agent defined in that file: read everything in its <required_reading>,
|
|
596
|
+
obey its <quality_gate>, and run its mandatory bash checklist before returning.
|
|
597
|
+
|
|
598
|
+
Never modify application source code (src/, app/, lib/, or equivalent
|
|
599
|
+
directories) unless the agent's <scope> section explicitly permits it. If the
|
|
600
|
+
agent's contract restricts what files it can touch, honor that restriction —
|
|
601
|
+
even if a fix seems obvious or trivial.
|
|
602
|
+
|
|
603
|
+
If you cannot read ~/.claude/qaa/agents/qaa-bug-detective.md (file missing, corrupted, parse
|
|
604
|
+
error), STOP and report the failure. Do NOT proceed with default agent
|
|
605
|
+
behavior — that defeats the purpose of this directive.
|
|
606
|
+
|
|
607
|
+
Do NOT improvise, substitute steps, apply changes the agent isn't allowed to
|
|
608
|
+
make, or skip any of the above. Skipping the agent's required_reading,
|
|
609
|
+
quality_gate, or bash checklist makes the run INVALID — its output cannot be
|
|
610
|
+
trusted.
|
|
611
|
+
</critical_directive>
|
|
612
|
+
|
|
613
|
+
<objective>Auto-fix the confirmed TEST CODE ERRORS from the analysis phase. Use Playwright MCP to reproduce E2E failures in the browser when available. Query Context7 MCP to verify correct framework syntax before applying any fix.</objective>
|
|
614
|
+
<execution_context>~/.claude/qaa/agents/qaa-bug-detective.md</execution_context>
|
|
615
|
+
<files_to_read>
|
|
616
|
+
- CLAUDE.md
|
|
617
|
+
- ~/.claude/qaa/MY_PREFERENCES.md (if exists)
|
|
618
|
+
- .qa-output/locators/LOCATOR_REGISTRY.md (if exists)
|
|
619
|
+
- .qa-output/codebase/CODE_PATTERNS.md (if exists)
|
|
620
|
+
- .qa-output/codebase/API_CONTRACTS.md (if exists)
|
|
621
|
+
- .qa-output/codebase/TEST_SURFACE.md (if exists)
|
|
622
|
+
- .qa-output/codebase/TESTABILITY.md (if exists)
|
|
623
|
+
- .qa-output/research/FRAMEWORK_CAPABILITIES.md (if exists)
|
|
624
|
+
- .qa-output/research/TESTING_STACK.md (if exists)
|
|
625
|
+
- .qa-output/FAILURE_CLASSIFICATION_REPORT.md
|
|
626
|
+
</files_to_read>
|
|
627
|
+
<parameters>
|
|
628
|
+
user_input: $ARGUMENTS
|
|
629
|
+
mode: fix
|
|
630
|
+
app_url: {auto-detect from test config baseURL, or ask user}
|
|
631
|
+
</parameters>
|
|
632
|
+
"
|
|
633
|
+
)
|
|
634
|
+
|
|
635
|
+
**Classification categories:**
|
|
636
|
+
- **APPLICATION BUG** — error in production code → Report only, NEVER auto-fix
|
|
637
|
+
- **TEST CODE ERROR** — error in test code → Auto-fix if HIGH confidence
|
|
638
|
+
- **ENVIRONMENT ISSUE** — missing env, connection refused → Report with resolution steps
|
|
639
|
+
- **INCONCLUSIVE** — ambiguous → Report what's known, ask for more info
|
|
640
|
+
|
|
641
|
+
**Auto-fix rules:**
|
|
642
|
+
- Only TEST CODE ERROR at HIGH confidence
|
|
643
|
+
- Allowed fixes: import paths, selectors, assertion values, config, missing await, fixture paths
|
|
644
|
+
- Every fix verified by re-running the specific test
|
|
645
|
+
- Never modify application code (src/, app/, lib/)
|
|
646
|
+
|
|
647
|
+
**Browser reproduction (when Playwright MCP connected):**
|
|
648
|
+
- Navigate to failing page → snapshot DOM → reproduce action → screenshot
|
|
649
|
+
- Element not in DOM → TEST CODE ERROR (HIGH confidence)
|
|
650
|
+
- Element exists, wrong behavior → APPLICATION BUG
|
|
651
|
+
- Page doesn't load → ENVIRONMENT ISSUE
|
|
652
|
+
|
|
653
|
+
6. Present results. APPLICATION BUGs are reported for developer action, not auto-fixed.
|
|
654
|
+
|
|
655
|
+
$ARGUMENTS
|
|
656
|
+
|
|
657
|
+
## MANDATORY verification — run ALL commands below, no exceptions, no skipping
|
|
658
|
+
|
|
659
|
+
Before returning control, copy-paste and run this ENTIRE block. Do NOT decide which commands "apply" — run all of them every time. The output confirms what happened; you do not get to assume the answer.
|
|
660
|
+
|
|
661
|
+
```bash
|
|
662
|
+
echo "=== QA-FIX CHECKLIST START ==="
|
|
663
|
+
echo "1. Failure classification report:"
|
|
664
|
+
ls .qa-output/FAILURE_CLASSIFICATION_REPORT.md 2>/dev/null || echo "NO_CLASSIFICATION_REPORT"
|
|
665
|
+
echo "2. Locator Registry:"
|
|
666
|
+
ls .qa-output/locators/ 2>/dev/null || echo "NO_LOCATORS_FOUND"
|
|
667
|
+
echo "3. MY_PREFERENCES.md:"
|
|
668
|
+
cat ~/.claude/qaa/MY_PREFERENCES.md 2>/dev/null || echo "FILE_NOT_FOUND"
|
|
669
|
+
echo "4. Codebase map (context for bug-detective):"
|
|
670
|
+
ls .qa-output/codebase/ 2>/dev/null || echo "NO_CODEBASE_MAP"
|
|
671
|
+
echo "5. Test files in scope:"
|
|
672
|
+
find tests/ cypress/ __tests__/ e2e/ spec/ -type f -name "*.spec.*" -o -name "*.test.*" -o -name "*.e2e.*" 2>/dev/null | head -20 || echo "NO_TEST_FILES_FOUND"
|
|
673
|
+
echo "6. MCP evidence (if browser was used):"
|
|
674
|
+
ls .qa-output/mcp-evidence/ 2>/dev/null || echo "NO_MCP_EVIDENCE"
|
|
675
|
+
echo "7. Classification categories in report:"
|
|
676
|
+
grep -cE "APPLICATION BUG|TEST CODE ERROR|ENVIRONMENT ISSUE|INCONCLUSIVE" .qa-output/FAILURE_CLASSIFICATION_REPORT.md 2>/dev/null || echo "NO_CLASSIFICATIONS"
|
|
677
|
+
echo "=== QA-FIX CHECKLIST END ==="
|
|
678
|
+
```
|
|
679
|
+
|
|
680
|
+
**Rules:**
|
|
681
|
+
- Run the block AS-IS. Do not modify it. Do not split it. Do not skip lines.
|
|
682
|
+
- If any output shows a problem (NO_CLASSIFICATION_REPORT after fix mode completed), fix it before returning.
|
|
683
|
+
- If output shows expected "not found" results (e.g., NO_MCP_EVIDENCE when no browser was used), that is fine — the point is you RAN the command instead of assuming the answer.
|
|
684
|
+
- Do NOT mark this task as complete until the block has been executed and you have read every line of output.
|