qaa-agent 1.9.2 → 1.9.5
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +206 -179
- package/CLAUDE.md +718 -557
- package/README.md +384 -357
- package/VERSION +1 -1
- package/agents/qa-pipeline-orchestrator.md +1739 -1425
- package/agents/qaa-analyzer.md +0 -1
- package/agents/qaa-bug-detective.md +790 -655
- package/agents/qaa-codebase-mapper.md +50 -1
- package/agents/qaa-discovery.md +421 -384
- package/agents/qaa-e2e-runner.md +715 -577
- package/agents/qaa-executor.md +947 -830
- package/agents/qaa-planner.md +14 -1
- package/agents/qaa-project-researcher.md +533 -400
- package/agents/qaa-scanner.md +77 -1
- package/agents/qaa-testid-injector.md +0 -1
- package/agents/qaa-validator.md +86 -1
- package/bin/install.cjs +375 -253
- package/bin/lib/context7-cache.cjs +299 -0
- package/bin/lib/intent-detector.cjs +488 -0
- package/commands/qa-audit.md +255 -126
- package/commands/qa-create-test.md +666 -365
- package/commands/qa-fix.md +684 -513
- package/commands/qa-map.md +283 -139
- package/commands/qa-pr.md +63 -0
- package/commands/qa-research.md +181 -157
- package/commands/qa-start.md +62 -6
- package/commands/qa-test-report.md +219 -219
- package/frameworks/cypress.json +54 -0
- package/frameworks/jest.json +59 -0
- package/frameworks/playwright.json +58 -0
- package/frameworks/pytest.json +59 -0
- package/frameworks/robot-framework.json +54 -0
- package/frameworks/selenium.json +65 -0
- package/frameworks/vitest.json +57 -0
- package/package.json +5 -2
- package/workflows/qa-analyze.md +100 -4
- package/workflows/qa-from-ticket.md +50 -2
- package/workflows/qa-gap.md +50 -2
- package/workflows/qa-start.md +2048 -1405
- package/workflows/qa-testid.md +50 -2
- package/workflows/qa-validate.md +50 -2
package/workflows/qa-start.md
CHANGED
|
@@ -1,1405 +1,2048 @@
|
|
|
1
|
-
<
|
|
2
|
-
|
|
3
|
-
|
|
4
|
-
|
|
5
|
-
|
|
6
|
-
|
|
7
|
-
|
|
8
|
-
|
|
9
|
-
|
|
10
|
-
|
|
11
|
-
|
|
12
|
-
|
|
13
|
-
-
|
|
14
|
-
|
|
15
|
-
|
|
16
|
-
|
|
17
|
-
|
|
18
|
-
|
|
19
|
-
|
|
20
|
-
|
|
21
|
-
|
|
22
|
-
|
|
23
|
-
|
|
24
|
-
|
|
25
|
-
|
|
26
|
-
|
|
27
|
-
|
|
28
|
-
|
|
29
|
-
|
|
30
|
-
|
|
31
|
-
|
|
32
|
-
|
|
33
|
-
|
|
34
|
-
|
|
35
|
-
|
|
36
|
-
|
|
37
|
-
|
|
38
|
-
|
|
39
|
-
|
|
40
|
-
|
|
41
|
-
|
|
42
|
-
|
|
43
|
-
|
|
44
|
-
|
|
45
|
-
```
|
|
46
|
-
|
|
47
|
-
|
|
48
|
-
|
|
49
|
-
|
|
50
|
-
|
|
51
|
-
|
|
52
|
-
|
|
53
|
-
|
|
54
|
-
|
|
55
|
-
|
|
56
|
-
|
|
57
|
-
|
|
58
|
-
|
|
59
|
-
|
|
60
|
-
|
|
61
|
-
|
|
62
|
-
|
|
63
|
-
|
|
64
|
-
|
|
65
|
-
|
|
66
|
-
|
|
67
|
-
|
|
68
|
-
|
|
69
|
-
|
|
70
|
-
|
|
71
|
-
|
|
72
|
-
|
|
73
|
-
|
|
74
|
-
|
|
75
|
-
|
|
76
|
-
|
|
77
|
-
|
|
78
|
-
|
|
79
|
-
|
|
80
|
-
|
|
81
|
-
|
|
82
|
-
|
|
83
|
-
|
|
84
|
-
|
|
85
|
-
|
|
86
|
-
|
|
87
|
-
|
|
88
|
-
|
|
89
|
-
|
|
90
|
-
|
|
91
|
-
|
|
92
|
-
|
|
93
|
-
|
|
94
|
-
|
|
95
|
-
|
|
96
|
-
|
|
97
|
-
|
|
98
|
-
|
|
99
|
-
|
|
100
|
-
|
|
101
|
-
|
|
102
|
-
|
|
103
|
-
|
|
104
|
-
|
|
105
|
-
|
|
106
|
-
|
|
107
|
-
-
|
|
108
|
-
|
|
109
|
-
|
|
110
|
-
|
|
111
|
-
|
|
112
|
-
|
|
113
|
-
|
|
114
|
-
|
|
115
|
-
|
|
116
|
-
|
|
117
|
-
|
|
118
|
-
|
|
119
|
-
|
|
120
|
-
|
|
121
|
-
|
|
122
|
-
|
|
123
|
-
|
|
124
|
-
|
|
125
|
-
|
|
126
|
-
|
|
127
|
-
|
|
128
|
-
|
|
129
|
-
|
|
130
|
-
|
|
131
|
-
|
|
132
|
-
|
|
133
|
-
|
|
134
|
-
[ -
|
|
135
|
-
|
|
136
|
-
|
|
137
|
-
|
|
138
|
-
|
|
139
|
-
|
|
140
|
-
|
|
141
|
-
|
|
142
|
-
|
|
143
|
-
|
|
144
|
-
|
|
145
|
-
#
|
|
146
|
-
|
|
147
|
-
|
|
148
|
-
|
|
149
|
-
|
|
150
|
-
|
|
151
|
-
|
|
152
|
-
|
|
153
|
-
|
|
154
|
-
|
|
155
|
-
|
|
156
|
-
|
|
157
|
-
|
|
158
|
-
|
|
159
|
-
|
|
160
|
-
|
|
161
|
-
"
|
|
162
|
-
|
|
163
|
-
|
|
164
|
-
|
|
165
|
-
|
|
166
|
-
|
|
167
|
-
|
|
168
|
-
|
|
169
|
-
|
|
170
|
-
|
|
171
|
-
|
|
172
|
-
|
|
173
|
-
|
|
174
|
-
|
|
175
|
-
|
|
176
|
-
|
|
177
|
-
|
|
178
|
-
|
|
179
|
-
|
|
180
|
-
|
|
181
|
-
|
|
182
|
-
|
|
183
|
-
|
|
184
|
-
|
|
185
|
-
|
|
186
|
-
|
|
187
|
-
|
|
188
|
-
|
|
189
|
-
|
|
190
|
-
```
|
|
191
|
-
|
|
192
|
-
|
|
193
|
-
|
|
194
|
-
|
|
195
|
-
|
|
196
|
-
|
|
197
|
-
|
|
198
|
-
|
|
199
|
-
|
|
200
|
-
|
|
201
|
-
|
|
202
|
-
|
|
203
|
-
|
|
204
|
-
|
|
205
|
-
|
|
206
|
-
|
|
207
|
-
|
|
208
|
-
|
|
209
|
-
|
|
210
|
-
|
|
211
|
-
|
|
212
|
-
|
|
213
|
-
|
|
214
|
-
|
|
215
|
-
|
|
216
|
-
|
|
217
|
-
|
|
218
|
-
|
|
219
|
-
|
|
220
|
-
|
|
221
|
-
|
|
222
|
-
|
|
223
|
-
|
|
224
|
-
|
|
225
|
-
|
|
226
|
-
|
|
227
|
-
|
|
228
|
-
|
|
229
|
-
|
|
230
|
-
|
|
231
|
-
|
|
232
|
-
|
|
233
|
-
|
|
234
|
-
|
|
235
|
-
|
|
236
|
-
|
|
237
|
-
|
|
238
|
-
|
|
239
|
-
|
|
240
|
-
|
|
241
|
-
|
|
242
|
-
|
|
243
|
-
|
|
244
|
-
|
|
245
|
-
|
|
246
|
-
|
|
247
|
-
|
|
248
|
-
|
|
249
|
-
|
|
250
|
-
|
|
251
|
-
|
|
252
|
-
|
|
253
|
-
|
|
254
|
-
|
|
255
|
-
|
|
256
|
-
|
|
257
|
-
|
|
258
|
-
|
|
259
|
-
|
|
260
|
-
|
|
261
|
-
|
|
262
|
-
|
|
263
|
-
|
|
264
|
-
|
|
265
|
-
|
|
266
|
-
|
|
267
|
-
|
|
268
|
-
|
|
269
|
-
|
|
270
|
-
|
|
271
|
-
|
|
272
|
-
</step>
|
|
273
|
-
|
|
274
|
-
<step name="
|
|
275
|
-
|
|
276
|
-
## Step
|
|
277
|
-
|
|
278
|
-
|
|
279
|
-
|
|
280
|
-
|
|
281
|
-
```
|
|
282
|
-
|
|
283
|
-
|
|
284
|
-
|
|
285
|
-
|
|
286
|
-
|
|
287
|
-
|
|
288
|
-
|
|
289
|
-
|
|
290
|
-
|
|
291
|
-
|
|
292
|
-
|
|
293
|
-
|
|
294
|
-
|
|
295
|
-
|
|
296
|
-
|
|
297
|
-
|
|
298
|
-
|
|
299
|
-
|
|
300
|
-
|
|
301
|
-
|
|
302
|
-
|
|
303
|
-
|
|
304
|
-
|
|
305
|
-
|
|
306
|
-
|
|
307
|
-
|
|
308
|
-
|
|
309
|
-
|
|
310
|
-
|
|
311
|
-
|
|
312
|
-
|
|
313
|
-
|
|
314
|
-
)
|
|
315
|
-
|
|
316
|
-
|
|
317
|
-
|
|
318
|
-
|
|
319
|
-
|
|
320
|
-
|
|
321
|
-
|
|
322
|
-
|
|
323
|
-
|
|
324
|
-
|
|
325
|
-
|
|
326
|
-
|
|
327
|
-
|
|
328
|
-
|
|
329
|
-
|
|
330
|
-
|
|
331
|
-
|
|
332
|
-
|
|
333
|
-
|
|
334
|
-
|
|
335
|
-
|
|
336
|
-
|
|
337
|
-
|
|
338
|
-
|
|
339
|
-
|
|
340
|
-
|
|
341
|
-
|
|
342
|
-
|
|
343
|
-
|
|
344
|
-
|
|
345
|
-
|
|
346
|
-
|
|
347
|
-
|
|
348
|
-
|
|
349
|
-
|
|
350
|
-
|
|
351
|
-
|
|
352
|
-
|
|
353
|
-
|
|
354
|
-
|
|
355
|
-
|
|
356
|
-
|
|
357
|
-
|
|
358
|
-
|
|
359
|
-
|
|
360
|
-
|
|
361
|
-
|
|
362
|
-
|
|
363
|
-
|
|
364
|
-
|
|
365
|
-
|
|
366
|
-
|
|
367
|
-
|
|
368
|
-
|
|
369
|
-
|
|
370
|
-
|
|
371
|
-
|
|
372
|
-
|
|
373
|
-
|
|
374
|
-
|
|
375
|
-
|
|
376
|
-
|
|
377
|
-
|
|
378
|
-
|
|
379
|
-
|
|
380
|
-
|
|
381
|
-
|
|
382
|
-
|
|
383
|
-
|
|
384
|
-
|
|
385
|
-
|
|
386
|
-
|
|
387
|
-
|
|
388
|
-
|
|
389
|
-
|
|
390
|
-
|
|
391
|
-
|
|
392
|
-
|
|
393
|
-
|
|
394
|
-
|
|
395
|
-
<
|
|
396
|
-
|
|
397
|
-
|
|
398
|
-
|
|
399
|
-
|
|
400
|
-
|
|
401
|
-
|
|
402
|
-
|
|
403
|
-
|
|
404
|
-
|
|
405
|
-
|
|
406
|
-
|
|
407
|
-
|
|
408
|
-
|
|
409
|
-
|
|
410
|
-
```
|
|
411
|
-
|
|
412
|
-
|
|
413
|
-
|
|
414
|
-
|
|
415
|
-
|
|
416
|
-
|
|
417
|
-
|
|
418
|
-
|
|
419
|
-
|
|
420
|
-
|
|
421
|
-
|
|
422
|
-
|
|
423
|
-
<
|
|
424
|
-
|
|
425
|
-
|
|
426
|
-
|
|
427
|
-
|
|
428
|
-
|
|
429
|
-
|
|
430
|
-
|
|
431
|
-
|
|
432
|
-
|
|
433
|
-
|
|
434
|
-
|
|
435
|
-
</
|
|
436
|
-
|
|
437
|
-
|
|
438
|
-
|
|
439
|
-
|
|
440
|
-
|
|
441
|
-
|
|
442
|
-
|
|
443
|
-
|
|
444
|
-
|
|
445
|
-
|
|
446
|
-
|
|
447
|
-
|
|
448
|
-
|
|
449
|
-
|
|
450
|
-
|
|
451
|
-
|
|
452
|
-
|
|
453
|
-
|
|
454
|
-
|
|
455
|
-
|
|
456
|
-
|
|
457
|
-
|
|
458
|
-
|
|
459
|
-
|
|
460
|
-
|
|
461
|
-
|
|
462
|
-
|
|
463
|
-
|
|
464
|
-
|
|
465
|
-
|
|
466
|
-
|
|
467
|
-
|
|
468
|
-
|
|
469
|
-
|
|
470
|
-
|
|
471
|
-
|
|
472
|
-
|
|
473
|
-
```
|
|
474
|
-
|
|
475
|
-
|
|
476
|
-
|
|
477
|
-
|
|
478
|
-
```
|
|
479
|
-
|
|
480
|
-
|
|
481
|
-
|
|
482
|
-
|
|
483
|
-
|
|
484
|
-
|
|
485
|
-
|
|
486
|
-
|
|
487
|
-
|
|
488
|
-
|
|
489
|
-
|
|
490
|
-
|
|
491
|
-
|
|
492
|
-
|
|
493
|
-
|
|
494
|
-
|
|
495
|
-
|
|
496
|
-
|
|
497
|
-
|
|
498
|
-
|
|
499
|
-
|
|
500
|
-
|
|
501
|
-
|
|
502
|
-
|
|
503
|
-
|
|
504
|
-
|
|
505
|
-
|
|
506
|
-
|
|
507
|
-
|
|
508
|
-
)
|
|
509
|
-
|
|
510
|
-
|
|
511
|
-
|
|
512
|
-
|
|
513
|
-
|
|
514
|
-
|
|
515
|
-
|
|
516
|
-
|
|
517
|
-
|
|
518
|
-
|
|
519
|
-
|
|
520
|
-
|
|
521
|
-
|
|
522
|
-
|
|
523
|
-
|
|
524
|
-
|
|
525
|
-
|
|
526
|
-
|
|
527
|
-
|
|
528
|
-
|
|
529
|
-
|
|
530
|
-
|
|
531
|
-
|
|
532
|
-
|
|
533
|
-
|
|
534
|
-
|
|
535
|
-
|
|
536
|
-
|
|
537
|
-
|
|
538
|
-
|
|
539
|
-
|
|
540
|
-
|
|
541
|
-
|
|
542
|
-
|
|
543
|
-
|
|
544
|
-
|
|
545
|
-
|
|
546
|
-
|
|
547
|
-
|
|
548
|
-
|
|
549
|
-
|
|
550
|
-
|
|
551
|
-
|
|
552
|
-
|
|
553
|
-
|
|
554
|
-
|
|
555
|
-
|
|
556
|
-
|
|
557
|
-
|
|
558
|
-
|
|
559
|
-
|
|
560
|
-
|
|
561
|
-
|
|
562
|
-
|
|
563
|
-
|
|
564
|
-
|
|
565
|
-
|
|
566
|
-
|
|
567
|
-
|
|
568
|
-
|
|
569
|
-
|
|
570
|
-
|
|
571
|
-
|
|
572
|
-
|
|
573
|
-
|
|
574
|
-
|
|
575
|
-
|
|
576
|
-
|
|
577
|
-
|
|
578
|
-
|
|
579
|
-
|
|
580
|
-
|
|
581
|
-
<objective>
|
|
582
|
-
<execution_context
|
|
583
|
-
<files_to_read>
|
|
584
|
-
-
|
|
585
|
-
-
|
|
586
|
-
</files_to_read>
|
|
587
|
-
<parameters>
|
|
588
|
-
|
|
589
|
-
|
|
590
|
-
|
|
591
|
-
|
|
592
|
-
|
|
593
|
-
|
|
594
|
-
|
|
595
|
-
|
|
596
|
-
|
|
597
|
-
|
|
598
|
-
|
|
599
|
-
|
|
600
|
-
|
|
601
|
-
|
|
602
|
-
|
|
603
|
-
|
|
604
|
-
|
|
605
|
-
|
|
606
|
-
|
|
607
|
-
|
|
608
|
-
|
|
609
|
-
|
|
610
|
-
If
|
|
611
|
-
|
|
612
|
-
|
|
613
|
-
|
|
614
|
-
|
|
615
|
-
|
|
616
|
-
|
|
617
|
-
|
|
618
|
-
|
|
619
|
-
|
|
620
|
-
|
|
621
|
-
|
|
622
|
-
|
|
623
|
-
|
|
624
|
-
|
|
625
|
-
|
|
626
|
-
|
|
627
|
-
|
|
628
|
-
|
|
629
|
-
|
|
630
|
-
|
|
631
|
-
|
|
632
|
-
|
|
633
|
-
|
|
634
|
-
|
|
635
|
-
|
|
636
|
-
|
|
637
|
-
|
|
638
|
-
|
|
639
|
-
|
|
640
|
-
|
|
641
|
-
|
|
642
|
-
|
|
643
|
-
|
|
644
|
-
|
|
645
|
-
|
|
646
|
-
|
|
647
|
-
|
|
648
|
-
|
|
649
|
-
|
|
650
|
-
|
|
651
|
-
|
|
652
|
-
|
|
653
|
-
|
|
654
|
-
|
|
655
|
-
|
|
656
|
-
|
|
657
|
-
-
|
|
658
|
-
|
|
659
|
-
|
|
660
|
-
|
|
661
|
-
|
|
662
|
-
|
|
663
|
-
|
|
664
|
-
|
|
665
|
-
|
|
666
|
-
|
|
667
|
-
|
|
668
|
-
|
|
669
|
-
|
|
670
|
-
|
|
671
|
-
|
|
672
|
-
|
|
673
|
-
|
|
674
|
-
|
|
675
|
-
|
|
676
|
-
|
|
677
|
-
|
|
678
|
-
|
|
679
|
-
|
|
680
|
-
|
|
681
|
-
|
|
682
|
-
|
|
683
|
-
|
|
684
|
-
|
|
685
|
-
|
|
686
|
-
|
|
687
|
-
|
|
688
|
-
|
|
689
|
-
|
|
690
|
-
[
|
|
691
|
-
|
|
692
|
-
|
|
693
|
-
|
|
694
|
-
|
|
695
|
-
|
|
696
|
-
|
|
697
|
-
|
|
698
|
-
|
|
699
|
-
|
|
700
|
-
|
|
701
|
-
|
|
702
|
-
|
|
703
|
-
|
|
704
|
-
|
|
705
|
-
|
|
706
|
-
|
|
707
|
-
|
|
708
|
-
|
|
709
|
-
|
|
710
|
-
|
|
711
|
-
|
|
712
|
-
|
|
713
|
-
|
|
714
|
-
|
|
715
|
-
|
|
716
|
-
|
|
717
|
-
|
|
718
|
-
|
|
719
|
-
|
|
720
|
-
|
|
721
|
-
```
|
|
722
|
-
|
|
723
|
-
|
|
724
|
-
|
|
725
|
-
|
|
726
|
-
|
|
727
|
-
|
|
728
|
-
|
|
729
|
-
|
|
730
|
-
|
|
731
|
-
|
|
732
|
-
|
|
733
|
-
-
|
|
734
|
-
|
|
735
|
-
|
|
736
|
-
|
|
737
|
-
|
|
738
|
-
|
|
739
|
-
|
|
740
|
-
|
|
741
|
-
|
|
742
|
-
|
|
743
|
-
|
|
744
|
-
|
|
745
|
-
|
|
746
|
-
|
|
747
|
-
|
|
748
|
-
|
|
749
|
-
|
|
750
|
-
|
|
751
|
-
|
|
752
|
-
|
|
753
|
-
|
|
754
|
-
|
|
755
|
-
|
|
756
|
-
|
|
757
|
-
|
|
758
|
-
|
|
759
|
-
- {output_dir}/
|
|
760
|
-
- {output_dir}/
|
|
761
|
-
-
|
|
762
|
-
- {output_dir}/
|
|
763
|
-
- {output_dir}/
|
|
764
|
-
- {output_dir}/
|
|
765
|
-
- {output_dir}/codebase/
|
|
766
|
-
- {output_dir}/codebase/
|
|
767
|
-
|
|
768
|
-
|
|
769
|
-
|
|
770
|
-
|
|
771
|
-
|
|
772
|
-
|
|
773
|
-
|
|
774
|
-
|
|
775
|
-
|
|
776
|
-
|
|
777
|
-
|
|
778
|
-
|
|
779
|
-
|
|
780
|
-
|
|
781
|
-
|
|
782
|
-
|
|
783
|
-
|
|
784
|
-
|
|
785
|
-
|
|
786
|
-
|
|
787
|
-
|
|
788
|
-
|
|
789
|
-
|
|
790
|
-
|
|
791
|
-
|
|
792
|
-
|
|
793
|
-
|
|
794
|
-
|
|
795
|
-
|
|
796
|
-
|
|
797
|
-
|
|
798
|
-
|
|
799
|
-
|
|
800
|
-
|
|
801
|
-
|
|
802
|
-
```
|
|
803
|
-
|
|
804
|
-
|
|
805
|
-
|
|
806
|
-
|
|
807
|
-
|
|
808
|
-
|
|
809
|
-
|
|
810
|
-
|
|
811
|
-
|
|
812
|
-
|
|
813
|
-
|
|
814
|
-
|
|
815
|
-
|
|
816
|
-
|
|
817
|
-
|
|
818
|
-
|
|
819
|
-
|
|
820
|
-
|
|
821
|
-
|
|
822
|
-
|
|
823
|
-
|
|
824
|
-
|
|
825
|
-
|
|
826
|
-
|
|
827
|
-
|
|
828
|
-
|
|
829
|
-
|
|
830
|
-
|
|
831
|
-
|
|
832
|
-
|
|
833
|
-
|
|
834
|
-
|
|
835
|
-
|
|
836
|
-
|
|
837
|
-
|
|
838
|
-
|
|
839
|
-
|
|
840
|
-
|
|
841
|
-
|
|
842
|
-
|
|
843
|
-
|
|
844
|
-
<
|
|
845
|
-
|
|
846
|
-
|
|
847
|
-
|
|
848
|
-
|
|
849
|
-
|
|
850
|
-
)
|
|
851
|
-
|
|
852
|
-
|
|
853
|
-
|
|
854
|
-
|
|
855
|
-
|
|
856
|
-
|
|
857
|
-
|
|
858
|
-
|
|
859
|
-
|
|
860
|
-
|
|
861
|
-
|
|
862
|
-
|
|
863
|
-
|
|
864
|
-
|
|
865
|
-
|
|
866
|
-
|
|
867
|
-
|
|
868
|
-
|
|
869
|
-
|
|
870
|
-
|
|
871
|
-
|
|
872
|
-
|
|
873
|
-
|
|
874
|
-
|
|
875
|
-
|
|
876
|
-
|
|
877
|
-
|
|
878
|
-
|
|
879
|
-
|
|
880
|
-
|
|
881
|
-
|
|
882
|
-
|
|
883
|
-
|
|
884
|
-
|
|
885
|
-
|
|
886
|
-
|
|
887
|
-
|
|
888
|
-
|
|
889
|
-
|
|
890
|
-
|
|
891
|
-
|
|
892
|
-
|
|
893
|
-
|
|
894
|
-
|
|
895
|
-
|
|
896
|
-
|
|
897
|
-
|
|
898
|
-
|
|
899
|
-
|
|
900
|
-
|
|
901
|
-
|
|
902
|
-
|
|
903
|
-
|
|
904
|
-
|
|
905
|
-
|
|
906
|
-
|
|
907
|
-
|
|
908
|
-
|
|
909
|
-
|
|
910
|
-
|
|
911
|
-
|
|
912
|
-
|
|
913
|
-
|
|
914
|
-
|
|
915
|
-
|
|
916
|
-
|
|
917
|
-
|
|
918
|
-
|
|
919
|
-
|
|
920
|
-
|
|
921
|
-
|
|
922
|
-
|
|
923
|
-
|
|
924
|
-
|
|
925
|
-
|
|
926
|
-
|
|
927
|
-
|
|
928
|
-
|
|
929
|
-
|
|
930
|
-
|
|
931
|
-
|
|
932
|
-
|
|
933
|
-
|
|
934
|
-
|
|
935
|
-
|
|
936
|
-
|
|
937
|
-
|
|
938
|
-
|
|
939
|
-
|
|
940
|
-
|
|
941
|
-
|
|
942
|
-
|
|
943
|
-
|
|
944
|
-
|
|
945
|
-
|
|
946
|
-
|
|
947
|
-
|
|
948
|
-
|
|
949
|
-
|
|
950
|
-
|
|
951
|
-
|
|
952
|
-
|
|
953
|
-
|
|
954
|
-
|
|
955
|
-
|
|
956
|
-
|
|
957
|
-
|
|
958
|
-
|
|
959
|
-
|
|
960
|
-
|
|
961
|
-
</
|
|
962
|
-
<
|
|
963
|
-
|
|
964
|
-
|
|
965
|
-
|
|
966
|
-
|
|
967
|
-
|
|
968
|
-
|
|
969
|
-
|
|
970
|
-
|
|
971
|
-
|
|
972
|
-
|
|
973
|
-
```
|
|
974
|
-
|
|
975
|
-
|
|
976
|
-
|
|
977
|
-
|
|
978
|
-
|
|
979
|
-
|
|
980
|
-
|
|
981
|
-
|
|
982
|
-
|
|
983
|
-
|
|
984
|
-
|
|
985
|
-
|
|
986
|
-
|
|
987
|
-
|
|
988
|
-
|
|
989
|
-
|
|
990
|
-
|
|
991
|
-
|
|
992
|
-
|
|
993
|
-
|
|
994
|
-
|
|
995
|
-
```
|
|
996
|
-
|
|
997
|
-
|
|
998
|
-
|
|
999
|
-
|
|
1000
|
-
|
|
1001
|
-
|
|
1002
|
-
|
|
1003
|
-
|
|
1004
|
-
|
|
1005
|
-
|
|
1006
|
-
|
|
1007
|
-
|
|
1008
|
-
|
|
1009
|
-
|
|
1010
|
-
|
|
1011
|
-
|
|
1012
|
-
|
|
1013
|
-
|
|
1014
|
-
|
|
1015
|
-
|
|
1016
|
-
|
|
1017
|
-
|
|
1018
|
-
|
|
1019
|
-
|
|
1020
|
-
|
|
1021
|
-
|
|
1022
|
-
|
|
1023
|
-
|
|
1024
|
-
|
|
1025
|
-
|
|
1026
|
-
|
|
1027
|
-
|
|
1028
|
-
-
|
|
1029
|
-
|
|
1030
|
-
|
|
1031
|
-
|
|
1032
|
-
|
|
1033
|
-
|
|
1034
|
-
|
|
1035
|
-
|
|
1036
|
-
|
|
1037
|
-
|
|
1038
|
-
-
|
|
1039
|
-
|
|
1040
|
-
|
|
1041
|
-
|
|
1042
|
-
|
|
1043
|
-
|
|
1044
|
-
|
|
1045
|
-
|
|
1046
|
-
|
|
1047
|
-
|
|
1048
|
-
|
|
1049
|
-
|
|
1050
|
-
|
|
1051
|
-
|
|
1052
|
-
|
|
1053
|
-
|
|
1054
|
-
|
|
1055
|
-
|
|
1056
|
-
|
|
1057
|
-
|
|
1058
|
-
|
|
1059
|
-
|
|
1060
|
-
|
|
1061
|
-
|
|
1062
|
-
|
|
1063
|
-
|
|
1064
|
-
|
|
1065
|
-
|
|
1066
|
-
|
|
1067
|
-
|
|
1068
|
-
|
|
1069
|
-
|
|
1070
|
-
|
|
1071
|
-
|
|
1072
|
-
|
|
1073
|
-
|
|
1074
|
-
|
|
1075
|
-
|
|
1076
|
-
|
|
1077
|
-
|
|
1078
|
-
|
|
1079
|
-
|
|
1080
|
-
|
|
1081
|
-
|
|
1082
|
-
|
|
1083
|
-
|
|
1084
|
-
|
|
1085
|
-
|
|
1086
|
-
|
|
1087
|
-
|
|
1088
|
-
|
|
1089
|
-
|
|
1090
|
-
|
|
1091
|
-
|
|
1092
|
-
|
|
1093
|
-
|
|
1094
|
-
|
|
1095
|
-
|
|
1096
|
-
|
|
1097
|
-
|
|
1098
|
-
|
|
1099
|
-
|
|
1100
|
-
|
|
1101
|
-
|
|
1102
|
-
|
|
1103
|
-
|
|
1104
|
-
|
|
1105
|
-
|
|
1106
|
-
|
|
1107
|
-
|
|
1108
|
-
|
|
1109
|
-
|
|
1110
|
-
|
|
1111
|
-
|
|
1112
|
-
|
|
1113
|
-
|
|
1114
|
-
|
|
1115
|
-
|
|
1116
|
-
|
|
1117
|
-
|
|
1118
|
-
|
|
1119
|
-
|
|
1120
|
-
|
|
1121
|
-
|
|
1122
|
-
|
|
1123
|
-
|
|
1124
|
-
|
|
1125
|
-
|
|
1126
|
-
|
|
1127
|
-
|
|
1128
|
-
|
|
1129
|
-
|
|
1130
|
-
|
|
1131
|
-
|
|
1132
|
-
|
|
1133
|
-
|
|
1134
|
-
|
|
1135
|
-
|
|
1136
|
-
|
|
1137
|
-
|
|
1138
|
-
|
|
1139
|
-
|
|
1140
|
-
|
|
1141
|
-
|
|
1142
|
-
|
|
1143
|
-
|
|
1144
|
-
|
|
1145
|
-
|
|
1146
|
-
|
|
1147
|
-
|
|
1148
|
-
|
|
1149
|
-
|
|
1150
|
-
|
|
1151
|
-
|
|
1152
|
-
|
|
1153
|
-
|
|
1154
|
-
|
|
1155
|
-
|
|
1156
|
-
|
|
1157
|
-
|
|
1158
|
-
|
|
1159
|
-
|
|
1160
|
-
|
|
1161
|
-
|
|
1162
|
-
|
|
1163
|
-
|
|
1164
|
-
-
|
|
1165
|
-
|
|
1166
|
-
|
|
1167
|
-
|
|
1168
|
-
|
|
1169
|
-
|
|
1170
|
-
|
|
1171
|
-
|
|
1172
|
-
|
|
1173
|
-
|
|
1174
|
-
|
|
1175
|
-
|
|
1176
|
-
|
|
1177
|
-
|
|
1178
|
-
|
|
1179
|
-
|
|
1180
|
-
|
|
1181
|
-
|
|
1182
|
-
|
|
1183
|
-
|
|
1184
|
-
|
|
1185
|
-
|
|
1186
|
-
|
|
1187
|
-
|
|
1188
|
-
|
|
1189
|
-
|
|
1190
|
-
|
|
1191
|
-
|
|
1192
|
-
|
|
1193
|
-
|
|
1194
|
-
|
|
1195
|
-
|
|
1196
|
-
|
|
1197
|
-
|
|
1198
|
-
|
|
1199
|
-
|
|
1200
|
-
|
|
1201
|
-
|
|
1202
|
-
|
|
1203
|
-
|
|
1204
|
-
|
|
1205
|
-
|
|
1206
|
-
|
|
1207
|
-
|
|
1208
|
-
|
|
1209
|
-
|
|
1210
|
-
|
|
1211
|
-
|
|
1212
|
-
|
|
1213
|
-
|
|
1214
|
-
|
|
1215
|
-
|
|
1216
|
-
|
|
1217
|
-
|
|
1218
|
-
|
|
1219
|
-
|
|
1220
|
-
|
|
1221
|
-
|
|
1222
|
-
|
|
1223
|
-
|
|
1224
|
-
|
|
1225
|
-
|
|
1226
|
-
|
|
1227
|
-
|
|
1228
|
-
|
|
1229
|
-
|
|
1230
|
-
|
|
1231
|
-
|
|
1232
|
-
|
|
1233
|
-
|
|
1234
|
-
|
|
1235
|
-
|
|
1236
|
-
|
|
1237
|
-
|
|
1238
|
-
|
|
1239
|
-
|
|
1240
|
-
|
|
1241
|
-
|
|
1242
|
-
|
|
1243
|
-
|
|
1244
|
-
|
|
1245
|
-
|
|
1246
|
-
|
|
1247
|
-
|
|
1248
|
-
|
|
1249
|
-
|
|
1250
|
-
|
|
1251
|
-
|
|
1252
|
-
|
|
1253
|
-
|
|
1254
|
-
|
|
1255
|
-
|
|
1256
|
-
|
|
1257
|
-
|
|
1258
|
-
|
|
1259
|
-
|
|
1260
|
-
|
|
1261
|
-
|
|
1262
|
-
|
|
1263
|
-
|
|
1264
|
-
|
|
1265
|
-
|
|
1266
|
-
|
|
1267
|
-
|
|
1268
|
-
|
|
1269
|
-
|
|
1270
|
-
|
|
1271
|
-
|
|
1272
|
-
|
|
1273
|
-
|
|
1274
|
-
|
|
1275
|
-
|
|
1276
|
-
|
|
1277
|
-
|
|
1278
|
-
|
|
1279
|
-
|
|
1280
|
-
|
|
1281
|
-
|
|
1282
|
-
|
|
1283
|
-
|
|
1284
|
-
|
|
1285
|
-
|
|
1286
|
-
|
|
1287
|
-
|
|
1288
|
-
|
|
1289
|
-
|
|
1290
|
-
|
|
1291
|
-
|
|
1292
|
-
|
|
1293
|
-
|
|
1294
|
-
|
|
1295
|
-
|
|
1296
|
-
|
|
1297
|
-
|
|
1298
|
-
|
|
1299
|
-
|
|
1300
|
-
|
|
1301
|
-
|
|
1302
|
-
|
|
1303
|
-
|
|
1304
|
-
|
|
1305
|
-
|
|
1306
|
-
|
|
1307
|
-
|
|
1308
|
-
|
|
1309
|
-
|
|
1310
|
-
|
|
1311
|
-
|
|
1312
|
-
|
|
1313
|
-
|
|
1314
|
-
|
|
1315
|
-
-
|
|
1316
|
-
|
|
1317
|
-
|
|
1318
|
-
|
|
1319
|
-
|
|
1320
|
-
|
|
1321
|
-
|
|
1322
|
-
|
|
1323
|
-
|
|
1324
|
-
|
|
1325
|
-
|
|
1326
|
-
|
|
1327
|
-
|
|
1328
|
-
|
|
1329
|
-
|
|
1330
|
-
|
|
1331
|
-
|
|
1332
|
-
|
|
1333
|
-
|
|
1334
|
-
|
|
1335
|
-
|
|
1336
|
-
|
|
1337
|
-
|
|
1338
|
-
|
|
1339
|
-
|
|
1340
|
-
|
|
1341
|
-
|
|
1342
|
-
|
|
1343
|
-
|
|
1344
|
-
|
|
1345
|
-
|
|
1346
|
-
|
|
1347
|
-
|
|
1348
|
-
|
|
1349
|
-
|
|
1350
|
-
|
|
1351
|
-
|
|
1352
|
-
|
|
1353
|
-
|
|
1354
|
-
|
|
1355
|
-
|
|
1356
|
-
|
|
1357
|
-
|
|
1358
|
-
|
|
1359
|
-
|
|
1360
|
-
|
|
1361
|
-
|
|
1362
|
-
|
|
1363
|
-
|
|
1364
|
-
|
|
1365
|
-
|
|
1366
|
-
|
|
1367
|
-
|
|
1368
|
-
|
|
1369
|
-
|
|
1370
|
-
|
|
1371
|
-
|
|
1372
|
-
|
|
1373
|
-
|
|
1374
|
-
|
|
1375
|
-
|
|
1376
|
-
|
|
1377
|
-
|
|
1378
|
-
|
|
1379
|
-
|
|
1380
|
-
|
|
1381
|
-
|
|
1382
|
-
|
|
1383
|
-
|
|
1384
|
-
|
|
1385
|
-
|
|
1386
|
-
|
|
1387
|
-
|
|
1388
|
-
|
|
1389
|
-
|
|
1390
|
-
|
|
1391
|
-
|
|
1392
|
-
|
|
1393
|
-
|
|
1394
|
-
|
|
1395
|
-
|
|
1396
|
-
|
|
1397
|
-
|
|
1398
|
-
|
|
1399
|
-
|
|
1400
|
-
|
|
1401
|
-
|
|
1402
|
-
|
|
1403
|
-
|
|
1404
|
-
|
|
1405
|
-
|
|
1
|
+
<critical_directive priority="MANDATORY">
|
|
2
|
+
|
|
3
|
+
## ⚠ MANDATORY EXECUTION RULES — READ AND OBEY BEFORE ANYTHING ELSE
|
|
4
|
+
|
|
5
|
+
These rules are non-negotiable. Read them before any other section of this file.
|
|
6
|
+
|
|
7
|
+
### Rule 1 — Execute every step literally
|
|
8
|
+
|
|
9
|
+
This workflow is a script, NOT a guide. Run each `<step>` in order. Do NOT skip steps. Do NOT reorder steps. Do NOT adapt steps to "fit" the user's situation.
|
|
10
|
+
|
|
11
|
+
### Rule 2 — Your FIRST tool call MUST be the bash block in Step 1 (initialize)
|
|
12
|
+
|
|
13
|
+
Specifically: run `node ~/.claude/qaa/bin/lib/intent-detector.cjs --resolve ...` as your very first action. Do NOT explore the repo first. Do NOT analyze the user's prompt independently. Do NOT make assumptions about whether the pipeline "applies".
|
|
14
|
+
|
|
15
|
+
### Rule 3 — NEVER improvise an alternative pipeline
|
|
16
|
+
|
|
17
|
+
If you find yourself thinking "the standard pipeline doesn't apply because [empty repo / external URL / different framework / X / Y]", **STOP**. The pipeline applies. Run it as written.
|
|
18
|
+
|
|
19
|
+
If a stage genuinely cannot complete, the workflow itself handles that case (with non-blocking failures, fallbacks, or stop-pipeline directives). Your job is to **execute**, not to redesign.
|
|
20
|
+
|
|
21
|
+
### Rule 4 — If a step fails, report failure — do not adapt
|
|
22
|
+
|
|
23
|
+
When a bash command fails or a sub-agent returns an error, report it as a pipeline failure following the `<error_handling>` rules at the bottom of this file. Do NOT "work around" it. Do NOT scaffold an alternative.
|
|
24
|
+
|
|
25
|
+
### Rule 5 — Empty target repos are NORMAL and expected
|
|
26
|
+
|
|
27
|
+
The pipeline handles empty repos. Many stages produce skeleton output or report "nothing to do" for empty repos. This is **by design**. Do NOT interpret "empty repo" as a signal to abandon the workflow and start scaffolding things directly.
|
|
28
|
+
|
|
29
|
+
### Rule 6 — Your role is execution, not architecture
|
|
30
|
+
|
|
31
|
+
You are running a defined process. The user chose this process by invoking `/qa-start`. Do NOT second-guess the design. Do NOT decide that a different approach would be better. Run the workflow.
|
|
32
|
+
|
|
33
|
+
### Rule 7 — Pass `$ARGUMENTS` to the intent detector LITERALLY
|
|
34
|
+
|
|
35
|
+
When Step 1 says `node ~/.claude/qaa/bin/lib/intent-detector.cjs --resolve "$ARGUMENTS" ...`, the placeholder `$ARGUMENTS` must be substituted with the **raw user input as received**, with no mental translation, parsing, or restructuring.
|
|
36
|
+
|
|
37
|
+
**WRONG** — pre-translating NL to flag form before invoking the detector:
|
|
38
|
+
```
|
|
39
|
+
# User prompt: "from URL https://x.com to C:\foo using cypress"
|
|
40
|
+
# DON'T DO THIS:
|
|
41
|
+
node ~/.claude/qaa/bin/lib/intent-detector.cjs --resolve '--app-url https://x.com --dev-repo "C:\foo" --framework cypress' ...
|
|
42
|
+
```
|
|
43
|
+
|
|
44
|
+
**RIGHT** — passing the raw prompt as-is so the detector does its job:
|
|
45
|
+
```
|
|
46
|
+
node ~/.claude/qaa/bin/lib/intent-detector.cjs --resolve 'from URL https://x.com to C:\foo using cypress' ...
|
|
47
|
+
```
|
|
48
|
+
|
|
49
|
+
The detector handles NL parsing. If you pre-translate, the detector sees flags and reports `source: flag` for everything, which **bypasses the confirmation checkpoint** that exists specifically to catch NL-misinterpretation errors. That defeats the purpose.
|
|
50
|
+
|
|
51
|
+
### Failure mode to recognize and prevent: IMPROVISATION
|
|
52
|
+
|
|
53
|
+
If, after reading this workflow, you find yourself doing any of the following, you have FAILED:
|
|
54
|
+
|
|
55
|
+
- Listing the target repo and saying "this is empty, I'll scaffold files directly"
|
|
56
|
+
- Saying "the standard pipeline assumes X, I'll adapt"
|
|
57
|
+
- Calling sub-agents that the workflow does not explicitly invoke
|
|
58
|
+
- Generating files (.spec.ts, .robot, etc.) before reaching the `<step name="generate">` stage
|
|
59
|
+
- Skipping the `<step name="initialize">` bash block and going straight to other steps
|
|
60
|
+
|
|
61
|
+
If you catch yourself about to do any of these, **stop, return to the top of this workflow, and execute Step 1 first**.
|
|
62
|
+
|
|
63
|
+
</critical_directive>
|
|
64
|
+
|
|
65
|
+
<purpose>
|
|
66
|
+
|
|
67
|
+
Orchestrate the full QA automation pipeline: scan -> analyze -> [testid-inject if frontend] -> plan -> generate -> validate -> [e2e-runner if E2E tests AND app URL] -> [bug-detective if failures] -> deliver. Detects workflow option (1/2/3) from arguments, spawns specialized agents for each stage, manages state transitions, handles checkpoints (safe auto-approve, risky always pause), and delivers a draft PR with per-stage atomic commits.
|
|
68
|
+
|
|
69
|
+
Invoked by the `/qa-start` slash command. Accepts `--dev-repo`, `--qa-repo`, and `--auto` flags.
|
|
70
|
+
|
|
71
|
+
</purpose>
|
|
72
|
+
|
|
73
|
+
<required_reading>
|
|
74
|
+
|
|
75
|
+
Read these files BEFORE executing any pipeline stage. Do NOT skip.
|
|
76
|
+
|
|
77
|
+
- **CLAUDE.md** -- Agent pipeline stages, module boundaries, quality gates, stage transitions, auto-advance rules, agent coordination, data-testid convention. Read the full file.
|
|
78
|
+
- **agents/qa-pipeline-orchestrator.md** -- Full orchestrator logic, checkpoint classification, error handling, delivery sub-steps.
|
|
79
|
+
|
|
80
|
+
</required_reading>
|
|
81
|
+
|
|
82
|
+
<process>
|
|
83
|
+
|
|
84
|
+
<step name="initialize" priority="first">
|
|
85
|
+
|
|
86
|
+
## Step 1: Initialize Pipeline
|
|
87
|
+
|
|
88
|
+
Parse `$ARGUMENTS` using the intent detector (handles flags + natural language + paths with spaces):
|
|
89
|
+
|
|
90
|
+
```bash
|
|
91
|
+
# Resolve flags + NL + defaults via shared intent detector
|
|
92
|
+
# IMPORTANT: $ARGUMENTS must be the RAW user input, NOT a pre-translated flag string.
|
|
93
|
+
# The detector handles NL parsing — pre-translation defeats the confirmation checkpoint.
|
|
94
|
+
RESOLVED=$(node ~/.claude/qaa/bin/lib/intent-detector.cjs --resolve "$ARGUMENTS" --aliases '{"dev_repo":"dev-repo","qa_repo":"qa-repo","app_url":"app-url","framework":"framework"}')
|
|
95
|
+
|
|
96
|
+
# Extract resolved values
|
|
97
|
+
DEV_REPO=$(echo "$RESOLVED" | node -e "const j=JSON.parse(require('fs').readFileSync(0,'utf8')); console.log(j.resolved.dev_repo?.value || '')")
|
|
98
|
+
QA_REPO=$(echo "$RESOLVED" | node -e "const j=JSON.parse(require('fs').readFileSync(0,'utf8')); console.log(j.resolved.qa_repo?.value || '')")
|
|
99
|
+
APP_URL=$(echo "$RESOLVED" | node -e "const j=JSON.parse(require('fs').readFileSync(0,'utf8')); console.log(j.resolved.app_url?.value || '')")
|
|
100
|
+
FRAMEWORK=$(echo "$RESOLVED" | node -e "const j=JSON.parse(require('fs').readFileSync(0,'utf8')); console.log(j.resolved.framework?.value || '')")
|
|
101
|
+
IS_AUTO=$(echo "$RESOLVED" | node -e "const j=JSON.parse(require('fs').readFileSync(0,'utf8')); console.log(j.resolved.auto?.value === true ? 'true' : 'false')")
|
|
102
|
+
|
|
103
|
+
# Print the INPUT DETECTION banner (always shown — with flags, with NL, or defaults)
|
|
104
|
+
echo "$RESOLVED" | node -e "const j=JSON.parse(require('fs').readFileSync(0,'utf8')); console.log(j.banner)"
|
|
105
|
+
```
|
|
106
|
+
|
|
107
|
+
**If no --dev-repo provided and not detected from NL**, use the current working directory:
|
|
108
|
+
```bash
|
|
109
|
+
if [ -z "$DEV_REPO" ]; then
|
|
110
|
+
DEV_REPO=$(pwd)
|
|
111
|
+
echo " (no dev-repo specified, defaulting to cwd: $DEV_REPO)"
|
|
112
|
+
fi
|
|
113
|
+
```
|
|
114
|
+
|
|
115
|
+
**Input Validation (5 pre-pipeline checks — Capa 1 of 3-layer validation system):**
|
|
116
|
+
|
|
117
|
+
```bash
|
|
118
|
+
echo "=== INPUT VALIDATION ==="
|
|
119
|
+
VALIDATION_FAIL=false
|
|
120
|
+
|
|
121
|
+
# Check 1: URL reachable (warning only, not fatal)
|
|
122
|
+
if [ -n "$APP_URL" ]; then
|
|
123
|
+
if curl -sf -o /dev/null -m 5 "$APP_URL" 2>/dev/null; then
|
|
124
|
+
echo " OK: AUT URL reachable ($APP_URL)"
|
|
125
|
+
else
|
|
126
|
+
echo " WARN: AUT URL not reachable from this network — proceeding anyway"
|
|
127
|
+
fi
|
|
128
|
+
fi
|
|
129
|
+
|
|
130
|
+
# Check 2: Path destino writable or creatable
|
|
131
|
+
if [ -n "$DEV_REPO" ]; then
|
|
132
|
+
if [ -d "$DEV_REPO" ] && [ -w "$DEV_REPO" ]; then
|
|
133
|
+
echo " OK: dev-repo path exists and is writable"
|
|
134
|
+
elif [ ! -d "$DEV_REPO" ]; then
|
|
135
|
+
parent=$(dirname "$DEV_REPO")
|
|
136
|
+
if [ -d "$parent" ] && [ -w "$parent" ]; then
|
|
137
|
+
echo " OK: dev-repo path can be created"
|
|
138
|
+
else
|
|
139
|
+
echo " FAIL: dev-repo path is not writable: $DEV_REPO"
|
|
140
|
+
VALIDATION_FAIL=true
|
|
141
|
+
fi
|
|
142
|
+
fi
|
|
143
|
+
fi
|
|
144
|
+
|
|
145
|
+
# Check 3: URL not ambiguous with ticket platforms
|
|
146
|
+
if [ -n "$APP_URL" ]; then
|
|
147
|
+
if echo "$APP_URL" | grep -qE "github\.com.*(issues|pull)|atlassian\.net|linear\.app|dev\.azure\.com"; then
|
|
148
|
+
echo " WARN: $APP_URL looks like a ticket URL — did you mean /qa-from-ticket? Proceeding anyway."
|
|
149
|
+
fi
|
|
150
|
+
fi
|
|
151
|
+
|
|
152
|
+
# Check 4: No multiple AUT URLs detected
|
|
153
|
+
URL_COUNT=$(echo "$RESOLVED" | node -e "const j=JSON.parse(require('fs').readFileSync(0,'utf8')); console.log(j.intent.aut_urls.length)")
|
|
154
|
+
if [ "$URL_COUNT" -gt 1 ]; then
|
|
155
|
+
echo " FAIL: $URL_COUNT AUT URLs detected, expected at most 1"
|
|
156
|
+
echo " Found: $(echo "$RESOLVED" | node -e "const j=JSON.parse(require('fs').readFileSync(0,'utf8')); console.log(j.intent.aut_urls.join(', '))")"
|
|
157
|
+
VALIDATION_FAIL=true
|
|
158
|
+
fi
|
|
159
|
+
|
|
160
|
+
# Check 5: Framework target exists in registry (warn if not, falls back to Context7)
|
|
161
|
+
if [ -n "$FRAMEWORK" ]; then
|
|
162
|
+
if [ -f "frameworks/${FRAMEWORK}.json" ]; then
|
|
163
|
+
echo " OK: framework '$FRAMEWORK' found in registry"
|
|
164
|
+
else
|
|
165
|
+
echo " WARN: framework '$FRAMEWORK' not in frameworks/ registry — relying on Context7 only"
|
|
166
|
+
fi
|
|
167
|
+
fi
|
|
168
|
+
|
|
169
|
+
if [ "$VALIDATION_FAIL" = "true" ]; then
|
|
170
|
+
echo "=== VALIDATION FAILED — pipeline stopped ==="
|
|
171
|
+
exit 1
|
|
172
|
+
fi
|
|
173
|
+
echo "=== VALIDATION OK ==="
|
|
174
|
+
|
|
175
|
+
# Detect if any value came from NL or default (i.e., not all explicit flags)
|
|
176
|
+
ALL_FROM_FLAGS=$(echo "$RESOLVED" | node -e "const j=JSON.parse(require('fs').readFileSync(0,'utf8')); const sources=Object.values(j.resolved).map(v=>v.source); console.log(sources.length>0 && sources.every(s=>s==='flag') ? 'true' : 'false')")
|
|
177
|
+
HAS_ANY_NL=$(echo "$RESOLVED" | node -e "const j=JSON.parse(require('fs').readFileSync(0,'utf8')); console.log(Object.values(j.resolved).some(v=>v.source==='NL') ? 'true' : 'false')")
|
|
178
|
+
```
|
|
179
|
+
|
|
180
|
+
**Confirmation checkpoint** (only in interactive mode):
|
|
181
|
+
|
|
182
|
+
If `IS_AUTO=false` AND `ALL_FROM_FLAGS=false` (i.e., some values came from NL or defaults), the agent **MUST PAUSE** and ask the user to confirm before proceeding to scan/codebase-map/research stages.
|
|
183
|
+
|
|
184
|
+
The pause is justified because:
|
|
185
|
+
- NL detection can be wrong (e.g., wrong path inferred from a sentence)
|
|
186
|
+
- All-defaults case may mean user is running from wrong cwd
|
|
187
|
+
|
|
188
|
+
Format the confirmation prompt like this:
|
|
189
|
+
|
|
190
|
+
```
|
|
191
|
+
The values above were resolved from {natural language | defaults | a mix of flags and NL}.
|
|
192
|
+
Some values were not explicitly specified via flags.
|
|
193
|
+
|
|
194
|
+
Do you want to continue with these inputs? (yes/no)
|
|
195
|
+
- yes → proceed with the pipeline
|
|
196
|
+
- no → cancel; re-run with explicit flags
|
|
197
|
+
```
|
|
198
|
+
|
|
199
|
+
**Skip the pause when:**
|
|
200
|
+
- `IS_AUTO=true` (auto mode bypasses all safe checkpoints)
|
|
201
|
+
- `ALL_FROM_FLAGS=true` (every resolved value has source: flag — user knows what they want)
|
|
202
|
+
|
|
203
|
+
In auto mode, log instead of prompt:
|
|
204
|
+
```
|
|
205
|
+
Auto-mode: proceeding with detected inputs (banner above for audit trail).
|
|
206
|
+
```
|
|
207
|
+
|
|
208
|
+
Wait for user response before invoking any sub-agent. If user says "no" or anything other than yes/y/proceed/continue, STOP the pipeline cleanly:
|
|
209
|
+
```bash
|
|
210
|
+
echo "Pipeline cancelled by user. No tokens spent on sub-agents."
|
|
211
|
+
exit 0
|
|
212
|
+
```
|
|
213
|
+
|
|
214
|
+
|
|
215
|
+
|
|
216
|
+
**Attempt to call qaa-tools init** (handle missing tool gracefully):
|
|
217
|
+
```bash
|
|
218
|
+
INIT_JSON=$(node bin/qaa-tools.cjs init qa-start 2>/dev/null || echo "")
|
|
219
|
+
```
|
|
220
|
+
|
|
221
|
+
If `INIT_JSON` is empty or the command fails, proceed with manual initialization:
|
|
222
|
+
- Set `output_dir` to `.qa-output`
|
|
223
|
+
- Set `date` to current date in `YYYY-MM-DD` format
|
|
224
|
+
- Create output directory: `mkdir -p "$output_dir"`
|
|
225
|
+
|
|
226
|
+
If `INIT_JSON` is valid, parse it for: `option`, `dev_repo_path`, `qa_repo_path`, `maturity_score`, `maturity_note`, `output_dir`, `date`, agent model assignments, `auto_advance`, `auto_chain_active`, `parallelization`, `commit_docs`.
|
|
227
|
+
|
|
228
|
+
**Detect workflow option based on inputs:**
|
|
229
|
+
|
|
230
|
+
- If `QA_REPO` is empty (no --qa-repo flag): **Option 1** (Dev-Only -- Full Pipeline)
|
|
231
|
+
- If `QA_REPO` is provided: Assess QA repo maturity
|
|
232
|
+
- Check for existing test files, configs, coverage reports in QA repo
|
|
233
|
+
- Count test files, evaluate framework setup, check for CI config
|
|
234
|
+
- Score 0-100 based on test count, framework config presence, CI setup, coverage data
|
|
235
|
+
- Score >= 60: **Option 3** (Dev + Mature QA -- Surgical)
|
|
236
|
+
- Score < 60: **Option 2** (Dev + Immature QA -- Gap-Fill)
|
|
237
|
+
|
|
238
|
+
**Determine auto-advance mode:**
|
|
239
|
+
```bash
|
|
240
|
+
# Check persistent config flag
|
|
241
|
+
AUTO_CFG=$(node bin/qaa-tools.cjs config-get workflow.auto_advance 2>/dev/null || echo "false")
|
|
242
|
+
AUTO_CHAIN=$(node bin/qaa-tools.cjs config-get workflow._auto_chain_active 2>/dev/null || echo "false")
|
|
243
|
+
|
|
244
|
+
if [ "$IS_AUTO" = "true" ] || [ "$AUTO_CFG" = "true" ] || [ "$AUTO_CHAIN" = "true" ]; then
|
|
245
|
+
IS_AUTO=true
|
|
246
|
+
node bin/qaa-tools.cjs config-set workflow._auto_chain_active true 2>/dev/null || true
|
|
247
|
+
fi
|
|
248
|
+
|
|
249
|
+
# Safety: clear stale chain flag if NOT in auto mode
|
|
250
|
+
if [ "$IS_AUTO" = "false" ]; then
|
|
251
|
+
node bin/qaa-tools.cjs config-set workflow._auto_chain_active false 2>/dev/null || true
|
|
252
|
+
fi
|
|
253
|
+
```
|
|
254
|
+
|
|
255
|
+
**Print initialization banner:**
|
|
256
|
+
```
|
|
257
|
+
=== QA Pipeline Orchestrator ===
|
|
258
|
+
Option: {option} ({description})
|
|
259
|
+
Dev Repo: {DEV_REPO}
|
|
260
|
+
QA Repo: {QA_REPO or 'N/A'}
|
|
261
|
+
Maturity Score: {maturity_score or 'N/A'}
|
|
262
|
+
Auto-Advance: {IS_AUTO}
|
|
263
|
+
Date: {date}
|
|
264
|
+
================================
|
|
265
|
+
```
|
|
266
|
+
|
|
267
|
+
Where `{description}` is:
|
|
268
|
+
- Option 1: "Dev-Only -- Full Pipeline"
|
|
269
|
+
- Option 2: "Dev + Immature QA -- Gap-Fill"
|
|
270
|
+
- Option 3: "Dev + Mature QA -- Surgical"
|
|
271
|
+
|
|
272
|
+
</step>
|
|
273
|
+
|
|
274
|
+
<step name="detect_framework">
|
|
275
|
+
|
|
276
|
+
## Step 2: Detect Framework
|
|
277
|
+
|
|
278
|
+
Before scanning, detect the project's language and test framework to guide all downstream agents.
|
|
279
|
+
|
|
280
|
+
**Read project config files:**
|
|
281
|
+
```bash
|
|
282
|
+
# Check for Node.js / JavaScript / TypeScript
|
|
283
|
+
[ -f "${DEV_REPO}/package.json" ] && cat "${DEV_REPO}/package.json"
|
|
284
|
+
|
|
285
|
+
# Check for Python
|
|
286
|
+
[ -f "${DEV_REPO}/requirements.txt" ] && cat "${DEV_REPO}/requirements.txt"
|
|
287
|
+
[ -f "${DEV_REPO}/pyproject.toml" ] && cat "${DEV_REPO}/pyproject.toml"
|
|
288
|
+
[ -f "${DEV_REPO}/setup.py" ] && cat "${DEV_REPO}/setup.py"
|
|
289
|
+
|
|
290
|
+
# Check for .NET
|
|
291
|
+
ls "${DEV_REPO}"/*.csproj 2>/dev/null
|
|
292
|
+
ls "${DEV_REPO}"/**/*.csproj 2>/dev/null
|
|
293
|
+
|
|
294
|
+
# Check for Java
|
|
295
|
+
[ -f "${DEV_REPO}/pom.xml" ] && echo "Maven project"
|
|
296
|
+
[ -f "${DEV_REPO}/build.gradle" ] && echo "Gradle project"
|
|
297
|
+
```
|
|
298
|
+
|
|
299
|
+
**Detect test framework from config files:**
|
|
300
|
+
```bash
|
|
301
|
+
# JavaScript/TypeScript ecosystem
|
|
302
|
+
[ -f "${DEV_REPO}/cypress.config.ts" ] || [ -f "${DEV_REPO}/cypress.config.js" ] && echo "FRAMEWORK=cypress"
|
|
303
|
+
[ -f "${DEV_REPO}/playwright.config.ts" ] || [ -f "${DEV_REPO}/playwright.config.js" ] && echo "FRAMEWORK=playwright"
|
|
304
|
+
[ -f "${DEV_REPO}/jest.config.ts" ] || [ -f "${DEV_REPO}/jest.config.js" ] && echo "FRAMEWORK=jest"
|
|
305
|
+
[ -f "${DEV_REPO}/vitest.config.ts" ] || [ -f "${DEV_REPO}/vitest.config.js" ] && echo "FRAMEWORK=vitest"
|
|
306
|
+
|
|
307
|
+
# Python ecosystem
|
|
308
|
+
[ -f "${DEV_REPO}/pytest.ini" ] || [ -f "${DEV_REPO}/conftest.py" ] && echo "FRAMEWORK=pytest"
|
|
309
|
+
|
|
310
|
+
# Check package.json devDependencies for test frameworks
|
|
311
|
+
node -e "
|
|
312
|
+
try {
|
|
313
|
+
const pkg = require('${DEV_REPO}/package.json');
|
|
314
|
+
const deps = {...(pkg.devDependencies||{}), ...(pkg.dependencies||{})};
|
|
315
|
+
const frameworks = [];
|
|
316
|
+
if (deps.cypress) frameworks.push('cypress');
|
|
317
|
+
if (deps['@playwright/test'] || deps.playwright) frameworks.push('playwright');
|
|
318
|
+
if (deps.jest) frameworks.push('jest');
|
|
319
|
+
if (deps.vitest) frameworks.push('vitest');
|
|
320
|
+
if (deps.mocha) frameworks.push('mocha');
|
|
321
|
+
console.log(frameworks.join(',') || 'none');
|
|
322
|
+
} catch { console.log('no-package-json'); }
|
|
323
|
+
" 2>/dev/null
|
|
324
|
+
```
|
|
325
|
+
|
|
326
|
+
**Assess detection confidence:**
|
|
327
|
+
- **HIGH**: Config file found AND matching dependency in package.json/requirements.txt
|
|
328
|
+
- **MEDIUM**: Only dependency found (no config file) OR only config file (no dependency)
|
|
329
|
+
- **LOW**: No test framework detected, or conflicting signals
|
|
330
|
+
|
|
331
|
+
**If no test framework found:**
|
|
332
|
+
- If `IS_AUTO` is false: Ask the user which framework to use. STOP and wait for response.
|
|
333
|
+
- If `IS_AUTO` is true: Select the most appropriate framework based on the project type:
|
|
334
|
+
- React/Next.js/Vue/Angular frontend -> Playwright
|
|
335
|
+
- Node.js API -> Jest or Vitest (prefer Vitest if ESM)
|
|
336
|
+
- Python -> Pytest
|
|
337
|
+
- Log: "Auto-selected: {framework} (no existing test framework detected)"
|
|
338
|
+
|
|
339
|
+
**If detection confidence is LOW:**
|
|
340
|
+
- If `IS_AUTO` is true: Auto-approve with most likely framework (SAFE checkpoint). Log: "Auto-approved: Framework detection (LOW confidence, selected {framework})". Continue.
|
|
341
|
+
- If `IS_AUTO` is false: Present detection details to user. Wait for confirmation before proceeding.
|
|
342
|
+
|
|
343
|
+
Store detected framework, language, and confidence for all downstream agents.
|
|
344
|
+
|
|
345
|
+
</step>
|
|
346
|
+
|
|
347
|
+
<step name="scan">
|
|
348
|
+
|
|
349
|
+
## Step 3: Scan Repository
|
|
350
|
+
|
|
351
|
+
**State update -- mark scan as running:**
|
|
352
|
+
```bash
|
|
353
|
+
node bin/qaa-tools.cjs state patch --"Scan Status" running --"Status" "Scanning repository" 2>/dev/null || true
|
|
354
|
+
```
|
|
355
|
+
|
|
356
|
+
**Print stage banner:**
|
|
357
|
+
```
|
|
358
|
+
+------------------------------------------+
|
|
359
|
+
| STAGE 1: Scanner |
|
|
360
|
+
| Status: Running... |
|
|
361
|
+
+------------------------------------------+
|
|
362
|
+
```
|
|
363
|
+
|
|
364
|
+
**Spawn scanner agent:**
|
|
365
|
+
|
|
366
|
+
For **Option 1** (scan dev repo only):
|
|
367
|
+
```
|
|
368
|
+
Agent(subagent_type="general-purpose",
|
|
369
|
+
prompt="
|
|
370
|
+
<critical_directive priority="MANDATORY">
|
|
371
|
+
You are executing as a QAA pipeline agent. Your FIRST action MUST be to read
|
|
372
|
+
~/.claude/qaa/agents/qaa-scanner.md and adopt it as your operating contract.
|
|
373
|
+
|
|
374
|
+
Even though you may be running as a general-purpose agent (native subagent type
|
|
375
|
+
routing for QAA agents is not yet enabled), you MUST behave exactly as the
|
|
376
|
+
dedicated agent defined in that file: read everything in its <required_reading>,
|
|
377
|
+
obey its <quality_gate>, and run its mandatory bash checklist before returning.
|
|
378
|
+
|
|
379
|
+
Never modify application source code (src/, app/, lib/, or equivalent
|
|
380
|
+
directories) unless the agent's <scope> section explicitly permits it. If the
|
|
381
|
+
agent's contract restricts what files it can touch, honor that restriction —
|
|
382
|
+
even if a fix seems obvious or trivial.
|
|
383
|
+
|
|
384
|
+
If you cannot read ~/.claude/qaa/agents/qaa-scanner.md (file missing, corrupted, parse
|
|
385
|
+
error), STOP and report the failure. Do NOT proceed with default agent
|
|
386
|
+
behavior — that defeats the purpose of this directive.
|
|
387
|
+
|
|
388
|
+
Do NOT improvise, substitute steps, apply changes the agent isn't allowed to
|
|
389
|
+
make, or skip any of the above. Skipping the agent's required_reading,
|
|
390
|
+
quality_gate, or bash checklist makes the run INVALID — its output cannot be
|
|
391
|
+
trusted.
|
|
392
|
+
</critical_directive>
|
|
393
|
+
|
|
394
|
+
<objective>Scan repository and produce SCAN_MANIFEST.md</objective>
|
|
395
|
+
<execution_context>~/.claude/qaa/agents/qaa-scanner.md</execution_context>
|
|
396
|
+
<files_to_read>
|
|
397
|
+
- CLAUDE.md
|
|
398
|
+
- frameworks/*.json (read all framework registry entries for detection)
|
|
399
|
+
</files_to_read>
|
|
400
|
+
<parameters>
|
|
401
|
+
dev_repo_path: {DEV_REPO}
|
|
402
|
+
qa_repo_path: null
|
|
403
|
+
output_path: {output_dir}/SCAN_MANIFEST.md
|
|
404
|
+
</parameters>
|
|
405
|
+
"
|
|
406
|
+
)
|
|
407
|
+
```
|
|
408
|
+
|
|
409
|
+
For **Options 2 and 3** (scan both repos):
|
|
410
|
+
```
|
|
411
|
+
Agent(subagent_type="general-purpose",
|
|
412
|
+
prompt="
|
|
413
|
+
<critical_directive priority="MANDATORY">
|
|
414
|
+
You are executing as a QAA pipeline agent. Your FIRST action MUST be to read
|
|
415
|
+
~/.claude/qaa/agents/qaa-scanner.md and adopt it as your operating contract.
|
|
416
|
+
|
|
417
|
+
Even though you may be running as a general-purpose agent (native subagent type
|
|
418
|
+
routing for QAA agents is not yet enabled), you MUST behave exactly as the
|
|
419
|
+
dedicated agent defined in that file: read everything in its <required_reading>,
|
|
420
|
+
obey its <quality_gate>, and run its mandatory bash checklist before returning.
|
|
421
|
+
|
|
422
|
+
Never modify application source code (src/, app/, lib/, or equivalent
|
|
423
|
+
directories) unless the agent's <scope> section explicitly permits it. If the
|
|
424
|
+
agent's contract restricts what files it can touch, honor that restriction —
|
|
425
|
+
even if a fix seems obvious or trivial.
|
|
426
|
+
|
|
427
|
+
If you cannot read ~/.claude/qaa/agents/qaa-scanner.md (file missing, corrupted, parse
|
|
428
|
+
error), STOP and report the failure. Do NOT proceed with default agent
|
|
429
|
+
behavior — that defeats the purpose of this directive.
|
|
430
|
+
|
|
431
|
+
Do NOT improvise, substitute steps, apply changes the agent isn't allowed to
|
|
432
|
+
make, or skip any of the above. Skipping the agent's required_reading,
|
|
433
|
+
quality_gate, or bash checklist makes the run INVALID — its output cannot be
|
|
434
|
+
trusted.
|
|
435
|
+
</critical_directive>
|
|
436
|
+
|
|
437
|
+
<objective>Scan both developer and QA repositories and produce SCAN_MANIFEST.md</objective>
|
|
438
|
+
<execution_context>~/.claude/qaa/agents/qaa-scanner.md</execution_context>
|
|
439
|
+
<files_to_read>
|
|
440
|
+
- CLAUDE.md
|
|
441
|
+
- frameworks/*.json (read all framework registry entries for detection)
|
|
442
|
+
</files_to_read>
|
|
443
|
+
<parameters>
|
|
444
|
+
dev_repo_path: {DEV_REPO}
|
|
445
|
+
qa_repo_path: {QA_REPO}
|
|
446
|
+
output_path: {output_dir}/SCAN_MANIFEST.md
|
|
447
|
+
</parameters>
|
|
448
|
+
"
|
|
449
|
+
)
|
|
450
|
+
```
|
|
451
|
+
|
|
452
|
+
**Parse scanner return:**
|
|
453
|
+
|
|
454
|
+
Expected return structure:
|
|
455
|
+
```
|
|
456
|
+
SCANNER_COMPLETE:
|
|
457
|
+
file_path: ".qa-output/SCAN_MANIFEST.md"
|
|
458
|
+
decision: PROCEED | STOP
|
|
459
|
+
has_frontend: true | false
|
|
460
|
+
detection_confidence: HIGH | MEDIUM | LOW
|
|
461
|
+
```
|
|
462
|
+
|
|
463
|
+
**Handle decision field:**
|
|
464
|
+
- If `decision` is `STOP`:
|
|
465
|
+
```bash
|
|
466
|
+
node bin/qaa-tools.cjs state patch --"Scan Status" failed --"Status" "Pipeline stopped: Scanner returned STOP" 2>/dev/null || true
|
|
467
|
+
```
|
|
468
|
+
Print failure banner and STOP PIPELINE ENTIRELY. Do NOT proceed to any further stage.
|
|
469
|
+
|
|
470
|
+
- If `decision` is `PROCEED`:
|
|
471
|
+
```bash
|
|
472
|
+
node bin/qaa-tools.cjs state patch --"Scan Status" complete 2>/dev/null || true
|
|
473
|
+
```
|
|
474
|
+
Capture `has_frontend` for testid-injector conditional (Step 5).
|
|
475
|
+
Capture `detection_confidence` for checkpoint handling.
|
|
476
|
+
|
|
477
|
+
**Verify artifact exists before continuing:**
|
|
478
|
+
```bash
|
|
479
|
+
[ -f "${output_dir}/SCAN_MANIFEST.md" ] && echo "OK: SCAN_MANIFEST.md exists" || echo "MISSING: SCAN_MANIFEST.md"
|
|
480
|
+
```
|
|
481
|
+
|
|
482
|
+
If SCAN_MANIFEST.md is missing, treat as stage failure. Set status to failed and STOP pipeline.
|
|
483
|
+
|
|
484
|
+
</step>
|
|
485
|
+
|
|
486
|
+
<step name="codebase_map">
|
|
487
|
+
|
|
488
|
+
## Step 3b: Codebase Map (Deep Analysis)
|
|
489
|
+
|
|
490
|
+
**State update -- mark codebase-map as running:**
|
|
491
|
+
```bash
|
|
492
|
+
node bin/qaa-tools.cjs state patch --"Map Status" running --"Status" "Mapping codebase" 2>/dev/null || true
|
|
493
|
+
```
|
|
494
|
+
|
|
495
|
+
**Print stage banner:**
|
|
496
|
+
```
|
|
497
|
+
+------------------------------------------+
|
|
498
|
+
| STAGE 1b: Codebase Mapper |
|
|
499
|
+
| Status: Running 4 parallel sub-agents...|
|
|
500
|
+
+------------------------------------------+
|
|
501
|
+
```
|
|
502
|
+
|
|
503
|
+
**Create output directory:**
|
|
504
|
+
```bash
|
|
505
|
+
mkdir -p ${output_dir}/codebase
|
|
506
|
+
```
|
|
507
|
+
|
|
508
|
+
**Spawn 4 parallel sub-agents** (one per focus area). Issue all 4 `Agent()` calls in a single message for parallel execution:
|
|
509
|
+
|
|
510
|
+
**Sub-agent 1 - testability focus** (produces TESTABILITY.md + TEST_SURFACE.md):
|
|
511
|
+
```
|
|
512
|
+
Agent(subagent_type="general-purpose",
|
|
513
|
+
prompt="
|
|
514
|
+
<critical_directive priority="MANDATORY">
|
|
515
|
+
You are executing as a QAA pipeline agent. Your FIRST action MUST be to read
|
|
516
|
+
~/.claude/qaa/agents/qaa-codebase-mapper.md and adopt it as your operating contract.
|
|
517
|
+
|
|
518
|
+
Even though you may be running as a general-purpose agent (native subagent type
|
|
519
|
+
routing for QAA agents is not yet enabled), you MUST behave exactly as the
|
|
520
|
+
dedicated agent defined in that file: read everything in its <required_reading>,
|
|
521
|
+
obey its <quality_gate>, and run its mandatory bash checklist before returning.
|
|
522
|
+
|
|
523
|
+
Never modify application source code (src/, app/, lib/, or equivalent
|
|
524
|
+
directories) unless the agent's <scope> section explicitly permits it. If the
|
|
525
|
+
agent's contract restricts what files it can touch, honor that restriction —
|
|
526
|
+
even if a fix seems obvious or trivial.
|
|
527
|
+
|
|
528
|
+
If you cannot read ~/.claude/qaa/agents/qaa-codebase-mapper.md (file missing, corrupted, parse
|
|
529
|
+
error), STOP and report the failure. Do NOT proceed with default agent
|
|
530
|
+
behavior — that defeats the purpose of this directive.
|
|
531
|
+
|
|
532
|
+
Do NOT improvise, substitute steps, apply changes the agent isn't allowed to
|
|
533
|
+
make, or skip any of the above. Skipping the agent's required_reading,
|
|
534
|
+
quality_gate, or bash checklist makes the run INVALID — its output cannot be
|
|
535
|
+
trusted.
|
|
536
|
+
</critical_directive>
|
|
537
|
+
|
|
538
|
+
<objective>Codebase analysis - testability focus. Produce TESTABILITY.md and TEST_SURFACE.md mapping what is testable in this codebase, mock boundaries, and exhaustive list of testable entry points.</objective>
|
|
539
|
+
<execution_context>~/.claude/qaa/agents/qaa-codebase-mapper.md</execution_context>
|
|
540
|
+
<files_to_read>
|
|
541
|
+
- CLAUDE.md
|
|
542
|
+
- ${output_dir}/SCAN_MANIFEST.md
|
|
543
|
+
</files_to_read>
|
|
544
|
+
<parameters>
|
|
545
|
+
focus_area: testability
|
|
546
|
+
dev_repo_path: {DEV_REPO}
|
|
547
|
+
output_dir: ${output_dir}/codebase
|
|
548
|
+
</parameters>
|
|
549
|
+
"
|
|
550
|
+
)
|
|
551
|
+
```
|
|
552
|
+
|
|
553
|
+
**Sub-agent 2 - risk focus** (produces RISK_MAP.md + CRITICAL_PATHS.md):
|
|
554
|
+
```
|
|
555
|
+
Agent(subagent_type="general-purpose",
|
|
556
|
+
prompt="
|
|
557
|
+
<critical_directive priority="MANDATORY">
|
|
558
|
+
You are executing as a QAA pipeline agent. Your FIRST action MUST be to read
|
|
559
|
+
~/.claude/qaa/agents/qaa-codebase-mapper.md and adopt it as your operating contract.
|
|
560
|
+
|
|
561
|
+
Even though you may be running as a general-purpose agent (native subagent type
|
|
562
|
+
routing for QAA agents is not yet enabled), you MUST behave exactly as the
|
|
563
|
+
dedicated agent defined in that file: read everything in its <required_reading>,
|
|
564
|
+
obey its <quality_gate>, and run its mandatory bash checklist before returning.
|
|
565
|
+
|
|
566
|
+
Never modify application source code (src/, app/, lib/, or equivalent
|
|
567
|
+
directories) unless the agent's <scope> section explicitly permits it. If the
|
|
568
|
+
agent's contract restricts what files it can touch, honor that restriction —
|
|
569
|
+
even if a fix seems obvious or trivial.
|
|
570
|
+
|
|
571
|
+
If you cannot read ~/.claude/qaa/agents/qaa-codebase-mapper.md (file missing, corrupted, parse
|
|
572
|
+
error), STOP and report the failure. Do NOT proceed with default agent
|
|
573
|
+
behavior — that defeats the purpose of this directive.
|
|
574
|
+
|
|
575
|
+
Do NOT improvise, substitute steps, apply changes the agent isn't allowed to
|
|
576
|
+
make, or skip any of the above. Skipping the agent's required_reading,
|
|
577
|
+
quality_gate, or bash checklist makes the run INVALID — its output cannot be
|
|
578
|
+
trusted.
|
|
579
|
+
</critical_directive>
|
|
580
|
+
|
|
581
|
+
<objective>Codebase analysis - risk focus. Produce RISK_MAP.md and CRITICAL_PATHS.md identifying business-critical paths, security-sensitive areas, and the exact user flows that E2E smoke tests must cover.</objective>
|
|
582
|
+
<execution_context>~/.claude/qaa/agents/qaa-codebase-mapper.md</execution_context>
|
|
583
|
+
<files_to_read>
|
|
584
|
+
- CLAUDE.md
|
|
585
|
+
- ${output_dir}/SCAN_MANIFEST.md
|
|
586
|
+
</files_to_read>
|
|
587
|
+
<parameters>
|
|
588
|
+
focus_area: risk
|
|
589
|
+
dev_repo_path: {DEV_REPO}
|
|
590
|
+
output_dir: ${output_dir}/codebase
|
|
591
|
+
</parameters>
|
|
592
|
+
"
|
|
593
|
+
)
|
|
594
|
+
```
|
|
595
|
+
|
|
596
|
+
**Sub-agent 3 - patterns focus** (produces CODE_PATTERNS.md + API_CONTRACTS.md):
|
|
597
|
+
```
|
|
598
|
+
Agent(subagent_type="general-purpose",
|
|
599
|
+
prompt="
|
|
600
|
+
<critical_directive priority="MANDATORY">
|
|
601
|
+
You are executing as a QAA pipeline agent. Your FIRST action MUST be to read
|
|
602
|
+
~/.claude/qaa/agents/qaa-codebase-mapper.md and adopt it as your operating contract.
|
|
603
|
+
|
|
604
|
+
Even though you may be running as a general-purpose agent (native subagent type
|
|
605
|
+
routing for QAA agents is not yet enabled), you MUST behave exactly as the
|
|
606
|
+
dedicated agent defined in that file: read everything in its <required_reading>,
|
|
607
|
+
obey its <quality_gate>, and run its mandatory bash checklist before returning.
|
|
608
|
+
|
|
609
|
+
Never modify application source code (src/, app/, lib/, or equivalent
|
|
610
|
+
directories) unless the agent's <scope> section explicitly permits it. If the
|
|
611
|
+
agent's contract restricts what files it can touch, honor that restriction —
|
|
612
|
+
even if a fix seems obvious or trivial.
|
|
613
|
+
|
|
614
|
+
If you cannot read ~/.claude/qaa/agents/qaa-codebase-mapper.md (file missing, corrupted, parse
|
|
615
|
+
error), STOP and report the failure. Do NOT proceed with default agent
|
|
616
|
+
behavior — that defeats the purpose of this directive.
|
|
617
|
+
|
|
618
|
+
Do NOT improvise, substitute steps, apply changes the agent isn't allowed to
|
|
619
|
+
make, or skip any of the above. Skipping the agent's required_reading,
|
|
620
|
+
quality_gate, or bash checklist makes the run INVALID — its output cannot be
|
|
621
|
+
trusted.
|
|
622
|
+
</critical_directive>
|
|
623
|
+
|
|
624
|
+
<objective>Codebase analysis - patterns focus. Produce CODE_PATTERNS.md and API_CONTRACTS.md documenting naming conventions, import style, and exact request/response shapes for API tests.</objective>
|
|
625
|
+
<execution_context>~/.claude/qaa/agents/qaa-codebase-mapper.md</execution_context>
|
|
626
|
+
<files_to_read>
|
|
627
|
+
- CLAUDE.md
|
|
628
|
+
- ${output_dir}/SCAN_MANIFEST.md
|
|
629
|
+
</files_to_read>
|
|
630
|
+
<parameters>
|
|
631
|
+
focus_area: patterns
|
|
632
|
+
dev_repo_path: {DEV_REPO}
|
|
633
|
+
output_dir: ${output_dir}/codebase
|
|
634
|
+
</parameters>
|
|
635
|
+
"
|
|
636
|
+
)
|
|
637
|
+
```
|
|
638
|
+
|
|
639
|
+
**Sub-agent 4 - existing-tests focus** (produces TEST_ASSESSMENT.md + COVERAGE_GAPS.md):
|
|
640
|
+
```
|
|
641
|
+
Agent(subagent_type="general-purpose",
|
|
642
|
+
prompt="
|
|
643
|
+
<critical_directive priority="MANDATORY">
|
|
644
|
+
You are executing as a QAA pipeline agent. Your FIRST action MUST be to read
|
|
645
|
+
~/.claude/qaa/agents/qaa-codebase-mapper.md and adopt it as your operating contract.
|
|
646
|
+
|
|
647
|
+
Even though you may be running as a general-purpose agent (native subagent type
|
|
648
|
+
routing for QAA agents is not yet enabled), you MUST behave exactly as the
|
|
649
|
+
dedicated agent defined in that file: read everything in its <required_reading>,
|
|
650
|
+
obey its <quality_gate>, and run its mandatory bash checklist before returning.
|
|
651
|
+
|
|
652
|
+
Never modify application source code (src/, app/, lib/, or equivalent
|
|
653
|
+
directories) unless the agent's <scope> section explicitly permits it. If the
|
|
654
|
+
agent's contract restricts what files it can touch, honor that restriction —
|
|
655
|
+
even if a fix seems obvious or trivial.
|
|
656
|
+
|
|
657
|
+
If you cannot read ~/.claude/qaa/agents/qaa-codebase-mapper.md (file missing, corrupted, parse
|
|
658
|
+
error), STOP and report the failure. Do NOT proceed with default agent
|
|
659
|
+
behavior — that defeats the purpose of this directive.
|
|
660
|
+
|
|
661
|
+
Do NOT improvise, substitute steps, apply changes the agent isn't allowed to
|
|
662
|
+
make, or skip any of the above. Skipping the agent's required_reading,
|
|
663
|
+
quality_gate, or bash checklist makes the run INVALID — its output cannot be
|
|
664
|
+
trusted.
|
|
665
|
+
</critical_directive>
|
|
666
|
+
|
|
667
|
+
<objective>Codebase analysis - existing-tests focus. Produce TEST_ASSESSMENT.md and COVERAGE_GAPS.md assessing current test quality, frameworks in use, and identifying which modules/functions/paths have no test coverage.</objective>
|
|
668
|
+
<execution_context>~/.claude/qaa/agents/qaa-codebase-mapper.md</execution_context>
|
|
669
|
+
<files_to_read>
|
|
670
|
+
- CLAUDE.md
|
|
671
|
+
- ${output_dir}/SCAN_MANIFEST.md
|
|
672
|
+
</files_to_read>
|
|
673
|
+
<parameters>
|
|
674
|
+
focus_area: existing-tests
|
|
675
|
+
dev_repo_path: {DEV_REPO}
|
|
676
|
+
output_dir: ${output_dir}/codebase
|
|
677
|
+
</parameters>
|
|
678
|
+
"
|
|
679
|
+
)
|
|
680
|
+
```
|
|
681
|
+
|
|
682
|
+
**Verify codebase docs exist:**
|
|
683
|
+
```bash
|
|
684
|
+
docs_count=$(ls ${output_dir}/codebase/*.md 2>/dev/null | wc -l)
|
|
685
|
+
echo "Codebase docs produced: $docs_count of 8 expected"
|
|
686
|
+
|
|
687
|
+
if [ "$docs_count" -eq 0 ]; then
|
|
688
|
+
echo "FAIL: No codebase docs produced. Pipeline continues with degraded context."
|
|
689
|
+
node bin/qaa-tools.cjs state patch --"Map Status" failed 2>/dev/null || true
|
|
690
|
+
elif [ "$docs_count" -lt 4 ]; then
|
|
691
|
+
echo "WARNING: Fewer than 4 codebase docs produced. Downstream agents will have limited context."
|
|
692
|
+
node bin/qaa-tools.cjs state patch --"Map Status" "partial" 2>/dev/null || true
|
|
693
|
+
else
|
|
694
|
+
echo "OK: codebase map sufficient ($docs_count docs)"
|
|
695
|
+
node bin/qaa-tools.cjs state patch --"Map Status" complete 2>/dev/null || true
|
|
696
|
+
fi
|
|
697
|
+
```
|
|
698
|
+
|
|
699
|
+
**Codebase map is non-blocking:** if zero docs are produced, the pipeline continues with degraded context (downstream agents fall back to whatever they can find). All `<files_to_read>` references to codebase docs in subsequent stages use `(if exists)` semantics.
|
|
700
|
+
|
|
701
|
+
</step>
|
|
702
|
+
|
|
703
|
+
<step name="research">
|
|
704
|
+
|
|
705
|
+
## Step 3c: Research Testing Ecosystem
|
|
706
|
+
|
|
707
|
+
**State update -- mark research as running:**
|
|
708
|
+
```bash
|
|
709
|
+
node bin/qaa-tools.cjs state patch --"Research Status" running --"Status" "Researching testing ecosystem" 2>/dev/null || true
|
|
710
|
+
```
|
|
711
|
+
|
|
712
|
+
**Print stage banner:**
|
|
713
|
+
```
|
|
714
|
+
+------------------------------------------+
|
|
715
|
+
| STAGE 1c: Project Researcher |
|
|
716
|
+
| Status: Running... |
|
|
717
|
+
+------------------------------------------+
|
|
718
|
+
```
|
|
719
|
+
|
|
720
|
+
**Create output directory:**
|
|
721
|
+
```bash
|
|
722
|
+
mkdir -p ${output_dir}/research
|
|
723
|
+
```
|
|
724
|
+
|
|
725
|
+
**Spawn researcher agent:**
|
|
726
|
+
```
|
|
727
|
+
Agent(subagent_type="general-purpose",
|
|
728
|
+
prompt="
|
|
729
|
+
<critical_directive priority="MANDATORY">
|
|
730
|
+
You are executing as a QAA pipeline agent. Your FIRST action MUST be to read
|
|
731
|
+
~/.claude/qaa/agents/qaa-project-researcher.md and adopt it as your operating contract.
|
|
732
|
+
|
|
733
|
+
Even though you may be running as a general-purpose agent (native subagent type
|
|
734
|
+
routing for QAA agents is not yet enabled), you MUST behave exactly as the
|
|
735
|
+
dedicated agent defined in that file: read everything in its <required_reading>,
|
|
736
|
+
obey its <quality_gate>, and run its mandatory bash checklist before returning.
|
|
737
|
+
|
|
738
|
+
Never modify application source code (src/, app/, lib/, or equivalent
|
|
739
|
+
directories) unless the agent's <scope> section explicitly permits it. If the
|
|
740
|
+
agent's contract restricts what files it can touch, honor that restriction —
|
|
741
|
+
even if a fix seems obvious or trivial.
|
|
742
|
+
|
|
743
|
+
If you cannot read ~/.claude/qaa/agents/qaa-project-researcher.md (file missing, corrupted, parse
|
|
744
|
+
error), STOP and report the failure. Do NOT proceed with default agent
|
|
745
|
+
behavior — that defeats the purpose of this directive.
|
|
746
|
+
|
|
747
|
+
Do NOT improvise, substitute steps, apply changes the agent isn't allowed to
|
|
748
|
+
make, or skip any of the above. Skipping the agent's required_reading,
|
|
749
|
+
quality_gate, or bash checklist makes the run INVALID — its output cannot be
|
|
750
|
+
trusted.
|
|
751
|
+
</critical_directive>
|
|
752
|
+
|
|
753
|
+
<objective>Research the testing ecosystem for this project. Use the codebase map findings (RISK_MAP, CODE_PATTERNS, API_CONTRACTS, TESTABILITY, etc., from ${output_dir}/codebase/) to ground your Context7 queries to the project specifics — query for the libraries/patterns the project actually uses (e.g., MSW integration, Stripe testing, OpenAPI contract testing) instead of generic framework questions. Use Context7 MCP as the primary source. Produce research documents consumed by downstream agents.</objective>
|
|
754
|
+
<execution_context>~/.claude/qaa/agents/qaa-project-researcher.md</execution_context>
|
|
755
|
+
<files_to_read>
|
|
756
|
+
- CLAUDE.md
|
|
757
|
+
- ~/.claude/qaa/MY_PREFERENCES.md (if exists)
|
|
758
|
+
- ${output_dir}/SCAN_MANIFEST.md
|
|
759
|
+
- ${output_dir}/codebase/TESTABILITY.md (if exists)
|
|
760
|
+
- ${output_dir}/codebase/TEST_SURFACE.md (if exists)
|
|
761
|
+
- ${output_dir}/codebase/RISK_MAP.md (if exists)
|
|
762
|
+
- ${output_dir}/codebase/CRITICAL_PATHS.md (if exists)
|
|
763
|
+
- ${output_dir}/codebase/CODE_PATTERNS.md (if exists)
|
|
764
|
+
- ${output_dir}/codebase/API_CONTRACTS.md (if exists)
|
|
765
|
+
- ${output_dir}/codebase/TEST_ASSESSMENT.md (if exists)
|
|
766
|
+
- ${output_dir}/codebase/COVERAGE_GAPS.md (if exists)
|
|
767
|
+
</files_to_read>
|
|
768
|
+
<parameters>
|
|
769
|
+
mode: stack-testing
|
|
770
|
+
dev_repo_path: {DEV_REPO}
|
|
771
|
+
output_dir: ${output_dir}/research
|
|
772
|
+
</parameters>
|
|
773
|
+
"
|
|
774
|
+
)
|
|
775
|
+
```
|
|
776
|
+
|
|
777
|
+
**Verify research artifacts exist:**
|
|
778
|
+
```bash
|
|
779
|
+
ls ${output_dir}/research/*.md 2>/dev/null && echo "OK: Research files produced" || echo "WARNING: No research files produced"
|
|
780
|
+
```
|
|
781
|
+
|
|
782
|
+
**Research is non-blocking:** If the researcher fails or produces no output, the pipeline continues — downstream agents will fall back to Context7 queries directly. Log the warning but do NOT stop the pipeline.
|
|
783
|
+
|
|
784
|
+
```bash
|
|
785
|
+
if [ ! -f "${output_dir}/research/TESTING_STACK.md" ]; then
|
|
786
|
+
echo "WARNING: Research stage produced no output. Downstream agents will query Context7 directly."
|
|
787
|
+
node bin/qaa-tools.cjs state patch --"Research Status" "skipped (no output)" 2>/dev/null || true
|
|
788
|
+
else
|
|
789
|
+
node bin/qaa-tools.cjs state patch --"Research Status" complete 2>/dev/null || true
|
|
790
|
+
fi
|
|
791
|
+
```
|
|
792
|
+
|
|
793
|
+
</step>
|
|
794
|
+
|
|
795
|
+
<step name="analyze">
|
|
796
|
+
|
|
797
|
+
## Step 4: Analyze Repository
|
|
798
|
+
|
|
799
|
+
**State update -- mark analyze as running:**
|
|
800
|
+
```bash
|
|
801
|
+
node bin/qaa-tools.cjs state patch --"Analyze Status" running --"Status" "Analyzing repository" 2>/dev/null || true
|
|
802
|
+
```
|
|
803
|
+
|
|
804
|
+
**Print stage banner:**
|
|
805
|
+
```
|
|
806
|
+
+------------------------------------------+
|
|
807
|
+
| STAGE 2: Analyzer |
|
|
808
|
+
| Status: Running... |
|
|
809
|
+
+------------------------------------------+
|
|
810
|
+
```
|
|
811
|
+
|
|
812
|
+
**Determine analyzer mode based on option:**
|
|
813
|
+
- Option 1: `mode = 'full'` (produces QA_ANALYSIS.md + TEST_INVENTORY.md + QA_REPO_BLUEPRINT.md)
|
|
814
|
+
- Options 2 and 3: `mode = 'gap'` (produces GAP_ANALYSIS.md)
|
|
815
|
+
|
|
816
|
+
**Spawn analyzer agent:**
|
|
817
|
+
```
|
|
818
|
+
Agent(subagent_type="general-purpose",
|
|
819
|
+
prompt="
|
|
820
|
+
<critical_directive priority="MANDATORY">
|
|
821
|
+
You are executing as a QAA pipeline agent. Your FIRST action MUST be to read
|
|
822
|
+
~/.claude/qaa/agents/qaa-analyzer.md and adopt it as your operating contract.
|
|
823
|
+
|
|
824
|
+
Even though you may be running as a general-purpose agent (native subagent type
|
|
825
|
+
routing for QAA agents is not yet enabled), you MUST behave exactly as the
|
|
826
|
+
dedicated agent defined in that file: read everything in its <required_reading>,
|
|
827
|
+
obey its <quality_gate>, and run its mandatory bash checklist before returning.
|
|
828
|
+
|
|
829
|
+
Never modify application source code (src/, app/, lib/, or equivalent
|
|
830
|
+
directories) unless the agent's <scope> section explicitly permits it. If the
|
|
831
|
+
agent's contract restricts what files it can touch, honor that restriction —
|
|
832
|
+
even if a fix seems obvious or trivial.
|
|
833
|
+
|
|
834
|
+
If you cannot read ~/.claude/qaa/agents/qaa-analyzer.md (file missing, corrupted, parse
|
|
835
|
+
error), STOP and report the failure. Do NOT proceed with default agent
|
|
836
|
+
behavior — that defeats the purpose of this directive.
|
|
837
|
+
|
|
838
|
+
Do NOT improvise, substitute steps, apply changes the agent isn't allowed to
|
|
839
|
+
make, or skip any of the above. Skipping the agent's required_reading,
|
|
840
|
+
quality_gate, or bash checklist makes the run INVALID — its output cannot be
|
|
841
|
+
trusted.
|
|
842
|
+
</critical_directive>
|
|
843
|
+
|
|
844
|
+
<objective>Analyze scanned repository and produce analysis artifacts</objective>
|
|
845
|
+
<execution_context>~/.claude/qaa/agents/qaa-analyzer.md</execution_context>
|
|
846
|
+
<files_to_read>
|
|
847
|
+
- {output_dir}/SCAN_MANIFEST.md
|
|
848
|
+
- CLAUDE.md
|
|
849
|
+
- {output_dir}/research/TESTING_STACK.md (if exists)
|
|
850
|
+
- {output_dir}/research/FRAMEWORK_CAPABILITIES.md (if exists)
|
|
851
|
+
- {output_dir}/codebase/RISK_MAP.md (if exists)
|
|
852
|
+
- {output_dir}/codebase/CRITICAL_PATHS.md (if exists)
|
|
853
|
+
- {output_dir}/codebase/TEST_ASSESSMENT.md (if exists)
|
|
854
|
+
- {output_dir}/codebase/COVERAGE_GAPS.md (if exists)
|
|
855
|
+
</files_to_read>
|
|
856
|
+
<parameters>
|
|
857
|
+
mode: {mode}
|
|
858
|
+
workflow_option: {option}
|
|
859
|
+
dev_repo_path: {DEV_REPO}
|
|
860
|
+
qa_repo_path: {QA_REPO or null}
|
|
861
|
+
output_path: {output_dir}/
|
|
862
|
+
</parameters>
|
|
863
|
+
"
|
|
864
|
+
)
|
|
865
|
+
```
|
|
866
|
+
|
|
867
|
+
**Parse analyzer return:**
|
|
868
|
+
|
|
869
|
+
Expected return structure:
|
|
870
|
+
```
|
|
871
|
+
ANALYZER_COMPLETE:
|
|
872
|
+
files_produced: [...]
|
|
873
|
+
total_test_count: N
|
|
874
|
+
pyramid_breakdown: {unit: N, integration: N, api: N, e2e: N}
|
|
875
|
+
risk_count: {high: N, medium: N, low: N}
|
|
876
|
+
commit_hash: "..."
|
|
877
|
+
```
|
|
878
|
+
|
|
879
|
+
Capture `files_produced`, `total_test_count`, `pyramid_breakdown` for downstream stages.
|
|
880
|
+
|
|
881
|
+
**Handle analyzer checkpoint -- assumptions review:**
|
|
882
|
+
- If `IS_AUTO` is true: Auto-approve all assumptions (SAFE checkpoint). Log: "Auto-approved: Analyzer assumptions". Continue pipeline.
|
|
883
|
+
- If `IS_AUTO` is false: Present assumptions to user for review. Wait for confirmation or corrections. On user response, incorporate corrections and continue.
|
|
884
|
+
|
|
885
|
+
**State update -- mark analyze as complete:**
|
|
886
|
+
```bash
|
|
887
|
+
node bin/qaa-tools.cjs state patch --"Analyze Status" complete 2>/dev/null || true
|
|
888
|
+
```
|
|
889
|
+
|
|
890
|
+
**Verify artifacts exist before continuing:**
|
|
891
|
+
|
|
892
|
+
For Option 1:
|
|
893
|
+
```bash
|
|
894
|
+
[ -f "${output_dir}/QA_ANALYSIS.md" ] && echo "OK" || echo "MISSING: QA_ANALYSIS.md"
|
|
895
|
+
[ -f "${output_dir}/TEST_INVENTORY.md" ] && echo "OK" || echo "MISSING: TEST_INVENTORY.md"
|
|
896
|
+
```
|
|
897
|
+
|
|
898
|
+
For Options 2/3:
|
|
899
|
+
```bash
|
|
900
|
+
[ -f "${output_dir}/GAP_ANALYSIS.md" ] && echo "OK" || echo "MISSING: GAP_ANALYSIS.md"
|
|
901
|
+
```
|
|
902
|
+
|
|
903
|
+
If required artifacts are missing, treat as stage failure. Set status to failed and STOP pipeline.
|
|
904
|
+
|
|
905
|
+
Print: "Analysis complete. {total_test_count} test cases identified. Pyramid: unit={unit}, integration={integration}, api={api}, e2e={e2e}."
|
|
906
|
+
|
|
907
|
+
</step>
|
|
908
|
+
|
|
909
|
+
<step name="testid_inject">
|
|
910
|
+
|
|
911
|
+
## Step 5: TestID Injection (Conditional)
|
|
912
|
+
|
|
913
|
+
**Condition:** Only execute if `has_frontend` is `true` from scanner return (Step 3).
|
|
914
|
+
|
|
915
|
+
**If `has_frontend` is false:**
|
|
916
|
+
Print: "Skipping TestID injection (no frontend detected)." Proceed directly to Step 6 (Plan).
|
|
917
|
+
|
|
918
|
+
**If `has_frontend` is true:**
|
|
919
|
+
|
|
920
|
+
**State update:**
|
|
921
|
+
```bash
|
|
922
|
+
node bin/qaa-tools.cjs state patch --"Status" "Injecting test IDs into frontend components" 2>/dev/null || true
|
|
923
|
+
```
|
|
924
|
+
|
|
925
|
+
**Print stage banner:**
|
|
926
|
+
```
|
|
927
|
+
+------------------------------------------+
|
|
928
|
+
| STAGE 3: TestID Injector |
|
|
929
|
+
| Status: Running... |
|
|
930
|
+
+------------------------------------------+
|
|
931
|
+
```
|
|
932
|
+
|
|
933
|
+
**Spawn testid-injector agent:**
|
|
934
|
+
```
|
|
935
|
+
Agent(subagent_type="general-purpose",
|
|
936
|
+
prompt="
|
|
937
|
+
<critical_directive priority="MANDATORY">
|
|
938
|
+
You are executing as a QAA pipeline agent. Your FIRST action MUST be to read
|
|
939
|
+
~/.claude/qaa/agents/qaa-testid-injector.md and adopt it as your operating contract.
|
|
940
|
+
|
|
941
|
+
Even though you may be running as a general-purpose agent (native subagent type
|
|
942
|
+
routing for QAA agents is not yet enabled), you MUST behave exactly as the
|
|
943
|
+
dedicated agent defined in that file: read everything in its <required_reading>,
|
|
944
|
+
obey its <quality_gate>, and run its mandatory bash checklist before returning.
|
|
945
|
+
|
|
946
|
+
Never modify application source code (src/, app/, lib/, or equivalent
|
|
947
|
+
directories) unless the agent's <scope> section explicitly permits it. If the
|
|
948
|
+
agent's contract restricts what files it can touch, honor that restriction —
|
|
949
|
+
even if a fix seems obvious or trivial.
|
|
950
|
+
|
|
951
|
+
If you cannot read ~/.claude/qaa/agents/qaa-testid-injector.md (file missing, corrupted, parse
|
|
952
|
+
error), STOP and report the failure. Do NOT proceed with default agent
|
|
953
|
+
behavior — that defeats the purpose of this directive.
|
|
954
|
+
|
|
955
|
+
Do NOT improvise, substitute steps, apply changes the agent isn't allowed to
|
|
956
|
+
make, or skip any of the above. Skipping the agent's required_reading,
|
|
957
|
+
quality_gate, or bash checklist makes the run INVALID — its output cannot be
|
|
958
|
+
trusted.
|
|
959
|
+
</critical_directive>
|
|
960
|
+
|
|
961
|
+
<objective>Audit and inject data-testid attributes into frontend components</objective>
|
|
962
|
+
<execution_context>~/.claude/qaa/agents/qaa-testid-injector.md</execution_context>
|
|
963
|
+
<files_to_read>
|
|
964
|
+
- {output_dir}/SCAN_MANIFEST.md
|
|
965
|
+
- CLAUDE.md
|
|
966
|
+
</files_to_read>
|
|
967
|
+
<parameters>
|
|
968
|
+
dev_repo_path: {DEV_REPO}
|
|
969
|
+
output_path: {output_dir}/TESTID_AUDIT_REPORT.md
|
|
970
|
+
</parameters>
|
|
971
|
+
"
|
|
972
|
+
)
|
|
973
|
+
```
|
|
974
|
+
|
|
975
|
+
**Parse return:**
|
|
976
|
+
|
|
977
|
+
Check for `INJECTOR_COMPLETE` vs `INJECTOR_SKIPPED`:
|
|
978
|
+
|
|
979
|
+
If `INJECTOR_COMPLETE`:
|
|
980
|
+
```
|
|
981
|
+
INJECTOR_COMPLETE:
|
|
982
|
+
report_path: "..."
|
|
983
|
+
coverage_before: N%
|
|
984
|
+
coverage_after: N%
|
|
985
|
+
elements_injected: N
|
|
986
|
+
components_modified: N
|
|
987
|
+
```
|
|
988
|
+
Log: "TestID injection complete. Coverage: {coverage_before}% -> {coverage_after}%. {elements_injected} elements injected."
|
|
989
|
+
|
|
990
|
+
If `INJECTOR_SKIPPED`:
|
|
991
|
+
```
|
|
992
|
+
INJECTOR_SKIPPED:
|
|
993
|
+
reason: "..."
|
|
994
|
+
action: "..."
|
|
995
|
+
```
|
|
996
|
+
Log the reason and continue pipeline.
|
|
997
|
+
|
|
998
|
+
**Handle injector checkpoint -- audit review:**
|
|
999
|
+
- If `IS_AUTO` is true: Auto-approve P0-only injection (SAFE checkpoint). Log: "Auto-approved: TestID injection (P0 elements only)". Continue pipeline.
|
|
1000
|
+
- If `IS_AUTO` is false: Present audit report to user. Wait for approval, element selection, or rejection. On user response, incorporate decisions and continue.
|
|
1001
|
+
|
|
1002
|
+
**Verify artifact exists:**
|
|
1003
|
+
```bash
|
|
1004
|
+
[ -f "${output_dir}/TESTID_AUDIT_REPORT.md" ] && echo "OK" || echo "MISSING: TESTID_AUDIT_REPORT.md"
|
|
1005
|
+
```
|
|
1006
|
+
|
|
1007
|
+
</step>
|
|
1008
|
+
|
|
1009
|
+
<step name="plan">
|
|
1010
|
+
|
|
1011
|
+
## Step 6: Plan Test Generation
|
|
1012
|
+
|
|
1013
|
+
**State update -- mark generation as running (planning is part of generate):**
|
|
1014
|
+
```bash
|
|
1015
|
+
node bin/qaa-tools.cjs state patch --"Generate Status" running --"Status" "Planning test generation" 2>/dev/null || true
|
|
1016
|
+
```
|
|
1017
|
+
|
|
1018
|
+
**Print stage banner:**
|
|
1019
|
+
```
|
|
1020
|
+
+------------------------------------------+
|
|
1021
|
+
| STAGE 4: Planner |
|
|
1022
|
+
| Status: Running... |
|
|
1023
|
+
+------------------------------------------+
|
|
1024
|
+
```
|
|
1025
|
+
|
|
1026
|
+
**Determine planner input based on option:**
|
|
1027
|
+
- Option 1: Input from `{output_dir}/TEST_INVENTORY.md` + `{output_dir}/QA_ANALYSIS.md`
|
|
1028
|
+
- Options 2 and 3: Input from `{output_dir}/GAP_ANALYSIS.md`
|
|
1029
|
+
|
|
1030
|
+
**Spawn planner agent:**
|
|
1031
|
+
```
|
|
1032
|
+
Agent(subagent_type="general-purpose",
|
|
1033
|
+
prompt="
|
|
1034
|
+
<critical_directive priority="MANDATORY">
|
|
1035
|
+
You are executing as a QAA pipeline agent. Your FIRST action MUST be to read
|
|
1036
|
+
~/.claude/qaa/agents/qaa-planner.md and adopt it as your operating contract.
|
|
1037
|
+
|
|
1038
|
+
Even though you may be running as a general-purpose agent (native subagent type
|
|
1039
|
+
routing for QAA agents is not yet enabled), you MUST behave exactly as the
|
|
1040
|
+
dedicated agent defined in that file: read everything in its <required_reading>,
|
|
1041
|
+
obey its <quality_gate>, and run its mandatory bash checklist before returning.
|
|
1042
|
+
|
|
1043
|
+
Never modify application source code (src/, app/, lib/, or equivalent
|
|
1044
|
+
directories) unless the agent's <scope> section explicitly permits it. If the
|
|
1045
|
+
agent's contract restricts what files it can touch, honor that restriction —
|
|
1046
|
+
even if a fix seems obvious or trivial.
|
|
1047
|
+
|
|
1048
|
+
If you cannot read ~/.claude/qaa/agents/qaa-planner.md (file missing, corrupted, parse
|
|
1049
|
+
error), STOP and report the failure. Do NOT proceed with default agent
|
|
1050
|
+
behavior — that defeats the purpose of this directive.
|
|
1051
|
+
|
|
1052
|
+
Do NOT improvise, substitute steps, apply changes the agent isn't allowed to
|
|
1053
|
+
make, or skip any of the above. Skipping the agent's required_reading,
|
|
1054
|
+
quality_gate, or bash checklist makes the run INVALID — its output cannot be
|
|
1055
|
+
trusted.
|
|
1056
|
+
</critical_directive>
|
|
1057
|
+
|
|
1058
|
+
<objective>Create test generation plan with task breakdown and dependencies</objective>
|
|
1059
|
+
<execution_context>~/.claude/qaa/agents/qaa-planner.md</execution_context>
|
|
1060
|
+
<files_to_read>
|
|
1061
|
+
- {input files based on option}
|
|
1062
|
+
- CLAUDE.md
|
|
1063
|
+
- {output_dir}/codebase/TESTABILITY.md (if exists)
|
|
1064
|
+
- {output_dir}/codebase/TEST_SURFACE.md (if exists)
|
|
1065
|
+
- {output_dir}/codebase/CRITICAL_PATHS.md (if exists)
|
|
1066
|
+
- {output_dir}/codebase/COVERAGE_GAPS.md (if exists)
|
|
1067
|
+
</files_to_read>
|
|
1068
|
+
<parameters>
|
|
1069
|
+
workflow_option: {option}
|
|
1070
|
+
output_path: {output_dir}/GENERATION_PLAN.md
|
|
1071
|
+
</parameters>
|
|
1072
|
+
"
|
|
1073
|
+
)
|
|
1074
|
+
```
|
|
1075
|
+
|
|
1076
|
+
**Parse planner return:**
|
|
1077
|
+
|
|
1078
|
+
Expected return structure:
|
|
1079
|
+
```
|
|
1080
|
+
PLANNER_COMPLETE:
|
|
1081
|
+
file_path: "..."
|
|
1082
|
+
total_tasks: N
|
|
1083
|
+
total_files: N
|
|
1084
|
+
feature_count: N
|
|
1085
|
+
dependency_depth: N
|
|
1086
|
+
test_case_count: N
|
|
1087
|
+
commit_hash: "..."
|
|
1088
|
+
```
|
|
1089
|
+
|
|
1090
|
+
Capture `total_tasks`, `total_files`, `feature_count` for executor stage and pipeline summary.
|
|
1091
|
+
|
|
1092
|
+
**Verify artifact exists:**
|
|
1093
|
+
```bash
|
|
1094
|
+
[ -f "${output_dir}/GENERATION_PLAN.md" ] && echo "OK" || echo "MISSING: GENERATION_PLAN.md"
|
|
1095
|
+
```
|
|
1096
|
+
|
|
1097
|
+
If GENERATION_PLAN.md is missing, treat as stage failure. Set status to failed and STOP pipeline.
|
|
1098
|
+
|
|
1099
|
+
Print: "Plan complete. {total_tasks} tasks, {total_files} files planned across {feature_count} features."
|
|
1100
|
+
|
|
1101
|
+
</step>
|
|
1102
|
+
|
|
1103
|
+
<step name="generate">
|
|
1104
|
+
|
|
1105
|
+
## Step 7: Generate Test Files
|
|
1106
|
+
|
|
1107
|
+
State update continues from planning (already set to `running` in Step 6).
|
|
1108
|
+
|
|
1109
|
+
**Print stage banner:**
|
|
1110
|
+
```
|
|
1111
|
+
+------------------------------------------+
|
|
1112
|
+
| STAGE 5: Executor |
|
|
1113
|
+
| Generating {total_files} test files |
|
|
1114
|
+
| Status: Running... |
|
|
1115
|
+
+------------------------------------------+
|
|
1116
|
+
```
|
|
1117
|
+
|
|
1118
|
+
**Determine execution strategy:**
|
|
1119
|
+
|
|
1120
|
+
Check if planner created multiple independent feature groups. If `feature_count > 1` AND parallelization is enabled (from init config):
|
|
1121
|
+
|
|
1122
|
+
**Parallel execution** (when feature_count > 1 and parallelization enabled):
|
|
1123
|
+
|
|
1124
|
+
For each independent feature group from the generation plan, spawn a separate executor agent:
|
|
1125
|
+
```
|
|
1126
|
+
Agent(subagent_type="general-purpose",
|
|
1127
|
+
prompt="
|
|
1128
|
+
<critical_directive priority="MANDATORY">
|
|
1129
|
+
You are executing as a QAA pipeline agent. Your FIRST action MUST be to read
|
|
1130
|
+
~/.claude/qaa/agents/qaa-executor.md and adopt it as your operating contract.
|
|
1131
|
+
|
|
1132
|
+
Even though you may be running as a general-purpose agent (native subagent type
|
|
1133
|
+
routing for QAA agents is not yet enabled), you MUST behave exactly as the
|
|
1134
|
+
dedicated agent defined in that file: read everything in its <required_reading>,
|
|
1135
|
+
obey its <quality_gate>, and run its mandatory bash checklist before returning.
|
|
1136
|
+
|
|
1137
|
+
Never modify application source code (src/, app/, lib/, or equivalent
|
|
1138
|
+
directories) unless the agent's <scope> section explicitly permits it. If the
|
|
1139
|
+
agent's contract restricts what files it can touch, honor that restriction —
|
|
1140
|
+
even if a fix seems obvious or trivial.
|
|
1141
|
+
|
|
1142
|
+
If you cannot read ~/.claude/qaa/agents/qaa-executor.md (file missing, corrupted, parse
|
|
1143
|
+
error), STOP and report the failure. Do NOT proceed with default agent
|
|
1144
|
+
behavior — that defeats the purpose of this directive.
|
|
1145
|
+
|
|
1146
|
+
Do NOT improvise, substitute steps, apply changes the agent isn't allowed to
|
|
1147
|
+
make, or skip any of the above. Skipping the agent's required_reading,
|
|
1148
|
+
quality_gate, or bash checklist makes the run INVALID — its output cannot be
|
|
1149
|
+
trusted.
|
|
1150
|
+
</critical_directive>
|
|
1151
|
+
|
|
1152
|
+
<objective>Generate test files for {feature} feature</objective>
|
|
1153
|
+
<execution_context>~/.claude/qaa/agents/qaa-executor.md</execution_context>
|
|
1154
|
+
<files_to_read>
|
|
1155
|
+
- {output_dir}/GENERATION_PLAN.md
|
|
1156
|
+
- {output_dir}/TEST_INVENTORY.md (Option 1) or {output_dir}/GAP_ANALYSIS.md (Options 2/3)
|
|
1157
|
+
- CLAUDE.md
|
|
1158
|
+
- frameworks/{detected_framework}.json (registry metadata for conventions, extensions, context7_keys)
|
|
1159
|
+
- {output_dir}/research/FRAMEWORK_CAPABILITIES.md (if exists)
|
|
1160
|
+
- {output_dir}/research/E2E_STRATEGY.md (if exists)
|
|
1161
|
+
- {output_dir}/research/API_TESTING_STRATEGY.md (if exists)
|
|
1162
|
+
- {output_dir}/codebase/TEST_SURFACE.md (if exists)
|
|
1163
|
+
- {output_dir}/codebase/CODE_PATTERNS.md (if exists)
|
|
1164
|
+
- {output_dir}/codebase/API_CONTRACTS.md (if exists)
|
|
1165
|
+
</files_to_read>
|
|
1166
|
+
<parameters>
|
|
1167
|
+
workflow_option: {option}
|
|
1168
|
+
feature_group: {feature}
|
|
1169
|
+
dev_repo_path: {DEV_REPO}
|
|
1170
|
+
qa_repo_path: {QA_REPO or null}
|
|
1171
|
+
output_path: {output_dir}/
|
|
1172
|
+
</parameters>
|
|
1173
|
+
"
|
|
1174
|
+
)
|
|
1175
|
+
```
|
|
1176
|
+
|
|
1177
|
+
Multiple Agent() calls can be issued simultaneously for independent feature groups. Each executor handles one feature group and commits its files independently.
|
|
1178
|
+
|
|
1179
|
+
**Sequential execution** (when feature_count == 1 or parallelization disabled):
|
|
1180
|
+
|
|
1181
|
+
Spawn a single executor agent covering all tasks:
|
|
1182
|
+
```
|
|
1183
|
+
Agent(subagent_type="general-purpose",
|
|
1184
|
+
prompt="
|
|
1185
|
+
<critical_directive priority="MANDATORY">
|
|
1186
|
+
You are executing as a QAA pipeline agent. Your FIRST action MUST be to read
|
|
1187
|
+
~/.claude/qaa/agents/qaa-executor.md and adopt it as your operating contract.
|
|
1188
|
+
|
|
1189
|
+
Even though you may be running as a general-purpose agent (native subagent type
|
|
1190
|
+
routing for QAA agents is not yet enabled), you MUST behave exactly as the
|
|
1191
|
+
dedicated agent defined in that file: read everything in its <required_reading>,
|
|
1192
|
+
obey its <quality_gate>, and run its mandatory bash checklist before returning.
|
|
1193
|
+
|
|
1194
|
+
Never modify application source code (src/, app/, lib/, or equivalent
|
|
1195
|
+
directories) unless the agent's <scope> section explicitly permits it. If the
|
|
1196
|
+
agent's contract restricts what files it can touch, honor that restriction —
|
|
1197
|
+
even if a fix seems obvious or trivial.
|
|
1198
|
+
|
|
1199
|
+
If you cannot read ~/.claude/qaa/agents/qaa-executor.md (file missing, corrupted, parse
|
|
1200
|
+
error), STOP and report the failure. Do NOT proceed with default agent
|
|
1201
|
+
behavior — that defeats the purpose of this directive.
|
|
1202
|
+
|
|
1203
|
+
Do NOT improvise, substitute steps, apply changes the agent isn't allowed to
|
|
1204
|
+
make, or skip any of the above. Skipping the agent's required_reading,
|
|
1205
|
+
quality_gate, or bash checklist makes the run INVALID — its output cannot be
|
|
1206
|
+
trusted.
|
|
1207
|
+
</critical_directive>
|
|
1208
|
+
|
|
1209
|
+
<objective>Generate all test files from generation plan</objective>
|
|
1210
|
+
<execution_context>~/.claude/qaa/agents/qaa-executor.md</execution_context>
|
|
1211
|
+
<files_to_read>
|
|
1212
|
+
- {output_dir}/GENERATION_PLAN.md
|
|
1213
|
+
- {output_dir}/TEST_INVENTORY.md (Option 1) or {output_dir}/GAP_ANALYSIS.md (Options 2/3)
|
|
1214
|
+
- CLAUDE.md
|
|
1215
|
+
- frameworks/{detected_framework}.json (registry metadata for conventions, extensions, context7_keys)
|
|
1216
|
+
- {output_dir}/research/FRAMEWORK_CAPABILITIES.md (if exists)
|
|
1217
|
+
- {output_dir}/research/E2E_STRATEGY.md (if exists)
|
|
1218
|
+
- {output_dir}/research/API_TESTING_STRATEGY.md (if exists)
|
|
1219
|
+
- {output_dir}/codebase/TEST_SURFACE.md (if exists)
|
|
1220
|
+
- {output_dir}/codebase/CODE_PATTERNS.md (if exists)
|
|
1221
|
+
- {output_dir}/codebase/API_CONTRACTS.md (if exists)
|
|
1222
|
+
</files_to_read>
|
|
1223
|
+
<parameters>
|
|
1224
|
+
workflow_option: {option}
|
|
1225
|
+
dev_repo_path: {DEV_REPO}
|
|
1226
|
+
qa_repo_path: {QA_REPO or null}
|
|
1227
|
+
output_path: {output_dir}/
|
|
1228
|
+
</parameters>
|
|
1229
|
+
"
|
|
1230
|
+
)
|
|
1231
|
+
```
|
|
1232
|
+
|
|
1233
|
+
**Option 3 specific -- skip existing tests:**
|
|
1234
|
+
|
|
1235
|
+
For Option 3, pass `skip_existing_test_ids: true` to the executor so it checks existing test files by test ID before generating. If a test ID already exists in the QA repo, skip generating that test case:
|
|
1236
|
+
```
|
|
1237
|
+
<parameters>
|
|
1238
|
+
workflow_option: 3
|
|
1239
|
+
skip_existing_test_ids: true
|
|
1240
|
+
dev_repo_path: {DEV_REPO}
|
|
1241
|
+
qa_repo_path: {QA_REPO}
|
|
1242
|
+
output_path: {output_dir}/
|
|
1243
|
+
</parameters>
|
|
1244
|
+
```
|
|
1245
|
+
|
|
1246
|
+
**Parse executor return:**
|
|
1247
|
+
|
|
1248
|
+
Expected return structure:
|
|
1249
|
+
```
|
|
1250
|
+
EXECUTOR_COMPLETE:
|
|
1251
|
+
files_created: [{path, type}, ...]
|
|
1252
|
+
total_files: N
|
|
1253
|
+
commit_count: N
|
|
1254
|
+
features_covered: [...]
|
|
1255
|
+
test_case_count: N
|
|
1256
|
+
```
|
|
1257
|
+
|
|
1258
|
+
Capture `files_created`, `total_files`, `commit_count` for validation stage and pipeline summary.
|
|
1259
|
+
|
|
1260
|
+
**State update -- mark generate as complete:**
|
|
1261
|
+
```bash
|
|
1262
|
+
node bin/qaa-tools.cjs state patch --"Generate Status" complete --"Status" "Test generation complete" 2>/dev/null || true
|
|
1263
|
+
```
|
|
1264
|
+
|
|
1265
|
+
Print: "Generation complete. {total_files} files created across {features_covered_count} features. {commit_count} commits."
|
|
1266
|
+
|
|
1267
|
+
</step>
|
|
1268
|
+
|
|
1269
|
+
<step name="validate">
|
|
1270
|
+
|
|
1271
|
+
## Step 8: Validate Generated Tests
|
|
1272
|
+
|
|
1273
|
+
**State update -- mark validate as running:**
|
|
1274
|
+
```bash
|
|
1275
|
+
node bin/qaa-tools.cjs state patch --"Validate Status" running --"Status" "Validating generated tests" 2>/dev/null || true
|
|
1276
|
+
```
|
|
1277
|
+
|
|
1278
|
+
**Print stage banner:**
|
|
1279
|
+
```
|
|
1280
|
+
+------------------------------------------+
|
|
1281
|
+
| STAGE 6: Validator |
|
|
1282
|
+
| Validating {total_files} test files |
|
|
1283
|
+
| Status: Running... |
|
|
1284
|
+
+------------------------------------------+
|
|
1285
|
+
```
|
|
1286
|
+
|
|
1287
|
+
**Spawn validator agent:**
|
|
1288
|
+
```
|
|
1289
|
+
Agent(subagent_type="general-purpose",
|
|
1290
|
+
prompt="
|
|
1291
|
+
<critical_directive priority="MANDATORY">
|
|
1292
|
+
You are executing as a QAA pipeline agent. Your FIRST action MUST be to read
|
|
1293
|
+
~/.claude/qaa/agents/qaa-validator.md and adopt it as your operating contract.
|
|
1294
|
+
|
|
1295
|
+
Even though you may be running as a general-purpose agent (native subagent type
|
|
1296
|
+
routing for QAA agents is not yet enabled), you MUST behave exactly as the
|
|
1297
|
+
dedicated agent defined in that file: read everything in its <required_reading>,
|
|
1298
|
+
obey its <quality_gate>, and run its mandatory bash checklist before returning.
|
|
1299
|
+
|
|
1300
|
+
Never modify application source code (src/, app/, lib/, or equivalent
|
|
1301
|
+
directories) unless the agent's <scope> section explicitly permits it. If the
|
|
1302
|
+
agent's contract restricts what files it can touch, honor that restriction —
|
|
1303
|
+
even if a fix seems obvious or trivial.
|
|
1304
|
+
|
|
1305
|
+
If you cannot read ~/.claude/qaa/agents/qaa-validator.md (file missing, corrupted, parse
|
|
1306
|
+
error), STOP and report the failure. Do NOT proceed with default agent
|
|
1307
|
+
behavior — that defeats the purpose of this directive.
|
|
1308
|
+
|
|
1309
|
+
Do NOT improvise, substitute steps, apply changes the agent isn't allowed to
|
|
1310
|
+
make, or skip any of the above. Skipping the agent's required_reading,
|
|
1311
|
+
quality_gate, or bash checklist makes the run INVALID — its output cannot be
|
|
1312
|
+
trusted.
|
|
1313
|
+
</critical_directive>
|
|
1314
|
+
|
|
1315
|
+
<objective>Run 4-layer validation on all generated test files</objective>
|
|
1316
|
+
<execution_context>~/.claude/qaa/agents/qaa-validator.md</execution_context>
|
|
1317
|
+
<files_to_read>
|
|
1318
|
+
- {list all generated test files from executor return -- files_created paths}
|
|
1319
|
+
- {output_dir}/GENERATION_PLAN.md
|
|
1320
|
+
- CLAUDE.md
|
|
1321
|
+
- frameworks/{detected_framework}.json (registry metadata for validation_commands)</files_to_read>
|
|
1322
|
+
<parameters>
|
|
1323
|
+
mode: validation
|
|
1324
|
+
max_fix_loops: 3
|
|
1325
|
+
output_path: {output_dir}/VALIDATION_REPORT.md
|
|
1326
|
+
</parameters>
|
|
1327
|
+
"
|
|
1328
|
+
)
|
|
1329
|
+
```
|
|
1330
|
+
|
|
1331
|
+
**4-layer validation:**
|
|
1332
|
+
1. **Syntax** -- File parses without errors
|
|
1333
|
+
2. **Structure** -- Follows POM rules, naming conventions, locator hierarchy
|
|
1334
|
+
3. **Dependencies** -- Imports resolve, fixtures exist, configs present
|
|
1335
|
+
4. **Logic** -- Assertions are concrete, test IDs are unique, no assertions in page objects
|
|
1336
|
+
|
|
1337
|
+
**5. Browser verification (if app URL available and Playwright MCP connected):**
|
|
1338
|
+
|
|
1339
|
+
After the 4-layer static validation, use Playwright MCP to verify E2E tests against the live app:
|
|
1340
|
+
|
|
1341
|
+
1. Navigate to each page referenced in the E2E tests:
|
|
1342
|
+
```
|
|
1343
|
+
mcp__playwright__browser_navigate({ url: "{app_url}/{route}" })
|
|
1344
|
+
```
|
|
1345
|
+
|
|
1346
|
+
2. Take an accessibility snapshot to verify locators used in tests actually exist:
|
|
1347
|
+
```
|
|
1348
|
+
mcp__playwright__browser_snapshot()
|
|
1349
|
+
```
|
|
1350
|
+
|
|
1351
|
+
3. Cross-reference locators in generated tests against the real DOM:
|
|
1352
|
+
- Verify `data-testid` values exist on the page
|
|
1353
|
+
- Verify ARIA roles and names match test expectations
|
|
1354
|
+
- Flag any test locator that does not match a real DOM element
|
|
1355
|
+
|
|
1356
|
+
4. If mismatches are found, fix the test locators to match the real DOM and count as a fix loop iteration.
|
|
1357
|
+
|
|
1358
|
+
This browser verification step prevents delivering tests with locators that will immediately fail at runtime.
|
|
1359
|
+
|
|
1360
|
+
**Fix loop:** The validator automatically attempts to fix issues it finds. Maximum 3 fix loop iterations. After each fix attempt, re-validate.
|
|
1361
|
+
|
|
1362
|
+
**Parse validator return:**
|
|
1363
|
+
|
|
1364
|
+
Expected return structure:
|
|
1365
|
+
```
|
|
1366
|
+
VALIDATOR_COMPLETE:
|
|
1367
|
+
report_path: "..."
|
|
1368
|
+
overall_status: PASS | PASS_WITH_WARNINGS | FAIL
|
|
1369
|
+
confidence: HIGH | MEDIUM | LOW
|
|
1370
|
+
layers_summary: {syntax: PASS|FAIL, structure: PASS|FAIL, dependencies: PASS|FAIL, logic: PASS|FAIL}
|
|
1371
|
+
fix_loops_used: N
|
|
1372
|
+
issues_found: N
|
|
1373
|
+
issues_fixed: N
|
|
1374
|
+
unresolved_count: N
|
|
1375
|
+
```
|
|
1376
|
+
|
|
1377
|
+
**RISKY CHECKPOINT -- Validator escalation:**
|
|
1378
|
+
|
|
1379
|
+
If `unresolved_count > 0` after max fix loops (3):
|
|
1380
|
+
- **ALWAYS pause, even in auto mode** (RISKY checkpoint -- locked decision)
|
|
1381
|
+
- Present unresolved issues to user with full details from VALIDATION_REPORT.md
|
|
1382
|
+
- Wait for user decision:
|
|
1383
|
+
- `"approve-with-warnings"`: Accept the validation with warnings. Set Validate Status to complete. Continue to deliver.
|
|
1384
|
+
- `"abort"`: Set Validate Status to failed. STOP PIPELINE ENTIRELY.
|
|
1385
|
+
- Manual guidance: User provides specific fix instructions. Spawn fresh continuation agent to apply fixes and re-validate.
|
|
1386
|
+
|
|
1387
|
+
If `overall_status` is `PASS` or `PASS_WITH_WARNINGS` (and unresolved_count is 0):
|
|
1388
|
+
```bash
|
|
1389
|
+
node bin/qaa-tools.cjs state patch --"Validate Status" complete --"Status" "Validation passed" 2>/dev/null || true
|
|
1390
|
+
```
|
|
1391
|
+
|
|
1392
|
+
**Verify artifact exists:**
|
|
1393
|
+
```bash
|
|
1394
|
+
[ -f "${output_dir}/VALIDATION_REPORT.md" ] && echo "OK" || echo "MISSING: VALIDATION_REPORT.md"
|
|
1395
|
+
```
|
|
1396
|
+
|
|
1397
|
+
Print: "Validation complete. Status: {overall_status}. Confidence: {confidence}. {issues_found} issues found, {issues_fixed} fixed, {unresolved_count} unresolved."
|
|
1398
|
+
|
|
1399
|
+
</step>
|
|
1400
|
+
|
|
1401
|
+
<step name="e2e_runner">
|
|
1402
|
+
|
|
1403
|
+
## Step 8b: E2E Runner (Conditional)
|
|
1404
|
+
|
|
1405
|
+
**Condition:** Only execute if E2E test files were generated AND a live app URL is available:
|
|
1406
|
+
- `files_created` from the executor (Step 7) contains `*.e2e.*` / `*.cy.*` spec files, AND
|
|
1407
|
+
- `APP_URL` is set (`--app-url` flag or NL).
|
|
1408
|
+
|
|
1409
|
+
**If no E2E files were generated:**
|
|
1410
|
+
Print: "Skipping E2E Runner (no E2E files generated)." → pipeline-summary E2E status = `skipped (no E2E files)`. Proceed to Step 9.
|
|
1411
|
+
|
|
1412
|
+
**If no app URL is available:**
|
|
1413
|
+
Print: "Skipping E2E Runner (no app URL)." → pipeline-summary E2E status = `skipped (no app URL)`. Proceed to Step 9.
|
|
1414
|
+
|
|
1415
|
+
**State update:**
|
|
1416
|
+
```bash
|
|
1417
|
+
node bin/qaa-tools.cjs state patch --"Status" "Running E2E tests against live app" 2>/dev/null || true
|
|
1418
|
+
```
|
|
1419
|
+
|
|
1420
|
+
**Print stage banner:**
|
|
1421
|
+
```
|
|
1422
|
+
+------------------------------------------+
|
|
1423
|
+
| STAGE 6b: E2E Runner |
|
|
1424
|
+
| Status: Running... |
|
|
1425
|
+
+------------------------------------------+
|
|
1426
|
+
```
|
|
1427
|
+
|
|
1428
|
+
**Spawn e2e-runner agent:**
|
|
1429
|
+
```
|
|
1430
|
+
Agent(subagent_type="general-purpose",
|
|
1431
|
+
prompt="
|
|
1432
|
+
<critical_directive priority="MANDATORY">
|
|
1433
|
+
You are executing as a QAA pipeline agent. Your FIRST action MUST be to read
|
|
1434
|
+
~/.claude/qaa/agents/qaa-e2e-runner.md and adopt it as your operating contract.
|
|
1435
|
+
|
|
1436
|
+
Even though you may be running as a general-purpose agent (native subagent type
|
|
1437
|
+
routing for QAA agents is not yet enabled), you MUST behave exactly as the
|
|
1438
|
+
dedicated agent defined in that file: read everything in its <required_reading>,
|
|
1439
|
+
obey its <quality_gate>, and run its mandatory bash checklist before returning.
|
|
1440
|
+
|
|
1441
|
+
Never modify application source code (src/, app/, lib/, or equivalent
|
|
1442
|
+
directories) unless the agent's <scope> section explicitly permits it. If the
|
|
1443
|
+
agent's contract restricts what files it can touch, honor that restriction —
|
|
1444
|
+
even if a fix seems obvious or trivial.
|
|
1445
|
+
|
|
1446
|
+
If you cannot read ~/.claude/qaa/agents/qaa-e2e-runner.md (file missing, corrupted, parse
|
|
1447
|
+
error), STOP and report the failure. Do NOT proceed with default agent
|
|
1448
|
+
behavior — that defeats the purpose of this directive.
|
|
1449
|
+
|
|
1450
|
+
Do NOT improvise, substitute steps, apply changes the agent isn't allowed to
|
|
1451
|
+
make, or skip any of the above. Skipping the agent's required_reading,
|
|
1452
|
+
quality_gate, or bash checklist makes the run INVALID — its output cannot be
|
|
1453
|
+
trusted.
|
|
1454
|
+
</critical_directive>
|
|
1455
|
+
|
|
1456
|
+
<objective>Run the generated E2E tests against the live application, capture real locators, fix mismatches, loop until pass or failures are classified. Query Context7 MCP to verify framework selector syntax before fixing locators. READ ~/.claude/qaa/MY_PREFERENCES.md FIRST — on this machine Cypress MUST be invoked with the `env -u ELECTRON_RUN_AS_NODE` prefix, and the app URL is PRODUCTION (read-only: never submit forms / create data).</objective>
|
|
1457
|
+
<execution_context>~/.claude/qaa/agents/qaa-e2e-runner.md</execution_context>
|
|
1458
|
+
<files_to_read>
|
|
1459
|
+
- C:\Users\marti\.claude\CLAUDE.md
|
|
1460
|
+
- ~/.claude/qaa/MY_PREFERENCES.md (if exists)
|
|
1461
|
+
- ${output_dir}/locators/LOCATOR_REGISTRY.md (if exists)
|
|
1462
|
+
- ${output_dir}/research/FRAMEWORK_CAPABILITIES.md (if exists)
|
|
1463
|
+
- ${output_dir}/research/E2E_STRATEGY.md (if exists)
|
|
1464
|
+
- {generated E2E test files from executor return}
|
|
1465
|
+
- {generated POM files from executor return}
|
|
1466
|
+
</files_to_read>
|
|
1467
|
+
<parameters>
|
|
1468
|
+
app_url: {APP_URL}
|
|
1469
|
+
output_dir: ${output_dir}
|
|
1470
|
+
dev_repo_path: {DEV_REPO}
|
|
1471
|
+
</parameters>
|
|
1472
|
+
"
|
|
1473
|
+
)
|
|
1474
|
+
```
|
|
1475
|
+
|
|
1476
|
+
**Parse e2e-runner return:**
|
|
1477
|
+
```
|
|
1478
|
+
E2E_RUNNER_COMPLETE:
|
|
1479
|
+
app_url: "..."
|
|
1480
|
+
total_tests: N
|
|
1481
|
+
passed: N
|
|
1482
|
+
failed: N
|
|
1483
|
+
locator_fixes: N
|
|
1484
|
+
app_bugs_found: N
|
|
1485
|
+
fix_loops_used: N
|
|
1486
|
+
runner_executed: true | false
|
|
1487
|
+
runner_exit_code: N
|
|
1488
|
+
runner_status: OK | BROKEN
|
|
1489
|
+
report_path: "..."
|
|
1490
|
+
screenshots: [...]
|
|
1491
|
+
```
|
|
1492
|
+
|
|
1493
|
+
**Transition: validate → e2e-runner → { bug-detective | deliver | HALT }.**
|
|
1494
|
+
|
|
1495
|
+
**HARD STOP on broken runner (#47/#48):** If the e2e-runner returns `runner_executed: false` (or `runner_status: BROKEN`, or an ENVIRONMENT ISSUE — e.g. Cypress not installed, binary cannot launch), the test runner could not execute. **HALT the pipeline** — do NOT advance to bug-detective and do NOT advance to deliver:
|
|
1496
|
+
```bash
|
|
1497
|
+
node bin/qaa-tools.cjs state patch --"Status" "Pipeline halted: E2E runner unavailable (env issue)" 2>/dev/null || true
|
|
1498
|
+
```
|
|
1499
|
+
- Set the pipeline-summary E2E Runner row to `halted (runner unavailable — env issue)`.
|
|
1500
|
+
- Print the halt banner (see Error Handling) and STOP. A DOM probe is NEVER accepted as a substitute for a real run; if the report presents probe evidence as "verified" while `runner_executed: false`, treat the run as INVALID.
|
|
1501
|
+
|
|
1502
|
+
**Otherwise (`runner_executed: true`):**
|
|
1503
|
+
- Pipeline-summary E2E Runner status = `{passed}/{total_tests} passed ({fix_loops_used} fix loops)`.
|
|
1504
|
+
- If `app_bugs_found > 0`: surface the application bugs to the user (developer action required; the e2e-runner never modifies app code), then continue.
|
|
1505
|
+
- If `failed > 0` and `app_bugs_found == 0`: proceed to Step 9 (Bug Detective) for classification.
|
|
1506
|
+
- Else: proceed to Step 10 (Deliver).
|
|
1507
|
+
|
|
1508
|
+
**Verify artifact exists:**
|
|
1509
|
+
```bash
|
|
1510
|
+
[ -f "${output_dir}/E2E_RUN_REPORT.md" ] && echo "OK" || echo "MISSING: E2E_RUN_REPORT.md"
|
|
1511
|
+
```
|
|
1512
|
+
|
|
1513
|
+
Print: "E2E run complete. {passed}/{total_tests} passed. {locator_fixes} locators fixed. {app_bugs_found} app bugs found."
|
|
1514
|
+
|
|
1515
|
+
</step>
|
|
1516
|
+
|
|
1517
|
+
<step name="bug_detective">
|
|
1518
|
+
|
|
1519
|
+
## Step 9: Bug Detective (Conditional)
|
|
1520
|
+
|
|
1521
|
+
**Condition:** Only execute if test failures were detected during validation. Check:
|
|
1522
|
+
- `overall_status === 'FAIL'` in validator return, OR
|
|
1523
|
+
- Generated tests have runtime failures that need classification
|
|
1524
|
+
|
|
1525
|
+
**If no failures to classify:**
|
|
1526
|
+
Print: "Skipping Bug Detective (no test failures detected)." Proceed directly to Step 10 (Deliver).
|
|
1527
|
+
|
|
1528
|
+
**If failures need classification:**
|
|
1529
|
+
|
|
1530
|
+
**State update:**
|
|
1531
|
+
```bash
|
|
1532
|
+
node bin/qaa-tools.cjs state patch --"Status" "Classifying test failures" 2>/dev/null || true
|
|
1533
|
+
```
|
|
1534
|
+
|
|
1535
|
+
**Print stage banner:**
|
|
1536
|
+
```
|
|
1537
|
+
+------------------------------------------+
|
|
1538
|
+
| STAGE 7: Bug Detective |
|
|
1539
|
+
| Status: Running... |
|
|
1540
|
+
+------------------------------------------+
|
|
1541
|
+
```
|
|
1542
|
+
|
|
1543
|
+
**Spawn bug-detective agent:**
|
|
1544
|
+
```
|
|
1545
|
+
Agent(subagent_type="general-purpose",
|
|
1546
|
+
prompt="
|
|
1547
|
+
<critical_directive priority="MANDATORY">
|
|
1548
|
+
You are executing as a QAA pipeline agent. Your FIRST action MUST be to read
|
|
1549
|
+
~/.claude/qaa/agents/qaa-bug-detective.md and adopt it as your operating contract.
|
|
1550
|
+
|
|
1551
|
+
Even though you may be running as a general-purpose agent (native subagent type
|
|
1552
|
+
routing for QAA agents is not yet enabled), you MUST behave exactly as the
|
|
1553
|
+
dedicated agent defined in that file: read everything in its <required_reading>,
|
|
1554
|
+
obey its <quality_gate>, and run its mandatory bash checklist before returning.
|
|
1555
|
+
|
|
1556
|
+
Never modify application source code (src/, app/, lib/, or equivalent
|
|
1557
|
+
directories) unless the agent's <scope> section explicitly permits it. If the
|
|
1558
|
+
agent's contract restricts what files it can touch, honor that restriction —
|
|
1559
|
+
even if a fix seems obvious or trivial.
|
|
1560
|
+
|
|
1561
|
+
If you cannot read ~/.claude/qaa/agents/qaa-bug-detective.md (file missing, corrupted, parse
|
|
1562
|
+
error), STOP and report the failure. Do NOT proceed with default agent
|
|
1563
|
+
behavior — that defeats the purpose of this directive.
|
|
1564
|
+
|
|
1565
|
+
Do NOT improvise, substitute steps, apply changes the agent isn't allowed to
|
|
1566
|
+
make, or skip any of the above. Skipping the agent's required_reading,
|
|
1567
|
+
quality_gate, or bash checklist makes the run INVALID — its output cannot be
|
|
1568
|
+
trusted.
|
|
1569
|
+
</critical_directive>
|
|
1570
|
+
|
|
1571
|
+
<objective>Classify test failures and attempt auto-fixes for test errors. Use Playwright MCP to reproduce E2E failures in the browser when available.</objective>
|
|
1572
|
+
<execution_context>~/.claude/qaa/agents/qaa-bug-detective.md</execution_context>
|
|
1573
|
+
<files_to_read>
|
|
1574
|
+
- {test execution results -- from validator or direct test run}
|
|
1575
|
+
- {failing test source files -- paths from executor return}
|
|
1576
|
+
- CLAUDE.md
|
|
1577
|
+
- {output_dir}/research/FRAMEWORK_CAPABILITIES.md (if exists)
|
|
1578
|
+
- {output_dir}/research/TESTING_STACK.md (if exists)
|
|
1579
|
+
</files_to_read>
|
|
1580
|
+
<parameters>
|
|
1581
|
+
output_path: {output_dir}/FAILURE_CLASSIFICATION_REPORT.md
|
|
1582
|
+
app_url: {app_url if available}
|
|
1583
|
+
</parameters>
|
|
1584
|
+
"
|
|
1585
|
+
)
|
|
1586
|
+
```
|
|
1587
|
+
|
|
1588
|
+
**Parse bug-detective return:**
|
|
1589
|
+
|
|
1590
|
+
Expected return structure:
|
|
1591
|
+
```
|
|
1592
|
+
DETECTIVE_COMPLETE:
|
|
1593
|
+
report_path: "..."
|
|
1594
|
+
total_failures: N
|
|
1595
|
+
classification_breakdown: {app_bug: N, test_error: N, env_issue: N, inconclusive: N}
|
|
1596
|
+
auto_fixes_applied: N
|
|
1597
|
+
auto_fixes_verified: N
|
|
1598
|
+
commit_hash: "..."
|
|
1599
|
+
```
|
|
1600
|
+
|
|
1601
|
+
**RISKY CHECKPOINT -- Application bugs detected:**
|
|
1602
|
+
|
|
1603
|
+
If `classification_breakdown.app_bug > 0`:
|
|
1604
|
+
- **ALWAYS pause, even in auto mode** (RISKY checkpoint -- locked decision)
|
|
1605
|
+
- Present APPLICATION BUG classifications to user with full evidence from FAILURE_CLASSIFICATION_REPORT.md
|
|
1606
|
+
- These are genuine bugs in the application code discovered during test execution
|
|
1607
|
+
- The bug detective never touches application code -- it only reports
|
|
1608
|
+
- User must review and decide how to proceed:
|
|
1609
|
+
- Acknowledge bugs and continue pipeline (bugs will be in the PR description for developer attention)
|
|
1610
|
+
- Abort pipeline to fix bugs first
|
|
1611
|
+
|
|
1612
|
+
**Verify artifact exists:**
|
|
1613
|
+
```bash
|
|
1614
|
+
[ -f "${output_dir}/FAILURE_CLASSIFICATION_REPORT.md" ] && echo "OK" || echo "MISSING: FAILURE_CLASSIFICATION_REPORT.md"
|
|
1615
|
+
```
|
|
1616
|
+
|
|
1617
|
+
Print: "Bug Detective complete. {total_failures} failures classified: {app_bug} APP BUG, {test_error} TEST ERROR, {env_issue} ENV ISSUE, {inconclusive} INCONCLUSIVE. {auto_fixes_applied} auto-fixes applied."
|
|
1618
|
+
|
|
1619
|
+
</step>
|
|
1620
|
+
|
|
1621
|
+
<step name="deliver">
|
|
1622
|
+
|
|
1623
|
+
## Step 10: Deliver
|
|
1624
|
+
|
|
1625
|
+
**State update -- mark deliver as running:**
|
|
1626
|
+
```bash
|
|
1627
|
+
node bin/qaa-tools.cjs state patch --"Deliver Status" running --"Status" "Preparing delivery" 2>/dev/null || true
|
|
1628
|
+
```
|
|
1629
|
+
|
|
1630
|
+
**Print stage banner:**
|
|
1631
|
+
```
|
|
1632
|
+
+------------------------------------------+
|
|
1633
|
+
| STAGE 8: Deliver |
|
|
1634
|
+
| Status: Running... |
|
|
1635
|
+
+------------------------------------------+
|
|
1636
|
+
```
|
|
1637
|
+
|
|
1638
|
+
### Sub-step 1: Pre-flight checks
|
|
1639
|
+
|
|
1640
|
+
**Check for git remote:**
|
|
1641
|
+
```bash
|
|
1642
|
+
REMOTE_URL=$(git remote get-url origin 2>/dev/null || echo "")
|
|
1643
|
+
```
|
|
1644
|
+
|
|
1645
|
+
If `REMOTE_URL` is empty:
|
|
1646
|
+
- Print: "No git remote found. Artifacts committed locally but PR creation skipped."
|
|
1647
|
+
- Set `LOCAL_ONLY=true`
|
|
1648
|
+
|
|
1649
|
+
**Check for gh CLI authentication:**
|
|
1650
|
+
```bash
|
|
1651
|
+
gh auth status 2>/dev/null
|
|
1652
|
+
```
|
|
1653
|
+
|
|
1654
|
+
If `gh auth status` fails:
|
|
1655
|
+
- Print: "gh CLI not authenticated. Run 'gh auth login' first. Artifacts committed locally."
|
|
1656
|
+
- Set `LOCAL_ONLY=true`
|
|
1657
|
+
|
|
1658
|
+
If both checks pass, set `LOCAL_ONLY=false`.
|
|
1659
|
+
|
|
1660
|
+
### Sub-step 2: Derive project name
|
|
1661
|
+
|
|
1662
|
+
```bash
|
|
1663
|
+
# Read from package.json
|
|
1664
|
+
PROJECT_NAME=$(node -e "try { const p = require('${DEV_REPO}/package.json'); console.log(p.name || ''); } catch { console.log(''); }" 2>/dev/null)
|
|
1665
|
+
|
|
1666
|
+
# Fallback to directory basename
|
|
1667
|
+
if [ -z "$PROJECT_NAME" ]; then
|
|
1668
|
+
PROJECT_NAME=$(basename "${DEV_REPO}")
|
|
1669
|
+
fi
|
|
1670
|
+
|
|
1671
|
+
# Sanitize for branch naming
|
|
1672
|
+
PROJECT_NAME=$(echo "$PROJECT_NAME" | tr '[:upper:]' '[:lower:]' | sed 's/[^a-z0-9]/-/g' | sed 's/--*/-/g' | sed 's/^-//' | sed 's/-$//')
|
|
1673
|
+
```
|
|
1674
|
+
|
|
1675
|
+
### Sub-step 3: Detect default branch
|
|
1676
|
+
|
|
1677
|
+
```bash
|
|
1678
|
+
DEFAULT_BRANCH=$(gh repo view --json defaultBranchRef --jq '.defaultBranchRef.name' 2>/dev/null || echo "main")
|
|
1679
|
+
```
|
|
1680
|
+
|
|
1681
|
+
### Sub-step 4: Create feature branch
|
|
1682
|
+
|
|
1683
|
+
```bash
|
|
1684
|
+
BRANCH="qa/auto-${PROJECT_NAME}-${date}"
|
|
1685
|
+
|
|
1686
|
+
# Handle branch name collision
|
|
1687
|
+
if git rev-parse --verify "$BRANCH" 2>/dev/null || git rev-parse --verify "origin/$BRANCH" 2>/dev/null; then
|
|
1688
|
+
SUFFIX=2
|
|
1689
|
+
while git rev-parse --verify "${BRANCH}-${SUFFIX}" 2>/dev/null || git rev-parse --verify "origin/${BRANCH}-${SUFFIX}" 2>/dev/null; do
|
|
1690
|
+
SUFFIX=$((SUFFIX + 1))
|
|
1691
|
+
done
|
|
1692
|
+
BRANCH="${BRANCH}-${SUFFIX}"
|
|
1693
|
+
fi
|
|
1694
|
+
|
|
1695
|
+
git checkout -b "$BRANCH" "$DEFAULT_BRANCH"
|
|
1696
|
+
```
|
|
1697
|
+
|
|
1698
|
+
### Sub-step 5: Per-stage atomic commits
|
|
1699
|
+
|
|
1700
|
+
For each pipeline stage that produced artifacts, commit using `qaa-tools.cjs commit`. Check file existence before each commit.
|
|
1701
|
+
|
|
1702
|
+
**Scanner:**
|
|
1703
|
+
```bash
|
|
1704
|
+
if [ -f "${output_dir}/SCAN_MANIFEST.md" ]; then
|
|
1705
|
+
node bin/qaa-tools.cjs commit "qa(scanner): produce SCAN_MANIFEST.md for ${PROJECT_NAME}" --files ${output_dir}/SCAN_MANIFEST.md
|
|
1706
|
+
fi
|
|
1707
|
+
```
|
|
1708
|
+
|
|
1709
|
+
**Analyzer (Option 1):**
|
|
1710
|
+
```bash
|
|
1711
|
+
if [ -f "${output_dir}/QA_ANALYSIS.md" ]; then
|
|
1712
|
+
ANALYZER_FILES="${output_dir}/QA_ANALYSIS.md ${output_dir}/TEST_INVENTORY.md"
|
|
1713
|
+
[ -f "${output_dir}/QA_REPO_BLUEPRINT.md" ] && ANALYZER_FILES="${ANALYZER_FILES} ${output_dir}/QA_REPO_BLUEPRINT.md"
|
|
1714
|
+
node bin/qaa-tools.cjs commit "qa(analyzer): produce QA_ANALYSIS.md and TEST_INVENTORY.md" --files ${ANALYZER_FILES}
|
|
1715
|
+
fi
|
|
1716
|
+
```
|
|
1717
|
+
|
|
1718
|
+
**Analyzer (Options 2/3):**
|
|
1719
|
+
```bash
|
|
1720
|
+
if [ -f "${output_dir}/GAP_ANALYSIS.md" ]; then
|
|
1721
|
+
node bin/qaa-tools.cjs commit "qa(analyzer): produce GAP_ANALYSIS.md" --files ${output_dir}/GAP_ANALYSIS.md
|
|
1722
|
+
fi
|
|
1723
|
+
```
|
|
1724
|
+
|
|
1725
|
+
**TestID Injector (if ran):**
|
|
1726
|
+
```bash
|
|
1727
|
+
if [ -f "${output_dir}/TESTID_AUDIT_REPORT.md" ]; then
|
|
1728
|
+
node bin/qaa-tools.cjs commit "qa(testid-injector): inject ${elements_injected} data-testid attributes across ${components_modified} components" --files ${output_dir}/TESTID_AUDIT_REPORT.md ${modified_source_files}
|
|
1729
|
+
fi
|
|
1730
|
+
```
|
|
1731
|
+
|
|
1732
|
+
**Executor:**
|
|
1733
|
+
```bash
|
|
1734
|
+
if [ -n "${generated_file_paths}" ]; then
|
|
1735
|
+
node bin/qaa-tools.cjs commit "qa(executor): generate ${total_files} test files with POMs and fixtures" --files ${generated_file_paths}
|
|
1736
|
+
fi
|
|
1737
|
+
```
|
|
1738
|
+
|
|
1739
|
+
**Planner:**
|
|
1740
|
+
```bash
|
|
1741
|
+
if [ -f "${output_dir}/GENERATION_PLAN.md" ]; then
|
|
1742
|
+
node bin/qaa-tools.cjs commit "qa(planner): produce GENERATION_PLAN.md" --files ${output_dir}/GENERATION_PLAN.md
|
|
1743
|
+
fi
|
|
1744
|
+
```
|
|
1745
|
+
|
|
1746
|
+
**Validator:**
|
|
1747
|
+
```bash
|
|
1748
|
+
if [ -f "${output_dir}/VALIDATION_REPORT.md" ]; then
|
|
1749
|
+
node bin/qaa-tools.cjs commit "qa(validator): validate generated tests - ${overall_status} with ${confidence} confidence" --files ${output_dir}/VALIDATION_REPORT.md
|
|
1750
|
+
fi
|
|
1751
|
+
```
|
|
1752
|
+
|
|
1753
|
+
**Bug Detective (if ran):**
|
|
1754
|
+
```bash
|
|
1755
|
+
if [ -f "${output_dir}/FAILURE_CLASSIFICATION_REPORT.md" ]; then
|
|
1756
|
+
node bin/qaa-tools.cjs commit "qa(bug-detective): classify ${total_failures} failures - ${classification_summary}" --files ${output_dir}/FAILURE_CLASSIFICATION_REPORT.md
|
|
1757
|
+
fi
|
|
1758
|
+
```
|
|
1759
|
+
|
|
1760
|
+
### Sub-step 6: Push branch
|
|
1761
|
+
|
|
1762
|
+
If `LOCAL_ONLY` is true, skip this sub-step.
|
|
1763
|
+
|
|
1764
|
+
```bash
|
|
1765
|
+
git push -u origin "$BRANCH"
|
|
1766
|
+
```
|
|
1767
|
+
|
|
1768
|
+
If push fails:
|
|
1769
|
+
- Print: "Push failed: {error_message}. Artifacts committed locally on branch ${BRANCH}."
|
|
1770
|
+
- Set `LOCAL_ONLY=true`
|
|
1771
|
+
|
|
1772
|
+
### Sub-step 7: Build PR body
|
|
1773
|
+
|
|
1774
|
+
If `LOCAL_ONLY` is true, skip this sub-step.
|
|
1775
|
+
|
|
1776
|
+
Read the PR template:
|
|
1777
|
+
```bash
|
|
1778
|
+
PR_BODY=$(cat templates/pr-template.md)
|
|
1779
|
+
```
|
|
1780
|
+
|
|
1781
|
+
Replace all `{placeholder}` tokens with actual values collected during pipeline execution:
|
|
1782
|
+
- `{architecture_type}` -- from QA_ANALYSIS.md or SCAN_MANIFEST.md
|
|
1783
|
+
- `{framework}` -- detected test framework
|
|
1784
|
+
- `{risk_summary}` -- risk assessment counts (e.g., "3 HIGH, 5 MEDIUM, 2 LOW")
|
|
1785
|
+
- `{unit_count}` -- from pyramid_breakdown.unit
|
|
1786
|
+
- `{integration_count}` -- from pyramid_breakdown.integration
|
|
1787
|
+
- `{api_count}` -- from pyramid_breakdown.api
|
|
1788
|
+
- `{e2e_count}` -- from pyramid_breakdown.e2e
|
|
1789
|
+
- `{total_count}` -- from total_test_count
|
|
1790
|
+
- `{modules_covered}` -- count of modules with tests
|
|
1791
|
+
- `{coverage_estimate}` -- estimated coverage percentage
|
|
1792
|
+
- `{validation_result}` -- PASS, PASS_WITH_WARNINGS, or FAIL
|
|
1793
|
+
- `{confidence}` -- HIGH, MEDIUM, or LOW
|
|
1794
|
+
- `{fix_loops_used}` -- number 0-3
|
|
1795
|
+
- `{issues_found}` -- total issues found during validation
|
|
1796
|
+
- `{issues_fixed}` -- total issues auto-fixed
|
|
1797
|
+
- `{file_list}` -- if total files <= 50, list each file; if > 50, use summary
|
|
1798
|
+
|
|
1799
|
+
### Sub-step 8: Create draft PR
|
|
1800
|
+
|
|
1801
|
+
If `LOCAL_ONLY` is true, skip this sub-step.
|
|
1802
|
+
|
|
1803
|
+
```bash
|
|
1804
|
+
PR_URL=$(gh pr create \
|
|
1805
|
+
--draft \
|
|
1806
|
+
--title "qa: automated test suite for ${PROJECT_NAME}" \
|
|
1807
|
+
--body "${PR_BODY}" \
|
|
1808
|
+
--label "qa-automation" \
|
|
1809
|
+
--label "auto-generated" \
|
|
1810
|
+
--assignee "@me" 2>&1)
|
|
1811
|
+
```
|
|
1812
|
+
|
|
1813
|
+
Do NOT pass `--base` flag. Let gh auto-detect the default branch.
|
|
1814
|
+
|
|
1815
|
+
On failure:
|
|
1816
|
+
- Print: "PR creation failed: ${PR_URL}. Artifacts remain on branch ${BRANCH}."
|
|
1817
|
+
- Do NOT stop the pipeline -- artifacts are committed and pushed.
|
|
1818
|
+
|
|
1819
|
+
### Sub-step 9: Print result
|
|
1820
|
+
|
|
1821
|
+
If PR was created successfully:
|
|
1822
|
+
```
|
|
1823
|
+
PR created: ${PR_URL}
|
|
1824
|
+
```
|
|
1825
|
+
|
|
1826
|
+
If `LOCAL_ONLY` is true:
|
|
1827
|
+
```
|
|
1828
|
+
PR: not created (local-only mode). Artifacts committed on branch: ${BRANCH}
|
|
1829
|
+
```
|
|
1830
|
+
|
|
1831
|
+
**State update -- mark deliver as complete:**
|
|
1832
|
+
```bash
|
|
1833
|
+
node bin/qaa-tools.cjs state patch --"Deliver Status" complete --"Status" "Pipeline complete" 2>/dev/null || true
|
|
1834
|
+
```
|
|
1835
|
+
|
|
1836
|
+
**Clear auto-chain flag at pipeline completion:**
|
|
1837
|
+
```bash
|
|
1838
|
+
node bin/qaa-tools.cjs config-set workflow._auto_chain_active false 2>/dev/null || true
|
|
1839
|
+
```
|
|
1840
|
+
|
|
1841
|
+
### Print pipeline summary banner:
|
|
1842
|
+
|
|
1843
|
+
```
|
|
1844
|
+
======================================================
|
|
1845
|
+
QA PIPELINE COMPLETE
|
|
1846
|
+
======================================================
|
|
1847
|
+
|
|
1848
|
+
Option: {option} ({option_description})
|
|
1849
|
+
Repository: {DEV_REPO}
|
|
1850
|
+
QA Repo: {QA_REPO or 'N/A'}
|
|
1851
|
+
Maturity Score: {maturity_score or 'N/A'}
|
|
1852
|
+
|
|
1853
|
+
Stages Completed:
|
|
1854
|
+
[{check}] Scan -- {scan_result}
|
|
1855
|
+
[{check}] Analyze -- {analyze_result} ({test_count} test cases)
|
|
1856
|
+
[{check}] TestID Inject -- {inject_result or 'skipped'}
|
|
1857
|
+
[{check}] Plan -- {plan_result} ({file_count} files planned)
|
|
1858
|
+
[{check}] Generate -- {generate_result} ({files_created} files created)
|
|
1859
|
+
[{check}] Validate -- {validate_result} ({confidence} confidence)
|
|
1860
|
+
[{check}] E2E Runner -- {e2e_status}
|
|
1861
|
+
[{check}] Bug Detective -- {detective_result or 'skipped'}
|
|
1862
|
+
[{check}] Deliver -- {deliver_result}
|
|
1863
|
+
|
|
1864
|
+
PR: {pr_url or 'not created (local-only)'}
|
|
1865
|
+
|
|
1866
|
+
Artifacts:
|
|
1867
|
+
{list all produced .md files in output_dir}
|
|
1868
|
+
|
|
1869
|
+
Total Time: {total_duration}
|
|
1870
|
+
======================================================
|
|
1871
|
+
```
|
|
1872
|
+
|
|
1873
|
+
Where: `[x]` = completed, `[ ]` = skipped, `[!]` = failed.
|
|
1874
|
+
|
|
1875
|
+
</step>
|
|
1876
|
+
|
|
1877
|
+
</process>
|
|
1878
|
+
|
|
1879
|
+
<error_handling>
|
|
1880
|
+
|
|
1881
|
+
## Error Handling
|
|
1882
|
+
|
|
1883
|
+
### Stage Failure Protocol
|
|
1884
|
+
|
|
1885
|
+
When any agent returns a failure or error:
|
|
1886
|
+
|
|
1887
|
+
1. **Set stage status to failed:**
|
|
1888
|
+
```bash
|
|
1889
|
+
node bin/qaa-tools.cjs state patch --"{Stage} Status" failed --"Status" "Pipeline stopped: {Stage} failed - {reason}" 2>/dev/null || true
|
|
1890
|
+
```
|
|
1891
|
+
|
|
1892
|
+
2. **Print failure banner:**
|
|
1893
|
+
```
|
|
1894
|
+
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
|
|
1895
|
+
! PIPELINE STOPPED !
|
|
1896
|
+
! Stage: {stage_name} !
|
|
1897
|
+
! Reason: {failure_reason} !
|
|
1898
|
+
! !
|
|
1899
|
+
! Completed: {completed_stages} !
|
|
1900
|
+
! Artifacts: {artifacts_so_far} !
|
|
1901
|
+
! !
|
|
1902
|
+
! Action required: Review and re-run !
|
|
1903
|
+
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
|
|
1904
|
+
```
|
|
1905
|
+
|
|
1906
|
+
3. **DO NOT continue to next stage.** The pipeline stops entirely at the failed stage.
|
|
1907
|
+
|
|
1908
|
+
4. **DO NOT create partial PR.** No branch, no commit, no PR with incomplete results.
|
|
1909
|
+
|
|
1910
|
+
5. **Preserve all artifacts produced so far.** They remain on disk in `{output_dir}/` for debugging.
|
|
1911
|
+
|
|
1912
|
+
### Artifact Verification
|
|
1913
|
+
|
|
1914
|
+
After EVERY agent spawn, before advancing to next stage, verify the expected output artifact exists on disk:
|
|
1915
|
+
```bash
|
|
1916
|
+
[ -f "{expected_artifact_path}" ] && echo "OK" || echo "MISSING"
|
|
1917
|
+
```
|
|
1918
|
+
|
|
1919
|
+
If artifacts are missing, treat as stage failure and STOP pipeline.
|
|
1920
|
+
|
|
1921
|
+
### qaa-tools.cjs Graceful Fallback
|
|
1922
|
+
|
|
1923
|
+
All `node bin/qaa-tools.cjs` calls use `2>/dev/null || true` to handle cases where the tool is not installed or not found. The pipeline must not break due to missing state management tooling -- it logs a warning and continues.
|
|
1924
|
+
|
|
1925
|
+
</error_handling>
|
|
1926
|
+
|
|
1927
|
+
<auto_advance>
|
|
1928
|
+
|
|
1929
|
+
## Auto-Advance Mode
|
|
1930
|
+
|
|
1931
|
+
Auto-advance is enabled when ANY of these is true:
|
|
1932
|
+
- `--auto` flag passed to the `/qa-start` invocation
|
|
1933
|
+
- `config.json` has `workflow.auto_advance = true` (persistent user preference)
|
|
1934
|
+
- `workflow._auto_chain_active = true` in config (ephemeral chain flag from ongoing auto run)
|
|
1935
|
+
|
|
1936
|
+
### Behavior in Auto Mode
|
|
1937
|
+
|
|
1938
|
+
**SAFE checkpoints are auto-approved.** The pipeline continues without pausing. A log message records the auto-approval:
|
|
1939
|
+
```
|
|
1940
|
+
Auto-approved: {checkpoint_description}
|
|
1941
|
+
```
|
|
1942
|
+
|
|
1943
|
+
**RISKY checkpoints ALWAYS pause.** Even in auto mode, the pipeline stops and presents the checkpoint to the user.
|
|
1944
|
+
|
|
1945
|
+
### Safe vs Risky Checkpoint Classification
|
|
1946
|
+
|
|
1947
|
+
**SAFE (auto-approve in auto mode):**
|
|
1948
|
+
|
|
1949
|
+
| Checkpoint | Agent | Auto-Action |
|
|
1950
|
+
|------------|-------|-------------|
|
|
1951
|
+
| Framework detection uncertain (LOW confidence) | Scanner | Approve with most likely framework |
|
|
1952
|
+
| Analyzer assumptions review | Analyzer | Approve all assumptions |
|
|
1953
|
+
| TestID audit review | TestID Injector | Approve P0-only injection |
|
|
1954
|
+
|
|
1955
|
+
**RISKY (ALWAYS pause, even in auto mode):**
|
|
1956
|
+
|
|
1957
|
+
| Checkpoint | Agent | User Action Required |
|
|
1958
|
+
|------------|-------|---------------------|
|
|
1959
|
+
| Validator escalation (unresolved issues after 3 fix loops) | Validator | approve-with-warnings, abort, or fix guidance |
|
|
1960
|
+
| APPLICATION BUG classification | Bug Detective | Review bugs, continue or fix first |
|
|
1961
|
+
| Any checkpoint with "unresolved" or "failed" blocking text | Any | Review specific blocking issue |
|
|
1962
|
+
|
|
1963
|
+
### Checkpoint Handling Flow
|
|
1964
|
+
|
|
1965
|
+
```
|
|
1966
|
+
On agent return with checkpoint data:
|
|
1967
|
+
1. Extract checkpoint blocking field content
|
|
1968
|
+
2. Classify as SAFE or RISKY:
|
|
1969
|
+
- "framework detection" -> SAFE
|
|
1970
|
+
- "assumptions" -> SAFE
|
|
1971
|
+
- "audit" or "data-testid" -> SAFE
|
|
1972
|
+
- "unresolved" -> RISKY
|
|
1973
|
+
- "failed" -> RISKY
|
|
1974
|
+
- "APPLICATION BUG" -> RISKY
|
|
1975
|
+
- Default (no pattern match) -> RISKY (conservative)
|
|
1976
|
+
3. If IS_AUTO and SAFE:
|
|
1977
|
+
- Auto-approve with default action
|
|
1978
|
+
- Log the auto-approval
|
|
1979
|
+
- Continue pipeline
|
|
1980
|
+
4. If IS_AUTO and RISKY:
|
|
1981
|
+
- PAUSE pipeline
|
|
1982
|
+
- Print checkpoint details with full context
|
|
1983
|
+
- Wait for user input
|
|
1984
|
+
5. If NOT auto (manual mode):
|
|
1985
|
+
- PAUSE pipeline
|
|
1986
|
+
- Print checkpoint details
|
|
1987
|
+
- Wait for user input
|
|
1988
|
+
```
|
|
1989
|
+
|
|
1990
|
+
### Resume After Checkpoint
|
|
1991
|
+
|
|
1992
|
+
When resuming after a checkpoint, spawn a FRESH agent with explicit state:
|
|
1993
|
+
```
|
|
1994
|
+
Agent(subagent_type="general-purpose",
|
|
1995
|
+
prompt="
|
|
1996
|
+
<critical_directive priority="MANDATORY">
|
|
1997
|
+
You are executing as a QAA pipeline agent. Your FIRST action MUST be to read
|
|
1998
|
+
~/.claude/qaa/agents/qa-pipeline-orchestrator.md and adopt it as your operating contract.
|
|
1999
|
+
|
|
2000
|
+
Even though you may be running as a general-purpose agent (native subagent type
|
|
2001
|
+
routing for QAA agents is not yet enabled), you MUST behave exactly as the
|
|
2002
|
+
dedicated agent defined in that file: read everything in its <required_reading>,
|
|
2003
|
+
obey its <quality_gate>, and run its mandatory bash checklist before returning.
|
|
2004
|
+
|
|
2005
|
+
Never modify application source code (src/, app/, lib/, or equivalent
|
|
2006
|
+
directories) unless the agent's <scope> section explicitly permits it. If the
|
|
2007
|
+
agent's contract restricts what files it can touch, honor that restriction —
|
|
2008
|
+
even if a fix seems obvious or trivial.
|
|
2009
|
+
|
|
2010
|
+
If you cannot read ~/.claude/qaa/agents/qa-pipeline-orchestrator.md (file missing, corrupted, parse
|
|
2011
|
+
error), STOP and report the failure. Do NOT proceed with default agent
|
|
2012
|
+
behavior — that defeats the purpose of this directive.
|
|
2013
|
+
|
|
2014
|
+
Do NOT improvise, substitute steps, apply changes the agent isn't allowed to
|
|
2015
|
+
make, or skip any of the above. Skipping the agent's required_reading,
|
|
2016
|
+
quality_gate, or bash checklist makes the run INVALID — its output cannot be
|
|
2017
|
+
trusted.
|
|
2018
|
+
</critical_directive>
|
|
2019
|
+
|
|
2020
|
+
<objective>Continue QA pipeline from {stage} stage</objective>
|
|
2021
|
+
<execution_context>~/.claude/qaa/agents/qa-pipeline-orchestrator.md</execution_context>
|
|
2022
|
+
<resume_context>
|
|
2023
|
+
Pipeline state:
|
|
2024
|
+
- Completed stages: {list of completed stages with their results}
|
|
2025
|
+
- Current stage: {stage that triggered checkpoint}
|
|
2026
|
+
- Checkpoint response: {user's response or decision}
|
|
2027
|
+
- Artifacts produced so far: {list of files with paths}
|
|
2028
|
+
|
|
2029
|
+
Resume from: {exact step in pipeline to resume from}
|
|
2030
|
+
User decision: {what user chose at checkpoint}
|
|
2031
|
+
</resume_context>
|
|
2032
|
+
"
|
|
2033
|
+
)
|
|
2034
|
+
```
|
|
2035
|
+
|
|
2036
|
+
### Stale Chain Flag Protection
|
|
2037
|
+
|
|
2038
|
+
At orchestrator init, if `--auto` was NOT passed AND `auto_advance` config is false:
|
|
2039
|
+
```bash
|
|
2040
|
+
node bin/qaa-tools.cjs config-set workflow._auto_chain_active false 2>/dev/null || true
|
|
2041
|
+
```
|
|
2042
|
+
|
|
2043
|
+
At pipeline completion (success or failure):
|
|
2044
|
+
```bash
|
|
2045
|
+
node bin/qaa-tools.cjs config-set workflow._auto_chain_active false 2>/dev/null || true
|
|
2046
|
+
```
|
|
2047
|
+
|
|
2048
|
+
</auto_advance>
|