qaa-agent 1.9.1 → 1.9.5
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +43 -22
- package/CLAUDE.md +170 -9
- package/README.md +384 -357
- package/VERSION +1 -0
- package/agents/qa-pipeline-orchestrator.md +336 -22
- package/agents/qaa-analyzer.md +0 -1
- package/agents/qaa-bug-detective.md +163 -4
- package/agents/qaa-codebase-mapper.md +50 -1
- package/agents/qaa-discovery.md +421 -384
- package/agents/qaa-e2e-runner.md +163 -1
- package/agents/qaa-executor.md +142 -1
- package/agents/qaa-planner.md +14 -1
- package/agents/qaa-project-researcher.md +194 -0
- package/agents/qaa-scanner.md +77 -1
- package/agents/qaa-testid-injector.md +0 -1
- package/agents/qaa-validator.md +86 -1
- package/bin/install.cjs +375 -253
- package/bin/lib/context7-cache.cjs +299 -0
- package/bin/lib/intent-detector.cjs +488 -0
- package/commands/qa-audit.md +255 -126
- package/commands/qa-create-test.md +666 -365
- package/commands/qa-fix.md +684 -513
- package/commands/qa-map.md +283 -139
- package/commands/qa-pr.md +63 -0
- package/commands/qa-research.md +181 -157
- package/commands/qa-start.md +62 -6
- package/commands/qa-test-report.md +219 -219
- package/frameworks/cypress.json +54 -0
- package/frameworks/jest.json +59 -0
- package/frameworks/playwright.json +58 -0
- package/frameworks/pytest.json +59 -0
- package/frameworks/robot-framework.json +54 -0
- package/frameworks/selenium.json +65 -0
- package/frameworks/vitest.json +57 -0
- package/package.json +7 -3
- package/workflows/qa-analyze.md +100 -4
- package/workflows/qa-from-ticket.md +50 -2
- package/workflows/qa-gap.md +50 -2
- package/workflows/qa-start.md +819 -33
- package/workflows/qa-testid.md +50 -2
- package/workflows/qa-validate.md +50 -2
package/VERSION
ADDED
|
@@ -0,0 +1 @@
|
|
|
1
|
+
1.9.5
|
|
@@ -6,13 +6,13 @@ skills:
|
|
|
6
6
|
---
|
|
7
7
|
|
|
8
8
|
<purpose>
|
|
9
|
-
Single orchestrator for the QA automation pipeline. Coordinates all
|
|
9
|
+
Single orchestrator for the QA automation pipeline. Coordinates all 8 agent types (scanner, analyzer, planner, executor, validator, e2e-runner, bug-detective, testid-injector) across 3 workflow options. Owns all pipeline state transitions -- agents never update state directly. The orchestrator sets stage status to 'running' before spawning an agent and 'complete' or 'failed' after the agent returns.
|
|
10
10
|
|
|
11
11
|
Invoked by the `/qa-start` slash command (Phase 6) or directly via Task() with this file as execution_context. Accepts 0-2 repo paths: 0 paths uses cwd as dev repo (Option 1), 1 path is dev-only (Option 1), 2 paths triggers maturity scoring to determine Option 2 or 3.
|
|
12
12
|
|
|
13
13
|
**Pipeline stages in order:**
|
|
14
14
|
```
|
|
15
|
-
scan -> codebase-map -> analyze -> [testid-inject if frontend] -> plan -> generate -> validate -> [e2e-runner if E2E tests] -> [bug-detective if failures] -> deliver
|
|
15
|
+
scan -> codebase-map -> research -> analyze -> [testid-inject if frontend] -> plan -> generate -> validate -> [e2e-runner if E2E tests AND app URL] -> [bug-detective if failures] -> deliver
|
|
16
16
|
```
|
|
17
17
|
|
|
18
18
|
**Workflow options:**
|
|
@@ -116,7 +116,7 @@ Based on `option` value from init, select the stage sequence. Each option shares
|
|
|
116
116
|
|
|
117
117
|
**Option 1 stages:**
|
|
118
118
|
```
|
|
119
|
-
scan(dev) -> codebase-map -> analyze(full) -> [testid-inject if frontend] -> plan -> generate -> validate -> [bug-detective if failures] -> deliver
|
|
119
|
+
scan(dev) -> codebase-map -> research -> analyze(full) -> [testid-inject if frontend] -> plan -> generate -> validate -> [e2e-runner if E2E tests AND app URL] -> [bug-detective if failures] -> deliver
|
|
120
120
|
```
|
|
121
121
|
- Scanner: scan DEV repo only
|
|
122
122
|
- Codebase Map: deep-scan codebase for testability, risk, patterns, existing tests
|
|
@@ -127,7 +127,7 @@ scan(dev) -> codebase-map -> analyze(full) -> [testid-inject if frontend] -> pla
|
|
|
127
127
|
|
|
128
128
|
**Option 2 stages:**
|
|
129
129
|
```
|
|
130
|
-
scan(both) -> codebase-map -> analyze(gap) -> [testid-inject if frontend] -> plan(gap) -> generate(gap) -> validate -> [bug-detective if failures] -> deliver
|
|
130
|
+
scan(both) -> codebase-map -> research -> analyze(gap) -> [testid-inject if frontend] -> plan(gap) -> generate(gap) -> validate -> [e2e-runner if E2E tests AND app URL] -> [bug-detective if failures] -> deliver
|
|
131
131
|
```
|
|
132
132
|
- Scanner: scan BOTH dev_repo_path and qa_repo_path
|
|
133
133
|
- Codebase Map: deep-scan dev codebase for testability, risk, patterns, existing tests
|
|
@@ -138,7 +138,7 @@ scan(both) -> codebase-map -> analyze(gap) -> [testid-inject if frontend] -> pla
|
|
|
138
138
|
|
|
139
139
|
**Option 3 stages:**
|
|
140
140
|
```
|
|
141
|
-
scan(both) -> codebase-map -> analyze(gap) -> [testid-inject if frontend] -> plan(gap) -> generate(skip-existing) -> validate -> [bug-detective if failures] -> deliver
|
|
141
|
+
scan(both) -> codebase-map -> research -> analyze(gap) -> [testid-inject if frontend] -> plan(gap) -> generate(skip-existing) -> validate -> [e2e-runner if E2E tests AND app URL] -> [bug-detective if failures] -> deliver
|
|
142
142
|
```
|
|
143
143
|
- Scanner: scan BOTH dev_repo_path and qa_repo_path
|
|
144
144
|
- Codebase Map: deep-scan dev codebase for testability, risk, patterns, existing tests
|
|
@@ -176,8 +176,32 @@ For **Option 1** -- scan DEV repo only:
|
|
|
176
176
|
```
|
|
177
177
|
Task(
|
|
178
178
|
prompt="
|
|
179
|
+
<critical_directive priority="MANDATORY">
|
|
180
|
+
You are executing as a QAA pipeline agent. Your FIRST action MUST be to read
|
|
181
|
+
~/.claude/qaa/agents/qaa-scanner.md and adopt it as your operating contract.
|
|
182
|
+
|
|
183
|
+
Even though you may be running as a general-purpose agent (native subagent type
|
|
184
|
+
routing for QAA agents is not yet enabled), you MUST behave exactly as the
|
|
185
|
+
dedicated agent defined in that file: read everything in its <required_reading>,
|
|
186
|
+
obey its <quality_gate>, and run its mandatory bash checklist before returning.
|
|
187
|
+
|
|
188
|
+
Never modify application source code (src/, app/, lib/, or equivalent
|
|
189
|
+
directories) unless the agent's <scope> section explicitly permits it. If the
|
|
190
|
+
agent's contract restricts what files it can touch, honor that restriction —
|
|
191
|
+
even if a fix seems obvious or trivial.
|
|
192
|
+
|
|
193
|
+
If you cannot read ~/.claude/qaa/agents/qaa-scanner.md (file missing, corrupted, parse
|
|
194
|
+
error), STOP and report the failure. Do NOT proceed with default agent
|
|
195
|
+
behavior — that defeats the purpose of this directive.
|
|
196
|
+
|
|
197
|
+
Do NOT improvise, substitute steps, apply changes the agent isn't allowed to
|
|
198
|
+
make, or skip any of the above. Skipping the agent's required_reading,
|
|
199
|
+
quality_gate, or bash checklist makes the run INVALID — its output cannot be
|
|
200
|
+
trusted.
|
|
201
|
+
</critical_directive>
|
|
202
|
+
|
|
179
203
|
<objective>Scan repository and produce SCAN_MANIFEST.md</objective>
|
|
180
|
-
<execution_context
|
|
204
|
+
<execution_context>~/.claude/qaa/agents/qaa-scanner.md</execution_context>
|
|
181
205
|
<files_to_read>
|
|
182
206
|
- CLAUDE.md
|
|
183
207
|
</files_to_read>
|
|
@@ -194,8 +218,32 @@ For **Options 2 and 3** -- scan BOTH repos:
|
|
|
194
218
|
```
|
|
195
219
|
Task(
|
|
196
220
|
prompt="
|
|
221
|
+
<critical_directive priority="MANDATORY">
|
|
222
|
+
You are executing as a QAA pipeline agent. Your FIRST action MUST be to read
|
|
223
|
+
~/.claude/qaa/agents/qaa-scanner.md and adopt it as your operating contract.
|
|
224
|
+
|
|
225
|
+
Even though you may be running as a general-purpose agent (native subagent type
|
|
226
|
+
routing for QAA agents is not yet enabled), you MUST behave exactly as the
|
|
227
|
+
dedicated agent defined in that file: read everything in its <required_reading>,
|
|
228
|
+
obey its <quality_gate>, and run its mandatory bash checklist before returning.
|
|
229
|
+
|
|
230
|
+
Never modify application source code (src/, app/, lib/, or equivalent
|
|
231
|
+
directories) unless the agent's <scope> section explicitly permits it. If the
|
|
232
|
+
agent's contract restricts what files it can touch, honor that restriction —
|
|
233
|
+
even if a fix seems obvious or trivial.
|
|
234
|
+
|
|
235
|
+
If you cannot read ~/.claude/qaa/agents/qaa-scanner.md (file missing, corrupted, parse
|
|
236
|
+
error), STOP and report the failure. Do NOT proceed with default agent
|
|
237
|
+
behavior — that defeats the purpose of this directive.
|
|
238
|
+
|
|
239
|
+
Do NOT improvise, substitute steps, apply changes the agent isn't allowed to
|
|
240
|
+
make, or skip any of the above. Skipping the agent's required_reading,
|
|
241
|
+
quality_gate, or bash checklist makes the run INVALID — its output cannot be
|
|
242
|
+
trusted.
|
|
243
|
+
</critical_directive>
|
|
244
|
+
|
|
197
245
|
<objective>Scan both developer and QA repositories and produce SCAN_MANIFEST.md</objective>
|
|
198
|
-
<execution_context
|
|
246
|
+
<execution_context>~/.claude/qaa/agents/qaa-scanner.md</execution_context>
|
|
199
247
|
<files_to_read>
|
|
200
248
|
- CLAUDE.md
|
|
201
249
|
</files_to_read>
|
|
@@ -266,8 +314,32 @@ For each focus in [testability, risk, patterns, existing-tests]:
|
|
|
266
314
|
|
|
267
315
|
Task(
|
|
268
316
|
prompt="
|
|
317
|
+
<critical_directive priority="MANDATORY">
|
|
318
|
+
You are executing as a QAA pipeline agent. Your FIRST action MUST be to read
|
|
319
|
+
~/.claude/qaa/agents/qaa-codebase-mapper.md and adopt it as your operating contract.
|
|
320
|
+
|
|
321
|
+
Even though you may be running as a general-purpose agent (native subagent type
|
|
322
|
+
routing for QAA agents is not yet enabled), you MUST behave exactly as the
|
|
323
|
+
dedicated agent defined in that file: read everything in its <required_reading>,
|
|
324
|
+
obey its <quality_gate>, and run its mandatory bash checklist before returning.
|
|
325
|
+
|
|
326
|
+
Never modify application source code (src/, app/, lib/, or equivalent
|
|
327
|
+
directories) unless the agent's <scope> section explicitly permits it. If the
|
|
328
|
+
agent's contract restricts what files it can touch, honor that restriction —
|
|
329
|
+
even if a fix seems obvious or trivial.
|
|
330
|
+
|
|
331
|
+
If you cannot read ~/.claude/qaa/agents/qaa-codebase-mapper.md (file missing, corrupted, parse
|
|
332
|
+
error), STOP and report the failure. Do NOT proceed with default agent
|
|
333
|
+
behavior — that defeats the purpose of this directive.
|
|
334
|
+
|
|
335
|
+
Do NOT improvise, substitute steps, apply changes the agent isn't allowed to
|
|
336
|
+
make, or skip any of the above. Skipping the agent's required_reading,
|
|
337
|
+
quality_gate, or bash checklist makes the run INVALID — its output cannot be
|
|
338
|
+
trusted.
|
|
339
|
+
</critical_directive>
|
|
340
|
+
|
|
269
341
|
<objective>Analyze codebase for QA purposes. Focus area: {focus}. Write documents to {output_dir}/codebase/.</objective>
|
|
270
|
-
<execution_context
|
|
342
|
+
<execution_context>~/.claude/qaa/agents/qaa-codebase-mapper.md</execution_context>
|
|
271
343
|
<files_to_read>
|
|
272
344
|
- CLAUDE.md
|
|
273
345
|
- {output_dir}/SCAN_MANIFEST.md
|
|
@@ -336,8 +408,32 @@ node bin/qaa-tools.cjs state patch --"Analyze Status" running --"Status" "Analyz
|
|
|
336
408
|
```
|
|
337
409
|
Task(
|
|
338
410
|
prompt="
|
|
411
|
+
<critical_directive priority="MANDATORY">
|
|
412
|
+
You are executing as a QAA pipeline agent. Your FIRST action MUST be to read
|
|
413
|
+
~/.claude/qaa/agents/qaa-analyzer.md and adopt it as your operating contract.
|
|
414
|
+
|
|
415
|
+
Even though you may be running as a general-purpose agent (native subagent type
|
|
416
|
+
routing for QAA agents is not yet enabled), you MUST behave exactly as the
|
|
417
|
+
dedicated agent defined in that file: read everything in its <required_reading>,
|
|
418
|
+
obey its <quality_gate>, and run its mandatory bash checklist before returning.
|
|
419
|
+
|
|
420
|
+
Never modify application source code (src/, app/, lib/, or equivalent
|
|
421
|
+
directories) unless the agent's <scope> section explicitly permits it. If the
|
|
422
|
+
agent's contract restricts what files it can touch, honor that restriction —
|
|
423
|
+
even if a fix seems obvious or trivial.
|
|
424
|
+
|
|
425
|
+
If you cannot read ~/.claude/qaa/agents/qaa-analyzer.md (file missing, corrupted, parse
|
|
426
|
+
error), STOP and report the failure. Do NOT proceed with default agent
|
|
427
|
+
behavior — that defeats the purpose of this directive.
|
|
428
|
+
|
|
429
|
+
Do NOT improvise, substitute steps, apply changes the agent isn't allowed to
|
|
430
|
+
make, or skip any of the above. Skipping the agent's required_reading,
|
|
431
|
+
quality_gate, or bash checklist makes the run INVALID — its output cannot be
|
|
432
|
+
trusted.
|
|
433
|
+
</critical_directive>
|
|
434
|
+
|
|
339
435
|
<objective>Analyze scanned repository and produce analysis artifacts</objective>
|
|
340
|
-
<execution_context
|
|
436
|
+
<execution_context>~/.claude/qaa/agents/qaa-analyzer.md</execution_context>
|
|
341
437
|
<files_to_read>
|
|
342
438
|
- {output_dir}/SCAN_MANIFEST.md
|
|
343
439
|
- CLAUDE.md
|
|
@@ -413,8 +509,32 @@ node bin/qaa-tools.cjs state patch --"Status" "Injecting test IDs into frontend
|
|
|
413
509
|
```
|
|
414
510
|
Task(
|
|
415
511
|
prompt="
|
|
512
|
+
<critical_directive priority="MANDATORY">
|
|
513
|
+
You are executing as a QAA pipeline agent. Your FIRST action MUST be to read
|
|
514
|
+
~/.claude/qaa/agents/qaa-testid-injector.md and adopt it as your operating contract.
|
|
515
|
+
|
|
516
|
+
Even though you may be running as a general-purpose agent (native subagent type
|
|
517
|
+
routing for QAA agents is not yet enabled), you MUST behave exactly as the
|
|
518
|
+
dedicated agent defined in that file: read everything in its <required_reading>,
|
|
519
|
+
obey its <quality_gate>, and run its mandatory bash checklist before returning.
|
|
520
|
+
|
|
521
|
+
Never modify application source code (src/, app/, lib/, or equivalent
|
|
522
|
+
directories) unless the agent's <scope> section explicitly permits it. If the
|
|
523
|
+
agent's contract restricts what files it can touch, honor that restriction —
|
|
524
|
+
even if a fix seems obvious or trivial.
|
|
525
|
+
|
|
526
|
+
If you cannot read ~/.claude/qaa/agents/qaa-testid-injector.md (file missing, corrupted, parse
|
|
527
|
+
error), STOP and report the failure. Do NOT proceed with default agent
|
|
528
|
+
behavior — that defeats the purpose of this directive.
|
|
529
|
+
|
|
530
|
+
Do NOT improvise, substitute steps, apply changes the agent isn't allowed to
|
|
531
|
+
make, or skip any of the above. Skipping the agent's required_reading,
|
|
532
|
+
quality_gate, or bash checklist makes the run INVALID — its output cannot be
|
|
533
|
+
trusted.
|
|
534
|
+
</critical_directive>
|
|
535
|
+
|
|
416
536
|
<objective>Audit and inject data-testid attributes into frontend components</objective>
|
|
417
|
-
<execution_context
|
|
537
|
+
<execution_context>~/.claude/qaa/agents/qaa-testid-injector.md</execution_context>
|
|
418
538
|
<files_to_read>
|
|
419
539
|
- {output_dir}/SCAN_MANIFEST.md
|
|
420
540
|
- CLAUDE.md
|
|
@@ -479,8 +599,32 @@ node bin/qaa-tools.cjs state patch --"Generate Status" running --"Status" "Plann
|
|
|
479
599
|
```
|
|
480
600
|
Task(
|
|
481
601
|
prompt="
|
|
602
|
+
<critical_directive priority="MANDATORY">
|
|
603
|
+
You are executing as a QAA pipeline agent. Your FIRST action MUST be to read
|
|
604
|
+
~/.claude/qaa/agents/qaa-planner.md and adopt it as your operating contract.
|
|
605
|
+
|
|
606
|
+
Even though you may be running as a general-purpose agent (native subagent type
|
|
607
|
+
routing for QAA agents is not yet enabled), you MUST behave exactly as the
|
|
608
|
+
dedicated agent defined in that file: read everything in its <required_reading>,
|
|
609
|
+
obey its <quality_gate>, and run its mandatory bash checklist before returning.
|
|
610
|
+
|
|
611
|
+
Never modify application source code (src/, app/, lib/, or equivalent
|
|
612
|
+
directories) unless the agent's <scope> section explicitly permits it. If the
|
|
613
|
+
agent's contract restricts what files it can touch, honor that restriction —
|
|
614
|
+
even if a fix seems obvious or trivial.
|
|
615
|
+
|
|
616
|
+
If you cannot read ~/.claude/qaa/agents/qaa-planner.md (file missing, corrupted, parse
|
|
617
|
+
error), STOP and report the failure. Do NOT proceed with default agent
|
|
618
|
+
behavior — that defeats the purpose of this directive.
|
|
619
|
+
|
|
620
|
+
Do NOT improvise, substitute steps, apply changes the agent isn't allowed to
|
|
621
|
+
make, or skip any of the above. Skipping the agent's required_reading,
|
|
622
|
+
quality_gate, or bash checklist makes the run INVALID — its output cannot be
|
|
623
|
+
trusted.
|
|
624
|
+
</critical_directive>
|
|
625
|
+
|
|
482
626
|
<objective>Create test generation plan with task breakdown and dependencies</objective>
|
|
483
|
-
<execution_context
|
|
627
|
+
<execution_context>~/.claude/qaa/agents/qaa-planner.md</execution_context>
|
|
484
628
|
<files_to_read>
|
|
485
629
|
- {input files based on option -- see above}
|
|
486
630
|
- CLAUDE.md
|
|
@@ -541,8 +685,32 @@ For each independent feature group from the generation plan, spawn a separate ex
|
|
|
541
685
|
```
|
|
542
686
|
Task(
|
|
543
687
|
prompt="
|
|
688
|
+
<critical_directive priority="MANDATORY">
|
|
689
|
+
You are executing as a QAA pipeline agent. Your FIRST action MUST be to read
|
|
690
|
+
~/.claude/qaa/agents/qaa-executor.md and adopt it as your operating contract.
|
|
691
|
+
|
|
692
|
+
Even though you may be running as a general-purpose agent (native subagent type
|
|
693
|
+
routing for QAA agents is not yet enabled), you MUST behave exactly as the
|
|
694
|
+
dedicated agent defined in that file: read everything in its <required_reading>,
|
|
695
|
+
obey its <quality_gate>, and run its mandatory bash checklist before returning.
|
|
696
|
+
|
|
697
|
+
Never modify application source code (src/, app/, lib/, or equivalent
|
|
698
|
+
directories) unless the agent's <scope> section explicitly permits it. If the
|
|
699
|
+
agent's contract restricts what files it can touch, honor that restriction —
|
|
700
|
+
even if a fix seems obvious or trivial.
|
|
701
|
+
|
|
702
|
+
If you cannot read ~/.claude/qaa/agents/qaa-executor.md (file missing, corrupted, parse
|
|
703
|
+
error), STOP and report the failure. Do NOT proceed with default agent
|
|
704
|
+
behavior — that defeats the purpose of this directive.
|
|
705
|
+
|
|
706
|
+
Do NOT improvise, substitute steps, apply changes the agent isn't allowed to
|
|
707
|
+
make, or skip any of the above. Skipping the agent's required_reading,
|
|
708
|
+
quality_gate, or bash checklist makes the run INVALID — its output cannot be
|
|
709
|
+
trusted.
|
|
710
|
+
</critical_directive>
|
|
711
|
+
|
|
544
712
|
<objective>Generate test files for {feature} feature</objective>
|
|
545
|
-
<execution_context
|
|
713
|
+
<execution_context>~/.claude/qaa/agents/qaa-executor.md</execution_context>
|
|
546
714
|
<files_to_read>
|
|
547
715
|
- {output_dir}/GENERATION_PLAN.md
|
|
548
716
|
- {output_dir}/TEST_INVENTORY.md (Option 1) or {output_dir}/GAP_ANALYSIS.md (Options 2/3)
|
|
@@ -571,8 +739,32 @@ Spawn a single executor agent covering all tasks:
|
|
|
571
739
|
```
|
|
572
740
|
Task(
|
|
573
741
|
prompt="
|
|
742
|
+
<critical_directive priority="MANDATORY">
|
|
743
|
+
You are executing as a QAA pipeline agent. Your FIRST action MUST be to read
|
|
744
|
+
~/.claude/qaa/agents/qaa-executor.md and adopt it as your operating contract.
|
|
745
|
+
|
|
746
|
+
Even though you may be running as a general-purpose agent (native subagent type
|
|
747
|
+
routing for QAA agents is not yet enabled), you MUST behave exactly as the
|
|
748
|
+
dedicated agent defined in that file: read everything in its <required_reading>,
|
|
749
|
+
obey its <quality_gate>, and run its mandatory bash checklist before returning.
|
|
750
|
+
|
|
751
|
+
Never modify application source code (src/, app/, lib/, or equivalent
|
|
752
|
+
directories) unless the agent's <scope> section explicitly permits it. If the
|
|
753
|
+
agent's contract restricts what files it can touch, honor that restriction —
|
|
754
|
+
even if a fix seems obvious or trivial.
|
|
755
|
+
|
|
756
|
+
If you cannot read ~/.claude/qaa/agents/qaa-executor.md (file missing, corrupted, parse
|
|
757
|
+
error), STOP and report the failure. Do NOT proceed with default agent
|
|
758
|
+
behavior — that defeats the purpose of this directive.
|
|
759
|
+
|
|
760
|
+
Do NOT improvise, substitute steps, apply changes the agent isn't allowed to
|
|
761
|
+
make, or skip any of the above. Skipping the agent's required_reading,
|
|
762
|
+
quality_gate, or bash checklist makes the run INVALID — its output cannot be
|
|
763
|
+
trusted.
|
|
764
|
+
</critical_directive>
|
|
765
|
+
|
|
574
766
|
<objective>Generate all test files from generation plan</objective>
|
|
575
|
-
<execution_context
|
|
767
|
+
<execution_context>~/.claude/qaa/agents/qaa-executor.md</execution_context>
|
|
576
768
|
<files_to_read>
|
|
577
769
|
- {output_dir}/GENERATION_PLAN.md
|
|
578
770
|
- {output_dir}/TEST_INVENTORY.md (Option 1) or {output_dir}/GAP_ANALYSIS.md (Options 2/3)
|
|
@@ -649,8 +841,32 @@ node bin/qaa-tools.cjs state patch --"Validate Status" running --"Status" "Valid
|
|
|
649
841
|
```
|
|
650
842
|
Task(
|
|
651
843
|
prompt="
|
|
844
|
+
<critical_directive priority="MANDATORY">
|
|
845
|
+
You are executing as a QAA pipeline agent. Your FIRST action MUST be to read
|
|
846
|
+
~/.claude/qaa/agents/qaa-validator.md and adopt it as your operating contract.
|
|
847
|
+
|
|
848
|
+
Even though you may be running as a general-purpose agent (native subagent type
|
|
849
|
+
routing for QAA agents is not yet enabled), you MUST behave exactly as the
|
|
850
|
+
dedicated agent defined in that file: read everything in its <required_reading>,
|
|
851
|
+
obey its <quality_gate>, and run its mandatory bash checklist before returning.
|
|
852
|
+
|
|
853
|
+
Never modify application source code (src/, app/, lib/, or equivalent
|
|
854
|
+
directories) unless the agent's <scope> section explicitly permits it. If the
|
|
855
|
+
agent's contract restricts what files it can touch, honor that restriction —
|
|
856
|
+
even if a fix seems obvious or trivial.
|
|
857
|
+
|
|
858
|
+
If you cannot read ~/.claude/qaa/agents/qaa-validator.md (file missing, corrupted, parse
|
|
859
|
+
error), STOP and report the failure. Do NOT proceed with default agent
|
|
860
|
+
behavior — that defeats the purpose of this directive.
|
|
861
|
+
|
|
862
|
+
Do NOT improvise, substitute steps, apply changes the agent isn't allowed to
|
|
863
|
+
make, or skip any of the above. Skipping the agent's required_reading,
|
|
864
|
+
quality_gate, or bash checklist makes the run INVALID — its output cannot be
|
|
865
|
+
trusted.
|
|
866
|
+
</critical_directive>
|
|
867
|
+
|
|
652
868
|
<objective>Run 4-layer validation on all generated test files</objective>
|
|
653
|
-
<execution_context
|
|
869
|
+
<execution_context>~/.claude/qaa/agents/qaa-validator.md</execution_context>
|
|
654
870
|
<files_to_read>
|
|
655
871
|
- {list all generated test files from executor return -- files_created paths}
|
|
656
872
|
- {output_dir}/GENERATION_PLAN.md
|
|
@@ -723,8 +939,32 @@ node bin/qaa-tools.cjs state patch --"Status" "Running E2E tests against live ap
|
|
|
723
939
|
```
|
|
724
940
|
Task(
|
|
725
941
|
prompt="
|
|
942
|
+
<critical_directive priority="MANDATORY">
|
|
943
|
+
You are executing as a QAA pipeline agent. Your FIRST action MUST be to read
|
|
944
|
+
~/.claude/qaa/agents/qaa-e2e-runner.md and adopt it as your operating contract.
|
|
945
|
+
|
|
946
|
+
Even though you may be running as a general-purpose agent (native subagent type
|
|
947
|
+
routing for QAA agents is not yet enabled), you MUST behave exactly as the
|
|
948
|
+
dedicated agent defined in that file: read everything in its <required_reading>,
|
|
949
|
+
obey its <quality_gate>, and run its mandatory bash checklist before returning.
|
|
950
|
+
|
|
951
|
+
Never modify application source code (src/, app/, lib/, or equivalent
|
|
952
|
+
directories) unless the agent's <scope> section explicitly permits it. If the
|
|
953
|
+
agent's contract restricts what files it can touch, honor that restriction —
|
|
954
|
+
even if a fix seems obvious or trivial.
|
|
955
|
+
|
|
956
|
+
If you cannot read ~/.claude/qaa/agents/qaa-e2e-runner.md (file missing, corrupted, parse
|
|
957
|
+
error), STOP and report the failure. Do NOT proceed with default agent
|
|
958
|
+
behavior — that defeats the purpose of this directive.
|
|
959
|
+
|
|
960
|
+
Do NOT improvise, substitute steps, apply changes the agent isn't allowed to
|
|
961
|
+
make, or skip any of the above. Skipping the agent's required_reading,
|
|
962
|
+
quality_gate, or bash checklist makes the run INVALID — its output cannot be
|
|
963
|
+
trusted.
|
|
964
|
+
</critical_directive>
|
|
965
|
+
|
|
726
966
|
<objective>Run E2E tests against live application, capture real locators, fix mismatches, loop until pass</objective>
|
|
727
|
-
<execution_context
|
|
967
|
+
<execution_context>~/.claude/qaa/agents/qaa-e2e-runner.md</execution_context>
|
|
728
968
|
<files_to_read>
|
|
729
969
|
- CLAUDE.md
|
|
730
970
|
- {e2e_test_files from executor return}
|
|
@@ -751,11 +991,28 @@ E2E_RUNNER_COMPLETE:
|
|
|
751
991
|
locator_fixes: N
|
|
752
992
|
app_bugs_found: N
|
|
753
993
|
fix_loops_used: N
|
|
994
|
+
runner_executed: true | false
|
|
995
|
+
runner_exit_code: N
|
|
996
|
+
runner_status: OK | BROKEN
|
|
754
997
|
report_path: "..."
|
|
755
998
|
screenshots: [...]
|
|
756
999
|
```
|
|
757
1000
|
|
|
758
|
-
Capture `passed`, `failed`, `app_bugs_found`, `locator_fixes` for pipeline summary.
|
|
1001
|
+
Capture `passed`, `failed`, `app_bugs_found`, `locator_fixes`, `runner_executed`, `runner_status` for pipeline summary.
|
|
1002
|
+
|
|
1003
|
+
**Transition: validate → e2e-runner → { bug-detective | deliver | HALT }.**
|
|
1004
|
+
|
|
1005
|
+
**HARD STOP on broken runner (#47/#48):** If the e2e-runner returns `runner_executed: false` (or `runner_status: BROKEN`, or an ENVIRONMENT ISSUE), the test runner could not execute. HALT the pipeline — do NOT advance to bug-detective and do NOT advance to deliver:
|
|
1006
|
+
|
|
1007
|
+
```bash
|
|
1008
|
+
node bin/qaa-tools.cjs state patch --"Status" "Pipeline halted: E2E runner unavailable (env issue)"
|
|
1009
|
+
```
|
|
1010
|
+
|
|
1011
|
+
- Set the pipeline_summary E2E Runner row to `halted (runner unavailable — env issue)`.
|
|
1012
|
+
- Print the halt banner (see Error Handling).
|
|
1013
|
+
- A DOM probe is NEVER accepted as a substitute for a real run; if the report presents probe evidence as "verified" while `runner_executed: false`, treat the run as INVALID.
|
|
1014
|
+
|
|
1015
|
+
Otherwise (`runner_executed: true`), continue with the normal flow below.
|
|
759
1016
|
|
|
760
1017
|
If `app_bugs_found > 0`:
|
|
761
1018
|
Print: "E2E Runner found {app_bugs_found} application bug(s). See E2E_RUN_REPORT.md for details with screenshots."
|
|
@@ -797,8 +1054,32 @@ node bin/qaa-tools.cjs state patch --"Status" "Classifying test failures"
|
|
|
797
1054
|
```
|
|
798
1055
|
Task(
|
|
799
1056
|
prompt="
|
|
1057
|
+
<critical_directive priority="MANDATORY">
|
|
1058
|
+
You are executing as a QAA pipeline agent. Your FIRST action MUST be to read
|
|
1059
|
+
~/.claude/qaa/agents/qaa-bug-detective.md and adopt it as your operating contract.
|
|
1060
|
+
|
|
1061
|
+
Even though you may be running as a general-purpose agent (native subagent type
|
|
1062
|
+
routing for QAA agents is not yet enabled), you MUST behave exactly as the
|
|
1063
|
+
dedicated agent defined in that file: read everything in its <required_reading>,
|
|
1064
|
+
obey its <quality_gate>, and run its mandatory bash checklist before returning.
|
|
1065
|
+
|
|
1066
|
+
Never modify application source code (src/, app/, lib/, or equivalent
|
|
1067
|
+
directories) unless the agent's <scope> section explicitly permits it. If the
|
|
1068
|
+
agent's contract restricts what files it can touch, honor that restriction —
|
|
1069
|
+
even if a fix seems obvious or trivial.
|
|
1070
|
+
|
|
1071
|
+
If you cannot read ~/.claude/qaa/agents/qaa-bug-detective.md (file missing, corrupted, parse
|
|
1072
|
+
error), STOP and report the failure. Do NOT proceed with default agent
|
|
1073
|
+
behavior — that defeats the purpose of this directive.
|
|
1074
|
+
|
|
1075
|
+
Do NOT improvise, substitute steps, apply changes the agent isn't allowed to
|
|
1076
|
+
make, or skip any of the above. Skipping the agent's required_reading,
|
|
1077
|
+
quality_gate, or bash checklist makes the run INVALID — its output cannot be
|
|
1078
|
+
trusted.
|
|
1079
|
+
</critical_directive>
|
|
1080
|
+
|
|
800
1081
|
<objective>Classify test failures and attempt auto-fixes for test errors</objective>
|
|
801
|
-
<execution_context
|
|
1082
|
+
<execution_context>~/.claude/qaa/agents/qaa-bug-detective.md</execution_context>
|
|
802
1083
|
<files_to_read>
|
|
803
1084
|
- {test execution results -- from validator or direct test run}
|
|
804
1085
|
- {failing test source files -- paths from executor return}
|
|
@@ -821,9 +1102,14 @@ DETECTIVE_COMPLETE:
|
|
|
821
1102
|
classification_breakdown: {app_bug: N, test_error: N, env_issue: N, inconclusive: N}
|
|
822
1103
|
auto_fixes_applied: N
|
|
823
1104
|
auto_fixes_verified: N
|
|
1105
|
+
runner_executed: true | false
|
|
1106
|
+
runner_exit_code: N
|
|
1107
|
+
runner_status: OK | BROKEN
|
|
824
1108
|
commit_hash: "..."
|
|
825
1109
|
```
|
|
826
1110
|
|
|
1111
|
+
**HARD STOP on broken runner (#47/#48):** If the bug-detective returns `runner_executed: false` / `runner_status: BROKEN` (the test suite never executed — ENV ISSUE), HALT the pipeline: do NOT advance to deliver, print the halt banner, and set Status to "Pipeline halted: test runner unavailable (env issue)". The bug-detective MUST NOT have substituted MCP DOM probes for a real run; a report claiming classifications with `runner_executed: false` is INVALID.
|
|
1112
|
+
|
|
827
1113
|
**RISKY CHECKPOINT -- Application bugs detected:**
|
|
828
1114
|
|
|
829
1115
|
If `classification_breakdown.app_bug > 0`:
|
|
@@ -1207,8 +1493,32 @@ When resuming after a checkpoint, spawn a FRESH agent (not serialized state). Th
|
|
|
1207
1493
|
```
|
|
1208
1494
|
Task(
|
|
1209
1495
|
prompt="
|
|
1496
|
+
<critical_directive priority="MANDATORY">
|
|
1497
|
+
You are executing as a QAA pipeline agent. Your FIRST action MUST be to read
|
|
1498
|
+
~/.claude/qaa/agents/qa-pipeline-orchestrator.md and adopt it as your operating contract.
|
|
1499
|
+
|
|
1500
|
+
Even though you may be running as a general-purpose agent (native subagent type
|
|
1501
|
+
routing for QAA agents is not yet enabled), you MUST behave exactly as the
|
|
1502
|
+
dedicated agent defined in that file: read everything in its <required_reading>,
|
|
1503
|
+
obey its <quality_gate>, and run its mandatory bash checklist before returning.
|
|
1504
|
+
|
|
1505
|
+
Never modify application source code (src/, app/, lib/, or equivalent
|
|
1506
|
+
directories) unless the agent's <scope> section explicitly permits it. If the
|
|
1507
|
+
agent's contract restricts what files it can touch, honor that restriction —
|
|
1508
|
+
even if a fix seems obvious or trivial.
|
|
1509
|
+
|
|
1510
|
+
If you cannot read ~/.claude/qaa/agents/qa-pipeline-orchestrator.md (file missing, corrupted, parse
|
|
1511
|
+
error), STOP and report the failure. Do NOT proceed with default agent
|
|
1512
|
+
behavior — that defeats the purpose of this directive.
|
|
1513
|
+
|
|
1514
|
+
Do NOT improvise, substitute steps, apply changes the agent isn't allowed to
|
|
1515
|
+
make, or skip any of the above. Skipping the agent's required_reading,
|
|
1516
|
+
quality_gate, or bash checklist makes the run INVALID — its output cannot be
|
|
1517
|
+
trusted.
|
|
1518
|
+
</critical_directive>
|
|
1519
|
+
|
|
1210
1520
|
<objective>Continue QA pipeline from {stage} stage</objective>
|
|
1211
|
-
<execution_context
|
|
1521
|
+
<execution_context>~/.claude/qaa/agents/qa-pipeline-orchestrator.md</execution_context>
|
|
1212
1522
|
<resume_context>
|
|
1213
1523
|
Pipeline state:
|
|
1214
1524
|
- Completed stages: {list of completed stages with their results}
|
|
@@ -1326,6 +1636,7 @@ After all stages complete (or on pipeline stop), print a summary banner:
|
|
|
1326
1636
|
[{check}] Plan -- {plan_duration} ({file_count} files planned)
|
|
1327
1637
|
[{check}] Generate -- {generate_duration} ({files_created} files created)
|
|
1328
1638
|
[{check}] Validate -- {validate_duration} ({confidence} confidence)
|
|
1639
|
+
[{check}] E2E Runner -- {e2e_status}
|
|
1329
1640
|
[{check}] Bug Detective-- {detective_duration or 'skipped'}
|
|
1330
1641
|
[{check}] Deliver -- {deliver_duration}
|
|
1331
1642
|
|
|
@@ -1343,6 +1654,8 @@ Where:
|
|
|
1343
1654
|
- `[ ]` = stage skipped (testid-inject when no frontend, bug-detective when no failures)
|
|
1344
1655
|
- `[!]` = stage failed
|
|
1345
1656
|
|
|
1657
|
+
`{e2e_status}` is one of: `skipped (no E2E files)`, `skipped (no app URL)`, `halted (runner unavailable — env issue)`, or `{e2e_duration}` on success. A `halted` status means the e2e-runner returned `runner_executed: false` / ENV ISSUE; the pipeline HALTs and does NOT advance to bug-detective or deliver (see #47/#48 transition).
|
|
1658
|
+
|
|
1346
1659
|
**On pipeline failure:** The summary still prints, but shows which stages completed and which failed, along with the failure reason.
|
|
1347
1660
|
|
|
1348
1661
|
**Artifact list includes:**
|
|
@@ -1354,6 +1667,7 @@ Where:
|
|
|
1354
1667
|
- GENERATION_PLAN.md (if plan stage completed)
|
|
1355
1668
|
- Generated test files (if generate stage completed)
|
|
1356
1669
|
- VALIDATION_REPORT.md (if validate stage completed)
|
|
1670
|
+
- E2E_RUN_REPORT.md (if e2e-runner ran)
|
|
1357
1671
|
- FAILURE_CLASSIFICATION_REPORT.md (if bug detective ran)
|
|
1358
1672
|
</pipeline_summary>
|
|
1359
1673
|
|
|
@@ -1363,12 +1677,13 @@ Where:
|
|
|
1363
1677
|
Before this orchestrator is considered complete, verify:
|
|
1364
1678
|
|
|
1365
1679
|
- [ ] All 3 workflow options route to correct stage sequences:
|
|
1366
|
-
- Option 1: scan(dev) -> analyze(full) -> [testid-inject] -> plan -> generate -> validate -> [bug-detective] -> deliver
|
|
1367
|
-
- Option 2: scan(both) -> analyze(gap) -> [testid-inject] -> plan(gap) -> generate(gap) -> validate -> [bug-detective] -> deliver
|
|
1368
|
-
- Option 3: scan(both) -> analyze(gap) -> [testid-inject] -> plan(gap) -> generate(skip-existing) -> validate -> [bug-detective] -> deliver
|
|
1680
|
+
- Option 1: scan(dev) -> codebase-map -> research -> analyze(full) -> [testid-inject] -> plan -> generate -> validate -> [e2e-runner if E2E tests AND app URL] -> [bug-detective] -> deliver
|
|
1681
|
+
- Option 2: scan(both) -> codebase-map -> research -> analyze(gap) -> [testid-inject] -> plan(gap) -> generate(gap) -> validate -> [e2e-runner if E2E tests AND app URL] -> [bug-detective] -> deliver
|
|
1682
|
+
- Option 3: scan(both) -> codebase-map -> research -> analyze(gap) -> [testid-inject] -> plan(gap) -> generate(skip-existing) -> validate -> [e2e-runner if E2E tests AND app URL] -> [bug-detective] -> deliver
|
|
1369
1683
|
- [ ] Every agent spawn is bracketed by state updates (running before, complete/failed after)
|
|
1370
1684
|
- [ ] Auto-advance correctly classifies safe vs risky checkpoints
|
|
1371
1685
|
- [ ] Pipeline stops entirely on any stage failure (no partial PR)
|
|
1686
|
+
- [ ] e2e-runner / bug-detective transition enforced: `runner_executed: false` / `runner_status: BROKEN` / ENV ISSUE → HALT (no advance to bug-detective or deliver), never a fallback to DOM probes (validate → e2e-runner → { bug-detective | deliver | HALT })
|
|
1372
1687
|
- [ ] Progress banners print for every stage even in auto mode
|
|
1373
1688
|
- [ ] Deliver stage creates branch, commits per-stage, pushes, and creates draft PR via gh CLI
|
|
1374
1689
|
- [ ] Resume spawns fresh agent with explicit state (no serialization)
|
|
@@ -1422,4 +1737,3 @@ echo "=== PIPELINE ORCHESTRATOR CHECKLIST END ==="
|
|
|
1422
1737
|
- If any output shows a problem (STATE_FILE_NOT_FOUND, NO_SCAN_MANIFEST), fix it before returning.
|
|
1423
1738
|
- If output shows expected "not found" results (e.g., NO_TESTID_REPORT when no frontend was detected), that is fine — the point is you RAN the command instead of assuming the answer.
|
|
1424
1739
|
- Do NOT return control to the user until the block has been executed and you have read every line of output.
|
|
1425
|
-
|
package/agents/qaa-analyzer.md
CHANGED
|
@@ -577,4 +577,3 @@ echo "=== ANALYZER CHECKLIST END ==="
|
|
|
577
577
|
- If any output shows a problem (SCAN_MANIFEST_NOT_FOUND, OUTPUTS_NOT_WRITTEN), fix it before returning.
|
|
578
578
|
- If output shows expected "not found" results (e.g., NO_CODEBASE_MAP when mapper hasn't run yet), that is fine — the point is you RAN the command instead of assuming the answer.
|
|
579
579
|
- Do NOT return control to the parent agent until the block has been executed and you have read every line of output.
|
|
580
|
-
|