qaa-agent 1.9.5 → 1.9.7
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +38 -0
- package/CLAUDE.md +5 -3
- package/VERSION +1 -1
- package/agents/qaa-bug-detective.md +2 -0
- package/agents/qaa-codebase-mapper.md +4 -0
- package/agents/qaa-discovery.md +10 -0
- package/agents/qaa-e2e-runner.md +2 -0
- package/agents/qaa-executor.md +19 -0
- package/agents/qaa-project-researcher.md +6 -0
- package/agents/qaa-scanner.md +9 -0
- package/bin/install.cjs +13 -3
- package/bin/lib/intent-detector.cjs +6 -4
- package/commands/qa-audit.md +77 -26
- package/commands/qa-create-test-ado.md +486 -404
- package/commands/qa-create-test.md +58 -0
- package/commands/qa-fix.md +73 -3
- package/commands/qa-map.md +76 -4
- package/commands/qa-pr.md +36 -0
- package/commands/qa-research.md +67 -0
- package/commands/qa-start.md +37 -0
- package/commands/qa-test-report.md +82 -0
- package/commands/qa-testid.md +85 -0
- package/package.json +41 -41
|
@@ -165,6 +165,58 @@ SUGGESTED_MODE=$(echo "$RESOLVED" | node -e "const j=JSON.parse(require('fs').re
|
|
|
165
165
|
|
|
166
166
|
The detector resolves the mode automatically when possible (from-ticket, from-aut, ado, etc.) and surfaces it in `SUGGESTED_MODE`. The mode detection logic in Step 1 below uses these resolved values.
|
|
167
167
|
|
|
168
|
+
## Pre-flight checks
|
|
169
|
+
|
|
170
|
+
Run these BEFORE the main flow. A **[STOP]** check that fails: print the error EXACTLY as shown (fill in the `{...}` placeholders) and STOP. A **[WARN]** check that fails: print the notice and continue in degraded mode.
|
|
171
|
+
|
|
172
|
+
### Dev-repo path does not exist [STOP]
|
|
173
|
+
If the resolved `--dev-repo` path is not an existing directory:
|
|
174
|
+
```
|
|
175
|
+
❌ Dev-repo path not found: {path}
|
|
176
|
+
|
|
177
|
+
The --dev-repo path you gave does not exist (or is not a directory), so there's
|
|
178
|
+
nothing to read the code from.
|
|
179
|
+
|
|
180
|
+
Options:
|
|
181
|
+
1. Re-run with a correct path: /qa-create-test <feature> --dev-repo <path>
|
|
182
|
+
2. Run from inside the repo with no --dev-repo (defaults to the current directory)
|
|
183
|
+
3. Check for a typo or an unexpanded ~ in the path
|
|
184
|
+
|
|
185
|
+
Try: ls {path}
|
|
186
|
+
```
|
|
187
|
+
|
|
188
|
+
### Playwright MCP not connected — from-aut mode [STOP]
|
|
189
|
+
If MODE is from-aut and the Playwright MCP tools are unavailable:
|
|
190
|
+
```
|
|
191
|
+
❌ Playwright MCP is not connected — from-aut mode needs it to explore the app.
|
|
192
|
+
|
|
193
|
+
from-aut generates tests by driving the live app with Playwright MCP. Without
|
|
194
|
+
it, there's no way to discover real elements and locators.
|
|
195
|
+
|
|
196
|
+
Options:
|
|
197
|
+
1. Connect Playwright MCP (it ships with QAA — check ~/.claude.json), then re-run
|
|
198
|
+
2. Reinstall to re-register the MCP servers: npx qaa-agent
|
|
199
|
+
3. Generate from the code instead (no live app needed): /qa-create-test <feature>
|
|
200
|
+
|
|
201
|
+
Try: /mcp (to see connected MCP servers)
|
|
202
|
+
```
|
|
203
|
+
|
|
204
|
+
### Framework not detectable — from-code mode [STOP]
|
|
205
|
+
If MODE is from-code and no framework can be detected:
|
|
206
|
+
```
|
|
207
|
+
❌ Could not detect a test framework in {dev-repo}.
|
|
208
|
+
|
|
209
|
+
from-code needs to know which framework to generate for. No config files or
|
|
210
|
+
manifest entries pointed to one.
|
|
211
|
+
|
|
212
|
+
Options:
|
|
213
|
+
1. Pass it explicitly: /qa-create-test <feature> --framework <playwright|cypress|jest|vitest|pytest|selenium|robot-framework>
|
|
214
|
+
2. Run /qa-map first to build the codebase map, then re-run
|
|
215
|
+
3. Add a framework to the repo (e.g. `npm i -D cypress`) and re-run
|
|
216
|
+
|
|
217
|
+
Try: /qa-map (to scan and detect the stack first)
|
|
218
|
+
```
|
|
219
|
+
|
|
168
220
|
## Instructions
|
|
169
221
|
|
|
170
222
|
### Step 1: Detect Mode
|
|
@@ -227,6 +279,7 @@ App URL: {url or "auto-detect"}
|
|
|
227
279
|
5. Invoke executor agent to generate test files:
|
|
228
280
|
|
|
229
281
|
Task(
|
|
282
|
+
subagent_type="qaa-executor",
|
|
230
283
|
prompt="
|
|
231
284
|
<critical_directive priority="MANDATORY">
|
|
232
285
|
You are executing as a QAA pipeline agent. Your FIRST action MUST be to read
|
|
@@ -280,6 +333,7 @@ Task(
|
|
|
280
333
|
6. If E2E test files were generated AND `--skip-run` was NOT passed, invoke E2E runner:
|
|
281
334
|
|
|
282
335
|
Task(
|
|
336
|
+
subagent_type="qaa-e2e-runner",
|
|
283
337
|
prompt="
|
|
284
338
|
<critical_directive priority="MANDATORY">
|
|
285
339
|
You are executing as a QAA pipeline agent. Your FIRST action MUST be to read
|
|
@@ -366,6 +420,7 @@ Generate tests directly from a running application URL — no ticket, no spec, n
|
|
|
366
420
|
6. After approval, invoke the executor to generate specs + POMs + fixtures:
|
|
367
421
|
|
|
368
422
|
Task(
|
|
423
|
+
subagent_type="qaa-executor",
|
|
369
424
|
prompt="
|
|
370
425
|
<critical_directive priority="MANDATORY">
|
|
371
426
|
You are executing as a QAA pipeline agent. Your FIRST action MUST be to read
|
|
@@ -485,6 +540,7 @@ For the complete step-by-step process, see `@commands/qa-create-test-ado.md`.
|
|
|
485
540
|
2. Invoke validator agent in audit mode:
|
|
486
541
|
|
|
487
542
|
Task(
|
|
543
|
+
subagent_type="qaa-validator",
|
|
488
544
|
prompt="
|
|
489
545
|
<critical_directive priority="MANDATORY">
|
|
490
546
|
You are executing as a QAA pipeline agent. Your FIRST action MUST be to read
|
|
@@ -528,6 +584,7 @@ Task(
|
|
|
528
584
|
4. Invoke executor agent to apply approved improvements:
|
|
529
585
|
|
|
530
586
|
Task(
|
|
587
|
+
subagent_type="qaa-executor",
|
|
531
588
|
prompt="
|
|
532
589
|
<critical_directive priority="MANDATORY">
|
|
533
590
|
You are executing as a QAA pipeline agent. Your FIRST action MUST be to read
|
|
@@ -587,6 +644,7 @@ Generate only Page Object Model files — no test specs.
|
|
|
587
644
|
2. Invoke executor agent in POM-only mode:
|
|
588
645
|
|
|
589
646
|
Task(
|
|
647
|
+
subagent_type="qaa-executor",
|
|
590
648
|
prompt="
|
|
591
649
|
<critical_directive priority="MANDATORY">
|
|
592
650
|
You are executing as a QAA pipeline agent. Your FIRST action MUST be to read
|
package/commands/qa-fix.md
CHANGED
|
@@ -115,6 +115,69 @@ TICKET=$(echo "$RESOLVED" | node -e "const j=JSON.parse(require('fs').readFileSy
|
|
|
115
115
|
|
|
116
116
|
If you see values you didn't intend in the banner above (interactive mode), you can press Ctrl-C and re-run with explicit flags.
|
|
117
117
|
|
|
118
|
+
## Pre-flight checks
|
|
119
|
+
|
|
120
|
+
Run these BEFORE the main flow. If a **[STOP]** check fails, print the error EXACTLY as shown (fill in the `{...}` placeholders) and STOP.
|
|
121
|
+
|
|
122
|
+
### Test files not found [STOP]
|
|
123
|
+
- If a path/directory WAS passed but does not exist, print:
|
|
124
|
+
```
|
|
125
|
+
❌ Test path not found: {path}
|
|
126
|
+
|
|
127
|
+
The path you passed to /qa-fix doesn't exist.
|
|
128
|
+
|
|
129
|
+
Options:
|
|
130
|
+
1. Point at an existing file or directory: /qa-fix <path/to/tests>
|
|
131
|
+
2. Run from your QA repo root and pass no path (it auto-detects the suite)
|
|
132
|
+
3. Check for a typo or an unexpanded ~ in the path
|
|
133
|
+
|
|
134
|
+
Try: ls {path}
|
|
135
|
+
```
|
|
136
|
+
- If NO path was passed and auto-detection found no test files, print (no `Try:` line — there's no path to show):
|
|
137
|
+
```
|
|
138
|
+
❌ No test files found to fix.
|
|
139
|
+
|
|
140
|
+
/qa-fix auto-detected from the current directory but found no test files.
|
|
141
|
+
|
|
142
|
+
Options:
|
|
143
|
+
1. Point at your tests explicitly: /qa-fix <path/to/tests>
|
|
144
|
+
2. Run from your QA repo root so auto-detection can find the suite
|
|
145
|
+
3. If you haven't generated tests yet, create them first: /qa-create-test <feature>
|
|
146
|
+
```
|
|
147
|
+
|
|
148
|
+
### Test runner can't execute [STOP]
|
|
149
|
+
If the test runner binary is missing or fails to launch (an infra failure, not a test failure):
|
|
150
|
+
```
|
|
151
|
+
❌ The test runner could not start.
|
|
152
|
+
|
|
153
|
+
The framework's runner (npx playwright / cypress / jest / vitest / pytest) is
|
|
154
|
+
missing, misconfigured, or its dependencies aren't installed — so tests never
|
|
155
|
+
ran. This is an ENVIRONMENT ISSUE, not a test failure.
|
|
156
|
+
|
|
157
|
+
Options:
|
|
158
|
+
1. Install dependencies in the repo first: npm install (or the project's equivalent)
|
|
159
|
+
2. Verify the runner works on its own before re-running /qa-fix
|
|
160
|
+
3. Run static validation only (no execution): /qa-fix --validate-only
|
|
161
|
+
|
|
162
|
+
Try: npx {runner} --version (to confirm the runner is installed)
|
|
163
|
+
```
|
|
164
|
+
|
|
165
|
+
### Playwright MCP not connected — E2E run requested [STOP]
|
|
166
|
+
If `--run` / `--app-url` was passed (E2E reproduction against the live app) but Playwright MCP is unavailable:
|
|
167
|
+
```
|
|
168
|
+
❌ Playwright MCP is not connected — E2E reproduction against the app needs it.
|
|
169
|
+
|
|
170
|
+
You asked to run E2E tests against the live app (--run / --app-url), which uses
|
|
171
|
+
Playwright MCP to reproduce failures in the browser. It isn't available.
|
|
172
|
+
|
|
173
|
+
Options:
|
|
174
|
+
1. Connect Playwright MCP (ships with QAA — check ~/.claude.json), then re-run
|
|
175
|
+
2. Reinstall to re-register the MCP servers: npx qaa-agent
|
|
176
|
+
3. Skip execution and only classify or validate: /qa-fix --classify (or --validate-only)
|
|
177
|
+
|
|
178
|
+
Try: /mcp
|
|
179
|
+
```
|
|
180
|
+
|
|
118
181
|
## Instructions
|
|
119
182
|
|
|
120
183
|
### Step 1: Detect Mode and Test Directory
|
|
@@ -396,9 +459,9 @@ The validator runs 4 layers per file:
|
|
|
396
459
|
3. **Dependencies** — all imports resolve
|
|
397
460
|
4. **Logic** — concrete assertions, Tier 1-2 locators, no assertions in POMs
|
|
398
461
|
|
|
399
|
-
**
|
|
462
|
+
**Conditional Layer 5 — labeled DOM-probe locator cross-check (runs ONLY if Playwright MCP connected + app URL available):**
|
|
400
463
|
|
|
401
|
-
This layer is a STATIC locator cross-check via DOM probe. It is NOT test execution and never reports that tests passed:
|
|
464
|
+
This layer is a STATIC locator cross-check via DOM probe. It is NOT test execution, MUST NEVER be substituted for a real test run, and never reports that tests passed:
|
|
402
465
|
|
|
403
466
|
```
|
|
404
467
|
[DOM_PROBE — NOT_TEST_EXECUTION]
|
|
@@ -429,6 +492,7 @@ Max 5 fix loop iterations. Produces VALIDATION_REPORT.md.
|
|
|
429
492
|
If `--run` flag is also present and E2E test files exist, invoke E2E runner after static validation:
|
|
430
493
|
|
|
431
494
|
Task(
|
|
495
|
+
subagent_type="qaa-e2e-runner",
|
|
432
496
|
prompt="
|
|
433
497
|
<critical_directive priority="MANDATORY">
|
|
434
498
|
You are executing as a QAA pipeline agent. Your FIRST action MUST be to read
|
|
@@ -490,6 +554,7 @@ Fix mode runs in two phases: **analyze first, then fix after user confirmation.*
|
|
|
490
554
|
2. Invoke bug-detective agent in **classify-only mode** (no auto-fixes):
|
|
491
555
|
|
|
492
556
|
Task(
|
|
557
|
+
subagent_type="qaa-bug-detective",
|
|
493
558
|
prompt="
|
|
494
559
|
<critical_directive priority="MANDATORY">
|
|
495
560
|
You are executing as a QAA pipeline agent. Your FIRST action MUST be to read
|
|
@@ -536,6 +601,8 @@ Task(
|
|
|
536
601
|
"
|
|
537
602
|
)
|
|
538
603
|
|
|
604
|
+
**HARD STOP on broken runner (#48):** If the bug-detective returns `runner_executed: false` / `runner_status: BROKEN` (the suite never executed — ENVIRONMENT ISSUE), do NOT present a FIX PLAN and do NOT proceed to Phase 2. Surface the ENVIRONMENT ISSUE with the runner's startup error and stop. A DOM probe is NEVER a substitute for a real run; a report claiming classifications with `runner_executed: false` is INVALID.
|
|
605
|
+
|
|
539
606
|
3. Present the analysis to the user as a **fix plan**:
|
|
540
607
|
|
|
541
608
|
```
|
|
@@ -564,7 +631,7 @@ Proceed with auto-fixes? [yes / modify / cancel]
|
|
|
564
631
|
================
|
|
565
632
|
```
|
|
566
633
|
|
|
567
|
-
4. **
|
|
634
|
+
4. **HARD STOP — Phase 1 → Phase 2 gate.** Do NOT enter Phase 2 (execute fixes) until the user EXPLICITLY approves the FIX PLAN. No approval = no fixes — this gate is mandatory and cannot be skipped, assumed, or auto-satisfied by silence. This is a refinement loop — repeat until the user is satisfied:
|
|
568
635
|
|
|
569
636
|
- **"yes"** / **"proceed"** / **"dale"** → continue to Phase 2
|
|
570
637
|
- **"cancel"** / **"no"** → stop, deliver only the FAILURE_CLASSIFICATION_REPORT.md (same as --classify)
|
|
@@ -582,9 +649,12 @@ Proceed with auto-fixes? [yes / modify / cancel]
|
|
|
582
649
|
|
|
583
650
|
#### Phase 2: Execute Fixes (only after user confirmation)
|
|
584
651
|
|
|
652
|
+
**Do NOT reach this phase without an explicit "yes" / "proceed" / "dale" from the Phase 1 → Phase 2 gate above.** If you have not received explicit approval, STOP — you are still in Phase 1.
|
|
653
|
+
|
|
585
654
|
5. Invoke bug-detective agent in **fix mode** with the confirmed plan:
|
|
586
655
|
|
|
587
656
|
Task(
|
|
657
|
+
subagent_type="qaa-bug-detective",
|
|
588
658
|
prompt="
|
|
589
659
|
<critical_directive priority="MANDATORY">
|
|
590
660
|
You are executing as a QAA pipeline agent. Your FIRST action MUST be to read
|
package/commands/qa-map.md
CHANGED
|
@@ -2,6 +2,40 @@
|
|
|
2
2
|
|
|
3
3
|
Deep-scan a codebase for QA-relevant information, produce a complete analysis, and generate test inventory. Runs codebase mapping (4 parallel agents) followed by full repository analysis. One command to fully understand a codebase before writing tests.
|
|
4
4
|
|
|
5
|
+
## ⚠ MANDATORY: How to Execute This Command
|
|
6
|
+
|
|
7
|
+
Execute these steps IN ORDER. Every step is mandatory. Do NOT improvise,
|
|
8
|
+
reorder, skip, or invent steps. This command is flag-driven — there is no intent
|
|
9
|
+
detector; read the flags from `$ARGUMENTS` literally.
|
|
10
|
+
|
|
11
|
+
1. Read `CLAUDE.md` (QA standards) before anything else.
|
|
12
|
+
2. Create the output directory: `mkdir -p .qa-output/codebase`.
|
|
13
|
+
3. STAGE 1 — Codebase Mapping (see "## Instructions" Step 3):
|
|
14
|
+
- If neither `--skip-map` nor `--focus` was passed: spawn ALL 4
|
|
15
|
+
codebase-mapper agents IN PARALLEL (testability, risk, patterns,
|
|
16
|
+
existing-tests), each via Task() with its `<critical_directive>` block.
|
|
17
|
+
- If `--focus <area>` was passed: spawn ONLY that one agent, then STOP
|
|
18
|
+
(skip Stage 2 entirely).
|
|
19
|
+
- If `--skip-map` was passed: skip Stage 1 and go straight to Stage 2.
|
|
20
|
+
4. Print the summary of documents produced by Stage 1.
|
|
21
|
+
5. STAGE 2 — Repository Analysis: initialize pipeline context, then invoke the
|
|
22
|
+
scanner agent, then the analyzer agent — IN THAT ORDER, each via Task() with
|
|
23
|
+
its `<critical_directive>` block. Do NOT start the analyzer before the
|
|
24
|
+
scanner has produced SCAN_MANIFEST.md.
|
|
25
|
+
6. Print the final summary of all documents produced across both stages.
|
|
26
|
+
|
|
27
|
+
### DO NOT
|
|
28
|
+
- Skip Stage 1 unless `--skip-map` was explicitly passed
|
|
29
|
+
- Run Stage 2 when `--focus` was passed (focus mode stops after Stage 1)
|
|
30
|
+
- Start the analyzer before the scanner completes
|
|
31
|
+
- Run any git operation or generate any test files — this command only maps and
|
|
32
|
+
analyzes
|
|
33
|
+
- Spawn the map/scan/analyze work inline instead of through the sub-agents
|
|
34
|
+
- Invent steps not in this command, or reorder the ones that are
|
|
35
|
+
|
|
36
|
+
If you find yourself improvising, reordering the stages, or generating tests —
|
|
37
|
+
STOP and restart from step 1.
|
|
38
|
+
|
|
5
39
|
## Usage
|
|
6
40
|
|
|
7
41
|
```
|
|
@@ -39,6 +73,42 @@ All documents written to `.qa-output/codebase/`.
|
|
|
39
73
|
| QA_REPO_BLUEPRINT.md | If no QA repo — full repo structure, configs, CI/CD strategy |
|
|
40
74
|
| GAP_ANALYSIS.md | If QA repo provided — coverage map, missing tests, broken tests, quality assessment |
|
|
41
75
|
|
|
76
|
+
## Pre-flight checks
|
|
77
|
+
|
|
78
|
+
Run these BEFORE the main flow. If a **[STOP]** check fails, print the error EXACTLY as shown (fill in the `{...}` placeholders) and STOP.
|
|
79
|
+
|
|
80
|
+
### Dev-repo path does not exist [STOP]
|
|
81
|
+
If the resolved `--dev-repo` path is not an existing directory:
|
|
82
|
+
```
|
|
83
|
+
❌ Dev-repo path not found: {path}
|
|
84
|
+
|
|
85
|
+
/qa-map scans a repository; the --dev-repo path you gave does not exist (or is
|
|
86
|
+
not a directory).
|
|
87
|
+
|
|
88
|
+
Options:
|
|
89
|
+
1. Re-run with a correct path: /qa-map --dev-repo <path>
|
|
90
|
+
2. Run from inside the repo with no --dev-repo (defaults to the current directory)
|
|
91
|
+
3. Check for a typo or an unexpanded ~ in the path
|
|
92
|
+
|
|
93
|
+
Try: ls {path}
|
|
94
|
+
```
|
|
95
|
+
|
|
96
|
+
### No framework or testable surfaces detected [STOP]
|
|
97
|
+
If the scan finds no framework and no source it can classify as a testable surface:
|
|
98
|
+
```
|
|
99
|
+
❌ Nothing testable detected in {dev-repo}.
|
|
100
|
+
|
|
101
|
+
The scan found no test framework and no source it can classify as a testable
|
|
102
|
+
surface, so there's nothing to map or analyze.
|
|
103
|
+
|
|
104
|
+
Options:
|
|
105
|
+
1. Point at the app sub-directory if this is a monorepo: /qa-map --dev-repo <path/to/app>
|
|
106
|
+
2. Add a framework config to the repo first (playwright/cypress/jest/vitest/pytest), then re-run
|
|
107
|
+
3. To analyze an existing test suite, pass a QA repo: /qa-map --qa-repo <path>
|
|
108
|
+
|
|
109
|
+
Try: ls {dev-repo} (to confirm you're pointing at the right directory)
|
|
110
|
+
```
|
|
111
|
+
|
|
42
112
|
## Instructions
|
|
43
113
|
|
|
44
114
|
1. Read `CLAUDE.md` — QA standards.
|
|
@@ -79,7 +149,7 @@ Agent(
|
|
|
79
149
|
trusted.
|
|
80
150
|
</critical_directive>
|
|
81
151
|
|
|
82
|
-
subagent_type="
|
|
152
|
+
subagent_type="qaa-codebase-mapper",
|
|
83
153
|
execution_context="~/.claude/qaa/agents/qaa-codebase-mapper.md"
|
|
84
154
|
)
|
|
85
155
|
|
|
@@ -109,7 +179,7 @@ Agent(
|
|
|
109
179
|
trusted.
|
|
110
180
|
</critical_directive>
|
|
111
181
|
|
|
112
|
-
subagent_type="
|
|
182
|
+
subagent_type="qaa-codebase-mapper",
|
|
113
183
|
execution_context="~/.claude/qaa/agents/qaa-codebase-mapper.md"
|
|
114
184
|
)
|
|
115
185
|
|
|
@@ -139,7 +209,7 @@ Agent(
|
|
|
139
209
|
trusted.
|
|
140
210
|
</critical_directive>
|
|
141
211
|
|
|
142
|
-
subagent_type="
|
|
212
|
+
subagent_type="qaa-codebase-mapper",
|
|
143
213
|
execution_context="~/.claude/qaa/agents/qaa-codebase-mapper.md"
|
|
144
214
|
)
|
|
145
215
|
|
|
@@ -169,7 +239,7 @@ Agent(
|
|
|
169
239
|
trusted.
|
|
170
240
|
</critical_directive>
|
|
171
241
|
|
|
172
|
-
subagent_type="
|
|
242
|
+
subagent_type="qaa-codebase-mapper",
|
|
173
243
|
execution_context="~/.claude/qaa/agents/qaa-codebase-mapper.md"
|
|
174
244
|
)
|
|
175
245
|
```
|
|
@@ -190,6 +260,7 @@ node bin/qaa-tools.cjs init qa-start 2>/dev/null || true
|
|
|
190
260
|
Invoke scanner agent:
|
|
191
261
|
|
|
192
262
|
Task(
|
|
263
|
+
subagent_type="qaa-scanner",
|
|
193
264
|
prompt="
|
|
194
265
|
<critical_directive priority="MANDATORY">
|
|
195
266
|
You are executing as a QAA pipeline agent. Your FIRST action MUST be to read
|
|
@@ -230,6 +301,7 @@ Task(
|
|
|
230
301
|
Invoke analyzer agent:
|
|
231
302
|
|
|
232
303
|
Task(
|
|
304
|
+
subagent_type="qaa-analyzer",
|
|
233
305
|
prompt="
|
|
234
306
|
<critical_directive priority="MANDATORY">
|
|
235
307
|
You are executing as a QAA pipeline agent. Your FIRST action MUST be to read
|
package/commands/qa-pr.md
CHANGED
|
@@ -75,6 +75,42 @@ TICKET=$(echo "$RESOLVED" | node -e "const j=JSON.parse(require('fs').readFileSy
|
|
|
75
75
|
|
|
76
76
|
If you see values you didn't intend in the banner above (interactive mode), you can press Ctrl-C and re-run with explicit flags.
|
|
77
77
|
|
|
78
|
+
## Pre-flight checks
|
|
79
|
+
|
|
80
|
+
Run these BEFORE the main flow. If a **[STOP]** check fails, print the error EXACTLY as shown and STOP.
|
|
81
|
+
|
|
82
|
+
### Not a git repository [STOP]
|
|
83
|
+
If the current directory is not inside a git repository:
|
|
84
|
+
```
|
|
85
|
+
❌ Not a git repository.
|
|
86
|
+
|
|
87
|
+
/qa-pr creates a branch, commits QA artifacts, and opens a pull request — all of
|
|
88
|
+
which need a git repo. The current directory isn't one.
|
|
89
|
+
|
|
90
|
+
Options:
|
|
91
|
+
1. Run from inside your QA repo (or the repo where the artifacts live)
|
|
92
|
+
2. Initialize git if this is a new repo: git init && git remote add origin <url>
|
|
93
|
+
3. If you just want the files locally, skip /qa-pr — the artifacts are already in .qa-output/
|
|
94
|
+
|
|
95
|
+
Try: git status (to confirm you're in a repo)
|
|
96
|
+
```
|
|
97
|
+
|
|
98
|
+
### GitHub CLI not available or not signed in [STOP]
|
|
99
|
+
If `gh` is not installed or `gh auth status` fails:
|
|
100
|
+
```
|
|
101
|
+
❌ GitHub CLI (gh) is not available or not signed in.
|
|
102
|
+
|
|
103
|
+
/qa-pr uses `gh` to open the draft pull request. It's either not installed or
|
|
104
|
+
not authenticated, so the PR step can't run.
|
|
105
|
+
|
|
106
|
+
Options:
|
|
107
|
+
1. Authenticate: gh auth login
|
|
108
|
+
2. Install the GitHub CLI first (https://cli.github.com), then re-run
|
|
109
|
+
3. Create the branch and commits locally, then open the PR by hand in your git host
|
|
110
|
+
|
|
111
|
+
Try: gh auth status
|
|
112
|
+
```
|
|
113
|
+
|
|
78
114
|
## Instructions
|
|
79
115
|
|
|
80
116
|
1. Read `CLAUDE.md` -- git workflow, commit conventions.
|
package/commands/qa-research.md
CHANGED
|
@@ -2,6 +2,36 @@
|
|
|
2
2
|
|
|
3
3
|
Research the testing ecosystem for the current project. Investigates framework capabilities, best practices, selector syntax, and testing patterns using Context7 MCP and official documentation. Produces research files consumed by all downstream QA agents (analyzer, planner, executor, validator).
|
|
4
4
|
|
|
5
|
+
## ⚠ MANDATORY: How to Execute This Command
|
|
6
|
+
|
|
7
|
+
Execute these steps IN ORDER. Every step is mandatory. Do NOT improvise,
|
|
8
|
+
reorder, skip, or invent steps. This command is flag-driven — there is no intent
|
|
9
|
+
detector; parse the flags from `$ARGUMENTS` literally.
|
|
10
|
+
|
|
11
|
+
1. STEP 1 — Initialize (see "## Instructions"): parse `$ARGUMENTS` for `--focus`
|
|
12
|
+
and `--dev-repo`, resolve MODE (default `stack-testing`) and DEV_REPO
|
|
13
|
+
(default: current directory), create `.qa-output/research/`, and print the
|
|
14
|
+
`=== QA Research ===` banner.
|
|
15
|
+
2. STEP 2 — Read existing context if present: `.qa-output/SCAN_MANIFEST.md`,
|
|
16
|
+
`CLAUDE.md`, `~/.claude/qaa/MY_PREFERENCES.md`.
|
|
17
|
+
3. STEP 3 — Spawn the project-researcher via Task() with its `<critical_directive>`
|
|
18
|
+
block, passing the resolved MODE and DEV_REPO. Do NOT do the research inline.
|
|
19
|
+
4. STEP 4 — After the researcher returns, print the `=== Research Complete ===`
|
|
20
|
+
summary.
|
|
21
|
+
5. Run the "## MANDATORY verification" checklist at the bottom of this file
|
|
22
|
+
before returning control.
|
|
23
|
+
|
|
24
|
+
### DO NOT
|
|
25
|
+
- Skip the `=== QA Research ===` banner
|
|
26
|
+
- Do the research yourself inline instead of spawning the project-researcher
|
|
27
|
+
- Invent a mode not in the `--focus` list, or reorder the steps
|
|
28
|
+
- Skip the bottom verification checklist
|
|
29
|
+
- Assume dev-repo path or mode from a previous command invocation — each
|
|
30
|
+
/qa-research call is self-contained
|
|
31
|
+
|
|
32
|
+
If you find yourself improvising, doing the research inline, or skipping the
|
|
33
|
+
verification checklist — STOP and restart from step 1.
|
|
34
|
+
|
|
5
35
|
## Usage
|
|
6
36
|
|
|
7
37
|
```
|
|
@@ -50,6 +80,42 @@ These files are consumed by downstream agents:
|
|
|
50
80
|
| `API_TESTING_STRATEGY.md` | Planner (API test case design), Executor (implementation patterns) |
|
|
51
81
|
| `E2E_STRATEGY.md` | Planner (E2E scope decisions), Executor (POM and selector patterns) |
|
|
52
82
|
|
|
83
|
+
## Pre-flight checks
|
|
84
|
+
|
|
85
|
+
Run these BEFORE the main flow. If a **[STOP]** check fails, print the error EXACTLY as shown (fill in the `{...}` placeholders) and STOP.
|
|
86
|
+
|
|
87
|
+
### Context7 MCP not connected [STOP]
|
|
88
|
+
If the Context7 MCP tools are unavailable:
|
|
89
|
+
```
|
|
90
|
+
❌ Context7 MCP is not connected — research needs it as the primary source.
|
|
91
|
+
|
|
92
|
+
/qa-research relies on Context7 to fetch current framework and library docs.
|
|
93
|
+
Without it, findings would fall back to training data and could be stale or wrong.
|
|
94
|
+
|
|
95
|
+
Options:
|
|
96
|
+
1. Connect Context7 (it ships with QAA — check that it registered in ~/.claude.json)
|
|
97
|
+
2. Reinstall to re-register the MCP servers: npx qaa-agent
|
|
98
|
+
3. Skip research for now — downstream agents fall back to cache/training data if no research docs exist
|
|
99
|
+
|
|
100
|
+
Try: /mcp (to see connected MCP servers)
|
|
101
|
+
```
|
|
102
|
+
|
|
103
|
+
### Dev-repo path does not exist [STOP]
|
|
104
|
+
If `--dev-repo` was passed but the path does not exist (default is the current directory):
|
|
105
|
+
```
|
|
106
|
+
❌ Dev-repo path not found: {path}
|
|
107
|
+
|
|
108
|
+
/qa-research reads the project's stack to focus its research. The --dev-repo path
|
|
109
|
+
you gave does not exist (or is not a directory).
|
|
110
|
+
|
|
111
|
+
Options:
|
|
112
|
+
1. Re-run with a correct path: /qa-research --dev-repo <path>
|
|
113
|
+
2. Run from inside the repo with no --dev-repo (defaults to the current directory)
|
|
114
|
+
3. Check for a typo or an unexpanded ~ in the path
|
|
115
|
+
|
|
116
|
+
Try: ls {path}
|
|
117
|
+
```
|
|
118
|
+
|
|
53
119
|
## Instructions
|
|
54
120
|
|
|
55
121
|
### Step 1: Initialize
|
|
@@ -95,6 +161,7 @@ Read these files if they exist — they provide context to the researcher:
|
|
|
95
161
|
### Step 3: Spawn Researcher Agent
|
|
96
162
|
|
|
97
163
|
Task(
|
|
164
|
+
subagent_type="qaa-project-researcher",
|
|
98
165
|
prompt="
|
|
99
166
|
<critical_directive priority="MANDATORY">
|
|
100
167
|
You are executing as a QAA pipeline agent. Your FIRST action MUST be to read
|
package/commands/qa-start.md
CHANGED
|
@@ -67,6 +67,43 @@ When the workflow's Step 1 invokes the intent detector with `$ARGUMENTS`, you mu
|
|
|
67
67
|
|
|
68
68
|
If you pre-translate, the detector sees only flags and reports every value as `source: flag`, which causes `ALL_FROM_FLAGS=true` and bypasses the confirmation checkpoint — defeating its purpose. Always pass raw input.
|
|
69
69
|
|
|
70
|
+
## Pre-flight checks
|
|
71
|
+
|
|
72
|
+
Run these BEFORE spawning any agent. A **[STOP]** check that fails: print the error EXACTLY as shown (fill in the `{...}` placeholders) and STOP. A **[WARN]** check that fails: print the notice and continue in degraded mode.
|
|
73
|
+
|
|
74
|
+
### Dev-repo path does not exist [STOP]
|
|
75
|
+
If the resolved `--dev-repo` path is not an existing directory:
|
|
76
|
+
```
|
|
77
|
+
❌ Dev-repo path not found: {path}
|
|
78
|
+
|
|
79
|
+
/qa-start needs a real directory to scan. The path resolved from your input
|
|
80
|
+
does not exist (or is not a directory).
|
|
81
|
+
|
|
82
|
+
Options:
|
|
83
|
+
1. Re-run with a correct path: /qa-start --dev-repo <path-to-your-repo>
|
|
84
|
+
2. Run from inside the repo with no path (defaults to the current directory)
|
|
85
|
+
3. Double-check the path — a typo or a ~ that did not expand is the usual cause
|
|
86
|
+
|
|
87
|
+
Try: ls {path}
|
|
88
|
+
```
|
|
89
|
+
|
|
90
|
+
### Framework not detectable [STOP]
|
|
91
|
+
If the scanner finds no test framework and no language/config it can map to one:
|
|
92
|
+
```
|
|
93
|
+
❌ Could not detect a test framework in {dev-repo}.
|
|
94
|
+
|
|
95
|
+
QAA looks for a framework via config files (playwright.config, cypress.config,
|
|
96
|
+
jest/vitest config, pytest.ini, etc.) and package manifests. None were found,
|
|
97
|
+
so it can't decide how to write tests.
|
|
98
|
+
|
|
99
|
+
Options:
|
|
100
|
+
1. Pass it explicitly: /qa-start --framework <playwright|cypress|jest|vitest|pytest|selenium|robot-framework>
|
|
101
|
+
2. Point at the app sub-directory if this is a monorepo: /qa-start --dev-repo <path/to/app>
|
|
102
|
+
3. Add a framework to the repo first (e.g. `npm i -D @playwright/test`), then re-run
|
|
103
|
+
|
|
104
|
+
Try: /qa-research (to investigate the stack before generating tests)
|
|
105
|
+
```
|
|
106
|
+
|
|
70
107
|
## Instructions
|
|
71
108
|
|
|
72
109
|
1. Read `CLAUDE.md` — all QA standards that govern the pipeline.
|
|
@@ -2,6 +2,41 @@
|
|
|
2
2
|
|
|
3
3
|
Generate a per-ticket QA execution summary and append it to the Azure DevOps work item's `Custom.QATestCasesReport` field. For each test case linked to the work item, pull the execution status from ADO Test Runs — or prompt the user when ADO has no run result — then render a bulleted "Tested Scenarios" list (with a separate "Failures" section when anything failed).
|
|
4
4
|
|
|
5
|
+
## ⚠ MANDATORY: How to Execute This Command
|
|
6
|
+
|
|
7
|
+
Execute these steps IN ORDER. Every step is mandatory. Do NOT improvise,
|
|
8
|
+
reorder, skip, or invent steps. This command reads Azure DevOps and writes ONLY
|
|
9
|
+
the `Custom.QATestCasesReport` field; it spawns no sub-agents and creates no
|
|
10
|
+
test cases.
|
|
11
|
+
|
|
12
|
+
1. PHASE 1 — Resolve the work item: if `$ARGUMENTS` has no work item ID, ask the
|
|
13
|
+
user before proceeding. Fetch it with `wit_get_work_item` (relations) plus
|
|
14
|
+
its comments.
|
|
15
|
+
2. PHASE 2 — Resolve the test-case list. ADO is authoritative: build the list
|
|
16
|
+
from the *Tested By* links; a local `test-cases.md` is only a hint. If no TCs
|
|
17
|
+
are linked, print "nothing to report" and STOP.
|
|
18
|
+
3. PHASE 3 — Resolve each TC's status (Passed/Failed/Blocked): try the ADO Test
|
|
19
|
+
Points REST endpoint first; for any TC that doesn't resolve, ask the user. For
|
|
20
|
+
Failed TCs, capture a reason (run data → comments → ask the user).
|
|
21
|
+
4. PHASE 4 — Synthesize a readable one-line scenario description per TC (not the
|
|
22
|
+
raw TC title).
|
|
23
|
+
5. PHASE 5 — Build BOTH the markdown (for chat) and HTML (for the field) reports.
|
|
24
|
+
Include the Failures section only if something failed.
|
|
25
|
+
6. PHASE 6 — APPEND the HTML report to `Custom.QATestCasesReport`: re-read the
|
|
26
|
+
current value and preserve it (existing + `<br><br>` + new). Update ONLY that
|
|
27
|
+
field — never overwrite others.
|
|
28
|
+
7. PHASE 7 — Print the markdown report + the append confirmation to the user.
|
|
29
|
+
|
|
30
|
+
### DO NOT
|
|
31
|
+
- OVERWRITE `Custom.QATestCasesReport` — always append, preserving prior content
|
|
32
|
+
- Write to any field other than `Custom.QATestCasesReport`
|
|
33
|
+
- Create test cases, change TC state, or create Test Runs (this command is read-mostly)
|
|
34
|
+
- Copy raw TC titles as scenario descriptions when they are terse
|
|
35
|
+
- Invent steps not in this command, or reorder the phases
|
|
36
|
+
|
|
37
|
+
If you find yourself overwriting the field, touching other fields, or creating or
|
|
38
|
+
modifying test cases — STOP and restart from step 1.
|
|
39
|
+
|
|
5
40
|
## Usage
|
|
6
41
|
|
|
7
42
|
```
|
|
@@ -22,6 +57,53 @@ Generate a per-ticket QA execution summary and append it to the Azure DevOps wor
|
|
|
22
57
|
|
|
23
58
|
---
|
|
24
59
|
|
|
60
|
+
## Pre-flight checks
|
|
61
|
+
|
|
62
|
+
Run these BEFORE the main flow. If a **[STOP]** check fails, print the error EXACTLY as shown (fill in the `{...}` placeholders) and STOP.
|
|
63
|
+
|
|
64
|
+
### Azure DevOps MCP not connected [STOP]
|
|
65
|
+
If the ADO MCP tools are unavailable:
|
|
66
|
+
```
|
|
67
|
+
❌ Azure DevOps MCP is not connected — this command reads from ADO.
|
|
68
|
+
|
|
69
|
+
/qa-test-report pulls test-case execution status from Azure DevOps via the ADO
|
|
70
|
+
MCP tools and REST API. They aren't available in this session.
|
|
71
|
+
|
|
72
|
+
Options:
|
|
73
|
+
1. Connect the Azure DevOps MCP server, then re-run
|
|
74
|
+
2. Verify it's configured in your Claude Code MCP settings
|
|
75
|
+
3. Confirm ADO_MCP_AUTH_TOKEN is set if you also need the REST fallback for run outcomes
|
|
76
|
+
|
|
77
|
+
Try: /mcp (to see connected MCP servers)
|
|
78
|
+
```
|
|
79
|
+
|
|
80
|
+
### Work item ID missing or not found [STOP]
|
|
81
|
+
- If NO work item ID was provided, print:
|
|
82
|
+
```
|
|
83
|
+
❌ No work item ID provided.
|
|
84
|
+
|
|
85
|
+
/qa-test-report needs an Azure DevOps work item ID to build the report from its
|
|
86
|
+
linked test cases.
|
|
87
|
+
|
|
88
|
+
Options:
|
|
89
|
+
1. Pass one: /qa-test-report <work-item-id>
|
|
90
|
+
2. Copy the numeric ID from the work item's ADO URL
|
|
91
|
+
3. Confirm you have access to that work item's project
|
|
92
|
+
|
|
93
|
+
Try: /qa-test-report 85508 (example)
|
|
94
|
+
```
|
|
95
|
+
- If an ID WAS provided but `wit_get_work_item` returns nothing for it, print:
|
|
96
|
+
```
|
|
97
|
+
❌ Work item not found: {id}
|
|
98
|
+
|
|
99
|
+
The ID you gave doesn't resolve to a work item you can access.
|
|
100
|
+
|
|
101
|
+
Options:
|
|
102
|
+
1. Double-check the numeric ID from the work item's ADO URL
|
|
103
|
+
2. Confirm you have access to that work item's project
|
|
104
|
+
3. Re-run with the correct ID: /qa-test-report <work-item-id>
|
|
105
|
+
```
|
|
106
|
+
|
|
25
107
|
## Process
|
|
26
108
|
|
|
27
109
|
### Phase 1 — Resolve the Work Item
|