qaa-agent 1.6.2 → 1.7.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.mcp.json +8 -8
- package/CHANGELOG.md +93 -71
- package/CLAUDE.md +553 -553
- package/agents/qa-pipeline-orchestrator.md +1378 -1378
- package/agents/qaa-analyzer.md +539 -524
- package/agents/qaa-bug-detective.md +479 -446
- package/agents/qaa-codebase-mapper.md +935 -935
- package/agents/qaa-discovery.md +384 -0
- package/agents/qaa-e2e-runner.md +416 -415
- package/agents/qaa-executor.md +651 -651
- package/agents/qaa-planner.md +405 -390
- package/agents/qaa-project-researcher.md +319 -319
- package/agents/qaa-scanner.md +424 -424
- package/agents/qaa-testid-injector.md +643 -585
- package/agents/qaa-validator.md +490 -452
- package/bin/install.cjs +200 -198
- package/bin/lib/commands.cjs +709 -709
- package/bin/lib/config.cjs +307 -307
- package/bin/lib/core.cjs +497 -497
- package/bin/lib/frontmatter.cjs +299 -299
- package/bin/lib/init.cjs +989 -989
- package/bin/lib/milestone.cjs +241 -241
- package/bin/lib/model-profiles.cjs +60 -60
- package/bin/lib/phase.cjs +911 -911
- package/bin/lib/roadmap.cjs +306 -306
- package/bin/lib/state.cjs +748 -748
- package/bin/lib/template.cjs +222 -222
- package/bin/lib/verify.cjs +842 -842
- package/bin/qaa-tools.cjs +607 -607
- package/commands/qa-audit.md +119 -0
- package/commands/qa-create-test.md +288 -0
- package/commands/qa-fix.md +147 -0
- package/commands/qa-map.md +137 -0
- package/{.claude/commands → commands}/qa-pr.md +23 -23
- package/{.claude/commands → commands}/qa-start.md +22 -22
- package/{.claude/commands → commands}/qa-testid.md +19 -19
- package/docs/COMMANDS.md +341 -341
- package/docs/DEMO.md +182 -182
- package/docs/TESTING.md +156 -156
- package/package.json +6 -7
- package/{.claude/settings.json → settings.json} +1 -2
- package/templates/failure-classification.md +391 -391
- package/templates/gap-analysis.md +409 -409
- package/templates/pr-template.md +48 -48
- package/templates/qa-analysis.md +381 -381
- package/templates/qa-audit-report.md +465 -465
- package/templates/qa-repo-blueprint.md +636 -636
- package/templates/scan-manifest.md +312 -312
- package/templates/test-inventory.md +582 -582
- package/templates/testid-audit-report.md +354 -354
- package/templates/validation-report.md +243 -243
- package/workflows/qa-analyze.md +296 -296
- package/workflows/qa-from-ticket.md +536 -536
- package/workflows/qa-gap.md +309 -303
- package/workflows/qa-pr.md +389 -389
- package/workflows/qa-start.md +1192 -1168
- package/workflows/qa-testid.md +384 -356
- package/workflows/qa-validate.md +299 -295
- package/.claude/commands/create-test.md +0 -164
- package/.claude/commands/qa-audit.md +0 -37
- package/.claude/commands/qa-blueprint.md +0 -54
- package/.claude/commands/qa-fix.md +0 -36
- package/.claude/commands/qa-from-ticket.md +0 -24
- package/.claude/commands/qa-gap.md +0 -20
- package/.claude/commands/qa-map.md +0 -47
- package/.claude/commands/qa-pom.md +0 -36
- package/.claude/commands/qa-pyramid.md +0 -37
- package/.claude/commands/qa-report.md +0 -38
- package/.claude/commands/qa-research.md +0 -33
- package/.claude/commands/qa-validate.md +0 -42
- package/.claude/commands/update-test.md +0 -58
- package/.claude/skills/qa-learner/SKILL.md +0 -150
- /package/{.claude/skills → skills}/qa-bug-detective/SKILL.md +0 -0
- /package/{.claude/skills → skills}/qa-repo-analyzer/SKILL.md +0 -0
- /package/{.claude/skills → skills}/qa-self-validator/SKILL.md +0 -0
- /package/{.claude/skills → skills}/qa-template-engine/SKILL.md +0 -0
- /package/{.claude/skills → skills}/qa-testid-injector/SKILL.md +0 -0
- /package/{.claude/skills → skills}/qa-workflow-documenter/SKILL.md +0 -0
|
@@ -0,0 +1,119 @@
|
|
|
1
|
+
# QA Audit & Report
|
|
2
|
+
|
|
3
|
+
Comprehensive quality audit of a test suite with 6-dimension scoring, testing pyramid analysis, and status reporting. Supports three output modes: full audit, pyramid analysis only, or status report adapted to audience.
|
|
4
|
+
|
|
5
|
+
## Usage
|
|
6
|
+
|
|
7
|
+
```
|
|
8
|
+
/qa-audit <path-to-tests> [options]
|
|
9
|
+
```
|
|
10
|
+
|
|
11
|
+
### Options
|
|
12
|
+
|
|
13
|
+
- `<path-to-tests>` — directory containing test files to audit
|
|
14
|
+
- `--dev-repo <path>` — path to developer repository (for coverage cross-reference)
|
|
15
|
+
- `--app-url <url>` — URL of running application for locator verification via Playwright MCP
|
|
16
|
+
- `--pyramid` — pyramid analysis only: compare actual vs target distribution with action plan
|
|
17
|
+
- `--report [team|management|client]` — generate status report adapted to audience (default: team)
|
|
18
|
+
|
|
19
|
+
### Mode Detection
|
|
20
|
+
|
|
21
|
+
```
|
|
22
|
+
if --pyramid:
|
|
23
|
+
MODE = "pyramid" → PYRAMID_ANALYSIS.md
|
|
24
|
+
elif --report:
|
|
25
|
+
MODE = "report" → QA_STATUS_REPORT.md (adapted to audience)
|
|
26
|
+
else:
|
|
27
|
+
MODE = "audit" → QA_AUDIT_REPORT.md (full 6-dimension audit, default)
|
|
28
|
+
```
|
|
29
|
+
|
|
30
|
+
## What It Produces
|
|
31
|
+
|
|
32
|
+
| Mode | Artifact | Description |
|
|
33
|
+
|------|----------|-------------|
|
|
34
|
+
| audit | QA_AUDIT_REPORT.md | 6-dimension scoring, critical issues, recommendations with effort estimates |
|
|
35
|
+
| pyramid | PYRAMID_ANALYSIS.md | Current vs target distribution, gap table, prioritized action plan |
|
|
36
|
+
| report | QA_STATUS_REPORT.md | Metrics, pyramid distribution, risk areas, adapted to audience level |
|
|
37
|
+
|
|
38
|
+
## Instructions
|
|
39
|
+
|
|
40
|
+
### AUDIT MODE (default)
|
|
41
|
+
|
|
42
|
+
Scores across 6 dimensions: Locator Quality (20%), Assertion Specificity (20%), POM Compliance (15%), Test Coverage (20%), Naming Convention (15%), Test Data Management (10%).
|
|
43
|
+
|
|
44
|
+
1. Read `CLAUDE.md` — quality gates, locator tiers, assertion rules, POM rules, naming conventions.
|
|
45
|
+
2. Invoke validator agent in audit mode:
|
|
46
|
+
|
|
47
|
+
Task(
|
|
48
|
+
prompt="
|
|
49
|
+
<objective>Audit test suite quality and produce QA_AUDIT_REPORT.md with 6-dimension scoring. If Playwright MCP is connected and an app URL is available, verify E2E test locators against the live DOM via browser_navigate + browser_snapshot. Flag stale locators (Tier 4 CSS/XPath that could be upgraded to Tier 1 data-testid) and locators that no longer match any DOM element.</objective>
|
|
50
|
+
<execution_context>@agents/qaa-validator.md</execution_context>
|
|
51
|
+
<files_to_read>
|
|
52
|
+
- CLAUDE.md
|
|
53
|
+
</files_to_read>
|
|
54
|
+
<parameters>
|
|
55
|
+
user_input: $ARGUMENTS
|
|
56
|
+
mode: audit
|
|
57
|
+
app_url: {auto-detect from test config baseURL, or ask user}
|
|
58
|
+
</parameters>
|
|
59
|
+
"
|
|
60
|
+
)
|
|
61
|
+
|
|
62
|
+
3. Present results with overall score and prioritized recommendations.
|
|
63
|
+
|
|
64
|
+
---
|
|
65
|
+
|
|
66
|
+
### PYRAMID MODE (`--pyramid`)
|
|
67
|
+
|
|
68
|
+
Analyze test distribution against the ideal testing pyramid from CLAUDE.md (Unit 60-70%, Integration 10-15%, API 20-25%, E2E 3-5%). Compares actual percentages to targets and produces an action plan.
|
|
69
|
+
|
|
70
|
+
1. Read `CLAUDE.md` — testing pyramid target percentages.
|
|
71
|
+
2. Invoke analyzer agent for pyramid analysis:
|
|
72
|
+
|
|
73
|
+
Task(
|
|
74
|
+
prompt="
|
|
75
|
+
<objective>Produce PYRAMID_ANALYSIS.md comparing actual test distribution to target pyramid. Count tests by type (unit, integration, API, E2E), calculate percentages, compare to CLAUDE.md targets, identify gaps, and produce a prioritized action plan to reach the recommended distribution. Adjust target percentages based on the actual app architecture.</objective>
|
|
76
|
+
<execution_context>@agents/qaa-analyzer.md</execution_context>
|
|
77
|
+
<files_to_read>
|
|
78
|
+
- CLAUDE.md
|
|
79
|
+
</files_to_read>
|
|
80
|
+
<parameters>
|
|
81
|
+
user_input: $ARGUMENTS
|
|
82
|
+
mode: pyramid-analysis
|
|
83
|
+
</parameters>
|
|
84
|
+
"
|
|
85
|
+
)
|
|
86
|
+
|
|
87
|
+
3. Present analysis with gap table and action plan.
|
|
88
|
+
|
|
89
|
+
---
|
|
90
|
+
|
|
91
|
+
### REPORT MODE (`--report`)
|
|
92
|
+
|
|
93
|
+
Generate a summary report of current QA status. Adapts detail level to audience.
|
|
94
|
+
|
|
95
|
+
**Audience levels:**
|
|
96
|
+
- `team` (default) — file-level details, specific locator/assertion issues, technical recommendations
|
|
97
|
+
- `management` — high-level metrics, risk areas, coverage percentages, trend indicators
|
|
98
|
+
- `client` — coverage summary, confidence level, test pass rates, risk mitigation status
|
|
99
|
+
|
|
100
|
+
1. Read `CLAUDE.md` — testing pyramid targets, quality gates.
|
|
101
|
+
2. Invoke analyzer agent for status reporting:
|
|
102
|
+
|
|
103
|
+
Task(
|
|
104
|
+
prompt="
|
|
105
|
+
<objective>Produce QA_STATUS_REPORT.md with current test suite metrics and coverage. Adapt detail level to the specified audience: team (file-level technical details), management (high-level metrics and risks), or client (coverage summary and confidence). Include testing pyramid distribution, pass/fail rates, risk areas, and actionable recommendations appropriate for the audience.</objective>
|
|
106
|
+
<execution_context>@agents/qaa-analyzer.md</execution_context>
|
|
107
|
+
<files_to_read>
|
|
108
|
+
- CLAUDE.md
|
|
109
|
+
</files_to_read>
|
|
110
|
+
<parameters>
|
|
111
|
+
user_input: $ARGUMENTS
|
|
112
|
+
mode: status-report
|
|
113
|
+
</parameters>
|
|
114
|
+
"
|
|
115
|
+
)
|
|
116
|
+
|
|
117
|
+
3. Present report to user.
|
|
118
|
+
|
|
119
|
+
$ARGUMENTS
|
|
@@ -0,0 +1,288 @@
|
|
|
1
|
+
# QA Create Test
|
|
2
|
+
|
|
3
|
+
Create, update, or generate tests from tickets — all in one command. Supports three modes: generate tests from code analysis, generate tests from a ticket (Jira/Linear/GitHub), or update/improve existing tests. Uses Playwright MCP to extract real locators from the live app when available.
|
|
4
|
+
|
|
5
|
+
## Usage
|
|
6
|
+
|
|
7
|
+
```
|
|
8
|
+
/qa-create-test <feature-or-source> [options]
|
|
9
|
+
```
|
|
10
|
+
|
|
11
|
+
### Modes (auto-detected from arguments)
|
|
12
|
+
|
|
13
|
+
| Mode | Trigger | Example |
|
|
14
|
+
|------|---------|---------|
|
|
15
|
+
| **From code** | Feature name (no URL, no path to tests) | `/qa-create-test login` |
|
|
16
|
+
| **From ticket** | URL, shorthand (#123), or `--ticket` flag | `/qa-create-test https://github.com/org/repo/issues/42` |
|
|
17
|
+
| **Update existing** | Path to existing test files or `--update` flag | `/qa-create-test --update tests/e2e/` |
|
|
18
|
+
| **POM only** | `--pom-only` flag | `/qa-create-test --pom-only src/pages/` |
|
|
19
|
+
|
|
20
|
+
### Options
|
|
21
|
+
|
|
22
|
+
- `--dev-repo <path>` — path to developer repository (default: current directory)
|
|
23
|
+
- `--app-url <url>` — URL of running application for E2E execution and locator extraction (auto-detects if not provided)
|
|
24
|
+
- `--skip-run` — skip E2E execution, only generate and statically validate
|
|
25
|
+
- `--ticket <source>` — force ticket mode with: URL, shorthand (#123, org/repo#123), file path, or plain text
|
|
26
|
+
- `--update <path>` — force update mode: audit and improve existing tests at path
|
|
27
|
+
- `--scope fix|improve|add|full` — for update mode only (default: full)
|
|
28
|
+
- `--pom-only [path]` — generate only Page Object Model files (BasePage + feature POMs), no test specs
|
|
29
|
+
- `--framework <name>` — override framework auto-detection (playwright, cypress, selenium) — used with --pom-only
|
|
30
|
+
|
|
31
|
+
### Mode Detection Logic
|
|
32
|
+
|
|
33
|
+
```
|
|
34
|
+
if --pom-only:
|
|
35
|
+
MODE = "pom-only"
|
|
36
|
+
elif argument matches URL pattern ...
|
|
37
|
+
if argument matches URL pattern (github.com, atlassian.net, linear.app) OR contains "#" + digits OR --ticket flag:
|
|
38
|
+
MODE = "from-ticket"
|
|
39
|
+
elif --update flag OR argument is path to existing test directory/files:
|
|
40
|
+
MODE = "update"
|
|
41
|
+
else:
|
|
42
|
+
MODE = "from-code"
|
|
43
|
+
```
|
|
44
|
+
|
|
45
|
+
## What It Produces
|
|
46
|
+
|
|
47
|
+
### From Code Mode
|
|
48
|
+
- Test spec files (unit, API, E2E as appropriate)
|
|
49
|
+
- Page Object Model files (for E2E tests)
|
|
50
|
+
- Fixture files (test data)
|
|
51
|
+
- Locator registry entries (`.qa-output/locators/`)
|
|
52
|
+
- E2E_RUN_REPORT.md (if E2E tests ran against live app)
|
|
53
|
+
|
|
54
|
+
### From Ticket Mode
|
|
55
|
+
- TEST_CASES_FROM_TICKET.md — traceability matrix (AC → test case)
|
|
56
|
+
- GENERATION_PLAN_TICKET.md — synthetic generation plan
|
|
57
|
+
- Test spec files with `traces_to` fields linking back to ticket ACs
|
|
58
|
+
- VALIDATION_REPORT.md
|
|
59
|
+
|
|
60
|
+
### Update Mode
|
|
61
|
+
- QA_AUDIT_REPORT.md — current quality assessment
|
|
62
|
+
- Improved test files (after user approval)
|
|
63
|
+
|
|
64
|
+
## Instructions
|
|
65
|
+
|
|
66
|
+
### Step 1: Detect Mode
|
|
67
|
+
|
|
68
|
+
Parse `$ARGUMENTS` to determine mode using the detection logic above.
|
|
69
|
+
|
|
70
|
+
Print mode banner:
|
|
71
|
+
```
|
|
72
|
+
=== QA Create Test ===
|
|
73
|
+
Mode: {from-code | from-ticket | update}
|
|
74
|
+
Target: {feature name | ticket URL | test path}
|
|
75
|
+
App URL: {url or "auto-detect"}
|
|
76
|
+
===========================
|
|
77
|
+
```
|
|
78
|
+
|
|
79
|
+
---
|
|
80
|
+
|
|
81
|
+
### FROM CODE MODE
|
|
82
|
+
|
|
83
|
+
1. Read `CLAUDE.md` — POM rules, locator tiers, assertion rules, naming conventions, quality gates.
|
|
84
|
+
2. Read existing analysis artifacts if available:
|
|
85
|
+
- `.qa-output/QA_ANALYSIS.md` — architecture context
|
|
86
|
+
- `.qa-output/TEST_INVENTORY.md` — pre-defined test cases for this feature
|
|
87
|
+
3. **Check for codebase map** (`.qa-output/codebase/`):
|
|
88
|
+
- Look for: `CODE_PATTERNS.md`, `API_CONTRACTS.md`, `TEST_SURFACE.md`, `TESTABILITY.md`
|
|
89
|
+
- If at least 2 of these files exist: read them all for project context (naming conventions, API shapes, testable surfaces).
|
|
90
|
+
- **If NONE of these files exist: STOP and tell the user:**
|
|
91
|
+
```
|
|
92
|
+
⚠ No codebase map found (.qa-output/codebase/ is empty or missing).
|
|
93
|
+
|
|
94
|
+
The codebase map provides critical context: naming conventions, API contracts,
|
|
95
|
+
testable surfaces, and project structure. Without it, generated tests will lack
|
|
96
|
+
project-specific context and may not follow your repo's conventions.
|
|
97
|
+
|
|
98
|
+
Run /qa-map first to generate the codebase map, then re-run /qa-create-test.
|
|
99
|
+
|
|
100
|
+
To skip this check and proceed without context: re-run with --skip-map
|
|
101
|
+
```
|
|
102
|
+
Only proceed without codebase map if the user explicitly passes `--skip-map`.
|
|
103
|
+
|
|
104
|
+
4. **Check existing locator registry and extract new locators from live app:**
|
|
105
|
+
|
|
106
|
+
a. Read `.qa-output/locators/LOCATOR_REGISTRY.md` if it exists.
|
|
107
|
+
|
|
108
|
+
b. If locators for this feature already exist in the registry AND no `--app-url` was provided: reuse cached locators.
|
|
109
|
+
|
|
110
|
+
c. If locators are missing or `--app-url` was provided: Use Playwright MCP to navigate the app and extract real locators:
|
|
111
|
+
```
|
|
112
|
+
mcp__playwright__browser_navigate({ url: "{app_url}/{feature_path}" })
|
|
113
|
+
mcp__playwright__browser_snapshot()
|
|
114
|
+
```
|
|
115
|
+
Extract all data-testid, ARIA roles, labels, placeholders.
|
|
116
|
+
Navigate through multi-page flows if needed.
|
|
117
|
+
Write per-feature locator file to `.qa-output/locators/{feature}.locators.md`.
|
|
118
|
+
Update the registry `.qa-output/locators/LOCATOR_REGISTRY.md`.
|
|
119
|
+
|
|
120
|
+
If no app URL available and no locators in registry, skip — executor proposes locators from source code.
|
|
121
|
+
|
|
122
|
+
5. Invoke executor agent to generate test files:
|
|
123
|
+
|
|
124
|
+
Task(
|
|
125
|
+
prompt="
|
|
126
|
+
<objective>Generate test files for the specified feature following CLAUDE.md standards, using codebase map for context</objective>
|
|
127
|
+
<execution_context>@agents/qaa-executor.md</execution_context>
|
|
128
|
+
<files_to_read>
|
|
129
|
+
- CLAUDE.md
|
|
130
|
+
- .qa-output/locators/LOCATOR_REGISTRY.md (if exists)
|
|
131
|
+
- .qa-output/locators/{feature}.locators.md (if exists)
|
|
132
|
+
- .qa-output/codebase/CODE_PATTERNS.md (if exists)
|
|
133
|
+
- .qa-output/codebase/API_CONTRACTS.md (if exists)
|
|
134
|
+
- .qa-output/codebase/TEST_SURFACE.md (if exists)
|
|
135
|
+
- .qa-output/codebase/TESTABILITY.md (if exists)
|
|
136
|
+
</files_to_read>
|
|
137
|
+
<parameters>
|
|
138
|
+
user_input: $ARGUMENTS
|
|
139
|
+
mode: feature-test
|
|
140
|
+
codebase_map_dir: .qa-output/codebase
|
|
141
|
+
locator_registry: .qa-output/locators/LOCATOR_REGISTRY.md
|
|
142
|
+
</parameters>
|
|
143
|
+
"
|
|
144
|
+
)
|
|
145
|
+
|
|
146
|
+
6. If E2E test files were generated AND `--skip-run` was NOT passed, invoke E2E runner:
|
|
147
|
+
|
|
148
|
+
Task(
|
|
149
|
+
prompt="
|
|
150
|
+
<objective>Run generated E2E tests against live application, capture real locators, fix mismatches, loop until pass</objective>
|
|
151
|
+
<execution_context>@agents/qaa-e2e-runner.md</execution_context>
|
|
152
|
+
<files_to_read>
|
|
153
|
+
- CLAUDE.md
|
|
154
|
+
- {generated E2E test files from executor return}
|
|
155
|
+
- {generated POM files from executor return}
|
|
156
|
+
</files_to_read>
|
|
157
|
+
<parameters>
|
|
158
|
+
app_url: {from --app-url flag or auto-detect}
|
|
159
|
+
output_dir: .qa-output
|
|
160
|
+
</parameters>
|
|
161
|
+
"
|
|
162
|
+
)
|
|
163
|
+
|
|
164
|
+
7. Present results with file counts and suggest `/qa-pr`.
|
|
165
|
+
|
|
166
|
+
---
|
|
167
|
+
|
|
168
|
+
### FROM TICKET MODE
|
|
169
|
+
|
|
170
|
+
1. Read `CLAUDE.md` — all QA standards.
|
|
171
|
+
2. Execute the ticket workflow end-to-end:
|
|
172
|
+
|
|
173
|
+
Follow the workflow defined in `@workflows/qa-from-ticket.md` end-to-end.
|
|
174
|
+
Preserve all workflow gates (ticket parsing, acceptance criteria extraction, traceability matrix, validation).
|
|
175
|
+
|
|
176
|
+
Key steps in the workflow:
|
|
177
|
+
- Parse ticket source (GitHub URL, Jira URL, Linear URL, file, or plain text)
|
|
178
|
+
- Fetch ticket content (via `gh issue view`, WebFetch, or file read)
|
|
179
|
+
- Extract acceptance criteria, user stories, edge cases
|
|
180
|
+
- Scan dev repo for related source files
|
|
181
|
+
- Extract locators from live app via Playwright MCP (if app URL available)
|
|
182
|
+
- Generate test cases with traceability matrix (every AC maps to ≥1 test case)
|
|
183
|
+
- Spawn executor agent to produce test files
|
|
184
|
+
- Spawn validator agent for 4-layer validation
|
|
185
|
+
- Print summary with AC coverage and traceability
|
|
186
|
+
|
|
187
|
+
**Traceability guarantee:** Every acceptance criterion maps to at least one test case via the `traces_to` field.
|
|
188
|
+
|
|
189
|
+
**Supported ticket sources:**
|
|
190
|
+
|
|
191
|
+
| Format | Example | Detection |
|
|
192
|
+
|--------|---------|-----------|
|
|
193
|
+
| GitHub Issue URL | `https://github.com/org/repo/issues/123` | Contains `github.com` + `/issues/` |
|
|
194
|
+
| GitHub shorthand | `org/repo#123` or `#123` | Contains `#` + digits |
|
|
195
|
+
| Jira URL | `https://company.atlassian.net/browse/PROJ-123` | Contains `.atlassian.net/browse/` |
|
|
196
|
+
| Linear URL | `https://linear.app/team/issue/TEAM-123` | Contains `linear.app` |
|
|
197
|
+
| File path | `./tickets/feature-spec.md` | Path exists on disk |
|
|
198
|
+
| Plain text | `"As a user I want to..."` | None of the above match |
|
|
199
|
+
|
|
200
|
+
---
|
|
201
|
+
|
|
202
|
+
### UPDATE MODE
|
|
203
|
+
|
|
204
|
+
1. Read `CLAUDE.md` — quality gates, locator tiers, assertion rules, POM rules.
|
|
205
|
+
2. Invoke validator agent in audit mode:
|
|
206
|
+
|
|
207
|
+
Task(
|
|
208
|
+
prompt="
|
|
209
|
+
<objective>Audit existing test quality and produce QA_AUDIT_REPORT.md. If Playwright MCP is connected, verify E2E test locators against the live DOM via browser_navigate + browser_snapshot.</objective>
|
|
210
|
+
<execution_context>@agents/qaa-validator.md</execution_context>
|
|
211
|
+
<files_to_read>
|
|
212
|
+
- CLAUDE.md
|
|
213
|
+
</files_to_read>
|
|
214
|
+
<parameters>
|
|
215
|
+
user_input: $ARGUMENTS
|
|
216
|
+
mode: audit
|
|
217
|
+
app_url: {auto-detect from test config baseURL, or ask user}
|
|
218
|
+
</parameters>
|
|
219
|
+
"
|
|
220
|
+
)
|
|
221
|
+
|
|
222
|
+
3. Present audit results and wait for user approval.
|
|
223
|
+
4. Invoke executor agent to apply approved improvements:
|
|
224
|
+
|
|
225
|
+
Task(
|
|
226
|
+
prompt="
|
|
227
|
+
<objective>Apply approved improvements to existing tests without deleting working tests. If Playwright MCP is connected, use browser_navigate + browser_snapshot to extract real locators when upgrading from Tier 4 to Tier 1.</objective>
|
|
228
|
+
<execution_context>@agents/qaa-executor.md</execution_context>
|
|
229
|
+
<files_to_read>
|
|
230
|
+
- CLAUDE.md
|
|
231
|
+
- .qa-output/QA_AUDIT_REPORT.md
|
|
232
|
+
- .qa-output/locators/LOCATOR_REGISTRY.md (if exists)
|
|
233
|
+
</files_to_read>
|
|
234
|
+
<parameters>
|
|
235
|
+
user_input: $ARGUMENTS
|
|
236
|
+
mode: update
|
|
237
|
+
app_url: {auto-detect from test config baseURL, or ask user}
|
|
238
|
+
</parameters>
|
|
239
|
+
"
|
|
240
|
+
)
|
|
241
|
+
|
|
242
|
+
**Update scopes:**
|
|
243
|
+
- `fix` — repair broken tests only
|
|
244
|
+
- `improve` — upgrade locators, assertions, POM structure
|
|
245
|
+
- `add` — add missing test cases without modifying existing
|
|
246
|
+
- `full` — audit everything, then improve with approval (default)
|
|
247
|
+
|
|
248
|
+
**Rule:** NEVER delete or rewrite working tests without user approval. Surgical: add, fix, improve — never replace.
|
|
249
|
+
|
|
250
|
+
---
|
|
251
|
+
|
|
252
|
+
### POM ONLY MODE (`--pom-only`)
|
|
253
|
+
|
|
254
|
+
Generate only Page Object Model files — no test specs.
|
|
255
|
+
|
|
256
|
+
1. Read `CLAUDE.md` — POM rules, locator tier hierarchy, naming conventions.
|
|
257
|
+
2. Invoke executor agent in POM-only mode:
|
|
258
|
+
|
|
259
|
+
Task(
|
|
260
|
+
prompt="
|
|
261
|
+
<objective>Generate Page Object Models following CLAUDE.md POM rules. If Playwright MCP is connected and an app URL is available, navigate each page first to extract real locators (data-testid, ARIA roles, labels) from the live DOM via browser_navigate + browser_snapshot before generating POMs. This ensures POM locators match the real app instead of guessing from source code.</objective>
|
|
262
|
+
<execution_context>@agents/qaa-executor.md</execution_context>
|
|
263
|
+
<files_to_read>
|
|
264
|
+
- CLAUDE.md
|
|
265
|
+
- .qa-output/locators/LOCATOR_REGISTRY.md (if exists)
|
|
266
|
+
</files_to_read>
|
|
267
|
+
<parameters>
|
|
268
|
+
user_input: $ARGUMENTS
|
|
269
|
+
mode: pom-only
|
|
270
|
+
app_url: {auto-detect from test config baseURL, or ask user}
|
|
271
|
+
</parameters>
|
|
272
|
+
"
|
|
273
|
+
)
|
|
274
|
+
|
|
275
|
+
**Produces:**
|
|
276
|
+
- BasePage file (if not already present)
|
|
277
|
+
- Feature-specific POM files following `[PageName]Page.[ext]` naming convention
|
|
278
|
+
- No test specs, no fixtures
|
|
279
|
+
|
|
280
|
+
**POM rules enforced:**
|
|
281
|
+
- One class per page — no god objects
|
|
282
|
+
- No assertions in page objects — assertions belong ONLY in test specs
|
|
283
|
+
- Locators as readonly properties — Tier 1 preferred (data-testid, ARIA roles)
|
|
284
|
+
- Actions return void or next page — for fluent chaining
|
|
285
|
+
- State queries return data — let the test decide what to assert
|
|
286
|
+
- Every POM extends BasePage
|
|
287
|
+
|
|
288
|
+
$ARGUMENTS
|
|
@@ -0,0 +1,147 @@
|
|
|
1
|
+
# QA Fix & Validate
|
|
2
|
+
|
|
3
|
+
Validate, diagnose, and fix test files — all in one command. Runs 4-layer static validation (syntax, structure, dependencies, logic), classifies failures, and auto-fixes TEST CODE ERRORS. Uses Playwright MCP to reproduce E2E failures and verify locators against the live app when available. Never touches application code.
|
|
4
|
+
|
|
5
|
+
## Usage
|
|
6
|
+
|
|
7
|
+
```
|
|
8
|
+
/qa-fix [<test-directory>] [options]
|
|
9
|
+
```
|
|
10
|
+
|
|
11
|
+
### Options
|
|
12
|
+
|
|
13
|
+
- `<test-directory>` — path to test files (auto-detects if omitted)
|
|
14
|
+
- `--validate-only` — run 4-layer static validation only, no test execution or classification
|
|
15
|
+
- `--classify` — run tests and classify failures, but do NOT auto-fix
|
|
16
|
+
- `--run --app-url <url>` — also execute E2E tests against live app after static validation
|
|
17
|
+
- `--app-url <url>` — URL of running application (auto-detects if not provided)
|
|
18
|
+
- `[error output]` — paste test runner output directly (skips running tests, classifies the pasted output)
|
|
19
|
+
|
|
20
|
+
### Mode Detection
|
|
21
|
+
|
|
22
|
+
```
|
|
23
|
+
if --validate-only:
|
|
24
|
+
MODE = "validate" → 4-layer static validation + VALIDATION_REPORT.md
|
|
25
|
+
elif --classify:
|
|
26
|
+
MODE = "classify" → run tests + classify failures (no auto-fix)
|
|
27
|
+
else:
|
|
28
|
+
MODE = "fix" → run tests + classify + auto-fix TEST CODE ERRORS (default)
|
|
29
|
+
```
|
|
30
|
+
|
|
31
|
+
## What It Produces
|
|
32
|
+
|
|
33
|
+
| Mode | Artifacts |
|
|
34
|
+
|------|-----------|
|
|
35
|
+
| validate | VALIDATION_REPORT.md (syntax, structure, dependencies, logic per file) |
|
|
36
|
+
| classify | FAILURE_CLASSIFICATION_REPORT.md (per-failure evidence, no fixes) |
|
|
37
|
+
| fix | FAILURE_CLASSIFICATION_REPORT.md + auto-fixed test files |
|
|
38
|
+
|
|
39
|
+
## Instructions
|
|
40
|
+
|
|
41
|
+
### Step 1: Detect Mode and Test Directory
|
|
42
|
+
|
|
43
|
+
Parse `$ARGUMENTS` for mode flags and test directory path.
|
|
44
|
+
|
|
45
|
+
If no test directory provided, auto-detect:
|
|
46
|
+
- Check for `tests/`, `cypress/`, `__tests__/`, `e2e/`, `spec/` directories
|
|
47
|
+
- Check test config files for `testDir` setting
|
|
48
|
+
|
|
49
|
+
Print banner:
|
|
50
|
+
```
|
|
51
|
+
=== QA Fix & Validate ===
|
|
52
|
+
Mode: {validate | classify | fix}
|
|
53
|
+
Test Directory: {path}
|
|
54
|
+
App URL: {url or "auto-detect"}
|
|
55
|
+
==========================
|
|
56
|
+
```
|
|
57
|
+
|
|
58
|
+
---
|
|
59
|
+
|
|
60
|
+
### VALIDATE MODE (`--validate-only`)
|
|
61
|
+
|
|
62
|
+
1. Read `CLAUDE.md` — quality gates, locator tiers, assertion rules.
|
|
63
|
+
2. Execute static validation workflow:
|
|
64
|
+
|
|
65
|
+
Follow the workflow defined in `@workflows/qa-validate.md` end-to-end.
|
|
66
|
+
Preserve all workflow gates (fix loops, layer checks).
|
|
67
|
+
|
|
68
|
+
The validator runs 4 layers per file:
|
|
69
|
+
1. **Syntax** — code compiles without errors
|
|
70
|
+
2. **Structure** — naming, folders, POM compliance
|
|
71
|
+
3. **Dependencies** — all imports resolve
|
|
72
|
+
4. **Logic** — concrete assertions, Tier 1-2 locators, no assertions in POMs
|
|
73
|
+
|
|
74
|
+
**Optional Layer 5 (if Playwright MCP connected + app URL available):**
|
|
75
|
+
- Navigate to each page referenced in E2E tests via `mcp__playwright__browser_navigate`
|
|
76
|
+
- Take accessibility snapshot via `mcp__playwright__browser_snapshot`
|
|
77
|
+
- Cross-reference test locators against real DOM elements
|
|
78
|
+
- Flag locators that don't match, auto-fix mismatches
|
|
79
|
+
|
|
80
|
+
Max 3 fix loop iterations. Produces VALIDATION_REPORT.md.
|
|
81
|
+
|
|
82
|
+
If `--run` flag is also present and E2E test files exist, invoke E2E runner after static validation:
|
|
83
|
+
|
|
84
|
+
Task(
|
|
85
|
+
prompt="
|
|
86
|
+
<objective>Run E2E tests against live application, capture real locators, fix mismatches, loop until pass</objective>
|
|
87
|
+
<execution_context>@agents/qaa-e2e-runner.md</execution_context>
|
|
88
|
+
<files_to_read>
|
|
89
|
+
- CLAUDE.md
|
|
90
|
+
- {E2E test files from validated directory}
|
|
91
|
+
- {POM files from validated directory}
|
|
92
|
+
</files_to_read>
|
|
93
|
+
<parameters>
|
|
94
|
+
app_url: {from --app-url flag or auto-detect}
|
|
95
|
+
output_dir: .qa-output
|
|
96
|
+
</parameters>
|
|
97
|
+
"
|
|
98
|
+
)
|
|
99
|
+
|
|
100
|
+
---
|
|
101
|
+
|
|
102
|
+
### CLASSIFY MODE (`--classify`)
|
|
103
|
+
|
|
104
|
+
Same as fix mode below but skip Step 4 (auto-fix). Only classify and report.
|
|
105
|
+
|
|
106
|
+
---
|
|
107
|
+
|
|
108
|
+
### FIX MODE (default)
|
|
109
|
+
|
|
110
|
+
1. Read `CLAUDE.md` — classification rules, locator tiers, assertion quality.
|
|
111
|
+
2. Invoke bug-detective agent:
|
|
112
|
+
|
|
113
|
+
Task(
|
|
114
|
+
prompt="
|
|
115
|
+
<objective>Run tests, classify failures, and auto-fix TEST CODE ERRORS. Use Playwright MCP to reproduce E2E failures in the browser when available — navigate to failing pages, snapshot DOM, reproduce actions, and screenshot failure state for evidence.</objective>
|
|
116
|
+
<execution_context>@agents/qaa-bug-detective.md</execution_context>
|
|
117
|
+
<files_to_read>
|
|
118
|
+
- CLAUDE.md
|
|
119
|
+
</files_to_read>
|
|
120
|
+
<parameters>
|
|
121
|
+
user_input: $ARGUMENTS
|
|
122
|
+
app_url: {auto-detect from test config baseURL, or ask user}
|
|
123
|
+
</parameters>
|
|
124
|
+
"
|
|
125
|
+
)
|
|
126
|
+
|
|
127
|
+
**Classification categories:**
|
|
128
|
+
- **APPLICATION BUG** — error in production code → Report only, NEVER auto-fix
|
|
129
|
+
- **TEST CODE ERROR** — error in test code → Auto-fix if HIGH confidence
|
|
130
|
+
- **ENVIRONMENT ISSUE** — missing env, connection refused → Report with resolution steps
|
|
131
|
+
- **INCONCLUSIVE** — ambiguous → Report what's known, ask for more info
|
|
132
|
+
|
|
133
|
+
**Auto-fix rules:**
|
|
134
|
+
- Only TEST CODE ERROR at HIGH confidence
|
|
135
|
+
- Allowed fixes: import paths, selectors, assertion values, config, missing await, fixture paths
|
|
136
|
+
- Every fix verified by re-running the specific test
|
|
137
|
+
- Never modify application code (src/, app/, lib/)
|
|
138
|
+
|
|
139
|
+
**Browser reproduction (when Playwright MCP connected):**
|
|
140
|
+
- Navigate to failing page → snapshot DOM → reproduce action → screenshot
|
|
141
|
+
- Element not in DOM → TEST CODE ERROR (HIGH confidence)
|
|
142
|
+
- Element exists, wrong behavior → APPLICATION BUG
|
|
143
|
+
- Page doesn't load → ENVIRONMENT ISSUE
|
|
144
|
+
|
|
145
|
+
3. Present results. APPLICATION BUGs are reported for developer action, not auto-fixed.
|
|
146
|
+
|
|
147
|
+
$ARGUMENTS
|
|
@@ -0,0 +1,137 @@
|
|
|
1
|
+
# QA Codebase Map & Analysis
|
|
2
|
+
|
|
3
|
+
Deep-scan a codebase for QA-relevant information, produce a complete analysis, and generate test inventory. Runs codebase mapping (4 parallel agents) followed by full repository analysis. One command to fully understand a codebase before writing tests.
|
|
4
|
+
|
|
5
|
+
## Usage
|
|
6
|
+
|
|
7
|
+
```
|
|
8
|
+
/qa-map [options]
|
|
9
|
+
```
|
|
10
|
+
|
|
11
|
+
### Options
|
|
12
|
+
|
|
13
|
+
- No arguments — runs full map + analysis on current directory
|
|
14
|
+
- `--focus <area>` — run a single map area only, skip analysis (testability, risk, patterns, existing-tests)
|
|
15
|
+
- `--dev-repo <path>` — explicit path to developer repository
|
|
16
|
+
- `--qa-repo <path>` — path to existing QA repository (produces gap analysis instead of blueprint)
|
|
17
|
+
- `--skip-map` — skip codebase mapping, only run analysis (lighter, faster)
|
|
18
|
+
|
|
19
|
+
## What It Produces
|
|
20
|
+
|
|
21
|
+
### Stage 1: Codebase Map (4 parallel agents)
|
|
22
|
+
|
|
23
|
+
| Focus Area | Documents Produced |
|
|
24
|
+
|------------|-------------------|
|
|
25
|
+
| **testability** | TESTABILITY.md + TEST_SURFACE.md — what's testable, entry points, mocking needs |
|
|
26
|
+
| **risk** | RISK_MAP.md + CRITICAL_PATHS.md — business-critical paths, error handling gaps |
|
|
27
|
+
| **patterns** | CODE_PATTERNS.md + API_CONTRACTS.md — naming conventions, API shapes, auth patterns |
|
|
28
|
+
| **existing-tests** | TEST_ASSESSMENT.md + COVERAGE_GAPS.md — existing test quality, what's missing |
|
|
29
|
+
|
|
30
|
+
All documents written to `.qa-output/codebase/`.
|
|
31
|
+
|
|
32
|
+
### Stage 2: Repository Analysis
|
|
33
|
+
|
|
34
|
+
| Document | Description |
|
|
35
|
+
|----------|-------------|
|
|
36
|
+
| SCAN_MANIFEST.md | File tree, framework detection, testable surfaces |
|
|
37
|
+
| QA_ANALYSIS.md | Architecture overview, risk assessment, top 10 unit targets, testing pyramid |
|
|
38
|
+
| TEST_INVENTORY.md | Every test case with ID, target, inputs, expected outcome, priority |
|
|
39
|
+
| QA_REPO_BLUEPRINT.md | If no QA repo — full repo structure, configs, CI/CD strategy |
|
|
40
|
+
| GAP_ANALYSIS.md | If QA repo provided — coverage map, missing tests, broken tests, quality assessment |
|
|
41
|
+
|
|
42
|
+
## Instructions
|
|
43
|
+
|
|
44
|
+
1. Read `CLAUDE.md` — QA standards.
|
|
45
|
+
|
|
46
|
+
2. Create output directories:
|
|
47
|
+
```bash
|
|
48
|
+
mkdir -p .qa-output/codebase
|
|
49
|
+
```
|
|
50
|
+
|
|
51
|
+
3. **Stage 1: Codebase Mapping**
|
|
52
|
+
|
|
53
|
+
If `--skip-map` was NOT passed and `--focus` was NOT specified, spawn 4 agents in parallel (one per focus area):
|
|
54
|
+
|
|
55
|
+
```
|
|
56
|
+
Agent(
|
|
57
|
+
prompt="Analyze this codebase for QA purposes. Focus area: testability. Write TESTABILITY.md and TEST_SURFACE.md to .qa-output/codebase/. Follow your agent definition process.",
|
|
58
|
+
subagent_type="general-purpose",
|
|
59
|
+
execution_context="@agents/qaa-codebase-mapper.md"
|
|
60
|
+
)
|
|
61
|
+
|
|
62
|
+
Agent(
|
|
63
|
+
prompt="Analyze this codebase for QA purposes. Focus area: risk. Write RISK_MAP.md and CRITICAL_PATHS.md to .qa-output/codebase/. Follow your agent definition process.",
|
|
64
|
+
subagent_type="general-purpose",
|
|
65
|
+
execution_context="@agents/qaa-codebase-mapper.md"
|
|
66
|
+
)
|
|
67
|
+
|
|
68
|
+
Agent(
|
|
69
|
+
prompt="Analyze this codebase for QA purposes. Focus area: patterns. Write CODE_PATTERNS.md and API_CONTRACTS.md to .qa-output/codebase/. Follow your agent definition process.",
|
|
70
|
+
subagent_type="general-purpose",
|
|
71
|
+
execution_context="@agents/qaa-codebase-mapper.md"
|
|
72
|
+
)
|
|
73
|
+
|
|
74
|
+
Agent(
|
|
75
|
+
prompt="Analyze this codebase for QA purposes. Focus area: existing-tests. Write TEST_ASSESSMENT.md and COVERAGE_GAPS.md to .qa-output/codebase/. Follow your agent definition process.",
|
|
76
|
+
subagent_type="general-purpose",
|
|
77
|
+
execution_context="@agents/qaa-codebase-mapper.md"
|
|
78
|
+
)
|
|
79
|
+
```
|
|
80
|
+
|
|
81
|
+
If `--focus <area>` was provided, spawn only that one agent and STOP after it completes (skip Stage 2).
|
|
82
|
+
|
|
83
|
+
If `--skip-map` was passed, skip Stage 1 entirely and go to Stage 2.
|
|
84
|
+
|
|
85
|
+
4. When all map agents complete, print summary of documents produced.
|
|
86
|
+
|
|
87
|
+
5. **Stage 2: Repository Analysis**
|
|
88
|
+
|
|
89
|
+
Initialize pipeline context:
|
|
90
|
+
```bash
|
|
91
|
+
node bin/qaa-tools.cjs init qa-start 2>/dev/null || true
|
|
92
|
+
```
|
|
93
|
+
|
|
94
|
+
Invoke scanner agent:
|
|
95
|
+
|
|
96
|
+
Task(
|
|
97
|
+
prompt="
|
|
98
|
+
<objective>Scan repository and produce SCAN_MANIFEST.md</objective>
|
|
99
|
+
<execution_context>@agents/qaa-scanner.md</execution_context>
|
|
100
|
+
<files_to_read>
|
|
101
|
+
- CLAUDE.md
|
|
102
|
+
</files_to_read>
|
|
103
|
+
<parameters>
|
|
104
|
+
user_input: $ARGUMENTS
|
|
105
|
+
</parameters>
|
|
106
|
+
"
|
|
107
|
+
)
|
|
108
|
+
|
|
109
|
+
Invoke analyzer agent:
|
|
110
|
+
|
|
111
|
+
Task(
|
|
112
|
+
prompt="
|
|
113
|
+
<objective>Analyze repository and produce QA_ANALYSIS.md, TEST_INVENTORY.md, and blueprint or gap analysis. Use codebase map documents from .qa-output/codebase/ if they exist for deeper, more accurate analysis.</objective>
|
|
114
|
+
<execution_context>@agents/qaa-analyzer.md</execution_context>
|
|
115
|
+
<files_to_read>
|
|
116
|
+
- CLAUDE.md
|
|
117
|
+
- .qa-output/SCAN_MANIFEST.md
|
|
118
|
+
- .qa-output/codebase/TESTABILITY.md (if exists)
|
|
119
|
+
- .qa-output/codebase/RISK_MAP.md (if exists)
|
|
120
|
+
- .qa-output/codebase/CODE_PATTERNS.md (if exists)
|
|
121
|
+
- .qa-output/codebase/TEST_ASSESSMENT.md (if exists)
|
|
122
|
+
- .qa-output/codebase/TEST_SURFACE.md (if exists)
|
|
123
|
+
- .qa-output/codebase/CRITICAL_PATHS.md (if exists)
|
|
124
|
+
- .qa-output/codebase/API_CONTRACTS.md (if exists)
|
|
125
|
+
- .qa-output/codebase/COVERAGE_GAPS.md (if exists)
|
|
126
|
+
</files_to_read>
|
|
127
|
+
<parameters>
|
|
128
|
+
user_input: $ARGUMENTS
|
|
129
|
+
</parameters>
|
|
130
|
+
"
|
|
131
|
+
)
|
|
132
|
+
|
|
133
|
+
6. Print final summary: all documents produced across both stages.
|
|
134
|
+
No git operations. No test generation.
|
|
135
|
+
Suggest `/qa-create-test` to generate tests from the analysis.
|
|
136
|
+
|
|
137
|
+
$ARGUMENTS
|