qaa-agent 1.9.1 → 1.9.5

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (41) hide show
  1. package/CHANGELOG.md +43 -22
  2. package/CLAUDE.md +170 -9
  3. package/README.md +384 -357
  4. package/VERSION +1 -0
  5. package/agents/qa-pipeline-orchestrator.md +336 -22
  6. package/agents/qaa-analyzer.md +0 -1
  7. package/agents/qaa-bug-detective.md +163 -4
  8. package/agents/qaa-codebase-mapper.md +50 -1
  9. package/agents/qaa-discovery.md +421 -384
  10. package/agents/qaa-e2e-runner.md +163 -1
  11. package/agents/qaa-executor.md +142 -1
  12. package/agents/qaa-planner.md +14 -1
  13. package/agents/qaa-project-researcher.md +194 -0
  14. package/agents/qaa-scanner.md +77 -1
  15. package/agents/qaa-testid-injector.md +0 -1
  16. package/agents/qaa-validator.md +86 -1
  17. package/bin/install.cjs +375 -253
  18. package/bin/lib/context7-cache.cjs +299 -0
  19. package/bin/lib/intent-detector.cjs +488 -0
  20. package/commands/qa-audit.md +255 -126
  21. package/commands/qa-create-test.md +666 -365
  22. package/commands/qa-fix.md +684 -513
  23. package/commands/qa-map.md +283 -139
  24. package/commands/qa-pr.md +63 -0
  25. package/commands/qa-research.md +181 -157
  26. package/commands/qa-start.md +62 -6
  27. package/commands/qa-test-report.md +219 -219
  28. package/frameworks/cypress.json +54 -0
  29. package/frameworks/jest.json +59 -0
  30. package/frameworks/playwright.json +58 -0
  31. package/frameworks/pytest.json +59 -0
  32. package/frameworks/robot-framework.json +54 -0
  33. package/frameworks/selenium.json +65 -0
  34. package/frameworks/vitest.json +57 -0
  35. package/package.json +7 -3
  36. package/workflows/qa-analyze.md +100 -4
  37. package/workflows/qa-from-ticket.md +50 -2
  38. package/workflows/qa-gap.md +50 -2
  39. package/workflows/qa-start.md +819 -33
  40. package/workflows/qa-testid.md +50 -2
  41. package/workflows/qa-validate.md +50 -2
@@ -1,365 +1,666 @@
1
- # QA Create Test
2
-
3
- Create, update, or generate tests from tickets — all in one command. Supports five modes: generate tests from code analysis, generate tests from a ticket (Jira/Linear/GitHub), create Test Cases in Azure DevOps from a work item, update/improve existing tests, or generate POM files only. Uses Playwright MCP to extract real locators from the live app when available.
4
-
5
- ## Usage
6
-
7
- ```
8
- /qa-create-test <feature-or-source> [options]
9
- ```
10
-
11
- ### Modes (auto-detected from arguments)
12
-
13
- | Mode | Trigger | Example |
14
- |------|---------|---------|
15
- | **From code** | Feature name (no URL, no path to tests) | `/qa-create-test login` |
16
- | **From ticket** | URL, shorthand (#123), or `--ticket` flag | `/qa-create-test https://github.com/org/repo/issues/42` |
17
- | **Azure DevOps** | `--ado` flag with work item ID or ADO URL | `/qa-create-test --ado 85508` |
18
- | **Update existing** | Path to existing test files or `--update` flag | `/qa-create-test --update tests/e2e/` |
19
- | **POM only** | `--pom-only` flag | `/qa-create-test --pom-only src/pages/` |
20
-
21
- ### Options
22
-
23
- - `--dev-repo <path>` path to developer repository (default: current directory)
24
- - `--app-url <url>` — URL of running application for E2E execution and locator extraction (auto-detects if not provided)
25
- - `--skip-run` skip E2E execution, only generate and statically validate
26
- - `--ticket <source>` force ticket mode with: URL, shorthand (#123, org/repo#123), file path, or plain text
27
- - `--update <path>` force update mode: audit and improve existing tests at path
28
- - `--scope fix|improve|add|full` for update mode only (default: full)
29
- - `--ado <work-item-id>` Azure DevOps mode: read a work item and create Test Cases in ADO (accepts ID or full ADO URL)
30
- - `--area-path <path>` (ADO mode) override area path for created test cases (default: source work item's area path)
31
- - `--iteration-path <path>` (ADO mode) override iteration path for created test cases (default: source work item's iteration path)
32
- - `--skip-dedup` — (ADO mode) skip deduplication check against existing linked test cases
33
- - `--pom-only [path]` generate only Page Object Model files (BasePage + feature POMs), no test specs
34
- - `--framework <name>` — override framework auto-detection (playwright, cypress, selenium) — used with --pom-only
35
-
36
- ### Mode Detection Logic
37
-
38
- ```
39
- if --pom-only:
40
- MODE = "pom-only"
41
- elif --ado flag OR argument matches ADO URL (dev.azure.com, *.visualstudio.com):
42
- MODE = "ado"
43
- elif argument matches URL pattern (github.com, atlassian.net, linear.app) OR contains "#" + digits OR --ticket flag:
44
- MODE = "from-ticket"
45
- elif --update flag OR argument is path to existing test directory/files:
46
- MODE = "update"
47
- else:
48
- MODE = "from-code"
49
- ```
50
-
51
- ## What It Produces
52
-
53
- ### From Code Mode
54
- - Test spec files (unit, API, E2E as appropriate)
55
- - Page Object Model files (for E2E tests)
56
- - Fixture files (test data)
57
- - Locator registry entries (`.qa-output/locators/`)
58
- - E2E_RUN_REPORT.md (if E2E tests ran against live app)
59
-
60
- ### From Ticket Mode
61
- - TEST_CASES_FROM_TICKET.md — traceability matrix (AC → test case)
62
- - GENERATION_PLAN_TICKET.md — synthetic generation plan
63
- - Test spec files with `traces_to` fields linking back to ticket ACs
64
- - VALIDATION_REPORT.md
65
-
66
- ### Azure DevOps Mode
67
- - Test Cases created directly in Azure DevOps (via `testplan_create_test_case`)
68
- - Test Cases linked to source work item via *Tested By* relationship
69
- - Tags applied: `Smoke`, `Regression`, `Critical`, `AutomationCandidate`, `NeedsReview`
70
- - `ai-tasks/ticket-{id}/test-cases.md` — structured report
71
- - Report attached to work item (if `ADO_MCP_AUTH_TOKEN` is set) or written to `Custom.QATestCasesReport` field (fallback)
72
-
73
- ### Update Mode
74
- - QA_AUDIT_REPORT.md current quality assessment
75
- - Improved test files (after user approval)
76
-
77
- ## Instructions
78
-
79
- ### Step 1: Detect Mode
80
-
81
- Parse `$ARGUMENTS` to determine mode using the detection logic above.
82
-
83
- Print mode banner:
84
- ```
85
- === QA Create Test ===
86
- Mode: {from-code | from-ticket | ado | update | pom-only}
87
- Target: {feature name | ticket URL | ADO work item ID | test path}
88
- App URL: {url or "auto-detect"}
89
- ===========================
90
- ```
91
-
92
- ---
93
-
94
- ### FROM CODE MODE
95
-
96
- 1. Read `CLAUDE.md` — POM rules, locator tiers, assertion rules, naming conventions, quality gates.
97
- 1b. Read `~/.claude/qaa/MY_PREFERENCES.md` if it exists — user's personal QA preferences override CLAUDE.md defaults when there is a conflict.
98
- 2. Read existing analysis artifacts if available:
99
- - `.qa-output/QA_ANALYSIS.md` — architecture context
100
- - `.qa-output/TEST_INVENTORY.md` pre-defined test cases for this feature
101
- 3. **Check for codebase map** (`.qa-output/codebase/`):
102
- - Look for: `CODE_PATTERNS.md`, `API_CONTRACTS.md`, `TEST_SURFACE.md`, `TESTABILITY.md`
103
- - If at least 2 of these files exist: read them all for project context (naming conventions, API shapes, testable surfaces).
104
- - **If NONE of these files exist: STOP and tell the user:**
105
- ```
106
- ⚠ No codebase map found (.qa-output/codebase/ is empty or missing).
107
-
108
- The codebase map provides critical context: naming conventions, API contracts,
109
- testable surfaces, and project structure. Without it, generated tests will lack
110
- project-specific context and may not follow your repo's conventions.
111
-
112
- Run /qa-map first to generate the codebase map, then re-run /qa-create-test.
113
-
114
- To skip this check and proceed without context: re-run with --skip-map
115
- ```
116
- Only proceed without codebase map if the user explicitly passes `--skip-map`.
117
-
118
- 4. **Check existing locator registry and extract new locators from live app:**
119
-
120
- a. Read `.qa-output/locators/LOCATOR_REGISTRY.md` if it exists.
121
-
122
- b. If locators for this feature already exist in the registry AND no `--app-url` was provided: reuse cached locators.
123
-
124
- c. If locators are missing or `--app-url` was provided: Use Playwright MCP to navigate the app and extract real locators:
125
- ```
126
- mcp__playwright__browser_navigate({ url: "{app_url}/{feature_path}" })
127
- mcp__playwright__browser_snapshot()
128
- ```
129
- Extract all data-testid, ARIA roles, labels, placeholders.
130
- Navigate through multi-page flows if needed.
131
- Write per-feature locator file to `.qa-output/locators/{feature}.locators.md`.
132
- Update the registry `.qa-output/locators/LOCATOR_REGISTRY.md`.
133
-
134
- If no app URL available and no locators in registry, skip — executor proposes locators from source code.
135
-
136
- 5. Invoke executor agent to generate test files:
137
-
138
- Task(
139
- prompt="
140
- <objective>Generate test files for the specified feature following CLAUDE.md standards, using codebase map and research documents for context. Query Context7 MCP to verify framework syntax before generating code.</objective>
141
- <execution_context>@agents/qaa-executor.md</execution_context>
142
- <files_to_read>
143
- - CLAUDE.md
144
- - ~/.claude/qaa/MY_PREFERENCES.md (if exists)
145
- - .qa-output/locators/LOCATOR_REGISTRY.md (if exists)
146
- - .qa-output/locators/{feature}.locators.md (if exists)
147
- - .qa-output/codebase/CODE_PATTERNS.md (if exists)
148
- - .qa-output/codebase/API_CONTRACTS.md (if exists)
149
- - .qa-output/codebase/TEST_SURFACE.md (if exists)
150
- - .qa-output/codebase/TESTABILITY.md (if exists)
151
- - .qa-output/research/FRAMEWORK_CAPABILITIES.md (if exists)
152
- - .qa-output/research/E2E_STRATEGY.md (if exists)
153
- - .qa-output/research/API_TESTING_STRATEGY.md (if exists)
154
- - .qa-output/research/TESTING_STACK.md (if exists)
155
- </files_to_read>
156
- <parameters>
157
- user_input: $ARGUMENTS
158
- mode: feature-test
159
- codebase_map_dir: .qa-output/codebase
160
- locator_registry: .qa-output/locators/LOCATOR_REGISTRY.md
161
- </parameters>
162
- "
163
- )
164
-
165
- 6. If E2E test files were generated AND `--skip-run` was NOT passed, invoke E2E runner:
166
-
167
- Task(
168
- prompt="
169
- <objective>Run generated E2E tests against live application, capture real locators, fix mismatches, loop until pass. Query Context7 MCP to verify framework selector syntax before fixing locators.</objective>
170
- <execution_context>@agents/qaa-e2e-runner.md</execution_context>
171
- <files_to_read>
172
- - CLAUDE.md
173
- - ~/.claude/qaa/MY_PREFERENCES.md (if exists)
174
- - .qa-output/locators/LOCATOR_REGISTRY.md (if exists)
175
- - .qa-output/research/FRAMEWORK_CAPABILITIES.md (if exists)
176
- - .qa-output/research/E2E_STRATEGY.md (if exists)
177
- - {generated E2E test files from executor return}
178
- - {generated POM files from executor return}
179
- </files_to_read>
180
- <parameters>
181
- app_url: {from --app-url flag or auto-detect}
182
- output_dir: .qa-output
183
- </parameters>
184
- "
185
- )
186
-
187
- 7. Present results with file counts and suggest `/qa-pr`.
188
-
189
- ---
190
-
191
- ### FROM TICKET MODE
192
-
193
- 1. Read `CLAUDE.md` all QA standards.
194
- 2. Execute the ticket workflow end-to-end:
195
-
196
- Follow the workflow defined in `@workflows/qa-from-ticket.md` end-to-end.
197
- Preserve all workflow gates (ticket parsing, acceptance criteria extraction, traceability matrix, validation).
198
-
199
- Key steps in the workflow:
200
- - Parse ticket source (GitHub URL, Jira URL, Linear URL, file, or plain text)
201
- - Fetch ticket content (via `gh issue view`, WebFetch, or file read)
202
- - Extract acceptance criteria, user stories, edge cases
203
- - Scan dev repo for related source files
204
- - Extract locators from live app via Playwright MCP (if app URL available)
205
- - Generate test cases with traceability matrix (every AC maps to ≥1 test case)
206
- - Spawn executor agent to produce test files
207
- - Spawn validator agent for 4-layer validation
208
- - Print summary with AC coverage and traceability
209
-
210
- **Traceability guarantee:** Every acceptance criterion maps to at least one test case via the `traces_to` field.
211
-
212
- **Supported ticket sources:**
213
-
214
- | Format | Example | Detection |
215
- |--------|---------|-----------|
216
- | GitHub Issue URL | `https://github.com/org/repo/issues/123` | Contains `github.com` + `/issues/` |
217
- | GitHub shorthand | `org/repo#123` or `#123` | Contains `#` + digits |
218
- | Jira URL | `https://company.atlassian.net/browse/PROJ-123` | Contains `.atlassian.net/browse/` |
219
- | Linear URL | `https://linear.app/team/issue/TEAM-123` | Contains `linear.app` |
220
- | File path | `./tickets/feature-spec.md` | Path exists on disk |
221
- | Plain text | `"As a user I want to..."` | None of the above match |
222
-
223
- ---
224
-
225
- ### ADO MODE (Azure DevOps)
226
-
227
- Create Test Cases directly in Azure DevOps from a work item. Reads the work item content (repro steps, acceptance criteria, comments, attachments), designs test cases with boundary detection and deduplication, and creates them in ADO with full traceability.
228
-
229
- **Prerequisites:** ADO MCP server must be connected (provides `wit_get_work_item`, `testplan_create_test_case`, etc.).
230
-
231
- Execute the full ADO workflow defined in `@commands/qa-create-test-ado.md`:
232
-
233
- 1. **Phase 1** Read pipeline context: CLAUDE.md, MY_PREFERENCES.md, codebase map, locator registry
234
- 2. **Phase 2** — Retrieve work item with relations, comments, and attachments
235
- 3. **Phase 2b** Deduplication check against existing linked test cases (skip with `--skip-dedup`)
236
- 4. **Phase 3** Extract test source content based on work item type (Bug → Repro Steps, User Story → Acceptance Criteria)
237
- 5. **Phase 4** Design test cases with boundary value detection, tagging rules, confidence scoring, and preconditions
238
- 6. **Phase 5** Create test cases in ADO via `testplan_create_test_case`, link via *Tested By*, set tags
239
- 7. **Phase 6** — Generate structured report to `ai-tasks/ticket-{id}/test-cases.md`
240
- 8. **Phase 7** Attach report to source work item
241
-
242
- **Key features:**
243
- - Boundary value triplets: detects `max`, `min`, `limit`, `threshold` keywords with numeric values → generates N-1, N, N+1 test cases
244
- - Deduplication: checks existing linked TCs before creating, prevents duplicates
245
- - Confidence scoring: `Specified` (explicit source) vs `Draft` (inferred, tagged `NeedsReview`)
246
- - Cross-references codebase map for project-specific context when available
247
- - Supports `--area-path` and `--iteration-path` overrides
248
-
249
- For the complete step-by-step process, see `@commands/qa-create-test-ado.md`.
250
-
251
- ---
252
-
253
- ### UPDATE MODE
254
-
255
- 1. Read `CLAUDE.md` quality gates, locator tiers, assertion rules, POM rules.
256
- 2. Invoke validator agent in audit mode:
257
-
258
- Task(
259
- prompt="
260
- <objective>Audit existing test quality and produce QA_AUDIT_REPORT.md. If Playwright MCP is connected, verify E2E test locators against the live DOM via browser_navigate + browser_snapshot.</objective>
261
- <execution_context>@agents/qaa-validator.md</execution_context>
262
- <files_to_read>
263
- - CLAUDE.md
264
- - ~/.claude/qaa/MY_PREFERENCES.md (if exists)
265
- </files_to_read>
266
- <parameters>
267
- user_input: $ARGUMENTS
268
- mode: audit
269
- app_url: {auto-detect from test config baseURL, or ask user}
270
- </parameters>
271
- "
272
- )
273
-
274
- 3. Present audit results and wait for user approval.
275
- 4. Invoke executor agent to apply approved improvements:
276
-
277
- Task(
278
- prompt="
279
- <objective>Apply approved improvements to existing tests without deleting working tests. If Playwright MCP is connected, use browser_navigate + browser_snapshot to extract real locators when upgrading from Tier 4 to Tier 1.</objective>
280
- <execution_context>@agents/qaa-executor.md</execution_context>
281
- <files_to_read>
282
- - CLAUDE.md
283
- - ~/.claude/qaa/MY_PREFERENCES.md (if exists)
284
- - .qa-output/QA_AUDIT_REPORT.md
285
- - .qa-output/locators/LOCATOR_REGISTRY.md (if exists)
286
- </files_to_read>
287
- <parameters>
288
- user_input: $ARGUMENTS
289
- mode: update
290
- app_url: {auto-detect from test config baseURL, or ask user}
291
- </parameters>
292
- "
293
- )
294
-
295
- **Update scopes:**
296
- - `fix` repair broken tests only
297
- - `improve` — upgrade locators, assertions, POM structure
298
- - `add` add missing test cases without modifying existing
299
- - `full` audit everything, then improve with approval (default)
300
-
301
- **Rule:** NEVER delete or rewrite working tests without user approval. Surgical: add, fix, improve — never replace.
302
-
303
- ---
304
-
305
- ### POM ONLY MODE (`--pom-only`)
306
-
307
- Generate only Page Object Model files — no test specs.
308
-
309
- 1. Read `CLAUDE.md` — POM rules, locator tier hierarchy, naming conventions.
310
- 2. Invoke executor agent in POM-only mode:
311
-
312
- Task(
313
- prompt="
314
- <objective>Generate Page Object Models following CLAUDE.md POM rules. If Playwright MCP is connected and an app URL is available, navigate each page first to extract real locators (data-testid, ARIA roles, labels) from the live DOM via browser_navigate + browser_snapshot before generating POMs. This ensures POM locators match the real app instead of guessing from source code.</objective>
315
- <execution_context>@agents/qaa-executor.md</execution_context>
316
- <files_to_read>
317
- - CLAUDE.md
318
- - ~/.claude/qaa/MY_PREFERENCES.md (if exists)
319
- - .qa-output/locators/LOCATOR_REGISTRY.md (if exists)
320
- </files_to_read>
321
- <parameters>
322
- user_input: $ARGUMENTS
323
- mode: pom-only
324
- app_url: {auto-detect from test config baseURL, or ask user}
325
- </parameters>
326
- "
327
- )
328
-
329
- **Produces:**
330
- - BasePage file (if not already present)
331
- - Feature-specific POM files following `[PageName]Page.[ext]` naming convention
332
- - No test specs, no fixtures
333
-
334
- **POM rules enforced:**
335
- - One class per page — no god objects
336
- - No assertions in page objects assertions belong ONLY in test specs
337
- - Locators as readonly properties — Tier 1 preferred (data-testid, ARIA roles)
338
- - Actions return void or next page for fluent chaining
339
- - State queries return data — let the test decide what to assert
340
- - Every POM extends BasePage
341
-
342
- $ARGUMENTS
343
-
344
- ## MANDATORY verification — run ALL commands below, no exceptions, no skipping
345
-
346
- Before returning control, copy-paste and run this ENTIRE block. Do NOT decide which commands "apply" — run all of them every time. The output confirms what happened; you do not get to assume the answer.
347
-
348
- ```bash
349
- echo "=== CHECKLIST START ==="
350
- echo "1. Locator Registry:"
351
- ls .qa-output/locators/ 2>/dev/null || echo "NO_LOCATORS_FOUND"
352
- echo "2. MY_PREFERENCES.md:"
353
- cat ~/.claude/qaa/MY_PREFERENCES.md 2>/dev/null || echo "FILE_NOT_FOUND"
354
- echo "3. Generated test files:"
355
- find tests/ pages/ fixtures/ -type f 2>/dev/null | head -20 || echo "NO_TEST_FILES_FOUND"
356
- echo "4. MCP evidence (if browser was used):"
357
- ls .qa-output/mcp-evidence/ 2>/dev/null || echo "NO_MCP_EVIDENCE"
358
- echo "=== CHECKLIST END ==="
359
- ```
360
-
361
- **Rules:**
362
- - Run the block AS-IS. Do not modify it. Do not split it. Do not skip lines.
363
- - If any output shows a problem (NO_LOCATORS when MCP was used, NO_TEST_FILES after generation), fix it before returning.
364
- - If output shows expected "not found" results (e.g., NO_MCP_EVIDENCE when no browser was used), that is finethe point is you RAN the command instead of assuming the answer.
365
- - Do NOT mark this task as complete until the block has been executed and you have read every line of output.
1
+ # QA Create Test
2
+
3
+ Create, update, or generate tests from tickets — all in one command. Supports six modes: generate tests from code analysis, generate tests from a live app URL (from-aut), generate tests from a ticket (Jira/Linear/GitHub), create Test Cases in Azure DevOps from a work item, update/improve existing tests, or generate POM files only. Uses Playwright MCP to extract real locators from the live app when available.
4
+
5
+
6
+ ## ⚠ MANDATORY: How to Execute This Command
7
+
8
+ Execute these steps IN ORDER. Every step is mandatory. Do NOT improvise,
9
+ reorder, skip, or invent steps.
10
+
11
+ 1. FIRST tool call: run the intent detector with `$ARGUMENTS` passed LITERALLY
12
+ (the actual command lives in the "## Intent Detection" section below).
13
+ 2. Print the INPUT DETECTION banner (exact output from intent-detector.cjs).
14
+ 3. Determine `ALL_FROM_FLAGS`. If interactive (`IS_AUTO=false`) AND any value
15
+ came from NL/default (`ALL_FROM_FLAGS=false`), PAUSE for yes/no confirmation.
16
+ Do NOT proceed without it.
17
+ ```bash
18
+ ALL_FROM_FLAGS=$(echo "$RESOLVED" | node -e "const j=JSON.parse(require('fs').readFileSync(0,'utf8')); const sources=Object.values(j.resolved).map(v=>v.source); console.log(sources.length>0 && sources.every(s=>s==='flag') ? 'true' : 'false')")
19
+ ```
20
+ Pause prompt (when applicable):
21
+ ```
22
+ The values above were resolved from natural language and/or defaults.
23
+ Do you want to continue with these inputs? (yes/no)
24
+ ```
25
+ 4. Determine MODE using the "Mode Detection Logic" section below
26
+ (pom-only ado from-ticket from-aut update from-code). Use the
27
+ detector's `intent.suggested_mode` as the heuristic do NOT decide the mode
28
+ by your own NL parsing.
29
+ 5. Print the mode banner (Step 1 of Instructions) and execute that mode's flow
30
+ IN ORDER, including its own approval gates (e.g. update mode presents the
31
+ audit and waits for approval). Modes that generate files spawn the
32
+ qaa-executor (or mode-specific) sub-agent do NOT hand-write test files
33
+ inline instead of the sub-agent.
34
+
35
+ ### DO NOT
36
+ - Skip the INPUT DETECTION banner or the mode banner
37
+ - Run AskUserQuestion or any tool before the intent detector
38
+ - Use AskUserQuestion in place of the yes/no confirmation pause — use the exact text prompt above
39
+ - Decide the mode by parsing NL in your own head — use the detector + Mode Detection Logic
40
+ - Generate or overwrite any test files before the inputs are confirmed (step 3)
41
+ - Hand-write test files inline instead of spawning the sub-agent the mode specifies
42
+ - Delete or rewrite working tests without user approval (update mode is surgical: add / fix / improve, never replace)
43
+ - Assume context, app URL, framework, ticket, or any value from previous command
44
+ invocations (/qa-start, prior /qa-create-test calls, etc.). Each invocation is
45
+ self-contained the detector returns defaults precisely because the user did
46
+ not pass them.
47
+ - Invent steps not in this command, or reorder the ones that are
48
+
49
+ If you find yourself improvising, deciding the mode before the detector runs,
50
+ generating files before confirmation, or asking questions out of order — STOP
51
+ and restart from step 1.
52
+
53
+ ### Pass user input literally to the intent detector
54
+
55
+ When you run the intent detector with `$ARGUMENTS`, substitute that placeholder
56
+ with the RAW user input as received, NOT a pre-translated flag string. If you
57
+ pre-translate, the detector reports every value as `source: flag`, sets
58
+ `ALL_FROM_FLAGS=true`, and bypasses the confirmation checkpoint.
59
+
60
+
61
+ ## Usage
62
+
63
+ ```
64
+ /qa-create-test <feature-or-source> [options]
65
+ ```
66
+
67
+ ### Modes (auto-detected from arguments)
68
+
69
+ | Mode | Trigger | Example |
70
+ |------|---------|---------|
71
+ | **From code** | Feature name (no URL, no path to tests) | `/qa-create-test login` |
72
+ | **From ticket** | URL, shorthand (#123), or `--ticket` flag | `/qa-create-test https://github.com/org/repo/issues/42` |
73
+ | **From AUT** | Non-ticket URL or `--app-url` (explore live app) | `/qa-create-test signup on https://staging.example.com` |
74
+ | **Azure DevOps** | `--ado` flag with work item ID or ADO URL | `/qa-create-test --ado 85508` |
75
+ | **Update existing** | Path to existing test files or `--update` flag | `/qa-create-test --update tests/e2e/` |
76
+ | **POM only** | `--pom-only` flag | `/qa-create-test --pom-only src/pages/` |
77
+
78
+ ### Options
79
+
80
+ - `--dev-repo <path>` — path to developer repository (default: current directory)
81
+ - `--app-url <url>` URL of running application for E2E execution and locator extraction (auto-detects if not provided)
82
+ - `--skip-run` — skip E2E execution, only generate and statically validate
83
+ - `--ticket <source>` — force ticket mode with: URL, shorthand (#123, org/repo#123), file path, or plain text
84
+ - `--update <path>` — force update mode: audit and improve existing tests at path
85
+ - `--scope fix|improve|add|full` for update mode only (default: full)
86
+ - `--ado <work-item-id>` Azure DevOps mode: read a work item and create Test Cases in ADO (accepts ID or full ADO URL)
87
+ - `--area-path <path>` (ADO mode) override area path for created test cases (default: source work item's area path)
88
+ - `--iteration-path <path>` — (ADO mode) override iteration path for created test cases (default: source work item's iteration path)
89
+ - `--skip-dedup` — (ADO mode) skip deduplication check against existing linked test cases
90
+ - `--pom-only [path]` — generate only Page Object Model files (BasePage + feature POMs), no test specs
91
+ - `--framework <name>` — override framework auto-detection (playwright, cypress, selenium) — used with --pom-only
92
+
93
+ ### Mode Detection Logic
94
+
95
+ ```
96
+ if --pom-only:
97
+ MODE = "pom-only"
98
+ elif --ado flag OR argument matches ADO URL (dev.azure.com, *.visualstudio.com):
99
+ MODE = "ado"
100
+ elif argument matches ticket URL pattern (github.com, atlassian.net, linear.app) OR contains "#" + digits OR --ticket flag:
101
+ MODE = "from-ticket"
102
+ elif argument matches URL pattern (https?://) AND NOT a ticket platform:
103
+ MODE = "from-aut" # NEW: lightweight URL tests, single-shot
104
+ elif --update flag OR argument is path to existing test directory/files:
105
+ MODE = "update"
106
+ else:
107
+ MODE = "from-code"
108
+ ```
109
+
110
+ **Mode detection uses `bin/lib/intent-detector.cjs`** for robust URL/path/framework parsing. The detector already classifies URLs as ticket vs AUT, so the workflow can branch cleanly between `from-ticket` and `from-aut` modes.
111
+
112
+ ## What It Produces
113
+
114
+ ### From Code Mode
115
+ - Test spec files (unit, API, E2E as appropriate)
116
+ - Page Object Model files (for E2E tests)
117
+ - Fixture files (test data)
118
+ - Locator registry entries (`.qa-output/locators/`)
119
+ - E2E_RUN_REPORT.md (if E2E tests ran against live app)
120
+
121
+ ### From Ticket Mode
122
+ - TEST_CASES_FROM_TICKET.md traceability matrix (AC test case)
123
+ - GENERATION_PLAN_TICKET.md — synthetic generation plan
124
+ - Test spec files with `traces_to` fields linking back to ticket ACs
125
+ - VALIDATION_REPORT.md
126
+
127
+ ### Azure DevOps Mode
128
+ - Test Cases created directly in Azure DevOps (via `testplan_create_test_case`)
129
+ - Test Cases linked to source work item via *Tested By* relationship
130
+ - Tags applied: `Smoke`, `Regression`, `Critical`, `AutomationCandidate`, `NeedsReview`
131
+ - `ai-tasks/ticket-{id}/test-cases.md` — structured report
132
+ - Report attached to work item (if `ADO_MCP_AUTH_TOKEN` is set) or written to `Custom.QATestCasesReport` field (fallback)
133
+
134
+ ### From AUT Mode
135
+
136
+ - Test spec files generated directly from URL exploration (no full pipeline)
137
+ - Page Object Model files for the discovered pages
138
+ - Locator registry entries from real DOM (via Playwright MCP)
139
+ - VALIDATION_REPORT.md
140
+ - No PR created local delivery only
141
+
142
+ The agent navigates the URL with Playwright MCP, discovers interactive elements, and generates tests targeting them. Skips codebase-map check, scan, and analyze stages. **Faster than `/qa-start --app-url`** when you only need tests for one specific URL.
143
+
144
+ ### Update Mode
145
+ - QA_AUDIT_REPORT.md current quality assessment
146
+ - Improved test files (after user approval)
147
+
148
+
149
+ ## Intent Detection
150
+
151
+ Before processing arguments, run the intent detector to support natural-language input alongside flags. Flags always win over NL.
152
+
153
+ ```bash
154
+ RESOLVED=$(node ~/.claude/qaa/bin/lib/intent-detector.cjs --resolve "$ARGUMENTS" --aliases '{"app_url":"app-url","framework":"framework","ticket":"ticket","ado":"ado","update":"update","pom_only":"pom-only","skip_map":"skip-map","skip_run":"skip-run"}')
155
+
156
+ # Print INPUT DETECTION banner (always shown, even if defaults are used)
157
+ echo "$RESOLVED" | node -e "const j=JSON.parse(require('fs').readFileSync(0,'utf8')); console.log(j.banner)"
158
+
159
+ # Extract resolved values
160
+ APP_URL=$(echo "$RESOLVED" | node -e "const j=JSON.parse(require('fs').readFileSync(0,'utf8')); console.log(j.resolved.app_url?.value || '')")
161
+ FRAMEWORK=$(echo "$RESOLVED" | node -e "const j=JSON.parse(require('fs').readFileSync(0,'utf8')); console.log(j.resolved.framework?.value || '')")
162
+ TICKET=$(echo "$RESOLVED" | node -e "const j=JSON.parse(require('fs').readFileSync(0,'utf8')); console.log(j.resolved.ticket?.value || '')")
163
+ SUGGESTED_MODE=$(echo "$RESOLVED" | node -e "const j=JSON.parse(require('fs').readFileSync(0,'utf8')); console.log(j.intent.suggested_mode || '')")
164
+ ```
165
+
166
+ The detector resolves the mode automatically when possible (from-ticket, from-aut, ado, etc.) and surfaces it in `SUGGESTED_MODE`. The mode detection logic in Step 1 below uses these resolved values.
167
+
168
+ ## Instructions
169
+
170
+ ### Step 1: Detect Mode
171
+
172
+ Parse `$ARGUMENTS` to determine mode using the detection logic above.
173
+
174
+ Print mode banner:
175
+ ```
176
+ === QA Create Test ===
177
+ Mode: {from-code | from-aut | from-ticket | ado | update | pom-only}
178
+ Target: {feature name | ticket URL | ADO work item ID | test path}
179
+ App URL: {url or "auto-detect"}
180
+ ===========================
181
+ ```
182
+
183
+ ---
184
+
185
+ ### FROM CODE MODE
186
+
187
+ 1. Read `CLAUDE.md` POM rules, locator tiers, assertion rules, naming conventions, quality gates.
188
+ 1b. Read `~/.claude/qaa/MY_PREFERENCES.md` if it exists — user's personal QA preferences override CLAUDE.md defaults when there is a conflict.
189
+ 2. Read existing analysis artifacts if available:
190
+ - `.qa-output/QA_ANALYSIS.md` — architecture context
191
+ - `.qa-output/TEST_INVENTORY.md` pre-defined test cases for this feature
192
+ 3. **Check for codebase map** (`.qa-output/codebase/`):
193
+ - Look for: `CODE_PATTERNS.md`, `API_CONTRACTS.md`, `TEST_SURFACE.md`, `TESTABILITY.md`
194
+ - If at least 2 of these files exist: read them all for project context (naming conventions, API shapes, testable surfaces).
195
+ - **If NONE of these files exist: STOP and tell the user:**
196
+ ```
197
+ No codebase map found (.qa-output/codebase/ is empty or missing).
198
+
199
+ The codebase map provides critical context: naming conventions, API contracts,
200
+ testable surfaces, and project structure. Without it, generated tests will lack
201
+ project-specific context and may not follow your repo's conventions.
202
+
203
+ Run /qa-map first to generate the codebase map, then re-run /qa-create-test.
204
+
205
+ To skip this check and proceed without context: re-run with --skip-map
206
+ ```
207
+ Only proceed without codebase map if the user explicitly passes `--skip-map`.
208
+
209
+ 4. **Check existing locator registry and extract new locators from live app:**
210
+
211
+ a. Read `.qa-output/locators/LOCATOR_REGISTRY.md` if it exists.
212
+
213
+ b. If locators for this feature already exist in the registry AND no `--app-url` was provided: reuse cached locators.
214
+
215
+ c. If locators are missing or `--app-url` was provided: Use Playwright MCP to navigate the app and extract real locators:
216
+ ```
217
+ mcp__playwright__browser_navigate({ url: "{app_url}/{feature_path}" })
218
+ mcp__playwright__browser_snapshot()
219
+ ```
220
+ Extract all data-testid, ARIA roles, labels, placeholders.
221
+ Navigate through multi-page flows if needed.
222
+ Write per-feature locator file to `.qa-output/locators/{feature}.locators.md`.
223
+ Update the registry `.qa-output/locators/LOCATOR_REGISTRY.md`.
224
+
225
+ If no app URL available and no locators in registry, skip — executor proposes locators from source code.
226
+
227
+ 5. Invoke executor agent to generate test files:
228
+
229
+ Task(
230
+ prompt="
231
+ <critical_directive priority="MANDATORY">
232
+ You are executing as a QAA pipeline agent. Your FIRST action MUST be to read
233
+ ~/.claude/qaa/agents/qaa-executor.md and adopt it as your operating contract.
234
+
235
+ Even though you may be running as a general-purpose agent (native subagent type
236
+ routing for QAA agents is not yet enabled), you MUST behave exactly as the
237
+ dedicated agent defined in that file: read everything in its <required_reading>,
238
+ obey its <quality_gate>, and run its mandatory bash checklist before returning.
239
+
240
+ Never modify application source code (src/, app/, lib/, or equivalent
241
+ directories) unless the agent's <scope> section explicitly permits it. If the
242
+ agent's contract restricts what files it can touch, honor that restriction —
243
+ even if a fix seems obvious or trivial.
244
+
245
+ If you cannot read ~/.claude/qaa/agents/qaa-executor.md (file missing, corrupted, parse
246
+ error), STOP and report the failure. Do NOT proceed with default agent
247
+ behavior that defeats the purpose of this directive.
248
+
249
+ Do NOT improvise, substitute steps, apply changes the agent isn't allowed to
250
+ make, or skip any of the above. Skipping the agent's required_reading,
251
+ quality_gate, or bash checklist makes the run INVALID — its output cannot be
252
+ trusted.
253
+ </critical_directive>
254
+
255
+ <objective>Generate test files for the specified feature following CLAUDE.md standards, using codebase map and research documents for context. Query Context7 MCP to verify framework syntax before generating code.</objective>
256
+ <execution_context>~/.claude/qaa/agents/qaa-executor.md</execution_context>
257
+ <files_to_read>
258
+ - CLAUDE.md
259
+ - ~/.claude/qaa/MY_PREFERENCES.md (if exists)
260
+ - .qa-output/locators/LOCATOR_REGISTRY.md (if exists)
261
+ - .qa-output/locators/{feature}.locators.md (if exists)
262
+ - .qa-output/codebase/CODE_PATTERNS.md (if exists)
263
+ - .qa-output/codebase/API_CONTRACTS.md (if exists)
264
+ - .qa-output/codebase/TEST_SURFACE.md (if exists)
265
+ - .qa-output/codebase/TESTABILITY.md (if exists)
266
+ - .qa-output/research/FRAMEWORK_CAPABILITIES.md (if exists)
267
+ - .qa-output/research/E2E_STRATEGY.md (if exists)
268
+ - .qa-output/research/API_TESTING_STRATEGY.md (if exists)
269
+ - .qa-output/research/TESTING_STACK.md (if exists)
270
+ </files_to_read>
271
+ <parameters>
272
+ user_input: $ARGUMENTS
273
+ mode: feature-test
274
+ codebase_map_dir: .qa-output/codebase
275
+ locator_registry: .qa-output/locators/LOCATOR_REGISTRY.md
276
+ </parameters>
277
+ "
278
+ )
279
+
280
+ 6. If E2E test files were generated AND `--skip-run` was NOT passed, invoke E2E runner:
281
+
282
+ Task(
283
+ prompt="
284
+ <critical_directive priority="MANDATORY">
285
+ You are executing as a QAA pipeline agent. Your FIRST action MUST be to read
286
+ ~/.claude/qaa/agents/qaa-e2e-runner.md and adopt it as your operating contract.
287
+
288
+ Even though you may be running as a general-purpose agent (native subagent type
289
+ routing for QAA agents is not yet enabled), you MUST behave exactly as the
290
+ dedicated agent defined in that file: read everything in its <required_reading>,
291
+ obey its <quality_gate>, and run its mandatory bash checklist before returning.
292
+
293
+ Never modify application source code (src/, app/, lib/, or equivalent
294
+ directories) unless the agent's <scope> section explicitly permits it. If the
295
+ agent's contract restricts what files it can touch, honor that restriction —
296
+ even if a fix seems obvious or trivial.
297
+
298
+ If you cannot read ~/.claude/qaa/agents/qaa-e2e-runner.md (file missing, corrupted, parse
299
+ error), STOP and report the failure. Do NOT proceed with default agent
300
+ behavior — that defeats the purpose of this directive.
301
+
302
+ Do NOT improvise, substitute steps, apply changes the agent isn't allowed to
303
+ make, or skip any of the above. Skipping the agent's required_reading,
304
+ quality_gate, or bash checklist makes the run INVALID — its output cannot be
305
+ trusted.
306
+ </critical_directive>
307
+
308
+ <objective>Run generated E2E tests against live application, capture real locators, fix mismatches, loop until pass. Query Context7 MCP to verify framework selector syntax before fixing locators.</objective>
309
+ <execution_context>~/.claude/qaa/agents/qaa-e2e-runner.md</execution_context>
310
+ <files_to_read>
311
+ - CLAUDE.md
312
+ - ~/.claude/qaa/MY_PREFERENCES.md (if exists)
313
+ - .qa-output/locators/LOCATOR_REGISTRY.md (if exists)
314
+ - .qa-output/research/FRAMEWORK_CAPABILITIES.md (if exists)
315
+ - .qa-output/research/E2E_STRATEGY.md (if exists)
316
+ - {generated E2E test files from executor return}
317
+ - {generated POM files from executor return}
318
+ </files_to_read>
319
+ <parameters>
320
+ app_url: {from --app-url flag or auto-detect}
321
+ output_dir: .qa-output
322
+ </parameters>
323
+ "
324
+ )
325
+
326
+ 7. Present results with file counts and suggest `/qa-pr`.
327
+
328
+ ---
329
+
330
+ ### FROM AUT MODE
331
+
332
+ Generate tests directly from a running application URL — no ticket, no spec, no full pipeline. Triggered when an AUT URL (non-ticket) is resolved (or `--app-url` in NL) and no other mode applies. Faster than `/qa-start --app-url` when you only need tests for one specific flow. **No PR is created — local delivery only.**
333
+
334
+ **Production-URL safety (read first):** if the AUT is a live/production site, exploration AND generated tests MUST be read-only. NEVER submit forms, create accounts/records, post data, or trigger destructive mutations against production. For flows that open a modal/form, assert on its presence/contents then CANCEL — never submit. Reserve any state-changing path for validation-failure assertions only. Never hardcode credentials.
335
+
336
+ 1. Read `CLAUDE.md` (POM rules, locator tiers, assertion rules, naming, quality gates) and `~/.claude/qaa/MY_PREFERENCES.md` if it exists.
337
+
338
+ 2. **Match existing conventions.** If a test repo already exists (cypress/, tests/, page-objects/, etc.), read 2-3 existing specs + the POM base + the selectors/fixtures registry so generated tests match the project's structure, naming, and locator style. Do NOT invent a parallel structure.
339
+
340
+ 3. **Explore the app with Playwright MCP.** For each route/flow in the request:
341
+ ```
342
+ mcp__playwright__browser_navigate({ url: "{app_url}{route}" })
343
+ mcp__playwright__browser_snapshot()
344
+ ```
345
+ Discover interactive elements and real locators (data-testid, ARIA roles, labels, placeholders) and the ACTUAL behavior. Reconcile the request against reality: if the live DOM contradicts the user's description (e.g. a button isn't rendered for anonymous users), record the discrepancy — do NOT generate a test that asserts behavior the app does not exhibit. Persist verified locators to `.qa-output/locators/{feature}.locators.md` and update `.qa-output/locators/LOCATOR_REGISTRY.md`.
346
+
347
+ 4. **Auth handling.** If a flow requires authentication: if `TEST_USER_*` creds are configured (cypress.env.json / env vars), use the project's login command (e.g. `cy.loginAs()`); otherwise generate the authenticated specs but guard them to skip cleanly (e.g. `cy.skipIfNoTestAccount()`) and tell the user they need creds to run them.
348
+
349
+ 5. **CHECKPOINT present the test plan and WAIT for approval.** Before writing ANY file, present the inferred plan and pause:
350
+ ```
351
+ === FROM-AUT TEST PLAN ===
352
+ App: {app_url}
353
+ Flow: {what was requested}
354
+ Framework: {detected}
355
+ Proposed test cases:
356
+ 1. [ID] [priority] {title} {concrete assertion(s)} (locators: tier {N})
357
+ 2. ...
358
+ Discrepancies vs request: {list, or "none"}
359
+ Files to create: {spec paths, POM paths, fixtures}
360
+ Auth: {creds available → live | no creds → generated-but-skipped}
361
+ ==========================
362
+ Proceed with generation? (yes / modify / cancel)
363
+ ```
364
+ Do NOT generate files until the user approves. On "modify" adjust and re-present. On "cancel" stop and deliver only the plan. (This gate is mandatoryfrom-aut writes specs against a live app; never generate a batch without approval.)
365
+
366
+ 6. After approval, invoke the executor to generate specs + POMs + fixtures:
367
+
368
+ Task(
369
+ prompt="
370
+ <critical_directive priority="MANDATORY">
371
+ You are executing as a QAA pipeline agent. Your FIRST action MUST be to read
372
+ ~/.claude/qaa/agents/qaa-executor.md and adopt it as your operating contract.
373
+
374
+ Even though you may be running as a general-purpose agent (native subagent type
375
+ routing for QAA agents is not yet enabled), you MUST behave exactly as the
376
+ dedicated agent defined in that file: read everything in its <required_reading>,
377
+ obey its <quality_gate>, and run its mandatory bash checklist before returning.
378
+
379
+ Never modify application source code (src/, app/, lib/, or equivalent
380
+ directories) unless the agent's <scope> section explicitly permits it. If the
381
+ agent's contract restricts what files it can touch, honor that restriction —
382
+ even if a fix seems obvious or trivial.
383
+
384
+ If you cannot read ~/.claude/qaa/agents/qaa-executor.md (file missing, corrupted, parse
385
+ error), STOP and report the failure. Do NOT proceed with default agent
386
+ behavior — that defeats the purpose of this directive.
387
+
388
+ Do NOT improvise, substitute steps, apply changes the agent isn't allowed to
389
+ make, or skip any of the above. Skipping the agent's required_reading,
390
+ quality_gate, or bash checklist makes the run INVALID — its output cannot be
391
+ trusted.
392
+ </critical_directive>
393
+
394
+ <objective>Generate the approved from-aut test cases following CLAUDE.md standards and the existing repo conventions, using the verified locators from the registry. Read-only against production — never generate a spec that submits/creates data. Query Context7 MCP to verify framework syntax before generating code.</objective>
395
+ <execution_context>~/.claude/qaa/agents/qaa-executor.md</execution_context>
396
+ <files_to_read>
397
+ - CLAUDE.md
398
+ - ~/.claude/qaa/MY_PREFERENCES.md (if exists)
399
+ - .qa-output/locators/LOCATOR_REGISTRY.md
400
+ - .qa-output/locators/{feature}.locators.md
401
+ - .qa-output/research/FRAMEWORK_CAPABILITIES.md (if exists)
402
+ - {2-3 existing spec files + the POM base, to match conventions}
403
+ </files_to_read>
404
+ <parameters>
405
+ user_input: $ARGUMENTS
406
+ mode: from-aut
407
+ app_url: {resolved app_url}
408
+ approved_plan: {the test cases the user approved in step 5}
409
+ locator_registry: .qa-output/locators/LOCATOR_REGISTRY.md
410
+ </parameters>
411
+ "
412
+ )
413
+
414
+ 7. If E2E specs were generated AND `--skip-run` was NOT passed AND an app URL is available, invoke the e2e-runner (same Task() spawn as FROM CODE MODE step 6). Honor the production-safety rule — read-only flows only.
415
+
416
+ 8. Present results (file counts, locator tiers, any discrepancies surfaced in step 3), then suggest `/qa-pr`. No PR is created in this mode — local delivery only.
417
+
418
+ ---
419
+
420
+ ### FROM TICKET MODE
421
+
422
+ 1. Read `CLAUDE.md` — all QA standards.
423
+ 2. Execute the ticket workflow end-to-end:
424
+
425
+ Follow the workflow defined in `@workflows/qa-from-ticket.md` end-to-end.
426
+ Preserve all workflow gates (ticket parsing, acceptance criteria extraction, traceability matrix, validation).
427
+
428
+ Key steps in the workflow:
429
+ - Parse ticket source (GitHub URL, Jira URL, Linear URL, file, or plain text)
430
+ - Fetch ticket content (via `gh issue view`, WebFetch, or file read)
431
+ - Extract acceptance criteria, user stories, edge cases
432
+ - Scan dev repo for related source files
433
+ - Extract locators from live app via Playwright MCP (if app URL available)
434
+ - Generate test cases with traceability matrix (every AC maps to ≥1 test case)
435
+ - Spawn executor agent to produce test files
436
+ - Spawn validator agent for 4-layer validation
437
+ - Print summary with AC coverage and traceability
438
+
439
+ **Traceability guarantee:** Every acceptance criterion maps to at least one test case via the `traces_to` field.
440
+
441
+ **Supported ticket sources:**
442
+
443
+ | Format | Example | Detection |
444
+ |--------|---------|-----------|
445
+ | GitHub Issue URL | `https://github.com/org/repo/issues/123` | Contains `github.com` + `/issues/` |
446
+ | GitHub shorthand | `org/repo#123` or `#123` | Contains `#` + digits |
447
+ | Jira URL | `https://company.atlassian.net/browse/PROJ-123` | Contains `.atlassian.net/browse/` |
448
+ | Linear URL | `https://linear.app/team/issue/TEAM-123` | Contains `linear.app` |
449
+ | File path | `./tickets/feature-spec.md` | Path exists on disk |
450
+ | Plain text | `"As a user I want to..."` | None of the above match |
451
+
452
+ ---
453
+
454
+ ### ADO MODE (Azure DevOps)
455
+
456
+ Create Test Cases directly in Azure DevOps from a work item. Reads the work item content (repro steps, acceptance criteria, comments, attachments), designs test cases with boundary detection and deduplication, and creates them in ADO with full traceability.
457
+
458
+ **Prerequisites:** ADO MCP server must be connected (provides `wit_get_work_item`, `testplan_create_test_case`, etc.).
459
+
460
+ Execute the full ADO workflow defined in `@commands/qa-create-test-ado.md`:
461
+
462
+ 1. **Phase 1** — Read pipeline context: CLAUDE.md, MY_PREFERENCES.md, codebase map, locator registry
463
+ 2. **Phase 2** — Retrieve work item with relations, comments, and attachments
464
+ 3. **Phase 2b** — Deduplication check against existing linked test cases (skip with `--skip-dedup`)
465
+ 4. **Phase 3** — Extract test source content based on work item type (Bug → Repro Steps, User Story → Acceptance Criteria)
466
+ 5. **Phase 4** — Design test cases with boundary value detection, tagging rules, confidence scoring, and preconditions
467
+ 6. **Phase 5** — Create test cases in ADO via `testplan_create_test_case`, link via *Tested By*, set tags
468
+ 7. **Phase 6** — Generate structured report to `ai-tasks/ticket-{id}/test-cases.md`
469
+ 8. **Phase 7** — Attach report to source work item
470
+
471
+ **Key features:**
472
+ - Boundary value triplets: detects `max`, `min`, `limit`, `threshold` keywords with numeric values → generates N-1, N, N+1 test cases
473
+ - Deduplication: checks existing linked TCs before creating, prevents duplicates
474
+ - Confidence scoring: `Specified` (explicit source) vs `Draft` (inferred, tagged `NeedsReview`)
475
+ - Cross-references codebase map for project-specific context when available
476
+ - Supports `--area-path` and `--iteration-path` overrides
477
+
478
+ For the complete step-by-step process, see `@commands/qa-create-test-ado.md`.
479
+
480
+ ---
481
+
482
+ ### UPDATE MODE
483
+
484
+ 1. Read `CLAUDE.md` — quality gates, locator tiers, assertion rules, POM rules.
485
+ 2. Invoke validator agent in audit mode:
486
+
487
+ Task(
488
+ prompt="
489
+ <critical_directive priority="MANDATORY">
490
+ You are executing as a QAA pipeline agent. Your FIRST action MUST be to read
491
+ ~/.claude/qaa/agents/qaa-validator.md and adopt it as your operating contract.
492
+
493
+ Even though you may be running as a general-purpose agent (native subagent type
494
+ routing for QAA agents is not yet enabled), you MUST behave exactly as the
495
+ dedicated agent defined in that file: read everything in its <required_reading>,
496
+ obey its <quality_gate>, and run its mandatory bash checklist before returning.
497
+
498
+ Never modify application source code (src/, app/, lib/, or equivalent
499
+ directories) unless the agent's <scope> section explicitly permits it. If the
500
+ agent's contract restricts what files it can touch, honor that restriction —
501
+ even if a fix seems obvious or trivial.
502
+
503
+ If you cannot read ~/.claude/qaa/agents/qaa-validator.md (file missing, corrupted, parse
504
+ error), STOP and report the failure. Do NOT proceed with default agent
505
+ behavior — that defeats the purpose of this directive.
506
+
507
+ Do NOT improvise, substitute steps, apply changes the agent isn't allowed to
508
+ make, or skip any of the above. Skipping the agent's required_reading,
509
+ quality_gate, or bash checklist makes the run INVALID — its output cannot be
510
+ trusted.
511
+ </critical_directive>
512
+
513
+ <objective>Audit existing test quality and produce QA_AUDIT_REPORT.md. If Playwright MCP is connected, verify E2E test locators against the live DOM via browser_navigate + browser_snapshot.</objective>
514
+ <execution_context>~/.claude/qaa/agents/qaa-validator.md</execution_context>
515
+ <files_to_read>
516
+ - CLAUDE.md
517
+ - ~/.claude/qaa/MY_PREFERENCES.md (if exists)
518
+ </files_to_read>
519
+ <parameters>
520
+ user_input: $ARGUMENTS
521
+ mode: audit
522
+ app_url: {auto-detect from test config baseURL, or ask user}
523
+ </parameters>
524
+ "
525
+ )
526
+
527
+ 3. Present audit results and wait for user approval.
528
+ 4. Invoke executor agent to apply approved improvements:
529
+
530
+ Task(
531
+ prompt="
532
+ <critical_directive priority="MANDATORY">
533
+ You are executing as a QAA pipeline agent. Your FIRST action MUST be to read
534
+ ~/.claude/qaa/agents/qaa-executor.md and adopt it as your operating contract.
535
+
536
+ Even though you may be running as a general-purpose agent (native subagent type
537
+ routing for QAA agents is not yet enabled), you MUST behave exactly as the
538
+ dedicated agent defined in that file: read everything in its <required_reading>,
539
+ obey its <quality_gate>, and run its mandatory bash checklist before returning.
540
+
541
+ Never modify application source code (src/, app/, lib/, or equivalent
542
+ directories) unless the agent's <scope> section explicitly permits it. If the
543
+ agent's contract restricts what files it can touch, honor that restriction —
544
+ even if a fix seems obvious or trivial.
545
+
546
+ If you cannot read ~/.claude/qaa/agents/qaa-executor.md (file missing, corrupted, parse
547
+ error), STOP and report the failure. Do NOT proceed with default agent
548
+ behavior — that defeats the purpose of this directive.
549
+
550
+ Do NOT improvise, substitute steps, apply changes the agent isn't allowed to
551
+ make, or skip any of the above. Skipping the agent's required_reading,
552
+ quality_gate, or bash checklist makes the run INVALID — its output cannot be
553
+ trusted.
554
+ </critical_directive>
555
+
556
+ <objective>Apply approved improvements to existing tests without deleting working tests. If Playwright MCP is connected, use browser_navigate + browser_snapshot to extract real locators when upgrading from Tier 4 to Tier 1.</objective>
557
+ <execution_context>~/.claude/qaa/agents/qaa-executor.md</execution_context>
558
+ <files_to_read>
559
+ - CLAUDE.md
560
+ - ~/.claude/qaa/MY_PREFERENCES.md (if exists)
561
+ - .qa-output/QA_AUDIT_REPORT.md
562
+ - .qa-output/locators/LOCATOR_REGISTRY.md (if exists)
563
+ </files_to_read>
564
+ <parameters>
565
+ user_input: $ARGUMENTS
566
+ mode: update
567
+ app_url: {auto-detect from test config baseURL, or ask user}
568
+ </parameters>
569
+ "
570
+ )
571
+
572
+ **Update scopes:**
573
+ - `fix` — repair broken tests only
574
+ - `improve` — upgrade locators, assertions, POM structure
575
+ - `add` — add missing test cases without modifying existing
576
+ - `full` — audit everything, then improve with approval (default)
577
+
578
+ **Rule:** NEVER delete or rewrite working tests without user approval. Surgical: add, fix, improve — never replace.
579
+
580
+ ---
581
+
582
+ ### POM ONLY MODE (`--pom-only`)
583
+
584
+ Generate only Page Object Model files — no test specs.
585
+
586
+ 1. Read `CLAUDE.md` — POM rules, locator tier hierarchy, naming conventions.
587
+ 2. Invoke executor agent in POM-only mode:
588
+
589
+ Task(
590
+ prompt="
591
+ <critical_directive priority="MANDATORY">
592
+ You are executing as a QAA pipeline agent. Your FIRST action MUST be to read
593
+ ~/.claude/qaa/agents/qaa-executor.md and adopt it as your operating contract.
594
+
595
+ Even though you may be running as a general-purpose agent (native subagent type
596
+ routing for QAA agents is not yet enabled), you MUST behave exactly as the
597
+ dedicated agent defined in that file: read everything in its <required_reading>,
598
+ obey its <quality_gate>, and run its mandatory bash checklist before returning.
599
+
600
+ Never modify application source code (src/, app/, lib/, or equivalent
601
+ directories) unless the agent's <scope> section explicitly permits it. If the
602
+ agent's contract restricts what files it can touch, honor that restriction —
603
+ even if a fix seems obvious or trivial.
604
+
605
+ If you cannot read ~/.claude/qaa/agents/qaa-executor.md (file missing, corrupted, parse
606
+ error), STOP and report the failure. Do NOT proceed with default agent
607
+ behavior — that defeats the purpose of this directive.
608
+
609
+ Do NOT improvise, substitute steps, apply changes the agent isn't allowed to
610
+ make, or skip any of the above. Skipping the agent's required_reading,
611
+ quality_gate, or bash checklist makes the run INVALID — its output cannot be
612
+ trusted.
613
+ </critical_directive>
614
+
615
+ <objective>Generate Page Object Models following CLAUDE.md POM rules. If Playwright MCP is connected and an app URL is available, navigate each page first to extract real locators (data-testid, ARIA roles, labels) from the live DOM via browser_navigate + browser_snapshot before generating POMs. This ensures POM locators match the real app instead of guessing from source code.</objective>
616
+ <execution_context>~/.claude/qaa/agents/qaa-executor.md</execution_context>
617
+ <files_to_read>
618
+ - CLAUDE.md
619
+ - ~/.claude/qaa/MY_PREFERENCES.md (if exists)
620
+ - .qa-output/locators/LOCATOR_REGISTRY.md (if exists)
621
+ </files_to_read>
622
+ <parameters>
623
+ user_input: $ARGUMENTS
624
+ mode: pom-only
625
+ app_url: {auto-detect from test config baseURL, or ask user}
626
+ </parameters>
627
+ "
628
+ )
629
+
630
+ **Produces:**
631
+ - BasePage file (if not already present)
632
+ - Feature-specific POM files following `[PageName]Page.[ext]` naming convention
633
+ - No test specs, no fixtures
634
+
635
+ **POM rules enforced:**
636
+ - One class per page — no god objects
637
+ - No assertions in page objects — assertions belong ONLY in test specs
638
+ - Locators as readonly properties — Tier 1 preferred (data-testid, ARIA roles)
639
+ - Actions return void or next page — for fluent chaining
640
+ - State queries return data — let the test decide what to assert
641
+ - Every POM extends BasePage
642
+
643
+ $ARGUMENTS
644
+
645
+ ## MANDATORY verification — run ALL commands below, no exceptions, no skipping
646
+
647
+ Before returning control, copy-paste and run this ENTIRE block. Do NOT decide which commands "apply" — run all of them every time. The output confirms what happened; you do not get to assume the answer.
648
+
649
+ ```bash
650
+ echo "=== CHECKLIST START ==="
651
+ echo "1. Locator Registry:"
652
+ ls .qa-output/locators/ 2>/dev/null || echo "NO_LOCATORS_FOUND"
653
+ echo "2. MY_PREFERENCES.md:"
654
+ cat ~/.claude/qaa/MY_PREFERENCES.md 2>/dev/null || echo "FILE_NOT_FOUND"
655
+ echo "3. Generated test files:"
656
+ find tests/ pages/ fixtures/ -type f 2>/dev/null | head -20 || echo "NO_TEST_FILES_FOUND"
657
+ echo "4. MCP evidence (if browser was used):"
658
+ ls .qa-output/mcp-evidence/ 2>/dev/null || echo "NO_MCP_EVIDENCE"
659
+ echo "=== CHECKLIST END ==="
660
+ ```
661
+
662
+ **Rules:**
663
+ - Run the block AS-IS. Do not modify it. Do not split it. Do not skip lines.
664
+ - If any output shows a problem (NO_LOCATORS when MCP was used, NO_TEST_FILES after generation), fix it before returning.
665
+ - If output shows expected "not found" results (e.g., NO_MCP_EVIDENCE when no browser was used), that is fine — the point is you RAN the command instead of assuming the answer.
666
+ - Do NOT mark this task as complete until the block has been executed and you have read every line of output.