qaa-agent 1.6.2 → 1.7.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (78) hide show
  1. package/.mcp.json +8 -8
  2. package/CHANGELOG.md +93 -71
  3. package/CLAUDE.md +553 -553
  4. package/agents/qa-pipeline-orchestrator.md +1378 -1378
  5. package/agents/qaa-analyzer.md +539 -524
  6. package/agents/qaa-bug-detective.md +479 -446
  7. package/agents/qaa-codebase-mapper.md +935 -935
  8. package/agents/qaa-discovery.md +384 -0
  9. package/agents/qaa-e2e-runner.md +416 -415
  10. package/agents/qaa-executor.md +651 -651
  11. package/agents/qaa-planner.md +405 -390
  12. package/agents/qaa-project-researcher.md +319 -319
  13. package/agents/qaa-scanner.md +424 -424
  14. package/agents/qaa-testid-injector.md +643 -585
  15. package/agents/qaa-validator.md +490 -452
  16. package/bin/install.cjs +200 -198
  17. package/bin/lib/commands.cjs +709 -709
  18. package/bin/lib/config.cjs +307 -307
  19. package/bin/lib/core.cjs +497 -497
  20. package/bin/lib/frontmatter.cjs +299 -299
  21. package/bin/lib/init.cjs +989 -989
  22. package/bin/lib/milestone.cjs +241 -241
  23. package/bin/lib/model-profiles.cjs +60 -60
  24. package/bin/lib/phase.cjs +911 -911
  25. package/bin/lib/roadmap.cjs +306 -306
  26. package/bin/lib/state.cjs +748 -748
  27. package/bin/lib/template.cjs +222 -222
  28. package/bin/lib/verify.cjs +842 -842
  29. package/bin/qaa-tools.cjs +607 -607
  30. package/commands/qa-audit.md +119 -0
  31. package/commands/qa-create-test.md +288 -0
  32. package/commands/qa-fix.md +147 -0
  33. package/commands/qa-map.md +137 -0
  34. package/{.claude/commands → commands}/qa-pr.md +23 -23
  35. package/{.claude/commands → commands}/qa-start.md +22 -22
  36. package/{.claude/commands → commands}/qa-testid.md +19 -19
  37. package/docs/COMMANDS.md +341 -341
  38. package/docs/DEMO.md +182 -182
  39. package/docs/TESTING.md +156 -156
  40. package/package.json +6 -7
  41. package/{.claude/settings.json → settings.json} +1 -2
  42. package/templates/failure-classification.md +391 -391
  43. package/templates/gap-analysis.md +409 -409
  44. package/templates/pr-template.md +48 -48
  45. package/templates/qa-analysis.md +381 -381
  46. package/templates/qa-audit-report.md +465 -465
  47. package/templates/qa-repo-blueprint.md +636 -636
  48. package/templates/scan-manifest.md +312 -312
  49. package/templates/test-inventory.md +582 -582
  50. package/templates/testid-audit-report.md +354 -354
  51. package/templates/validation-report.md +243 -243
  52. package/workflows/qa-analyze.md +296 -296
  53. package/workflows/qa-from-ticket.md +536 -536
  54. package/workflows/qa-gap.md +309 -303
  55. package/workflows/qa-pr.md +389 -389
  56. package/workflows/qa-start.md +1192 -1168
  57. package/workflows/qa-testid.md +384 -356
  58. package/workflows/qa-validate.md +299 -295
  59. package/.claude/commands/create-test.md +0 -164
  60. package/.claude/commands/qa-audit.md +0 -37
  61. package/.claude/commands/qa-blueprint.md +0 -54
  62. package/.claude/commands/qa-fix.md +0 -36
  63. package/.claude/commands/qa-from-ticket.md +0 -24
  64. package/.claude/commands/qa-gap.md +0 -20
  65. package/.claude/commands/qa-map.md +0 -47
  66. package/.claude/commands/qa-pom.md +0 -36
  67. package/.claude/commands/qa-pyramid.md +0 -37
  68. package/.claude/commands/qa-report.md +0 -38
  69. package/.claude/commands/qa-research.md +0 -33
  70. package/.claude/commands/qa-validate.md +0 -42
  71. package/.claude/commands/update-test.md +0 -58
  72. package/.claude/skills/qa-learner/SKILL.md +0 -150
  73. /package/{.claude/skills → skills}/qa-bug-detective/SKILL.md +0 -0
  74. /package/{.claude/skills → skills}/qa-repo-analyzer/SKILL.md +0 -0
  75. /package/{.claude/skills → skills}/qa-self-validator/SKILL.md +0 -0
  76. /package/{.claude/skills → skills}/qa-template-engine/SKILL.md +0 -0
  77. /package/{.claude/skills → skills}/qa-testid-injector/SKILL.md +0 -0
  78. /package/{.claude/skills → skills}/qa-workflow-documenter/SKILL.md +0 -0
package/CLAUDE.md CHANGED
@@ -1,553 +1,553 @@
1
- # QA Automation Standards
2
-
3
- This project follows strict QA automation standards. Every test, page object, and analysis produced MUST follow these rules.
4
-
5
- ## Framework Detection
6
-
7
- Before generating any code, **detect what the project already uses**:
8
-
9
- 1. Check for existing test config files: `cypress.config.ts`, `playwright.config.ts`, `jest.config.ts`, `vitest.config.ts`, `pytest.ini`, etc.
10
- 2. Check `package.json` or `requirements.txt` for test dependencies
11
- 3. Check existing test files for patterns and conventions
12
- 4. **Always match the project's existing framework, language, and conventions**
13
-
14
- If no framework exists yet, ask the user which one to use. Never assume.
15
-
16
- ## Testing Pyramid
17
-
18
- Target distribution for every project:
19
-
20
- ```
21
- / E2E \ 3-5% (critical path smoke only)
22
- / API \ 20-25% (endpoints + contracts)
23
- / Integration\ 10-15% (component interactions)
24
- / Unit \ 60-70% (business logic, pure functions)
25
- ```
26
-
27
- Adjust percentages based on the actual app architecture.
28
-
29
- ## Locator Strategy
30
-
31
- All UI test locators MUST follow this priority order. Never skip to a lower tier without written justification.
32
-
33
- **Tier 1 -- BEST (always try these first):**
34
- - Test IDs: `data-testid`, `data-cy`, `data-test` (adapt to framework)
35
- - Semantic roles: ARIA roles + accessible name
36
-
37
- **Tier 2 -- GOOD (when Tier 1 not available):**
38
- - Form labels, placeholders, visible text content
39
-
40
- **Tier 3 -- ACCEPTABLE (when Tier 1-2 not available):**
41
- - Alt text, title attributes
42
-
43
- **Tier 4 -- LAST RESORT (always add a TODO comment):**
44
- - CSS selectors, XPath -- mark with `// TODO: Request test ID for this element`
45
-
46
- ### Framework-Specific Examples
47
-
48
- **Playwright:**
49
- ```typescript
50
- page.getByTestId('submit') // Tier 1
51
- page.getByRole('button', {name: 'Log in'}) // Tier 1
52
- page.getByLabel('Email') // Tier 2
53
- page.locator('.btn') // Tier 4 -- add TODO
54
- ```
55
-
56
- **Cypress:**
57
- ```typescript
58
- cy.get('[data-cy="submit"]') // Tier 1
59
- cy.findByRole('button', {name: 'Log in'}) // Tier 1 (with testing-library)
60
- cy.get('[data-testid="submit"]') // Tier 1
61
- cy.contains('Submit') // Tier 2
62
- cy.get('.btn') // Tier 4 -- add TODO
63
- ```
64
-
65
- **Selenium / other:**
66
- ```
67
- driver.findElement(By.cssSelector('[data-testid="submit"]')) // Tier 1
68
- driver.findElement(By.className('btn')) // Tier 4 -- add TODO
69
- ```
70
-
71
- ## Page Object Model Rules
72
-
73
- These rules apply regardless of framework:
74
-
75
- 1. **One class/object per page or view** -- no god objects
76
- 2. **No assertions in page objects** -- assertions belong ONLY in test specs
77
- 3. **Locators are properties** -- defined in constructor or as class fields
78
- 4. **Actions return void or the next page** -- for fluent chaining
79
- 5. **State queries return data** -- let the test file decide what to assert
80
- 6. **Every POM extends a shared base** -- shared navigation, screenshots, waits
81
-
82
- ### POM File Structure
83
- ```
84
- [pages or page-objects or support/page-objects]/
85
- base/
86
- BasePage.[ext] -- shared methods
87
- [feature]/
88
- [Feature]Page.[ext] -- one file per page
89
- components/
90
- [Component].[ext] -- reusable UI components
91
- ```
92
-
93
- Adapt folder location to match the project's existing conventions.
94
-
95
- ## Test Spec Rules
96
-
97
- ### Every test case MUST have:
98
- - **Unique ID**: `UT-MODULE-001`, `API-AUTH-001`, `E2E-FLOW-001`
99
- - **Exact target**: file path + function name, or HTTP method + endpoint
100
- - **Concrete inputs**: actual values, not "valid data"
101
- - **Explicit expected outcome**: exact assertion, not "works correctly"
102
- - **Priority**: P0 (blocks release), P1 (should fix), P2 (nice to have)
103
-
104
- ### BAD assertions (never do this):
105
- ```
106
- expect(response.status).toBeTruthy()
107
- expect(data).toBeDefined()
108
- cy.get('.result').should('exist') // what should it contain?
109
- ```
110
-
111
- ### GOOD assertions (always do this):
112
- ```
113
- expect(response.status).toBe(200)
114
- expect(data.name).toBe('Test User')
115
- cy.get('[data-cy="result"]').should('have.text', 'Todo created successfully')
116
- ```
117
-
118
- ## Naming Conventions
119
-
120
- Adapt file extensions to match the project's language:
121
-
122
- | Type | Pattern | Example (.ts) | Example (.cy.ts) |
123
- |------|---------|---------------|-------------------|
124
- | Page Object | `[PageName]Page.[ext]` | `LoginPage.ts` | `LoginPage.ts` |
125
- | Component POM | `[ComponentName].[ext]` | `NavigationBar.ts` | `NavigationBar.ts` |
126
- | E2E test | `[feature].e2e.[ext]` | `login.e2e.spec.ts` | `login.e2e.cy.ts` |
127
- | API test | `[resource].api.[ext]` | `users.api.spec.ts` | `users.api.cy.ts` |
128
- | Unit test | `[module].unit.[ext]` | `validate.unit.spec.ts` | `validate.test.ts` |
129
- | Fixture | `[domain]-data.[ext]` | `auth-data.ts` | `auth-data.json` |
130
-
131
- If the project already has naming conventions, **follow those instead**.
132
-
133
- ## Repo Structure
134
-
135
- Recommended structure -- adapt to match what the project already has:
136
-
137
- ```
138
- tests/ or cypress/ or __tests__/
139
- e2e/
140
- smoke/ # P0 critical path (every PR)
141
- regression/ # Full suite (nightly)
142
- api/ # API-level tests
143
- unit/ # Unit tests
144
-
145
- pages/ or page-objects/ or support/page-objects/
146
- base/
147
- [feature]/
148
- components/
149
-
150
- fixtures/ # Test data & factories
151
- config/ # Test configs (if separate from root)
152
- reports/ # Generated reports (gitignored)
153
- ```
154
-
155
- ## Test Data Rules
156
-
157
- - **NEVER** hardcode real credentials
158
- - Use environment variables with test fallbacks
159
- - Fixtures go in dedicated folder
160
- - Each domain gets its own fixture file
161
-
162
- ## Analysis Documents
163
-
164
- When analyzing a repository, produce these documents:
165
-
166
- ### QA_ANALYSIS.md must include:
167
- - Architecture overview (system type, language, runtime, entry points, dependencies)
168
- - Risk assessment (HIGH / MEDIUM / LOW with justification)
169
- - Top 10 unit test targets with rationale
170
- - Recommended testing pyramid with percentages adjusted to this app
171
- - External dependencies with risk levels
172
-
173
- ### TEST_INVENTORY.md must include:
174
- - Every test case with ID, target, inputs, expected outcome, priority
175
- - Organized by pyramid level (unit -> integration -> API -> E2E)
176
- - No test case without an explicit expected outcome
177
-
178
- ## Quality Gates
179
-
180
- Before delivering ANY QA artifact, verify:
181
- - [ ] Framework matches what the project already uses
182
- - [ ] Every test case has an explicit expected outcome with a concrete value
183
- - [ ] No outcome says "correct", "proper", "appropriate", or "works" without defining what that means
184
- - [ ] All locators follow the tier hierarchy
185
- - [ ] No assertions inside page objects
186
- - [ ] No hardcoded credentials
187
- - [ ] File naming follows the project's existing conventions (or the standards above if none exist)
188
- - [ ] Test IDs are unique and follow naming convention
189
- - [ ] Priority assigned to every test case
190
-
191
- ---
192
-
193
- ## Agent Pipeline
194
-
195
- The QA automation system runs agents in a defined pipeline. Each stage produces artifacts consumed by the next stage.
196
-
197
- ### Pipeline Stages
198
-
199
- ```
200
- scan -> codebase-map -> analyze -> [testid-inject if frontend] -> plan -> generate -> validate -> [e2e-runner if E2E tests] -> [bug-detective if failures] -> deliver
201
- ```
202
-
203
- ### Workflow Options
204
-
205
- **Option 1: Dev-Only Repo (no existing QA repo)**
206
- Full pipeline from scratch:
207
- ```
208
- scan -> codebase-map -> analyze -> [testid-inject if frontend] -> plan -> generate -> validate -> [e2e-runner if E2E tests] -> [bug-detective if failures] -> deliver
209
- ```
210
- Produces: SCAN_MANIFEST.md -> 8 codebase map documents -> QA_ANALYSIS.md + TEST_INVENTORY.md + QA_REPO_BLUEPRINT.md -> [TESTID_AUDIT_REPORT.md] -> generation plan -> test files + POMs + fixtures + configs -> VALIDATION_REPORT.md -> [E2E_RUN_REPORT.md] -> branch + PR
211
-
212
- **Option 2: Dev + Immature QA Repo (existing QA repo with low coverage or quality)**
213
- Gap-fill and standardize:
214
- ```
215
- scan both repos -> gap analysis -> fix broken tests -> add missing coverage -> standardize existing -> validate -> deliver
216
- ```
217
- Produces: SCAN_MANIFEST.md (both repos) -> GAP_ANALYSIS.md -> fixed test files -> new test files -> standardized files -> VALIDATION_REPORT.md -> branch + PR
218
-
219
- **Option 3: Dev + Mature QA Repo (existing QA repo with solid coverage)**
220
- Surgical additions only:
221
- ```
222
- scan both repos -> identify thin coverage -> add only missing tests -> validate -> deliver
223
- ```
224
- Produces: SCAN_MANIFEST.md (both repos) -> GAP_ANALYSIS.md (thin areas only) -> new test files (targeted) -> VALIDATION_REPORT.md -> branch + PR
225
-
226
- ### Stage Transitions
227
-
228
- | From | To | Condition |
229
- |------|----|-----------|
230
- | scan | codebase-map | SCAN_MANIFEST.md exists with > 0 testable surfaces |
231
- | codebase-map | analyze | At least 4 of 8 codebase map documents exist |
232
- | analyze | testid-inject | QA_ANALYSIS.md exists AND frontend components detected |
233
- | analyze | plan | QA_ANALYSIS.md + TEST_INVENTORY.md exist (skip testid-inject if no frontend) |
234
- | testid-inject | plan | TESTID_AUDIT_REPORT.md exists with coverage score calculated |
235
- | plan | generate | Generation plan approved (or auto-approved in auto-advance mode) |
236
- | generate | validate | All planned test files exist on disk |
237
- | validate | e2e-runner | VALIDATION_REPORT.md shows PASS AND E2E test files were generated AND live app available |
238
- | validate | deliver | VALIDATION_REPORT.md shows PASS and no E2E files or --skip-run |
239
- | e2e-runner | bug-detective | E2E_RUN_REPORT.md exists AND failures need classification |
240
- | e2e-runner | deliver | E2E_RUN_REPORT.md exists AND all tests pass or failures classified |
241
- | bug-detective | deliver | FAILURE_CLASSIFICATION_REPORT.md exists |
242
-
243
- ---
244
-
245
- ## Module Boundaries
246
-
247
- Each agent owns specific artifacts. No agent may produce artifacts assigned to another agent.
248
-
249
- | Agent | Reads | Produces | Template |
250
- |-------|-------|----------|----------|
251
- | qa-scanner | repo source files, package.json, file tree | SCAN_MANIFEST.md | templates/scan-manifest.md |
252
- | qa-codebase-mapper | SCAN_MANIFEST.md, repo source files, CLAUDE.md | TESTABILITY.md, TEST_SURFACE.md, RISK_MAP.md, CRITICAL_PATHS.md, CODE_PATTERNS.md, API_CONTRACTS.md, TEST_ASSESSMENT.md, COVERAGE_GAPS.md | -- |
253
- | qa-analyzer | SCAN_MANIFEST.md, codebase map documents, CLAUDE.md | QA_ANALYSIS.md, TEST_INVENTORY.md, QA_REPO_BLUEPRINT.md (Option 1) or GAP_ANALYSIS.md (Option 2/3) | templates/qa-analysis.md, templates/test-inventory.md, templates/qa-repo-blueprint.md, templates/gap-analysis.md |
254
- | qa-planner | TEST_INVENTORY.md, QA_ANALYSIS.md, codebase map documents | Generation plan (internal) | -- |
255
- | qa-executor | TEST_INVENTORY.md, codebase map documents, CLAUDE.md | test files, POMs, fixtures, configs | qa-template-engine patterns |
256
- | qa-validator | generated test files, CLAUDE.md | VALIDATION_REPORT.md (validation mode) or QA_AUDIT_REPORT.md (audit mode) | templates/validation-report.md, templates/qa-audit-report.md |
257
- | qa-e2e-runner | generated E2E test files, POM files, CLAUDE.md, live application | E2E_RUN_REPORT.md, fixed test/POM files with real locators | -- |
258
- | qa-testid-injector | repo source files, SCAN_MANIFEST.md, CLAUDE.md | TESTID_AUDIT_REPORT.md, modified source files with data-testid attributes | templates/scan-manifest.md, templates/testid-audit-report.md |
259
- | qa-bug-detective | test execution results, test source files, CLAUDE.md | FAILURE_CLASSIFICATION_REPORT.md | templates/failure-classification.md |
260
- | qa-project-researcher | SCAN_MANIFEST.md, repo source files | TESTING_STACK.md, FRAMEWORK_CAPABILITIES.md, API_TESTING_STRATEGY.md, E2E_STRATEGY.md | -- |
261
-
262
- **Rule:** An agent MUST NOT produce artifacts assigned to another agent.
263
-
264
- **Rule:** An agent MUST read all artifacts listed in its "Reads" column before producing output.
265
-
266
- ---
267
-
268
- ## Verification Commands
269
-
270
- Every artifact must pass verification before the pipeline advances. Below are the validation rules per artifact type.
271
-
272
- ### SCAN_MANIFEST.md
273
- - Has > 0 files in File List table
274
- - Project Detection section is populated (framework, language, component patterns)
275
- - Testable Surfaces has at least 1 category with entries
276
- - File priority ordering is present (HIGH/MEDIUM/LOW)
277
-
278
- ### QA_ANALYSIS.md
279
- - All 6 sections present: Architecture Overview, External Dependencies, Risk Assessment, Top 10 Unit Test Targets, API/Contract Test Targets, Recommended Testing Pyramid
280
- - Top 10 has exactly 10 entries with module, rationale, and complexity
281
- - Testing pyramid percentages sum to 100%
282
- - Risk assessment uses only HIGH/MEDIUM/LOW with justification per item
283
-
284
- ### TEST_INVENTORY.md
285
- - Every test case has all mandatory fields: ID, target, inputs, expected outcome, priority
286
- - IDs are unique across the entire document (no duplicates)
287
- - IDs follow naming convention: UT-MODULE-NNN, INT-MODULE-NNN, API-RESOURCE-NNN, E2E-FLOW-NNN
288
- - Pyramid tier counts match the summary table
289
- - No expected outcome says "correct", "proper", "appropriate", or "works" without concrete value
290
-
291
- ### QA_REPO_BLUEPRINT.md
292
- - Folder structure tree is present with explanations per directory
293
- - Config files section has actual content (not placeholders)
294
- - npm scripts defined for smoke, regression, and API test runs
295
- - CI/CD strategy section includes PR-gate and nightly run configurations
296
- - Definition of Done checklist is present
297
-
298
- ### VALIDATION_REPORT.md
299
- - All 4 layers reported per file: Syntax, Structure, Dependencies, Logic
300
- - Each layer shows PASS or FAIL with details
301
- - Confidence level assigned: HIGH (all layers pass), MEDIUM (1-2 minor issues), LOW (structural problems)
302
- - Fix loop log shows iteration count and what was found/fixed per loop
303
- - Unresolved issues section documents anything not auto-fixed
304
-
305
- ### FAILURE_CLASSIFICATION_REPORT.md
306
- - Every failure has classification: APPLICATION BUG, TEST CODE ERROR, ENVIRONMENT ISSUE, or INCONCLUSIVE
307
- - Every failure has confidence level: HIGH, MEDIUM-HIGH, MEDIUM, or LOW
308
- - Every failure has evidence: code snippet + reasoning explaining the classification
309
- - No APPLICATION BUG is marked as auto-fixed (application bugs require developer action)
310
- - Auto-fix log documents what was fixed and at what confidence level
311
-
312
- ### TESTID_AUDIT_REPORT.md
313
- - Coverage score calculated: existing data-testid count / total interactive elements
314
- - All proposed data-testid values follow `{context}-{description}-{element-type}` naming convention
315
- - No duplicate data-testid values within the same page/route scope
316
- - Elements classified by priority: P0 (form inputs, buttons), P1 (links, images), P2 (containers, decorative)
317
- - Decision gate threshold applied: >90% SELECTIVE, 50-90% TARGETED, <50% FULL PASS, 0% P0 FIRST
318
-
319
- ### GAP_ANALYSIS.md
320
- - Coverage map shows all modules from SCAN_MANIFEST.md
321
- - Missing tests have IDs following naming convention and priorities assigned
322
- - Broken tests have failure reasons documented with file path and error
323
- - Quality assessment includes locator tier distribution and assertion quality rating
324
- - Recommendations are prioritized: fix broken first, then add P0, then P1
325
-
326
- ### QA_AUDIT_REPORT.md
327
- - All 6 dimensions scored: Locator Quality, Assertion Specificity, POM Compliance, Test Coverage, Naming Convention, Test Data Management
328
- - Weights sum to 100%: Locator 20%, Assertion 20%, POM 15%, Coverage 20%, Naming 15%, Test Data 10%
329
- - Overall score matches weighted calculation of dimension scores
330
- - Critical issues listed with file path, line number, and description
331
- - Each recommendation has effort estimate: S (small), M (medium), L (large)
332
-
333
- ---
334
-
335
- ## Git Workflow
336
-
337
- All QA automation output follows these git conventions.
338
-
339
- ### Branch Naming
340
-
341
- ```
342
- qa/auto-{project}-{date}
343
- ```
344
-
345
- Examples:
346
- - `qa/auto-shopflow-2026-03-18`
347
- - `qa/auto-acme-api-2026-04-01`
348
-
349
- ### Commit Message Format
350
-
351
- ```
352
- qa({agent}): {description}
353
- ```
354
-
355
- Examples:
356
- - `qa(scanner): produce SCAN_MANIFEST.md for shopflow`
357
- - `qa(analyzer): produce QA_ANALYSIS.md and TEST_INVENTORY.md`
358
- - `qa(executor): generate 24 test files with POMs and fixtures`
359
- - `qa(validator): validate generated tests - PASS with HIGH confidence`
360
- - `qa(testid-injector): inject 47 data-testid attributes across 12 components`
361
- - `qa(bug-detective): classify 5 failures - 2 APP BUG, 2 TEST ERROR, 1 ENV ISSUE`
362
-
363
- ### Commit Conventions
364
-
365
- - One commit per agent stage (scanner produces one commit, analyzer produces one commit, etc.)
366
- - Descriptive messages that include artifact names and counts
367
- - Never commit .env files, credentials, or secrets
368
- - Include modified file count in commit body when relevant
369
-
370
- ### PR Template
371
-
372
- PR description must include:
373
- - Analysis summary (architecture type, framework, risk areas)
374
- - Test counts by pyramid level (unit: N, integration: N, API: N, E2E: N)
375
- - Coverage metrics (modules covered, estimated line coverage)
376
- - Validation pass/fail status with confidence level
377
- - Link to VALIDATION_REPORT.md in the PR files
378
-
379
- ---
380
-
381
- ## Team Settings
382
-
383
- Configuration for multi-agent pipeline execution.
384
-
385
- ### Concurrent Execution
386
-
387
- Agents in the same pipeline stage can run in parallel when their inputs are independent. Examples:
388
- - qa-testid-injector and qa-analyzer can run simultaneously after scan completes (both read SCAN_MANIFEST.md)
389
- - Multiple qa-executor instances can generate tests for different modules in parallel
390
-
391
- Agents in different pipeline stages MUST respect stage ordering. A downstream agent cannot start until all its required inputs exist on disk.
392
-
393
- ### Worktree Isolation
394
-
395
- Each agent operates on the same branch. No worktree splits are needed for this system. Agents coordinate through file-based artifacts -- each agent writes its own files and reads other agents' files.
396
-
397
- ### Dependency Ordering
398
-
399
- Respect stage transitions from the Agent Pipeline section:
400
- 1. qa-scanner runs first (no dependencies)
401
- 2. qa-codebase-mapper runs after scanner (depends on SCAN_MANIFEST.md, spawns 4 parallel focus agents)
402
- 3. qa-analyzer and qa-testid-injector run after codebase-map (both depend on SCAN_MANIFEST.md + codebase map documents)
403
- 4. qa-planner runs after analyzer (depends on QA_ANALYSIS.md + TEST_INVENTORY.md + codebase map documents)
404
- 5. qa-executor runs after planner (depends on generation plan + codebase map documents)
405
- 6. qa-validator runs after executor (depends on generated test files)
406
- 7. qa-e2e-runner runs after validator (depends on E2E test files + live application)
407
- 8. qa-bug-detective runs after e2e-runner or validator if failures (depends on test results)
408
-
409
- ### Auto-Advance Mode
410
-
411
- When auto-advance is enabled, pipeline stages advance automatically when:
412
- 1. The previous stage completes
413
- 2. All output artifacts from the previous stage exist on disk
414
- 3. All output artifacts pass their verification commands (from Verification Commands section)
415
-
416
- No human confirmation is needed between stages in auto-advance mode. The pipeline pauses only at explicit checkpoint tasks or when verification fails.
417
-
418
- ---
419
-
420
- ## Agent Coordination
421
-
422
- Rules governing how agents communicate and hand off work through artifacts.
423
-
424
- ### Read-Before-Write Rules
425
-
426
- Every agent MUST read its required inputs before producing any output. Failure to read inputs produces low-quality, inconsistent artifacts.
427
-
428
- | Agent | MUST Read Before Producing Output |
429
- |-------|-----------------------------------|
430
- | qa-scanner | package.json (or equivalent), folder tree structure, all source file extensions to detect framework and language |
431
- | qa-analyzer | SCAN_MANIFEST.md (complete, verified), CLAUDE.md (all QA standards sections) |
432
- | qa-planner | TEST_INVENTORY.md (all test cases), QA_ANALYSIS.md (architecture and risk context) |
433
- | qa-executor | TEST_INVENTORY.md (test cases to implement), CLAUDE.md (POM rules, locator hierarchy, assertion rules, naming conventions, quality gates) |
434
- | qa-validator | CLAUDE.md (quality gates, locator tiers, assertion rules), all generated test files to validate |
435
- | qa-testid-injector | SCAN_MANIFEST.md (component file list), CLAUDE.md (data-testid Convention section for naming rules) |
436
- | qa-bug-detective | Test execution output (stdout/stderr, exit codes), test source files (to read the failing code), CLAUDE.md (classification rules) |
437
-
438
- ### Handoff Patterns
439
-
440
- Agents communicate exclusively through file-based artifacts:
441
-
442
- 1. **Producer writes** -- Agent completes its task and writes output artifact(s) to disk
443
- 2. **Pipeline verifies** -- Output artifacts pass verification commands before advancing
444
- 3. **Consumer reads** -- Next agent reads the artifact(s) as its first action
445
- 4. **No direct communication** -- Agents never pass data in memory or through environment variables between stages
446
-
447
- ### Quality Gates Per Artifact
448
-
449
- Before the pipeline advances past any stage, the produced artifact(s) must pass verification:
450
-
451
- | Stage Complete | Artifact | Gate |
452
- |----------------|----------|------|
453
- | scan | SCAN_MANIFEST.md | > 0 files listed, project detection populated |
454
- | analyze | QA_ANALYSIS.md + TEST_INVENTORY.md | All sections present, IDs unique, pyramid sums to 100% |
455
- | testid-inject | TESTID_AUDIT_REPORT.md | Coverage score calculated, naming convention compliant |
456
- | plan | Generation plan | Test cases mapped to output files, no unassigned cases |
457
- | generate | Test files + POMs | All planned files exist, imports resolve, syntax valid |
458
- | validate | VALIDATION_REPORT.md | All 4 layers checked per file, confidence level assigned |
459
- | deliver | Branch + PR | Branch pushed, PR created with required description sections |
460
-
461
- ### Error Recovery
462
-
463
- If an agent fails or produces an artifact that does not pass verification:
464
- 1. Log the failure with the specific verification check that failed
465
- 2. Retry the agent (max 3 attempts per stage)
466
- 3. If still failing after 3 attempts, pause the pipeline and report the blocked stage
467
- 4. Do not advance to the next stage with a failed artifact
468
-
469
- ---
470
-
471
- ## data-testid Convention
472
-
473
- All `data-testid` attributes injected by qa-testid-injector and referenced by generated tests MUST follow this naming convention.
474
-
475
- ### Naming Pattern
476
-
477
- ```
478
- {context}-{description}-{element-type}
479
- ```
480
-
481
- All values are **kebab-case**. No camelCase, no underscores, no periods.
482
-
483
- ### Context Derivation
484
-
485
- 1. **Page-level context**: Derived from the component filename or route
486
- - `LoginPage.tsx` -> context is `login`
487
- - `ProductDetailPage.tsx` -> context is `product-detail`
488
- - Route `/settings/profile` -> context is `settings-profile`
489
-
490
- 2. **Component-level context**: Derived from the component name
491
- - `<NavBar>` -> context is `navbar`
492
- - `<ShoppingCart>` -> context is `shopping-cart`
493
- - `<UserAvatar>` -> context is `user-avatar`
494
-
495
- 3. **Nested context**: Parent-child hierarchy, max 3 levels deep
496
- - `checkout-shipping-address-input` (page -> section -> field)
497
- - `dashboard-sidebar-nav-link` (page -> component -> element)
498
- - Never exceed 3 levels: `a-b-c-element` is the maximum depth
499
-
500
- 4. **Dynamic list items**: Use template literals with unique keys
501
- ```tsx
502
- data-testid={`product-${product.id}-card`}
503
- data-testid={`order-${order.id}-status-badge`}
504
- ```
505
-
506
- ### Element Type Suffix Table
507
-
508
- Every `data-testid` value ends with a suffix indicating the element type:
509
-
510
- | Element | Suffix | Example |
511
- |---------|--------|---------|
512
- | `<button>` | `-btn` | `login-submit-btn` |
513
- | `<input>` | `-input` | `login-email-input` |
514
- | `<select>` | `-select` | `settings-language-select` |
515
- | `<textarea>` | `-textarea` | `feedback-comment-textarea` |
516
- | `<a>` (link) | `-link` | `navbar-profile-link` |
517
- | `<form>` | `-form` | `checkout-payment-form` |
518
- | `<img>` | `-img` | `product-hero-img` |
519
- | `<table>` | `-table` | `users-list-table` |
520
- | `<tr>` (row) | `-row` | `users-item-row` |
521
- | `<dialog>/<modal>` | `-modal` | `confirm-delete-modal` |
522
- | `<div>` container | `-container` | `dashboard-stats-container` |
523
- | `<ul>/<ol>` list | `-list` | `notifications-list` |
524
- | `<li>` item | `-item` | `notifications-item` |
525
- | dropdown menu | `-dropdown` | `navbar-user-dropdown` |
526
- | tab | `-tab` | `settings-security-tab` |
527
- | checkbox | `-checkbox` | `terms-accept-checkbox` |
528
- | radio | `-radio` | `shipping-express-radio` |
529
- | toggle/switch | `-toggle` | `notifications-enabled-toggle` |
530
- | badge/chip | `-badge` | `cart-count-badge` |
531
- | alert/toast | `-alert` | `error-validation-alert` |
532
-
533
- ### Third-Party Component Handling
534
-
535
- When adding `data-testid` to third-party UI library components, use this priority order:
536
-
537
- 1. **Props passthrough** (preferred): If the library supports passing `data-testid` directly as a prop
538
- ```tsx
539
- <MuiButton data-testid="checkout-pay-btn">Pay</MuiButton>
540
- ```
541
-
542
- 2. **Wrapper div**: If the library does not support prop passthrough, wrap with a `<div>` that has the `data-testid`
543
- ```tsx
544
- <div data-testid="checkout-pay-container">
545
- <ThirdPartyButton>Pay</ThirdPartyButton>
546
- </div>
547
- ```
548
-
549
- 3. **inputProps / slotProps** (MUI-specific): Use component-specific prop APIs
550
- ```tsx
551
- <TextField inputProps={{ 'data-testid': 'login-email-input' }} />
552
- <Autocomplete slotProps={{ input: { 'data-testid': 'search-query-input' } }} />
553
- ```
1
+ # QA Automation Standards
2
+
3
+ This project follows strict QA automation standards. Every test, page object, and analysis produced MUST follow these rules.
4
+
5
+ ## Framework Detection
6
+
7
+ Before generating any code, **detect what the project already uses**:
8
+
9
+ 1. Check for existing test config files: `cypress.config.ts`, `playwright.config.ts`, `jest.config.ts`, `vitest.config.ts`, `pytest.ini`, etc.
10
+ 2. Check `package.json` or `requirements.txt` for test dependencies
11
+ 3. Check existing test files for patterns and conventions
12
+ 4. **Always match the project's existing framework, language, and conventions**
13
+
14
+ If no framework exists yet, ask the user which one to use. Never assume.
15
+
16
+ ## Testing Pyramid
17
+
18
+ Target distribution for every project:
19
+
20
+ ```
21
+ / E2E \ 3-5% (critical path smoke only)
22
+ / API \ 20-25% (endpoints + contracts)
23
+ / Integration\ 10-15% (component interactions)
24
+ / Unit \ 60-70% (business logic, pure functions)
25
+ ```
26
+
27
+ Adjust percentages based on the actual app architecture.
28
+
29
+ ## Locator Strategy
30
+
31
+ All UI test locators MUST follow this priority order. Never skip to a lower tier without written justification.
32
+
33
+ **Tier 1 -- BEST (always try these first):**
34
+ - Test IDs: `data-testid`, `data-cy`, `data-test` (adapt to framework)
35
+ - Semantic roles: ARIA roles + accessible name
36
+
37
+ **Tier 2 -- GOOD (when Tier 1 not available):**
38
+ - Form labels, placeholders, visible text content
39
+
40
+ **Tier 3 -- ACCEPTABLE (when Tier 1-2 not available):**
41
+ - Alt text, title attributes
42
+
43
+ **Tier 4 -- LAST RESORT (always add a TODO comment):**
44
+ - CSS selectors, XPath -- mark with `// TODO: Request test ID for this element`
45
+
46
+ ### Framework-Specific Examples
47
+
48
+ **Playwright:**
49
+ ```typescript
50
+ page.getByTestId('submit') // Tier 1
51
+ page.getByRole('button', {name: 'Log in'}) // Tier 1
52
+ page.getByLabel('Email') // Tier 2
53
+ page.locator('.btn') // Tier 4 -- add TODO
54
+ ```
55
+
56
+ **Cypress:**
57
+ ```typescript
58
+ cy.get('[data-cy="submit"]') // Tier 1
59
+ cy.findByRole('button', {name: 'Log in'}) // Tier 1 (with testing-library)
60
+ cy.get('[data-testid="submit"]') // Tier 1
61
+ cy.contains('Submit') // Tier 2
62
+ cy.get('.btn') // Tier 4 -- add TODO
63
+ ```
64
+
65
+ **Selenium / other:**
66
+ ```
67
+ driver.findElement(By.cssSelector('[data-testid="submit"]')) // Tier 1
68
+ driver.findElement(By.className('btn')) // Tier 4 -- add TODO
69
+ ```
70
+
71
+ ## Page Object Model Rules
72
+
73
+ These rules apply regardless of framework:
74
+
75
+ 1. **One class/object per page or view** -- no god objects
76
+ 2. **No assertions in page objects** -- assertions belong ONLY in test specs
77
+ 3. **Locators are properties** -- defined in constructor or as class fields
78
+ 4. **Actions return void or the next page** -- for fluent chaining
79
+ 5. **State queries return data** -- let the test file decide what to assert
80
+ 6. **Every POM extends a shared base** -- shared navigation, screenshots, waits
81
+
82
+ ### POM File Structure
83
+ ```
84
+ [pages or page-objects or support/page-objects]/
85
+ base/
86
+ BasePage.[ext] -- shared methods
87
+ [feature]/
88
+ [Feature]Page.[ext] -- one file per page
89
+ components/
90
+ [Component].[ext] -- reusable UI components
91
+ ```
92
+
93
+ Adapt folder location to match the project's existing conventions.
94
+
95
+ ## Test Spec Rules
96
+
97
+ ### Every test case MUST have:
98
+ - **Unique ID**: `UT-MODULE-001`, `API-AUTH-001`, `E2E-FLOW-001`
99
+ - **Exact target**: file path + function name, or HTTP method + endpoint
100
+ - **Concrete inputs**: actual values, not "valid data"
101
+ - **Explicit expected outcome**: exact assertion, not "works correctly"
102
+ - **Priority**: P0 (blocks release), P1 (should fix), P2 (nice to have)
103
+
104
+ ### BAD assertions (never do this):
105
+ ```
106
+ expect(response.status).toBeTruthy()
107
+ expect(data).toBeDefined()
108
+ cy.get('.result').should('exist') // what should it contain?
109
+ ```
110
+
111
+ ### GOOD assertions (always do this):
112
+ ```
113
+ expect(response.status).toBe(200)
114
+ expect(data.name).toBe('Test User')
115
+ cy.get('[data-cy="result"]').should('have.text', 'Todo created successfully')
116
+ ```
117
+
118
+ ## Naming Conventions
119
+
120
+ Adapt file extensions to match the project's language:
121
+
122
+ | Type | Pattern | Example (.ts) | Example (.cy.ts) |
123
+ |------|---------|---------------|-------------------|
124
+ | Page Object | `[PageName]Page.[ext]` | `LoginPage.ts` | `LoginPage.ts` |
125
+ | Component POM | `[ComponentName].[ext]` | `NavigationBar.ts` | `NavigationBar.ts` |
126
+ | E2E test | `[feature].e2e.[ext]` | `login.e2e.spec.ts` | `login.e2e.cy.ts` |
127
+ | API test | `[resource].api.[ext]` | `users.api.spec.ts` | `users.api.cy.ts` |
128
+ | Unit test | `[module].unit.[ext]` | `validate.unit.spec.ts` | `validate.test.ts` |
129
+ | Fixture | `[domain]-data.[ext]` | `auth-data.ts` | `auth-data.json` |
130
+
131
+ If the project already has naming conventions, **follow those instead**.
132
+
133
+ ## Repo Structure
134
+
135
+ Recommended structure -- adapt to match what the project already has:
136
+
137
+ ```
138
+ tests/ or cypress/ or __tests__/
139
+ e2e/
140
+ smoke/ # P0 critical path (every PR)
141
+ regression/ # Full suite (nightly)
142
+ api/ # API-level tests
143
+ unit/ # Unit tests
144
+
145
+ pages/ or page-objects/ or support/page-objects/
146
+ base/
147
+ [feature]/
148
+ components/
149
+
150
+ fixtures/ # Test data & factories
151
+ config/ # Test configs (if separate from root)
152
+ reports/ # Generated reports (gitignored)
153
+ ```
154
+
155
+ ## Test Data Rules
156
+
157
+ - **NEVER** hardcode real credentials
158
+ - Use environment variables with test fallbacks
159
+ - Fixtures go in dedicated folder
160
+ - Each domain gets its own fixture file
161
+
162
+ ## Analysis Documents
163
+
164
+ When analyzing a repository, produce these documents:
165
+
166
+ ### QA_ANALYSIS.md must include:
167
+ - Architecture overview (system type, language, runtime, entry points, dependencies)
168
+ - Risk assessment (HIGH / MEDIUM / LOW with justification)
169
+ - Top 10 unit test targets with rationale
170
+ - Recommended testing pyramid with percentages adjusted to this app
171
+ - External dependencies with risk levels
172
+
173
+ ### TEST_INVENTORY.md must include:
174
+ - Every test case with ID, target, inputs, expected outcome, priority
175
+ - Organized by pyramid level (unit -> integration -> API -> E2E)
176
+ - No test case without an explicit expected outcome
177
+
178
+ ## Quality Gates
179
+
180
+ Before delivering ANY QA artifact, verify:
181
+ - [ ] Framework matches what the project already uses
182
+ - [ ] Every test case has an explicit expected outcome with a concrete value
183
+ - [ ] No outcome says "correct", "proper", "appropriate", or "works" without defining what that means
184
+ - [ ] All locators follow the tier hierarchy
185
+ - [ ] No assertions inside page objects
186
+ - [ ] No hardcoded credentials
187
+ - [ ] File naming follows the project's existing conventions (or the standards above if none exist)
188
+ - [ ] Test IDs are unique and follow naming convention
189
+ - [ ] Priority assigned to every test case
190
+
191
+ ---
192
+
193
+ ## Agent Pipeline
194
+
195
+ The QA automation system runs agents in a defined pipeline. Each stage produces artifacts consumed by the next stage.
196
+
197
+ ### Pipeline Stages
198
+
199
+ ```
200
+ scan -> codebase-map -> analyze -> [testid-inject if frontend] -> plan -> generate -> validate -> [e2e-runner if E2E tests] -> [bug-detective if failures] -> deliver
201
+ ```
202
+
203
+ ### Workflow Options
204
+
205
+ **Option 1: Dev-Only Repo (no existing QA repo)**
206
+ Full pipeline from scratch:
207
+ ```
208
+ scan -> codebase-map -> analyze -> [testid-inject if frontend] -> plan -> generate -> validate -> [e2e-runner if E2E tests] -> [bug-detective if failures] -> deliver
209
+ ```
210
+ Produces: SCAN_MANIFEST.md -> 8 codebase map documents -> QA_ANALYSIS.md + TEST_INVENTORY.md + QA_REPO_BLUEPRINT.md -> [TESTID_AUDIT_REPORT.md] -> generation plan -> test files + POMs + fixtures + configs -> VALIDATION_REPORT.md -> [E2E_RUN_REPORT.md] -> branch + PR
211
+
212
+ **Option 2: Dev + Immature QA Repo (existing QA repo with low coverage or quality)**
213
+ Gap-fill and standardize:
214
+ ```
215
+ scan both repos -> gap analysis -> fix broken tests -> add missing coverage -> standardize existing -> validate -> deliver
216
+ ```
217
+ Produces: SCAN_MANIFEST.md (both repos) -> GAP_ANALYSIS.md -> fixed test files -> new test files -> standardized files -> VALIDATION_REPORT.md -> branch + PR
218
+
219
+ **Option 3: Dev + Mature QA Repo (existing QA repo with solid coverage)**
220
+ Surgical additions only:
221
+ ```
222
+ scan both repos -> identify thin coverage -> add only missing tests -> validate -> deliver
223
+ ```
224
+ Produces: SCAN_MANIFEST.md (both repos) -> GAP_ANALYSIS.md (thin areas only) -> new test files (targeted) -> VALIDATION_REPORT.md -> branch + PR
225
+
226
+ ### Stage Transitions
227
+
228
+ | From | To | Condition |
229
+ |------|----|-----------|
230
+ | scan | codebase-map | SCAN_MANIFEST.md exists with > 0 testable surfaces |
231
+ | codebase-map | analyze | At least 4 of 8 codebase map documents exist |
232
+ | analyze | testid-inject | QA_ANALYSIS.md exists AND frontend components detected |
233
+ | analyze | plan | QA_ANALYSIS.md + TEST_INVENTORY.md exist (skip testid-inject if no frontend) |
234
+ | testid-inject | plan | TESTID_AUDIT_REPORT.md exists with coverage score calculated |
235
+ | plan | generate | Generation plan approved (or auto-approved in auto-advance mode) |
236
+ | generate | validate | All planned test files exist on disk |
237
+ | validate | e2e-runner | VALIDATION_REPORT.md shows PASS AND E2E test files were generated AND live app available |
238
+ | validate | deliver | VALIDATION_REPORT.md shows PASS and no E2E files or --skip-run |
239
+ | e2e-runner | bug-detective | E2E_RUN_REPORT.md exists AND failures need classification |
240
+ | e2e-runner | deliver | E2E_RUN_REPORT.md exists AND all tests pass or failures classified |
241
+ | bug-detective | deliver | FAILURE_CLASSIFICATION_REPORT.md exists |
242
+
243
+ ---
244
+
245
+ ## Module Boundaries
246
+
247
+ Each agent owns specific artifacts. No agent may produce artifacts assigned to another agent.
248
+
249
+ | Agent | Reads | Produces | Template |
250
+ |-------|-------|----------|----------|
251
+ | qa-scanner | repo source files, package.json, file tree | SCAN_MANIFEST.md | templates/scan-manifest.md |
252
+ | qa-codebase-mapper | SCAN_MANIFEST.md, repo source files, CLAUDE.md | TESTABILITY.md, TEST_SURFACE.md, RISK_MAP.md, CRITICAL_PATHS.md, CODE_PATTERNS.md, API_CONTRACTS.md, TEST_ASSESSMENT.md, COVERAGE_GAPS.md | -- |
253
+ | qa-analyzer | SCAN_MANIFEST.md, codebase map documents, CLAUDE.md | QA_ANALYSIS.md, TEST_INVENTORY.md, QA_REPO_BLUEPRINT.md (Option 1) or GAP_ANALYSIS.md (Option 2/3) | templates/qa-analysis.md, templates/test-inventory.md, templates/qa-repo-blueprint.md, templates/gap-analysis.md |
254
+ | qa-planner | TEST_INVENTORY.md, QA_ANALYSIS.md, codebase map documents | Generation plan (internal) | -- |
255
+ | qa-executor | TEST_INVENTORY.md, codebase map documents, CLAUDE.md | test files, POMs, fixtures, configs | qa-template-engine patterns |
256
+ | qa-validator | generated test files, CLAUDE.md | VALIDATION_REPORT.md (validation mode) or QA_AUDIT_REPORT.md (audit mode) | templates/validation-report.md, templates/qa-audit-report.md |
257
+ | qa-e2e-runner | generated E2E test files, POM files, CLAUDE.md, live application | E2E_RUN_REPORT.md, fixed test/POM files with real locators | -- |
258
+ | qa-testid-injector | repo source files, SCAN_MANIFEST.md, CLAUDE.md | TESTID_AUDIT_REPORT.md, modified source files with data-testid attributes | templates/scan-manifest.md, templates/testid-audit-report.md |
259
+ | qa-bug-detective | test execution results, test source files, CLAUDE.md | FAILURE_CLASSIFICATION_REPORT.md | templates/failure-classification.md |
260
+ | qa-project-researcher | SCAN_MANIFEST.md, repo source files | TESTING_STACK.md, FRAMEWORK_CAPABILITIES.md, API_TESTING_STRATEGY.md, E2E_STRATEGY.md | -- |
261
+
262
+ **Rule:** An agent MUST NOT produce artifacts assigned to another agent.
263
+
264
+ **Rule:** An agent MUST read all artifacts listed in its "Reads" column before producing output.
265
+
266
+ ---
267
+
268
+ ## Verification Commands
269
+
270
+ Every artifact must pass verification before the pipeline advances. Below are the validation rules per artifact type.
271
+
272
+ ### SCAN_MANIFEST.md
273
+ - Has > 0 files in File List table
274
+ - Project Detection section is populated (framework, language, component patterns)
275
+ - Testable Surfaces has at least 1 category with entries
276
+ - File priority ordering is present (HIGH/MEDIUM/LOW)
277
+
278
+ ### QA_ANALYSIS.md
279
+ - All 6 sections present: Architecture Overview, External Dependencies, Risk Assessment, Top 10 Unit Test Targets, API/Contract Test Targets, Recommended Testing Pyramid
280
+ - Top 10 has exactly 10 entries with module, rationale, and complexity
281
+ - Testing pyramid percentages sum to 100%
282
+ - Risk assessment uses only HIGH/MEDIUM/LOW with justification per item
283
+
284
+ ### TEST_INVENTORY.md
285
+ - Every test case has all mandatory fields: ID, target, inputs, expected outcome, priority
286
+ - IDs are unique across the entire document (no duplicates)
287
+ - IDs follow naming convention: UT-MODULE-NNN, INT-MODULE-NNN, API-RESOURCE-NNN, E2E-FLOW-NNN
288
+ - Pyramid tier counts match the summary table
289
+ - No expected outcome says "correct", "proper", "appropriate", or "works" without concrete value
290
+
291
+ ### QA_REPO_BLUEPRINT.md
292
+ - Folder structure tree is present with explanations per directory
293
+ - Config files section has actual content (not placeholders)
294
+ - npm scripts defined for smoke, regression, and API test runs
295
+ - CI/CD strategy section includes PR-gate and nightly run configurations
296
+ - Definition of Done checklist is present
297
+
298
+ ### VALIDATION_REPORT.md
299
+ - All 4 layers reported per file: Syntax, Structure, Dependencies, Logic
300
+ - Each layer shows PASS or FAIL with details
301
+ - Confidence level assigned: HIGH (all layers pass), MEDIUM (1-2 minor issues), LOW (structural problems)
302
+ - Fix loop log shows iteration count and what was found/fixed per loop
303
+ - Unresolved issues section documents anything not auto-fixed
304
+
305
+ ### FAILURE_CLASSIFICATION_REPORT.md
306
+ - Every failure has classification: APPLICATION BUG, TEST CODE ERROR, ENVIRONMENT ISSUE, or INCONCLUSIVE
307
+ - Every failure has confidence level: HIGH, MEDIUM-HIGH, MEDIUM, or LOW
308
+ - Every failure has evidence: code snippet + reasoning explaining the classification
309
+ - No APPLICATION BUG is marked as auto-fixed (application bugs require developer action)
310
+ - Auto-fix log documents what was fixed and at what confidence level
311
+
312
+ ### TESTID_AUDIT_REPORT.md
313
+ - Coverage score calculated: existing data-testid count / total interactive elements
314
+ - All proposed data-testid values follow `{context}-{description}-{element-type}` naming convention
315
+ - No duplicate data-testid values within the same page/route scope
316
+ - Elements classified by priority: P0 (form inputs, buttons), P1 (links, images), P2 (containers, decorative)
317
+ - Decision gate threshold applied: >90% SELECTIVE, 50-90% TARGETED, <50% FULL PASS, 0% P0 FIRST
318
+
319
+ ### GAP_ANALYSIS.md
320
+ - Coverage map shows all modules from SCAN_MANIFEST.md
321
+ - Missing tests have IDs following naming convention and priorities assigned
322
+ - Broken tests have failure reasons documented with file path and error
323
+ - Quality assessment includes locator tier distribution and assertion quality rating
324
+ - Recommendations are prioritized: fix broken first, then add P0, then P1
325
+
326
+ ### QA_AUDIT_REPORT.md
327
+ - All 6 dimensions scored: Locator Quality, Assertion Specificity, POM Compliance, Test Coverage, Naming Convention, Test Data Management
328
+ - Weights sum to 100%: Locator 20%, Assertion 20%, POM 15%, Coverage 20%, Naming 15%, Test Data 10%
329
+ - Overall score matches weighted calculation of dimension scores
330
+ - Critical issues listed with file path, line number, and description
331
+ - Each recommendation has effort estimate: S (small), M (medium), L (large)
332
+
333
+ ---
334
+
335
+ ## Git Workflow
336
+
337
+ All QA automation output follows these git conventions.
338
+
339
+ ### Branch Naming
340
+
341
+ ```
342
+ qa/auto-{project}-{date}
343
+ ```
344
+
345
+ Examples:
346
+ - `qa/auto-shopflow-2026-03-18`
347
+ - `qa/auto-acme-api-2026-04-01`
348
+
349
+ ### Commit Message Format
350
+
351
+ ```
352
+ qa({agent}): {description}
353
+ ```
354
+
355
+ Examples:
356
+ - `qa(scanner): produce SCAN_MANIFEST.md for shopflow`
357
+ - `qa(analyzer): produce QA_ANALYSIS.md and TEST_INVENTORY.md`
358
+ - `qa(executor): generate 24 test files with POMs and fixtures`
359
+ - `qa(validator): validate generated tests - PASS with HIGH confidence`
360
+ - `qa(testid-injector): inject 47 data-testid attributes across 12 components`
361
+ - `qa(bug-detective): classify 5 failures - 2 APP BUG, 2 TEST ERROR, 1 ENV ISSUE`
362
+
363
+ ### Commit Conventions
364
+
365
+ - One commit per agent stage (scanner produces one commit, analyzer produces one commit, etc.)
366
+ - Descriptive messages that include artifact names and counts
367
+ - Never commit .env files, credentials, or secrets
368
+ - Include modified file count in commit body when relevant
369
+
370
+ ### PR Template
371
+
372
+ PR description must include:
373
+ - Analysis summary (architecture type, framework, risk areas)
374
+ - Test counts by pyramid level (unit: N, integration: N, API: N, E2E: N)
375
+ - Coverage metrics (modules covered, estimated line coverage)
376
+ - Validation pass/fail status with confidence level
377
+ - Link to VALIDATION_REPORT.md in the PR files
378
+
379
+ ---
380
+
381
+ ## Team Settings
382
+
383
+ Configuration for multi-agent pipeline execution.
384
+
385
+ ### Concurrent Execution
386
+
387
+ Agents in the same pipeline stage can run in parallel when their inputs are independent. Examples:
388
+ - qa-testid-injector and qa-analyzer can run simultaneously after scan completes (both read SCAN_MANIFEST.md)
389
+ - Multiple qa-executor instances can generate tests for different modules in parallel
390
+
391
+ Agents in different pipeline stages MUST respect stage ordering. A downstream agent cannot start until all its required inputs exist on disk.
392
+
393
+ ### Worktree Isolation
394
+
395
+ Each agent operates on the same branch. No worktree splits are needed for this system. Agents coordinate through file-based artifacts -- each agent writes its own files and reads other agents' files.
396
+
397
+ ### Dependency Ordering
398
+
399
+ Respect stage transitions from the Agent Pipeline section:
400
+ 1. qa-scanner runs first (no dependencies)
401
+ 2. qa-codebase-mapper runs after scanner (depends on SCAN_MANIFEST.md, spawns 4 parallel focus agents)
402
+ 3. qa-analyzer and qa-testid-injector run after codebase-map (both depend on SCAN_MANIFEST.md + codebase map documents)
403
+ 4. qa-planner runs after analyzer (depends on QA_ANALYSIS.md + TEST_INVENTORY.md + codebase map documents)
404
+ 5. qa-executor runs after planner (depends on generation plan + codebase map documents)
405
+ 6. qa-validator runs after executor (depends on generated test files)
406
+ 7. qa-e2e-runner runs after validator (depends on E2E test files + live application)
407
+ 8. qa-bug-detective runs after e2e-runner or validator if failures (depends on test results)
408
+
409
+ ### Auto-Advance Mode
410
+
411
+ When auto-advance is enabled, pipeline stages advance automatically when:
412
+ 1. The previous stage completes
413
+ 2. All output artifacts from the previous stage exist on disk
414
+ 3. All output artifacts pass their verification commands (from Verification Commands section)
415
+
416
+ No human confirmation is needed between stages in auto-advance mode. The pipeline pauses only at explicit checkpoint tasks or when verification fails.
417
+
418
+ ---
419
+
420
+ ## Agent Coordination
421
+
422
+ Rules governing how agents communicate and hand off work through artifacts.
423
+
424
+ ### Read-Before-Write Rules
425
+
426
+ Every agent MUST read its required inputs before producing any output. Failure to read inputs produces low-quality, inconsistent artifacts.
427
+
428
+ | Agent | MUST Read Before Producing Output |
429
+ |-------|-----------------------------------|
430
+ | qa-scanner | package.json (or equivalent), folder tree structure, all source file extensions to detect framework and language |
431
+ | qa-analyzer | SCAN_MANIFEST.md (complete, verified), CLAUDE.md (all QA standards sections) |
432
+ | qa-planner | TEST_INVENTORY.md (all test cases), QA_ANALYSIS.md (architecture and risk context) |
433
+ | qa-executor | TEST_INVENTORY.md (test cases to implement), CLAUDE.md (POM rules, locator hierarchy, assertion rules, naming conventions, quality gates) |
434
+ | qa-validator | CLAUDE.md (quality gates, locator tiers, assertion rules), all generated test files to validate |
435
+ | qa-testid-injector | SCAN_MANIFEST.md (component file list), CLAUDE.md (data-testid Convention section for naming rules) |
436
+ | qa-bug-detective | Test execution output (stdout/stderr, exit codes), test source files (to read the failing code), CLAUDE.md (classification rules) |
437
+
438
+ ### Handoff Patterns
439
+
440
+ Agents communicate exclusively through file-based artifacts:
441
+
442
+ 1. **Producer writes** -- Agent completes its task and writes output artifact(s) to disk
443
+ 2. **Pipeline verifies** -- Output artifacts pass verification commands before advancing
444
+ 3. **Consumer reads** -- Next agent reads the artifact(s) as its first action
445
+ 4. **No direct communication** -- Agents never pass data in memory or through environment variables between stages
446
+
447
+ ### Quality Gates Per Artifact
448
+
449
+ Before the pipeline advances past any stage, the produced artifact(s) must pass verification:
450
+
451
+ | Stage Complete | Artifact | Gate |
452
+ |----------------|----------|------|
453
+ | scan | SCAN_MANIFEST.md | > 0 files listed, project detection populated |
454
+ | analyze | QA_ANALYSIS.md + TEST_INVENTORY.md | All sections present, IDs unique, pyramid sums to 100% |
455
+ | testid-inject | TESTID_AUDIT_REPORT.md | Coverage score calculated, naming convention compliant |
456
+ | plan | Generation plan | Test cases mapped to output files, no unassigned cases |
457
+ | generate | Test files + POMs | All planned files exist, imports resolve, syntax valid |
458
+ | validate | VALIDATION_REPORT.md | All 4 layers checked per file, confidence level assigned |
459
+ | deliver | Branch + PR | Branch pushed, PR created with required description sections |
460
+
461
+ ### Error Recovery
462
+
463
+ If an agent fails or produces an artifact that does not pass verification:
464
+ 1. Log the failure with the specific verification check that failed
465
+ 2. Retry the agent (max 3 attempts per stage)
466
+ 3. If still failing after 3 attempts, pause the pipeline and report the blocked stage
467
+ 4. Do not advance to the next stage with a failed artifact
468
+
469
+ ---
470
+
471
+ ## data-testid Convention
472
+
473
+ All `data-testid` attributes injected by qa-testid-injector and referenced by generated tests MUST follow this naming convention.
474
+
475
+ ### Naming Pattern
476
+
477
+ ```
478
+ {context}-{description}-{element-type}
479
+ ```
480
+
481
+ All values are **kebab-case**. No camelCase, no underscores, no periods.
482
+
483
+ ### Context Derivation
484
+
485
+ 1. **Page-level context**: Derived from the component filename or route
486
+ - `LoginPage.tsx` -> context is `login`
487
+ - `ProductDetailPage.tsx` -> context is `product-detail`
488
+ - Route `/settings/profile` -> context is `settings-profile`
489
+
490
+ 2. **Component-level context**: Derived from the component name
491
+ - `<NavBar>` -> context is `navbar`
492
+ - `<ShoppingCart>` -> context is `shopping-cart`
493
+ - `<UserAvatar>` -> context is `user-avatar`
494
+
495
+ 3. **Nested context**: Parent-child hierarchy, max 3 levels deep
496
+ - `checkout-shipping-address-input` (page -> section -> field)
497
+ - `dashboard-sidebar-nav-link` (page -> component -> element)
498
+ - Never exceed 3 levels: `a-b-c-element` is the maximum depth
499
+
500
+ 4. **Dynamic list items**: Use template literals with unique keys
501
+ ```tsx
502
+ data-testid={`product-${product.id}-card`}
503
+ data-testid={`order-${order.id}-status-badge`}
504
+ ```
505
+
506
+ ### Element Type Suffix Table
507
+
508
+ Every `data-testid` value ends with a suffix indicating the element type:
509
+
510
+ | Element | Suffix | Example |
511
+ |---------|--------|---------|
512
+ | `<button>` | `-btn` | `login-submit-btn` |
513
+ | `<input>` | `-input` | `login-email-input` |
514
+ | `<select>` | `-select` | `settings-language-select` |
515
+ | `<textarea>` | `-textarea` | `feedback-comment-textarea` |
516
+ | `<a>` (link) | `-link` | `navbar-profile-link` |
517
+ | `<form>` | `-form` | `checkout-payment-form` |
518
+ | `<img>` | `-img` | `product-hero-img` |
519
+ | `<table>` | `-table` | `users-list-table` |
520
+ | `<tr>` (row) | `-row` | `users-item-row` |
521
+ | `<dialog>/<modal>` | `-modal` | `confirm-delete-modal` |
522
+ | `<div>` container | `-container` | `dashboard-stats-container` |
523
+ | `<ul>/<ol>` list | `-list` | `notifications-list` |
524
+ | `<li>` item | `-item` | `notifications-item` |
525
+ | dropdown menu | `-dropdown` | `navbar-user-dropdown` |
526
+ | tab | `-tab` | `settings-security-tab` |
527
+ | checkbox | `-checkbox` | `terms-accept-checkbox` |
528
+ | radio | `-radio` | `shipping-express-radio` |
529
+ | toggle/switch | `-toggle` | `notifications-enabled-toggle` |
530
+ | badge/chip | `-badge` | `cart-count-badge` |
531
+ | alert/toast | `-alert` | `error-validation-alert` |
532
+
533
+ ### Third-Party Component Handling
534
+
535
+ When adding `data-testid` to third-party UI library components, use this priority order:
536
+
537
+ 1. **Props passthrough** (preferred): If the library supports passing `data-testid` directly as a prop
538
+ ```tsx
539
+ <MuiButton data-testid="checkout-pay-btn">Pay</MuiButton>
540
+ ```
541
+
542
+ 2. **Wrapper div**: If the library does not support prop passthrough, wrap with a `<div>` that has the `data-testid`
543
+ ```tsx
544
+ <div data-testid="checkout-pay-container">
545
+ <ThirdPartyButton>Pay</ThirdPartyButton>
546
+ </div>
547
+ ```
548
+
549
+ 3. **inputProps / slotProps** (MUI-specific): Use component-specific prop APIs
550
+ ```tsx
551
+ <TextField inputProps={{ 'data-testid': 'login-email-input' }} />
552
+ <Autocomplete slotProps={{ input: { 'data-testid': 'search-query-input' } }} />
553
+ ```