qaa-agent 1.9.0 → 1.9.2

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CLAUDE.md CHANGED
@@ -1,557 +1,557 @@
1
- # QA Automation Standards
2
-
3
- This project follows strict QA automation standards. Every test, page object, and analysis produced MUST follow these rules.
4
-
5
- ## Framework Detection
6
-
7
- Before generating any code, **detect what the project already uses**:
8
-
9
- 1. Check for existing test config files: `cypress.config.ts`, `playwright.config.ts`, `jest.config.ts`, `vitest.config.ts`, `pytest.ini`, etc.
10
- 2. Check `package.json` or `requirements.txt` for test dependencies
11
- 3. Check existing test files for patterns and conventions
12
- 4. **Always match the project's existing framework, language, and conventions**
13
-
14
- If no framework exists yet, ask the user which one to use. Never assume.
15
-
16
- ## Testing Pyramid
17
-
18
- Target distribution for every project:
19
-
20
- ```
21
- / E2E \ 3-5% (critical path smoke only)
22
- / API \ 20-25% (endpoints + contracts)
23
- / Integration\ 10-15% (component interactions)
24
- / Unit \ 60-70% (business logic, pure functions)
25
- ```
26
-
27
- Adjust percentages based on the actual app architecture.
28
-
29
- ## Locator Strategy
30
-
31
- All UI test locators MUST follow this priority order. Never skip to a lower tier without written justification.
32
-
33
- **Tier 1 -- BEST (always try these first):**
34
- - Test IDs: `data-testid`, `data-cy`, `data-test` (adapt to framework)
35
- - Semantic roles: ARIA roles + accessible name
36
-
37
- **Tier 2 -- GOOD (when Tier 1 not available):**
38
- - Form labels, placeholders, visible text content
39
-
40
- **Tier 3 -- ACCEPTABLE (when Tier 1-2 not available):**
41
- - Alt text, title attributes
42
-
43
- **Tier 4 -- LAST RESORT (always add a TODO comment):**
44
- - CSS selectors, XPath -- mark with `// TODO: Request test ID for this element`
45
-
46
- ### Framework-Specific Examples
47
-
48
- **Playwright:**
49
- ```typescript
50
- page.getByTestId('submit') // Tier 1
51
- page.getByRole('button', {name: 'Log in'}) // Tier 1
52
- page.getByLabel('Email') // Tier 2
53
- page.locator('.btn') // Tier 4 -- add TODO
54
- ```
55
-
56
- **Cypress:**
57
- ```typescript
58
- cy.get('[data-cy="submit"]') // Tier 1
59
- cy.findByRole('button', {name: 'Log in'}) // Tier 1 (with testing-library)
60
- cy.get('[data-testid="submit"]') // Tier 1
61
- cy.contains('Submit') // Tier 2
62
- cy.get('.btn') // Tier 4 -- add TODO
63
- ```
64
-
65
- **Selenium / other:**
66
- ```
67
- driver.findElement(By.cssSelector('[data-testid="submit"]')) // Tier 1
68
- driver.findElement(By.className('btn')) // Tier 4 -- add TODO
69
- ```
70
-
71
- ## Page Object Model Rules
72
-
73
- These rules apply regardless of framework:
74
-
75
- 1. **One class/object per page or view** -- no god objects
76
- 2. **No assertions in page objects** -- assertions belong ONLY in test specs
77
- 3. **Locators are properties** -- defined in constructor or as class fields
78
- 4. **Actions return void or the next page** -- for fluent chaining
79
- 5. **State queries return data** -- let the test file decide what to assert
80
- 6. **Every POM extends a shared base** -- shared navigation, screenshots, waits
81
-
82
- ### POM File Structure
83
- ```
84
- [pages or page-objects or support/page-objects]/
85
- base/
86
- BasePage.[ext] -- shared methods
87
- [feature]/
88
- [Feature]Page.[ext] -- one file per page
89
- components/
90
- [Component].[ext] -- reusable UI components
91
- ```
92
-
93
- Adapt folder location to match the project's existing conventions.
94
-
95
- ## Test Spec Rules
96
-
97
- ### Every test case MUST have:
98
- - **Unique ID**: `UT-MODULE-001`, `API-AUTH-001`, `E2E-FLOW-001`
99
- - **Exact target**: file path + function name, or HTTP method + endpoint
100
- - **Concrete inputs**: actual values, not "valid data"
101
- - **Explicit expected outcome**: exact assertion, not "works correctly"
102
- - **Priority**: P0 (blocks release), P1 (should fix), P2 (nice to have)
103
-
104
- ### BAD assertions (never do this):
105
- ```
106
- expect(response.status).toBeTruthy()
107
- expect(data).toBeDefined()
108
- cy.get('.result').should('exist') // what should it contain?
109
- ```
110
-
111
- ### GOOD assertions (always do this):
112
- ```
113
- expect(response.status).toBe(200)
114
- expect(data.name).toBe('Test User')
115
- cy.get('[data-cy="result"]').should('have.text', 'Todo created successfully')
116
- ```
117
-
118
- ## Naming Conventions
119
-
120
- Adapt file extensions to match the project's language:
121
-
122
- | Type | Pattern | Example (.ts) | Example (.cy.ts) |
123
- |------|---------|---------------|-------------------|
124
- | Page Object | `[PageName]Page.[ext]` | `LoginPage.ts` | `LoginPage.ts` |
125
- | Component POM | `[ComponentName].[ext]` | `NavigationBar.ts` | `NavigationBar.ts` |
126
- | E2E test | `[feature].e2e.[ext]` | `login.e2e.spec.ts` | `login.e2e.cy.ts` |
127
- | API test | `[resource].api.[ext]` | `users.api.spec.ts` | `users.api.cy.ts` |
128
- | Unit test | `[module].unit.[ext]` | `validate.unit.spec.ts` | `validate.test.ts` |
129
- | Fixture | `[domain]-data.[ext]` | `auth-data.ts` | `auth-data.json` |
130
-
131
- If the project already has naming conventions, **follow those instead**.
132
-
133
- ## Repo Structure
134
-
135
- Recommended structure -- adapt to match what the project already has:
136
-
137
- ```
138
- tests/ or cypress/ or __tests__/
139
- e2e/
140
- smoke/ # P0 critical path (every PR)
141
- regression/ # Full suite (nightly)
142
- api/ # API-level tests
143
- unit/ # Unit tests
144
-
145
- pages/ or page-objects/ or support/page-objects/
146
- base/
147
- [feature]/
148
- components/
149
-
150
- fixtures/ # Test data & factories
151
- config/ # Test configs (if separate from root)
152
- reports/ # Generated reports (gitignored)
153
- ```
154
-
155
- ## Test Data Rules
156
-
157
- - **NEVER** hardcode real credentials
158
- - Use environment variables with test fallbacks
159
- - Fixtures go in dedicated folder
160
- - Each domain gets its own fixture file
161
-
162
- ## Analysis Documents
163
-
164
- When analyzing a repository, produce these documents:
165
-
166
- ### QA_ANALYSIS.md must include:
167
- - Architecture overview (system type, language, runtime, entry points, dependencies)
168
- - Risk assessment (HIGH / MEDIUM / LOW with justification)
169
- - Top 10 unit test targets with rationale
170
- - Recommended testing pyramid with percentages adjusted to this app
171
- - External dependencies with risk levels
172
-
173
- ### TEST_INVENTORY.md must include:
174
- - Every test case with ID, target, inputs, expected outcome, priority
175
- - Organized by pyramid level (unit -> integration -> API -> E2E)
176
- - No test case without an explicit expected outcome
177
-
178
- ## Quality Gates
179
-
180
- Before delivering ANY QA artifact, verify:
181
- - [ ] Framework matches what the project already uses
182
- - [ ] Every test case has an explicit expected outcome with a concrete value
183
- - [ ] No outcome says "correct", "proper", "appropriate", or "works" without defining what that means
184
- - [ ] All locators follow the tier hierarchy
185
- - [ ] No assertions inside page objects
186
- - [ ] No hardcoded credentials
187
- - [ ] File naming follows the project's existing conventions (or the standards above if none exist)
188
- - [ ] Test IDs are unique and follow naming convention
189
- - [ ] Priority assigned to every test case
190
-
191
- ---
192
-
193
- ## Agent Pipeline
194
-
195
- The QA automation system runs agents in a defined pipeline. Each stage produces artifacts consumed by the next stage.
196
-
197
- ### Pipeline Stages
198
-
199
- ```
200
- scan -> research -> codebase-map -> analyze -> [testid-inject if frontend] -> plan -> generate -> validate -> [e2e-runner if E2E tests] -> [bug-detective if failures] -> deliver
201
- ```
202
-
203
- ### Workflow Options
204
-
205
- **Option 1: Dev-Only Repo (no existing QA repo)**
206
- Full pipeline from scratch:
207
- ```
208
- scan -> research -> codebase-map -> analyze -> [testid-inject if frontend] -> plan -> generate -> validate -> [e2e-runner if E2E tests] -> [bug-detective if failures] -> deliver
209
- ```
210
- Produces: SCAN_MANIFEST.md -> 8 codebase map documents -> QA_ANALYSIS.md + TEST_INVENTORY.md + QA_REPO_BLUEPRINT.md -> [TESTID_AUDIT_REPORT.md] -> generation plan -> test files + POMs + fixtures + configs -> VALIDATION_REPORT.md -> [E2E_RUN_REPORT.md] -> branch + PR
211
-
212
- **Option 2: Dev + Immature QA Repo (existing QA repo with low coverage or quality)**
213
- Gap-fill and standardize:
214
- ```
215
- scan both repos -> gap analysis -> fix broken tests -> add missing coverage -> standardize existing -> validate -> deliver
216
- ```
217
- Produces: SCAN_MANIFEST.md (both repos) -> GAP_ANALYSIS.md -> fixed test files -> new test files -> standardized files -> VALIDATION_REPORT.md -> branch + PR
218
-
219
- **Option 3: Dev + Mature QA Repo (existing QA repo with solid coverage)**
220
- Surgical additions only:
221
- ```
222
- scan both repos -> identify thin coverage -> add only missing tests -> validate -> deliver
223
- ```
224
- Produces: SCAN_MANIFEST.md (both repos) -> GAP_ANALYSIS.md (thin areas only) -> new test files (targeted) -> VALIDATION_REPORT.md -> branch + PR
225
-
226
- ### Stage Transitions
227
-
228
- | From | To | Condition |
229
- |------|----|-----------|
230
- | scan | research | SCAN_MANIFEST.md exists with > 0 testable surfaces |
231
- | research | codebase-map | Research complete (non-blocking — continues even if no research output) |
232
- | codebase-map | analyze | At least 4 of 8 codebase map documents exist |
233
- | analyze | testid-inject | QA_ANALYSIS.md exists AND frontend components detected |
234
- | analyze | plan | QA_ANALYSIS.md + TEST_INVENTORY.md exist (skip testid-inject if no frontend) |
235
- | testid-inject | plan | TESTID_AUDIT_REPORT.md exists with coverage score calculated |
236
- | plan | generate | Generation plan approved (or auto-approved in auto-advance mode) |
237
- | generate | validate | All planned test files exist on disk |
238
- | validate | e2e-runner | VALIDATION_REPORT.md shows PASS AND E2E test files were generated AND live app available |
239
- | validate | deliver | VALIDATION_REPORT.md shows PASS and no E2E files or --skip-run |
240
- | e2e-runner | bug-detective | E2E_RUN_REPORT.md exists AND failures need classification |
241
- | e2e-runner | deliver | E2E_RUN_REPORT.md exists AND all tests pass or failures classified |
242
- | bug-detective | deliver | FAILURE_CLASSIFICATION_REPORT.md exists |
243
-
244
- ---
245
-
246
- ## Module Boundaries
247
-
248
- Each agent owns specific artifacts. No agent may produce artifacts assigned to another agent.
249
-
250
- | Agent | Reads | Produces | Template |
251
- |-------|-------|----------|----------|
252
- | qa-scanner | repo source files, package.json, file tree | SCAN_MANIFEST.md | templates/scan-manifest.md |
253
- | qa-project-researcher | SCAN_MANIFEST.md, repo source files, Context7 MCP, WebSearch, WebFetch | TESTING_STACK.md, FRAMEWORK_CAPABILITIES.md, API_TESTING_STRATEGY.md, E2E_STRATEGY.md | -- |
254
- | qa-codebase-mapper | SCAN_MANIFEST.md, repo source files, CLAUDE.md | TESTABILITY.md, TEST_SURFACE.md, RISK_MAP.md, CRITICAL_PATHS.md, CODE_PATTERNS.md, API_CONTRACTS.md, TEST_ASSESSMENT.md, COVERAGE_GAPS.md | -- |
255
- | qa-analyzer | SCAN_MANIFEST.md, codebase map documents, CLAUDE.md | QA_ANALYSIS.md, TEST_INVENTORY.md, QA_REPO_BLUEPRINT.md (Option 1) or GAP_ANALYSIS.md (Option 2/3) | templates/qa-analysis.md, templates/test-inventory.md, templates/qa-repo-blueprint.md, templates/gap-analysis.md |
256
- | qa-planner | TEST_INVENTORY.md, QA_ANALYSIS.md, codebase map documents | Generation plan (internal) | -- |
257
- | qa-executor | TEST_INVENTORY.md, codebase map documents, CLAUDE.md | test files, POMs, fixtures, configs | qa-template-engine patterns |
258
- | qa-validator | generated test files, CLAUDE.md | VALIDATION_REPORT.md (validation mode) or QA_AUDIT_REPORT.md (audit mode) | templates/validation-report.md, templates/qa-audit-report.md |
259
- | qa-e2e-runner | generated E2E test files, POM files, CLAUDE.md, live application | E2E_RUN_REPORT.md, fixed test/POM files with real locators | -- |
260
- | qa-testid-injector | repo source files, SCAN_MANIFEST.md, CLAUDE.md | TESTID_AUDIT_REPORT.md, modified source files with data-testid attributes | templates/scan-manifest.md, templates/testid-audit-report.md |
261
- | qa-bug-detective | test execution results, test source files, CLAUDE.md | FAILURE_CLASSIFICATION_REPORT.md | templates/failure-classification.md |
262
-
263
- **Rule:** An agent MUST NOT produce artifacts assigned to another agent.
264
-
265
- **Rule:** An agent MUST read all artifacts listed in its "Reads" column before producing output.
266
-
267
- ---
268
-
269
- ## Verification Commands
270
-
271
- Every artifact must pass verification before the pipeline advances. Below are the validation rules per artifact type.
272
-
273
- ### SCAN_MANIFEST.md
274
- - Has > 0 files in File List table
275
- - Project Detection section is populated (framework, language, component patterns)
276
- - Testable Surfaces has at least 1 category with entries
277
- - File priority ordering is present (HIGH/MEDIUM/LOW)
278
-
279
- ### QA_ANALYSIS.md
280
- - All 6 sections present: Architecture Overview, External Dependencies, Risk Assessment, Top 10 Unit Test Targets, API/Contract Test Targets, Recommended Testing Pyramid
281
- - Top 10 has exactly 10 entries with module, rationale, and complexity
282
- - Testing pyramid percentages sum to 100%
283
- - Risk assessment uses only HIGH/MEDIUM/LOW with justification per item
284
-
285
- ### TEST_INVENTORY.md
286
- - Every test case has all mandatory fields: ID, target, inputs, expected outcome, priority
287
- - IDs are unique across the entire document (no duplicates)
288
- - IDs follow naming convention: UT-MODULE-NNN, INT-MODULE-NNN, API-RESOURCE-NNN, E2E-FLOW-NNN
289
- - Pyramid tier counts match the summary table
290
- - No expected outcome says "correct", "proper", "appropriate", or "works" without concrete value
291
-
292
- ### QA_REPO_BLUEPRINT.md
293
- - Folder structure tree is present with explanations per directory
294
- - Config files section has actual content (not placeholders)
295
- - npm scripts defined for smoke, regression, and API test runs
296
- - CI/CD strategy section includes PR-gate and nightly run configurations
297
- - Definition of Done checklist is present
298
-
299
- ### VALIDATION_REPORT.md
300
- - All 4 layers reported per file: Syntax, Structure, Dependencies, Logic
301
- - Each layer shows PASS or FAIL with details
302
- - Confidence level assigned: HIGH (all layers pass), MEDIUM (1-2 minor issues), LOW (structural problems)
303
- - Fix loop log shows iteration count and what was found/fixed per loop
304
- - Unresolved issues section documents anything not auto-fixed
305
-
306
- ### FAILURE_CLASSIFICATION_REPORT.md
307
- - Every failure has classification: APPLICATION BUG, TEST CODE ERROR, ENVIRONMENT ISSUE, or INCONCLUSIVE
308
- - Every failure has confidence level: HIGH, MEDIUM-HIGH, MEDIUM, or LOW
309
- - Every failure has evidence: code snippet + reasoning explaining the classification
310
- - No APPLICATION BUG is marked as auto-fixed (application bugs require developer action)
311
- - Auto-fix log documents what was fixed and at what confidence level
312
-
313
- ### TESTID_AUDIT_REPORT.md
314
- - Coverage score calculated: existing data-testid count / total interactive elements
315
- - All proposed data-testid values follow `{context}-{description}-{element-type}` naming convention
316
- - No duplicate data-testid values within the same page/route scope
317
- - Elements classified by priority: P0 (form inputs, buttons), P1 (links, images), P2 (containers, decorative)
318
- - Decision gate threshold applied: >90% SELECTIVE, 50-90% TARGETED, <50% FULL PASS, 0% P0 FIRST
319
-
320
- ### GAP_ANALYSIS.md
321
- - Coverage map shows all modules from SCAN_MANIFEST.md
322
- - Missing tests have IDs following naming convention and priorities assigned
323
- - Broken tests have failure reasons documented with file path and error
324
- - Quality assessment includes locator tier distribution and assertion quality rating
325
- - Recommendations are prioritized: fix broken first, then add P0, then P1
326
-
327
- ### QA_AUDIT_REPORT.md
328
- - All 6 dimensions scored: Locator Quality, Assertion Specificity, POM Compliance, Test Coverage, Naming Convention, Test Data Management
329
- - Weights sum to 100%: Locator 20%, Assertion 20%, POM 15%, Coverage 20%, Naming 15%, Test Data 10%
330
- - Overall score matches weighted calculation of dimension scores
331
- - Critical issues listed with file path, line number, and description
332
- - Each recommendation has effort estimate: S (small), M (medium), L (large)
333
-
334
- ---
335
-
336
- ## Git Workflow
337
-
338
- All QA automation output follows these git conventions.
339
-
340
- ### Branch Naming
341
-
342
- ```
343
- qa/auto-{project}-{date}
344
- ```
345
-
346
- Examples:
347
- - `qa/auto-shopflow-2026-03-18`
348
- - `qa/auto-acme-api-2026-04-01`
349
-
350
- ### Commit Message Format
351
-
352
- ```
353
- qa({agent}): {description}
354
- ```
355
-
356
- Examples:
357
- - `qa(scanner): produce SCAN_MANIFEST.md for shopflow`
358
- - `qa(analyzer): produce QA_ANALYSIS.md and TEST_INVENTORY.md`
359
- - `qa(executor): generate 24 test files with POMs and fixtures`
360
- - `qa(validator): validate generated tests - PASS with HIGH confidence`
361
- - `qa(testid-injector): inject 47 data-testid attributes across 12 components`
362
- - `qa(bug-detective): classify 5 failures - 2 APP BUG, 2 TEST ERROR, 1 ENV ISSUE`
363
-
364
- ### Commit Conventions
365
-
366
- - One commit per agent stage (scanner produces one commit, analyzer produces one commit, etc.)
367
- - Descriptive messages that include artifact names and counts
368
- - Never commit .env files, credentials, or secrets
369
- - Include modified file count in commit body when relevant
370
-
371
- ### PR Template
372
-
373
- PR description must include:
374
- - Analysis summary (architecture type, framework, risk areas)
375
- - Test counts by pyramid level (unit: N, integration: N, API: N, E2E: N)
376
- - Coverage metrics (modules covered, estimated line coverage)
377
- - Validation pass/fail status with confidence level
378
- - Link to VALIDATION_REPORT.md in the PR files
379
-
380
- ---
381
-
382
- ## Team Settings
383
-
384
- Configuration for multi-agent pipeline execution.
385
-
386
- ### Concurrent Execution
387
-
388
- Agents in the same pipeline stage can run in parallel when their inputs are independent. Examples:
389
- - qa-testid-injector and qa-analyzer can run simultaneously after scan completes (both read SCAN_MANIFEST.md)
390
- - Multiple qa-executor instances can generate tests for different modules in parallel
391
-
392
- Agents in different pipeline stages MUST respect stage ordering. A downstream agent cannot start until all its required inputs exist on disk.
393
-
394
- ### Worktree Isolation
395
-
396
- Each agent operates on the same branch. No worktree splits are needed for this system. Agents coordinate through file-based artifacts -- each agent writes its own files and reads other agents' files.
397
-
398
- ### Dependency Ordering
399
-
400
- Respect stage transitions from the Agent Pipeline section:
401
- 1. qa-scanner runs first (no dependencies)
402
- 2. qa-project-researcher runs after scanner (depends on SCAN_MANIFEST.md, uses Context7 MCP — non-blocking, pipeline continues if it fails)
403
- 3. qa-codebase-mapper runs after researcher (depends on SCAN_MANIFEST.md, spawns 4 parallel focus agents)
404
- 4. qa-analyzer and qa-testid-injector run after codebase-map (both depend on SCAN_MANIFEST.md + codebase map documents + research documents if available)
405
- 5. qa-planner runs after analyzer (depends on QA_ANALYSIS.md + TEST_INVENTORY.md + codebase map documents)
406
- 6. qa-executor runs after planner (depends on generation plan + codebase map documents + research documents if available + Context7 MCP)
407
- 7. qa-validator runs after executor (depends on generated test files)
408
- 8. qa-e2e-runner runs after validator (depends on E2E test files + live application + Context7 MCP)
409
- 9. qa-bug-detective runs after e2e-runner or validator if failures (depends on test results + Context7 MCP)
410
-
411
- ### Auto-Advance Mode
412
-
413
- When auto-advance is enabled, pipeline stages advance automatically when:
414
- 1. The previous stage completes
415
- 2. All output artifacts from the previous stage exist on disk
416
- 3. All output artifacts pass their verification commands (from Verification Commands section)
417
-
418
- No human confirmation is needed between stages in auto-advance mode. The pipeline pauses only at explicit checkpoint tasks or when verification fails.
419
-
420
- ---
421
-
422
- ## Agent Coordination
423
-
424
- Rules governing how agents communicate and hand off work through artifacts.
425
-
426
- ### Read-Before-Write Rules
427
-
428
- Every agent MUST read its required inputs before producing any output. Failure to read inputs produces low-quality, inconsistent artifacts.
429
-
430
- | Agent | MUST Read Before Producing Output |
431
- |-------|-----------------------------------|
432
- | qa-scanner | package.json (or equivalent), folder tree structure, all source file extensions to detect framework and language |
433
- | qa-project-researcher | SCAN_MANIFEST.md, repo source files, MY_PREFERENCES.md. Uses Context7 MCP as primary source. |
434
- | qa-analyzer | SCAN_MANIFEST.md (complete, verified), CLAUDE.md (all QA standards sections), research documents (if available) |
435
- | qa-planner | TEST_INVENTORY.md (all test cases), QA_ANALYSIS.md (architecture and risk context) |
436
- | qa-executor | TEST_INVENTORY.md (test cases to implement), CLAUDE.md (POM rules, locator hierarchy, assertion rules, naming conventions, quality gates), research documents (if available). Must query Context7 MCP before generating code. |
437
- | qa-validator | CLAUDE.md (quality gates, locator tiers, assertion rules), all generated test files to validate |
438
- | qa-testid-injector | SCAN_MANIFEST.md (component file list), CLAUDE.md (data-testid Convention section for naming rules) |
439
- | qa-bug-detective | Test execution output (stdout/stderr, exit codes), test source files (to read the failing code), CLAUDE.md (classification rules), research documents (if available). Must query Context7 MCP before auto-fixing. |
440
- | qa-e2e-runner | Generated E2E test files, POM files, CLAUDE.md, live application, research documents (if available). Must query Context7 MCP before fixing locators. |
441
-
442
- ### Handoff Patterns
443
-
444
- Agents communicate exclusively through file-based artifacts:
445
-
446
- 1. **Producer writes** -- Agent completes its task and writes output artifact(s) to disk
447
- 2. **Pipeline verifies** -- Output artifacts pass verification commands before advancing
448
- 3. **Consumer reads** -- Next agent reads the artifact(s) as its first action
449
- 4. **No direct communication** -- Agents never pass data in memory or through environment variables between stages
450
-
451
- ### Quality Gates Per Artifact
452
-
453
- Before the pipeline advances past any stage, the produced artifact(s) must pass verification:
454
-
455
- | Stage Complete | Artifact | Gate |
456
- |----------------|----------|------|
457
- | scan | SCAN_MANIFEST.md | > 0 files listed, project detection populated |
458
- | analyze | QA_ANALYSIS.md + TEST_INVENTORY.md | All sections present, IDs unique, pyramid sums to 100% |
459
- | testid-inject | TESTID_AUDIT_REPORT.md | Coverage score calculated, naming convention compliant |
460
- | plan | Generation plan | Test cases mapped to output files, no unassigned cases |
461
- | generate | Test files + POMs | All planned files exist, imports resolve, syntax valid |
462
- | validate | VALIDATION_REPORT.md | All 4 layers checked per file, confidence level assigned |
463
- | deliver | Branch + PR | Branch pushed, PR created with required description sections |
464
-
465
- ### Error Recovery
466
-
467
- If an agent fails or produces an artifact that does not pass verification:
468
- 1. Log the failure with the specific verification check that failed
469
- 2. Retry the agent (max 3 attempts per stage)
470
- 3. If still failing after 3 attempts, pause the pipeline and report the blocked stage
471
- 4. Do not advance to the next stage with a failed artifact
472
-
473
- ---
474
-
475
- ## data-testid Convention
476
-
477
- All `data-testid` attributes injected by qa-testid-injector and referenced by generated tests MUST follow this naming convention.
478
-
479
- ### Naming Pattern
480
-
481
- ```
482
- {context}-{description}-{element-type}
483
- ```
484
-
485
- All values are **kebab-case**. No camelCase, no underscores, no periods.
486
-
487
- ### Context Derivation
488
-
489
- 1. **Page-level context**: Derived from the component filename or route
490
- - `LoginPage.tsx` -> context is `login`
491
- - `ProductDetailPage.tsx` -> context is `product-detail`
492
- - Route `/settings/profile` -> context is `settings-profile`
493
-
494
- 2. **Component-level context**: Derived from the component name
495
- - `<NavBar>` -> context is `navbar`
496
- - `<ShoppingCart>` -> context is `shopping-cart`
497
- - `<UserAvatar>` -> context is `user-avatar`
498
-
499
- 3. **Nested context**: Parent-child hierarchy, max 3 levels deep
500
- - `checkout-shipping-address-input` (page -> section -> field)
501
- - `dashboard-sidebar-nav-link` (page -> component -> element)
502
- - Never exceed 3 levels: `a-b-c-element` is the maximum depth
503
-
504
- 4. **Dynamic list items**: Use template literals with unique keys
505
- ```tsx
506
- data-testid={`product-${product.id}-card`}
507
- data-testid={`order-${order.id}-status-badge`}
508
- ```
509
-
510
- ### Element Type Suffix Table
511
-
512
- Every `data-testid` value ends with a suffix indicating the element type:
513
-
514
- | Element | Suffix | Example |
515
- |---------|--------|---------|
516
- | `<button>` | `-btn` | `login-submit-btn` |
517
- | `<input>` | `-input` | `login-email-input` |
518
- | `<select>` | `-select` | `settings-language-select` |
519
- | `<textarea>` | `-textarea` | `feedback-comment-textarea` |
520
- | `<a>` (link) | `-link` | `navbar-profile-link` |
521
- | `<form>` | `-form` | `checkout-payment-form` |
522
- | `<img>` | `-img` | `product-hero-img` |
523
- | `<table>` | `-table` | `users-list-table` |
524
- | `<tr>` (row) | `-row` | `users-item-row` |
525
- | `<dialog>/<modal>` | `-modal` | `confirm-delete-modal` |
526
- | `<div>` container | `-container` | `dashboard-stats-container` |
527
- | `<ul>/<ol>` list | `-list` | `notifications-list` |
528
- | `<li>` item | `-item` | `notifications-item` |
529
- | dropdown menu | `-dropdown` | `navbar-user-dropdown` |
530
- | tab | `-tab` | `settings-security-tab` |
531
- | checkbox | `-checkbox` | `terms-accept-checkbox` |
532
- | radio | `-radio` | `shipping-express-radio` |
533
- | toggle/switch | `-toggle` | `notifications-enabled-toggle` |
534
- | badge/chip | `-badge` | `cart-count-badge` |
535
- | alert/toast | `-alert` | `error-validation-alert` |
536
-
537
- ### Third-Party Component Handling
538
-
539
- When adding `data-testid` to third-party UI library components, use this priority order:
540
-
541
- 1. **Props passthrough** (preferred): If the library supports passing `data-testid` directly as a prop
542
- ```tsx
543
- <MuiButton data-testid="checkout-pay-btn">Pay</MuiButton>
544
- ```
545
-
546
- 2. **Wrapper div**: If the library does not support prop passthrough, wrap with a `<div>` that has the `data-testid`
547
- ```tsx
548
- <div data-testid="checkout-pay-container">
549
- <ThirdPartyButton>Pay</ThirdPartyButton>
550
- </div>
551
- ```
552
-
553
- 3. **inputProps / slotProps** (MUI-specific): Use component-specific prop APIs
554
- ```tsx
555
- <TextField inputProps={{ 'data-testid': 'login-email-input' }} />
556
- <Autocomplete slotProps={{ input: { 'data-testid': 'search-query-input' } }} />
557
- ```
1
+ # QA Automation Standards
2
+
3
+ This project follows strict QA automation standards. Every test, page object, and analysis produced MUST follow these rules.
4
+
5
+ ## Framework Detection
6
+
7
+ Before generating any code, **detect what the project already uses**:
8
+
9
+ 1. Check for existing test config files: `cypress.config.ts`, `playwright.config.ts`, `jest.config.ts`, `vitest.config.ts`, `pytest.ini`, etc.
10
+ 2. Check `package.json` or `requirements.txt` for test dependencies
11
+ 3. Check existing test files for patterns and conventions
12
+ 4. **Always match the project's existing framework, language, and conventions**
13
+
14
+ If no framework exists yet, ask the user which one to use. Never assume.
15
+
16
+ ## Testing Pyramid
17
+
18
+ Target distribution for every project:
19
+
20
+ ```
21
+ / E2E \ 3-5% (critical path smoke only)
22
+ / API \ 20-25% (endpoints + contracts)
23
+ / Integration\ 10-15% (component interactions)
24
+ / Unit \ 60-70% (business logic, pure functions)
25
+ ```
26
+
27
+ Adjust percentages based on the actual app architecture.
28
+
29
+ ## Locator Strategy
30
+
31
+ All UI test locators MUST follow this priority order. Never skip to a lower tier without written justification.
32
+
33
+ **Tier 1 -- BEST (always try these first):**
34
+ - Test IDs: `data-testid`, `data-cy`, `data-test` (adapt to framework)
35
+ - Semantic roles: ARIA roles + accessible name
36
+
37
+ **Tier 2 -- GOOD (when Tier 1 not available):**
38
+ - Form labels, placeholders, visible text content
39
+
40
+ **Tier 3 -- ACCEPTABLE (when Tier 1-2 not available):**
41
+ - Alt text, title attributes
42
+
43
+ **Tier 4 -- LAST RESORT (always add a TODO comment):**
44
+ - CSS selectors, XPath -- mark with `// TODO: Request test ID for this element`
45
+
46
+ ### Framework-Specific Examples
47
+
48
+ **Playwright:**
49
+ ```typescript
50
+ page.getByTestId('submit') // Tier 1
51
+ page.getByRole('button', {name: 'Log in'}) // Tier 1
52
+ page.getByLabel('Email') // Tier 2
53
+ page.locator('.btn') // Tier 4 -- add TODO
54
+ ```
55
+
56
+ **Cypress:**
57
+ ```typescript
58
+ cy.get('[data-cy="submit"]') // Tier 1
59
+ cy.findByRole('button', {name: 'Log in'}) // Tier 1 (with testing-library)
60
+ cy.get('[data-testid="submit"]') // Tier 1
61
+ cy.contains('Submit') // Tier 2
62
+ cy.get('.btn') // Tier 4 -- add TODO
63
+ ```
64
+
65
+ **Selenium / other:**
66
+ ```
67
+ driver.findElement(By.cssSelector('[data-testid="submit"]')) // Tier 1
68
+ driver.findElement(By.className('btn')) // Tier 4 -- add TODO
69
+ ```
70
+
71
+ ## Page Object Model Rules
72
+
73
+ These rules apply regardless of framework:
74
+
75
+ 1. **One class/object per page or view** -- no god objects
76
+ 2. **No assertions in page objects** -- assertions belong ONLY in test specs
77
+ 3. **Locators are properties** -- defined in constructor or as class fields
78
+ 4. **Actions return void or the next page** -- for fluent chaining
79
+ 5. **State queries return data** -- let the test file decide what to assert
80
+ 6. **Every POM extends a shared base** -- shared navigation, screenshots, waits
81
+
82
+ ### POM File Structure
83
+ ```
84
+ [pages or page-objects or support/page-objects]/
85
+ base/
86
+ BasePage.[ext] -- shared methods
87
+ [feature]/
88
+ [Feature]Page.[ext] -- one file per page
89
+ components/
90
+ [Component].[ext] -- reusable UI components
91
+ ```
92
+
93
+ Adapt folder location to match the project's existing conventions.
94
+
95
+ ## Test Spec Rules
96
+
97
+ ### Every test case MUST have:
98
+ - **Unique ID**: `UT-MODULE-001`, `API-AUTH-001`, `E2E-FLOW-001`
99
+ - **Exact target**: file path + function name, or HTTP method + endpoint
100
+ - **Concrete inputs**: actual values, not "valid data"
101
+ - **Explicit expected outcome**: exact assertion, not "works correctly"
102
+ - **Priority**: P0 (blocks release), P1 (should fix), P2 (nice to have)
103
+
104
+ ### BAD assertions (never do this):
105
+ ```
106
+ expect(response.status).toBeTruthy()
107
+ expect(data).toBeDefined()
108
+ cy.get('.result').should('exist') // what should it contain?
109
+ ```
110
+
111
+ ### GOOD assertions (always do this):
112
+ ```
113
+ expect(response.status).toBe(200)
114
+ expect(data.name).toBe('Test User')
115
+ cy.get('[data-cy="result"]').should('have.text', 'Todo created successfully')
116
+ ```
117
+
118
+ ## Naming Conventions
119
+
120
+ Adapt file extensions to match the project's language:
121
+
122
+ | Type | Pattern | Example (.ts) | Example (.cy.ts) |
123
+ |------|---------|---------------|-------------------|
124
+ | Page Object | `[PageName]Page.[ext]` | `LoginPage.ts` | `LoginPage.ts` |
125
+ | Component POM | `[ComponentName].[ext]` | `NavigationBar.ts` | `NavigationBar.ts` |
126
+ | E2E test | `[feature].e2e.[ext]` | `login.e2e.spec.ts` | `login.e2e.cy.ts` |
127
+ | API test | `[resource].api.[ext]` | `users.api.spec.ts` | `users.api.cy.ts` |
128
+ | Unit test | `[module].unit.[ext]` | `validate.unit.spec.ts` | `validate.test.ts` |
129
+ | Fixture | `[domain]-data.[ext]` | `auth-data.ts` | `auth-data.json` |
130
+
131
+ If the project already has naming conventions, **follow those instead**.
132
+
133
+ ## Repo Structure
134
+
135
+ Recommended structure -- adapt to match what the project already has:
136
+
137
+ ```
138
+ tests/ or cypress/ or __tests__/
139
+ e2e/
140
+ smoke/ # P0 critical path (every PR)
141
+ regression/ # Full suite (nightly)
142
+ api/ # API-level tests
143
+ unit/ # Unit tests
144
+
145
+ pages/ or page-objects/ or support/page-objects/
146
+ base/
147
+ [feature]/
148
+ components/
149
+
150
+ fixtures/ # Test data & factories
151
+ config/ # Test configs (if separate from root)
152
+ reports/ # Generated reports (gitignored)
153
+ ```
154
+
155
+ ## Test Data Rules
156
+
157
+ - **NEVER** hardcode real credentials
158
+ - Use environment variables with test fallbacks
159
+ - Fixtures go in dedicated folder
160
+ - Each domain gets its own fixture file
161
+
162
+ ## Analysis Documents
163
+
164
+ When analyzing a repository, produce these documents:
165
+
166
+ ### QA_ANALYSIS.md must include:
167
+ - Architecture overview (system type, language, runtime, entry points, dependencies)
168
+ - Risk assessment (HIGH / MEDIUM / LOW with justification)
169
+ - Top 10 unit test targets with rationale
170
+ - Recommended testing pyramid with percentages adjusted to this app
171
+ - External dependencies with risk levels
172
+
173
+ ### TEST_INVENTORY.md must include:
174
+ - Every test case with ID, target, inputs, expected outcome, priority
175
+ - Organized by pyramid level (unit -> integration -> API -> E2E)
176
+ - No test case without an explicit expected outcome
177
+
178
+ ## Quality Gates
179
+
180
+ Before delivering ANY QA artifact, verify:
181
+ - [ ] Framework matches what the project already uses
182
+ - [ ] Every test case has an explicit expected outcome with a concrete value
183
+ - [ ] No outcome says "correct", "proper", "appropriate", or "works" without defining what that means
184
+ - [ ] All locators follow the tier hierarchy
185
+ - [ ] No assertions inside page objects
186
+ - [ ] No hardcoded credentials
187
+ - [ ] File naming follows the project's existing conventions (or the standards above if none exist)
188
+ - [ ] Test IDs are unique and follow naming convention
189
+ - [ ] Priority assigned to every test case
190
+
191
+ ---
192
+
193
+ ## Agent Pipeline
194
+
195
+ The QA automation system runs agents in a defined pipeline. Each stage produces artifacts consumed by the next stage.
196
+
197
+ ### Pipeline Stages
198
+
199
+ ```
200
+ scan -> codebase-map -> research -> analyze -> [testid-inject if frontend] -> plan -> generate -> validate -> [e2e-runner if E2E tests] -> [bug-detective if failures] -> deliver
201
+ ```
202
+
203
+ ### Workflow Options
204
+
205
+ **Option 1: Dev-Only Repo (no existing QA repo)**
206
+ Full pipeline from scratch:
207
+ ```
208
+ scan -> codebase-map -> research -> analyze -> [testid-inject if frontend] -> plan -> generate -> validate -> [e2e-runner if E2E tests] -> [bug-detective if failures] -> deliver
209
+ ```
210
+ Produces: SCAN_MANIFEST.md -> 8 codebase map documents -> QA_ANALYSIS.md + TEST_INVENTORY.md + QA_REPO_BLUEPRINT.md -> [TESTID_AUDIT_REPORT.md] -> generation plan -> test files + POMs + fixtures + configs -> VALIDATION_REPORT.md -> [E2E_RUN_REPORT.md] -> branch + PR
211
+
212
+ **Option 2: Dev + Immature QA Repo (existing QA repo with low coverage or quality)**
213
+ Gap-fill and standardize:
214
+ ```
215
+ scan both repos -> gap analysis -> fix broken tests -> add missing coverage -> standardize existing -> validate -> deliver
216
+ ```
217
+ Produces: SCAN_MANIFEST.md (both repos) -> GAP_ANALYSIS.md -> fixed test files -> new test files -> standardized files -> VALIDATION_REPORT.md -> branch + PR
218
+
219
+ **Option 3: Dev + Mature QA Repo (existing QA repo with solid coverage)**
220
+ Surgical additions only:
221
+ ```
222
+ scan both repos -> identify thin coverage -> add only missing tests -> validate -> deliver
223
+ ```
224
+ Produces: SCAN_MANIFEST.md (both repos) -> GAP_ANALYSIS.md (thin areas only) -> new test files (targeted) -> VALIDATION_REPORT.md -> branch + PR
225
+
226
+ ### Stage Transitions
227
+
228
+ | From | To | Condition |
229
+ |------|----|-----------|
230
+ | scan | research | SCAN_MANIFEST.md exists with > 0 testable surfaces |
231
+ | research | codebase-map | Research complete (non-blocking — continues even if no research output) |
232
+ | codebase-map | analyze | At least 4 of 8 codebase map documents exist |
233
+ | analyze | testid-inject | QA_ANALYSIS.md exists AND frontend components detected |
234
+ | analyze | plan | QA_ANALYSIS.md + TEST_INVENTORY.md exist (skip testid-inject if no frontend) |
235
+ | testid-inject | plan | TESTID_AUDIT_REPORT.md exists with coverage score calculated |
236
+ | plan | generate | Generation plan approved (or auto-approved in auto-advance mode) |
237
+ | generate | validate | All planned test files exist on disk |
238
+ | validate | e2e-runner | VALIDATION_REPORT.md shows PASS AND E2E test files were generated AND live app available |
239
+ | validate | deliver | VALIDATION_REPORT.md shows PASS and no E2E files or --skip-run |
240
+ | e2e-runner | bug-detective | E2E_RUN_REPORT.md exists AND failures need classification |
241
+ | e2e-runner | deliver | E2E_RUN_REPORT.md exists AND all tests pass or failures classified |
242
+ | bug-detective | deliver | FAILURE_CLASSIFICATION_REPORT.md exists |
243
+
244
+ ---
245
+
246
+ ## Module Boundaries
247
+
248
+ Each agent owns specific artifacts. No agent may produce artifacts assigned to another agent.
249
+
250
+ | Agent | Reads | Produces | Template |
251
+ |-------|-------|----------|----------|
252
+ | qa-scanner | repo source files, package.json, file tree | SCAN_MANIFEST.md | templates/scan-manifest.md |
253
+ | qa-project-researcher | SCAN_MANIFEST.md, repo source files, Context7 MCP, WebSearch, WebFetch | TESTING_STACK.md, FRAMEWORK_CAPABILITIES.md, API_TESTING_STRATEGY.md, E2E_STRATEGY.md | -- |
254
+ | qa-codebase-mapper | SCAN_MANIFEST.md, repo source files, CLAUDE.md | TESTABILITY.md, TEST_SURFACE.md, RISK_MAP.md, CRITICAL_PATHS.md, CODE_PATTERNS.md, API_CONTRACTS.md, TEST_ASSESSMENT.md, COVERAGE_GAPS.md | -- |
255
+ | qa-analyzer | SCAN_MANIFEST.md, codebase map documents, CLAUDE.md | QA_ANALYSIS.md, TEST_INVENTORY.md, QA_REPO_BLUEPRINT.md (Option 1) or GAP_ANALYSIS.md (Option 2/3) | templates/qa-analysis.md, templates/test-inventory.md, templates/qa-repo-blueprint.md, templates/gap-analysis.md |
256
+ | qa-planner | TEST_INVENTORY.md, QA_ANALYSIS.md, codebase map documents | Generation plan (internal) | -- |
257
+ | qa-executor | TEST_INVENTORY.md, codebase map documents, CLAUDE.md | test files, POMs, fixtures, configs | qa-template-engine patterns |
258
+ | qa-validator | generated test files, CLAUDE.md | VALIDATION_REPORT.md (validation mode) or QA_AUDIT_REPORT.md (audit mode) | templates/validation-report.md, templates/qa-audit-report.md |
259
+ | qa-e2e-runner | generated E2E test files, POM files, CLAUDE.md, live application | E2E_RUN_REPORT.md, fixed test/POM files with real locators | -- |
260
+ | qa-testid-injector | repo source files, SCAN_MANIFEST.md, CLAUDE.md | TESTID_AUDIT_REPORT.md, modified source files with data-testid attributes | templates/scan-manifest.md, templates/testid-audit-report.md |
261
+ | qa-bug-detective | test execution results, test source files, CLAUDE.md | FAILURE_CLASSIFICATION_REPORT.md | templates/failure-classification.md |
262
+
263
+ **Rule:** An agent MUST NOT produce artifacts assigned to another agent.
264
+
265
+ **Rule:** An agent MUST read all artifacts listed in its "Reads" column before producing output.
266
+
267
+ ---
268
+
269
+ ## Verification Commands
270
+
271
+ Every artifact must pass verification before the pipeline advances. Below are the validation rules per artifact type.
272
+
273
+ ### SCAN_MANIFEST.md
274
+ - Has > 0 files in File List table
275
+ - Project Detection section is populated (framework, language, component patterns)
276
+ - Testable Surfaces has at least 1 category with entries
277
+ - File priority ordering is present (HIGH/MEDIUM/LOW)
278
+
279
+ ### QA_ANALYSIS.md
280
+ - All 6 sections present: Architecture Overview, External Dependencies, Risk Assessment, Top 10 Unit Test Targets, API/Contract Test Targets, Recommended Testing Pyramid
281
+ - Top 10 has exactly 10 entries with module, rationale, and complexity
282
+ - Testing pyramid percentages sum to 100%
283
+ - Risk assessment uses only HIGH/MEDIUM/LOW with justification per item
284
+
285
+ ### TEST_INVENTORY.md
286
+ - Every test case has all mandatory fields: ID, target, inputs, expected outcome, priority
287
+ - IDs are unique across the entire document (no duplicates)
288
+ - IDs follow naming convention: UT-MODULE-NNN, INT-MODULE-NNN, API-RESOURCE-NNN, E2E-FLOW-NNN
289
+ - Pyramid tier counts match the summary table
290
+ - No expected outcome says "correct", "proper", "appropriate", or "works" without concrete value
291
+
292
+ ### QA_REPO_BLUEPRINT.md
293
+ - Folder structure tree is present with explanations per directory
294
+ - Config files section has actual content (not placeholders)
295
+ - npm scripts defined for smoke, regression, and API test runs
296
+ - CI/CD strategy section includes PR-gate and nightly run configurations
297
+ - Definition of Done checklist is present
298
+
299
+ ### VALIDATION_REPORT.md
300
+ - All 4 layers reported per file: Syntax, Structure, Dependencies, Logic
301
+ - Each layer shows PASS or FAIL with details
302
+ - Confidence level assigned: HIGH (all layers pass), MEDIUM (1-2 minor issues), LOW (structural problems)
303
+ - Fix loop log shows iteration count and what was found/fixed per loop
304
+ - Unresolved issues section documents anything not auto-fixed
305
+
306
+ ### FAILURE_CLASSIFICATION_REPORT.md
307
+ - Every failure has classification: APPLICATION BUG, TEST CODE ERROR, ENVIRONMENT ISSUE, or INCONCLUSIVE
308
+ - Every failure has confidence level: HIGH, MEDIUM-HIGH, MEDIUM, or LOW
309
+ - Every failure has evidence: code snippet + reasoning explaining the classification
310
+ - No APPLICATION BUG is marked as auto-fixed (application bugs require developer action)
311
+ - Auto-fix log documents what was fixed and at what confidence level
312
+
313
+ ### TESTID_AUDIT_REPORT.md
314
+ - Coverage score calculated: existing data-testid count / total interactive elements
315
+ - All proposed data-testid values follow `{context}-{description}-{element-type}` naming convention
316
+ - No duplicate data-testid values within the same page/route scope
317
+ - Elements classified by priority: P0 (form inputs, buttons), P1 (links, images), P2 (containers, decorative)
318
+ - Decision gate threshold applied: >90% SELECTIVE, 50-90% TARGETED, <50% FULL PASS, 0% P0 FIRST
319
+
320
+ ### GAP_ANALYSIS.md
321
+ - Coverage map shows all modules from SCAN_MANIFEST.md
322
+ - Missing tests have IDs following naming convention and priorities assigned
323
+ - Broken tests have failure reasons documented with file path and error
324
+ - Quality assessment includes locator tier distribution and assertion quality rating
325
+ - Recommendations are prioritized: fix broken first, then add P0, then P1
326
+
327
+ ### QA_AUDIT_REPORT.md
328
+ - All 6 dimensions scored: Locator Quality, Assertion Specificity, POM Compliance, Test Coverage, Naming Convention, Test Data Management
329
+ - Weights sum to 100%: Locator 20%, Assertion 20%, POM 15%, Coverage 20%, Naming 15%, Test Data 10%
330
+ - Overall score matches weighted calculation of dimension scores
331
+ - Critical issues listed with file path, line number, and description
332
+ - Each recommendation has effort estimate: S (small), M (medium), L (large)
333
+
334
+ ---
335
+
336
+ ## Git Workflow
337
+
338
+ All QA automation output follows these git conventions.
339
+
340
+ ### Branch Naming
341
+
342
+ ```
343
+ qa/auto-{project}-{date}
344
+ ```
345
+
346
+ Examples:
347
+ - `qa/auto-shopflow-2026-03-18`
348
+ - `qa/auto-acme-api-2026-04-01`
349
+
350
+ ### Commit Message Format
351
+
352
+ ```
353
+ qa({agent}): {description}
354
+ ```
355
+
356
+ Examples:
357
+ - `qa(scanner): produce SCAN_MANIFEST.md for shopflow`
358
+ - `qa(analyzer): produce QA_ANALYSIS.md and TEST_INVENTORY.md`
359
+ - `qa(executor): generate 24 test files with POMs and fixtures`
360
+ - `qa(validator): validate generated tests - PASS with HIGH confidence`
361
+ - `qa(testid-injector): inject 47 data-testid attributes across 12 components`
362
+ - `qa(bug-detective): classify 5 failures - 2 APP BUG, 2 TEST ERROR, 1 ENV ISSUE`
363
+
364
+ ### Commit Conventions
365
+
366
+ - One commit per agent stage (scanner produces one commit, analyzer produces one commit, etc.)
367
+ - Descriptive messages that include artifact names and counts
368
+ - Never commit .env files, credentials, or secrets
369
+ - Include modified file count in commit body when relevant
370
+
371
+ ### PR Template
372
+
373
+ PR description must include:
374
+ - Analysis summary (architecture type, framework, risk areas)
375
+ - Test counts by pyramid level (unit: N, integration: N, API: N, E2E: N)
376
+ - Coverage metrics (modules covered, estimated line coverage)
377
+ - Validation pass/fail status with confidence level
378
+ - Link to VALIDATION_REPORT.md in the PR files
379
+
380
+ ---
381
+
382
+ ## Team Settings
383
+
384
+ Configuration for multi-agent pipeline execution.
385
+
386
+ ### Concurrent Execution
387
+
388
+ Agents in the same pipeline stage can run in parallel when their inputs are independent. Examples:
389
+ - qa-testid-injector and qa-analyzer can run simultaneously after scan completes (both read SCAN_MANIFEST.md)
390
+ - Multiple qa-executor instances can generate tests for different modules in parallel
391
+
392
+ Agents in different pipeline stages MUST respect stage ordering. A downstream agent cannot start until all its required inputs exist on disk.
393
+
394
+ ### Worktree Isolation
395
+
396
+ Each agent operates on the same branch. No worktree splits are needed for this system. Agents coordinate through file-based artifacts -- each agent writes its own files and reads other agents' files.
397
+
398
+ ### Dependency Ordering
399
+
400
+ Respect stage transitions from the Agent Pipeline section:
401
+ 1. qa-scanner runs first (no dependencies)
402
+ 2. qa-project-researcher runs after codebase-map (depends on SCAN_MANIFEST.md, uses Context7 MCP — non-blocking, pipeline continues if it fails)
403
+ 3. qa-codebase-mapper runs after scanner (depends on SCAN_MANIFEST.md, spawns 4 parallel focus agents)
404
+ 4. qa-analyzer and qa-testid-injector run after codebase-map (both depend on SCAN_MANIFEST.md + codebase map documents + research documents if available)
405
+ 5. qa-planner runs after analyzer (depends on QA_ANALYSIS.md + TEST_INVENTORY.md + codebase map documents)
406
+ 6. qa-executor runs after planner (depends on generation plan + codebase map documents + research documents if available + Context7 MCP)
407
+ 7. qa-validator runs after executor (depends on generated test files)
408
+ 8. qa-e2e-runner runs after validator (depends on E2E test files + live application + Context7 MCP)
409
+ 9. qa-bug-detective runs after e2e-runner or validator if failures (depends on test results + Context7 MCP)
410
+
411
+ ### Auto-Advance Mode
412
+
413
+ When auto-advance is enabled, pipeline stages advance automatically when:
414
+ 1. The previous stage completes
415
+ 2. All output artifacts from the previous stage exist on disk
416
+ 3. All output artifacts pass their verification commands (from Verification Commands section)
417
+
418
+ No human confirmation is needed between stages in auto-advance mode. The pipeline pauses only at explicit checkpoint tasks or when verification fails.
419
+
420
+ ---
421
+
422
+ ## Agent Coordination
423
+
424
+ Rules governing how agents communicate and hand off work through artifacts.
425
+
426
+ ### Read-Before-Write Rules
427
+
428
+ Every agent MUST read its required inputs before producing any output. Failure to read inputs produces low-quality, inconsistent artifacts.
429
+
430
+ | Agent | MUST Read Before Producing Output |
431
+ |-------|-----------------------------------|
432
+ | qa-scanner | package.json (or equivalent), folder tree structure, all source file extensions to detect framework and language |
433
+ | qa-project-researcher | SCAN_MANIFEST.md, repo source files, MY_PREFERENCES.md. Uses Context7 MCP as primary source. |
434
+ | qa-analyzer | SCAN_MANIFEST.md (complete, verified), CLAUDE.md (all QA standards sections), research documents (if available) |
435
+ | qa-planner | TEST_INVENTORY.md (all test cases), QA_ANALYSIS.md (architecture and risk context) |
436
+ | qa-executor | TEST_INVENTORY.md (test cases to implement), CLAUDE.md (POM rules, locator hierarchy, assertion rules, naming conventions, quality gates), research documents (if available). Must query Context7 MCP before generating code. |
437
+ | qa-validator | CLAUDE.md (quality gates, locator tiers, assertion rules), all generated test files to validate |
438
+ | qa-testid-injector | SCAN_MANIFEST.md (component file list), CLAUDE.md (data-testid Convention section for naming rules) |
439
+ | qa-bug-detective | Test execution output (stdout/stderr, exit codes), test source files (to read the failing code), CLAUDE.md (classification rules), research documents (if available). Must query Context7 MCP before auto-fixing. |
440
+ | qa-e2e-runner | Generated E2E test files, POM files, CLAUDE.md, live application, research documents (if available). Must query Context7 MCP before fixing locators. |
441
+
442
+ ### Handoff Patterns
443
+
444
+ Agents communicate exclusively through file-based artifacts:
445
+
446
+ 1. **Producer writes** -- Agent completes its task and writes output artifact(s) to disk
447
+ 2. **Pipeline verifies** -- Output artifacts pass verification commands before advancing
448
+ 3. **Consumer reads** -- Next agent reads the artifact(s) as its first action
449
+ 4. **No direct communication** -- Agents never pass data in memory or through environment variables between stages
450
+
451
+ ### Quality Gates Per Artifact
452
+
453
+ Before the pipeline advances past any stage, the produced artifact(s) must pass verification:
454
+
455
+ | Stage Complete | Artifact | Gate |
456
+ |----------------|----------|------|
457
+ | scan | SCAN_MANIFEST.md | > 0 files listed, project detection populated |
458
+ | analyze | QA_ANALYSIS.md + TEST_INVENTORY.md | All sections present, IDs unique, pyramid sums to 100% |
459
+ | testid-inject | TESTID_AUDIT_REPORT.md | Coverage score calculated, naming convention compliant |
460
+ | plan | Generation plan | Test cases mapped to output files, no unassigned cases |
461
+ | generate | Test files + POMs | All planned files exist, imports resolve, syntax valid |
462
+ | validate | VALIDATION_REPORT.md | All 4 layers checked per file, confidence level assigned |
463
+ | deliver | Branch + PR | Branch pushed, PR created with required description sections |
464
+
465
+ ### Error Recovery
466
+
467
+ If an agent fails or produces an artifact that does not pass verification:
468
+ 1. Log the failure with the specific verification check that failed
469
+ 2. Retry the agent (max 3 attempts per stage)
470
+ 3. If still failing after 3 attempts, pause the pipeline and report the blocked stage
471
+ 4. Do not advance to the next stage with a failed artifact
472
+
473
+ ---
474
+
475
+ ## data-testid Convention
476
+
477
+ All `data-testid` attributes injected by qa-testid-injector and referenced by generated tests MUST follow this naming convention.
478
+
479
+ ### Naming Pattern
480
+
481
+ ```
482
+ {context}-{description}-{element-type}
483
+ ```
484
+
485
+ All values are **kebab-case**. No camelCase, no underscores, no periods.
486
+
487
+ ### Context Derivation
488
+
489
+ 1. **Page-level context**: Derived from the component filename or route
490
+ - `LoginPage.tsx` -> context is `login`
491
+ - `ProductDetailPage.tsx` -> context is `product-detail`
492
+ - Route `/settings/profile` -> context is `settings-profile`
493
+
494
+ 2. **Component-level context**: Derived from the component name
495
+ - `<NavBar>` -> context is `navbar`
496
+ - `<ShoppingCart>` -> context is `shopping-cart`
497
+ - `<UserAvatar>` -> context is `user-avatar`
498
+
499
+ 3. **Nested context**: Parent-child hierarchy, max 3 levels deep
500
+ - `checkout-shipping-address-input` (page -> section -> field)
501
+ - `dashboard-sidebar-nav-link` (page -> component -> element)
502
+ - Never exceed 3 levels: `a-b-c-element` is the maximum depth
503
+
504
+ 4. **Dynamic list items**: Use template literals with unique keys
505
+ ```tsx
506
+ data-testid={`product-${product.id}-card`}
507
+ data-testid={`order-${order.id}-status-badge`}
508
+ ```
509
+
510
+ ### Element Type Suffix Table
511
+
512
+ Every `data-testid` value ends with a suffix indicating the element type:
513
+
514
+ | Element | Suffix | Example |
515
+ |---------|--------|---------|
516
+ | `<button>` | `-btn` | `login-submit-btn` |
517
+ | `<input>` | `-input` | `login-email-input` |
518
+ | `<select>` | `-select` | `settings-language-select` |
519
+ | `<textarea>` | `-textarea` | `feedback-comment-textarea` |
520
+ | `<a>` (link) | `-link` | `navbar-profile-link` |
521
+ | `<form>` | `-form` | `checkout-payment-form` |
522
+ | `<img>` | `-img` | `product-hero-img` |
523
+ | `<table>` | `-table` | `users-list-table` |
524
+ | `<tr>` (row) | `-row` | `users-item-row` |
525
+ | `<dialog>/<modal>` | `-modal` | `confirm-delete-modal` |
526
+ | `<div>` container | `-container` | `dashboard-stats-container` |
527
+ | `<ul>/<ol>` list | `-list` | `notifications-list` |
528
+ | `<li>` item | `-item` | `notifications-item` |
529
+ | dropdown menu | `-dropdown` | `navbar-user-dropdown` |
530
+ | tab | `-tab` | `settings-security-tab` |
531
+ | checkbox | `-checkbox` | `terms-accept-checkbox` |
532
+ | radio | `-radio` | `shipping-express-radio` |
533
+ | toggle/switch | `-toggle` | `notifications-enabled-toggle` |
534
+ | badge/chip | `-badge` | `cart-count-badge` |
535
+ | alert/toast | `-alert` | `error-validation-alert` |
536
+
537
+ ### Third-Party Component Handling
538
+
539
+ When adding `data-testid` to third-party UI library components, use this priority order:
540
+
541
+ 1. **Props passthrough** (preferred): If the library supports passing `data-testid` directly as a prop
542
+ ```tsx
543
+ <MuiButton data-testid="checkout-pay-btn">Pay</MuiButton>
544
+ ```
545
+
546
+ 2. **Wrapper div**: If the library does not support prop passthrough, wrap with a `<div>` that has the `data-testid`
547
+ ```tsx
548
+ <div data-testid="checkout-pay-container">
549
+ <ThirdPartyButton>Pay</ThirdPartyButton>
550
+ </div>
551
+ ```
552
+
553
+ 3. **inputProps / slotProps** (MUI-specific): Use component-specific prop APIs
554
+ ```tsx
555
+ <TextField inputProps={{ 'data-testid': 'login-email-input' }} />
556
+ <Autocomplete slotProps={{ input: { 'data-testid': 'search-query-input' } }} />
557
+ ```