agentic-qe 2.0.0 → 2.1.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.claude/agents/qx-partner.md +17 -4
- package/.claude/skills/accessibility-testing/SKILL.md +144 -692
- package/.claude/skills/agentic-quality-engineering/SKILL.md +176 -529
- package/.claude/skills/api-testing-patterns/SKILL.md +180 -560
- package/.claude/skills/brutal-honesty-review/SKILL.md +113 -603
- package/.claude/skills/bug-reporting-excellence/SKILL.md +116 -517
- package/.claude/skills/chaos-engineering-resilience/SKILL.md +127 -72
- package/.claude/skills/cicd-pipeline-qe-orchestrator/SKILL.md +209 -404
- package/.claude/skills/code-review-quality/SKILL.md +158 -608
- package/.claude/skills/compatibility-testing/SKILL.md +148 -38
- package/.claude/skills/compliance-testing/SKILL.md +132 -63
- package/.claude/skills/consultancy-practices/SKILL.md +114 -446
- package/.claude/skills/context-driven-testing/SKILL.md +117 -381
- package/.claude/skills/contract-testing/SKILL.md +176 -141
- package/.claude/skills/database-testing/SKILL.md +137 -130
- package/.claude/skills/exploratory-testing-advanced/SKILL.md +160 -629
- package/.claude/skills/holistic-testing-pact/SKILL.md +140 -188
- package/.claude/skills/localization-testing/SKILL.md +145 -33
- package/.claude/skills/mobile-testing/SKILL.md +132 -448
- package/.claude/skills/mutation-testing/SKILL.md +147 -41
- package/.claude/skills/performance-testing/SKILL.md +200 -546
- package/.claude/skills/quality-metrics/SKILL.md +164 -519
- package/.claude/skills/refactoring-patterns/SKILL.md +132 -699
- package/.claude/skills/regression-testing/SKILL.md +120 -926
- package/.claude/skills/risk-based-testing/SKILL.md +157 -660
- package/.claude/skills/security-testing/SKILL.md +199 -538
- package/.claude/skills/sherlock-review/SKILL.md +163 -699
- package/.claude/skills/shift-left-testing/SKILL.md +161 -465
- package/.claude/skills/shift-right-testing/SKILL.md +161 -519
- package/.claude/skills/six-thinking-hats/SKILL.md +175 -1110
- package/.claude/skills/skills-manifest.json +71 -20
- package/.claude/skills/tdd-london-chicago/SKILL.md +131 -448
- package/.claude/skills/technical-writing/SKILL.md +103 -154
- package/.claude/skills/test-automation-strategy/SKILL.md +166 -772
- package/.claude/skills/test-data-management/SKILL.md +126 -910
- package/.claude/skills/test-design-techniques/SKILL.md +179 -89
- package/.claude/skills/test-environment-management/SKILL.md +136 -91
- package/.claude/skills/test-reporting-analytics/SKILL.md +169 -92
- package/.claude/skills/testability-scoring/SKILL.md +172 -538
- package/.claude/skills/testability-scoring/scripts/generate-html-report.js +0 -0
- package/.claude/skills/visual-testing-advanced/SKILL.md +155 -78
- package/.claude/skills/xp-practices/SKILL.md +151 -587
- package/CHANGELOG.md +86 -0
- package/README.md +23 -16
- package/dist/agents/QXPartnerAgent.d.ts +47 -1
- package/dist/agents/QXPartnerAgent.d.ts.map +1 -1
- package/dist/agents/QXPartnerAgent.js +2086 -125
- package/dist/agents/QXPartnerAgent.js.map +1 -1
- package/dist/agents/lifecycle/AgentLifecycleManager.d.ts.map +1 -1
- package/dist/agents/lifecycle/AgentLifecycleManager.js +34 -31
- package/dist/agents/lifecycle/AgentLifecycleManager.js.map +1 -1
- package/dist/cli/commands/init-claude-md-template.d.ts.map +1 -1
- package/dist/cli/commands/init-claude-md-template.js +14 -0
- package/dist/cli/commands/init-claude-md-template.js.map +1 -1
- package/dist/core/SwarmCoordinator.d.ts +180 -0
- package/dist/core/SwarmCoordinator.d.ts.map +1 -0
- package/dist/core/SwarmCoordinator.js +473 -0
- package/dist/core/SwarmCoordinator.js.map +1 -0
- package/dist/core/memory/ReflexionMemoryAdapter.d.ts +109 -0
- package/dist/core/memory/ReflexionMemoryAdapter.d.ts.map +1 -0
- package/dist/core/memory/ReflexionMemoryAdapter.js +306 -0
- package/dist/core/memory/ReflexionMemoryAdapter.js.map +1 -0
- package/dist/core/memory/RuVectorPatternStore.d.ts +28 -0
- package/dist/core/memory/RuVectorPatternStore.d.ts.map +1 -1
- package/dist/core/memory/RuVectorPatternStore.js +70 -0
- package/dist/core/memory/RuVectorPatternStore.js.map +1 -1
- package/dist/core/memory/SparseVectorSearch.d.ts +55 -0
- package/dist/core/memory/SparseVectorSearch.d.ts.map +1 -0
- package/dist/core/memory/SparseVectorSearch.js +130 -0
- package/dist/core/memory/SparseVectorSearch.js.map +1 -0
- package/dist/core/memory/TieredCompression.d.ts +81 -0
- package/dist/core/memory/TieredCompression.d.ts.map +1 -0
- package/dist/core/memory/TieredCompression.js +270 -0
- package/dist/core/memory/TieredCompression.js.map +1 -0
- package/dist/core/memory/index.d.ts +6 -0
- package/dist/core/memory/index.d.ts.map +1 -1
- package/dist/core/memory/index.js +29 -1
- package/dist/core/memory/index.js.map +1 -1
- package/dist/core/metrics/MetricsAggregator.d.ts +228 -0
- package/dist/core/metrics/MetricsAggregator.d.ts.map +1 -0
- package/dist/core/metrics/MetricsAggregator.js +482 -0
- package/dist/core/metrics/MetricsAggregator.js.map +1 -0
- package/dist/core/metrics/index.d.ts +5 -0
- package/dist/core/metrics/index.d.ts.map +1 -0
- package/dist/core/metrics/index.js +11 -0
- package/dist/core/metrics/index.js.map +1 -0
- package/dist/core/optimization/SwarmOptimizer.d.ts +5 -0
- package/dist/core/optimization/SwarmOptimizer.d.ts.map +1 -1
- package/dist/core/optimization/SwarmOptimizer.js +17 -0
- package/dist/core/optimization/SwarmOptimizer.js.map +1 -1
- package/dist/core/orchestration/AdaptiveScheduler.d.ts +190 -0
- package/dist/core/orchestration/AdaptiveScheduler.d.ts.map +1 -0
- package/dist/core/orchestration/AdaptiveScheduler.js +460 -0
- package/dist/core/orchestration/AdaptiveScheduler.js.map +1 -0
- package/dist/core/orchestration/WorkflowOrchestrator.d.ts +13 -0
- package/dist/core/orchestration/WorkflowOrchestrator.d.ts.map +1 -1
- package/dist/core/orchestration/WorkflowOrchestrator.js +32 -0
- package/dist/core/orchestration/WorkflowOrchestrator.js.map +1 -1
- package/dist/core/recovery/CircuitBreaker.d.ts +176 -0
- package/dist/core/recovery/CircuitBreaker.d.ts.map +1 -0
- package/dist/core/recovery/CircuitBreaker.js +382 -0
- package/dist/core/recovery/CircuitBreaker.js.map +1 -0
- package/dist/core/recovery/RecoveryOrchestrator.d.ts +186 -0
- package/dist/core/recovery/RecoveryOrchestrator.d.ts.map +1 -0
- package/dist/core/recovery/RecoveryOrchestrator.js +476 -0
- package/dist/core/recovery/RecoveryOrchestrator.js.map +1 -0
- package/dist/core/recovery/RetryStrategy.d.ts +127 -0
- package/dist/core/recovery/RetryStrategy.d.ts.map +1 -0
- package/dist/core/recovery/RetryStrategy.js +314 -0
- package/dist/core/recovery/RetryStrategy.js.map +1 -0
- package/dist/core/recovery/index.d.ts +8 -0
- package/dist/core/recovery/index.d.ts.map +1 -0
- package/dist/core/recovery/index.js +27 -0
- package/dist/core/recovery/index.js.map +1 -0
- package/dist/core/skills/DependencyResolver.d.ts +99 -0
- package/dist/core/skills/DependencyResolver.d.ts.map +1 -0
- package/dist/core/skills/DependencyResolver.js +260 -0
- package/dist/core/skills/DependencyResolver.js.map +1 -0
- package/dist/core/skills/ManifestGenerator.d.ts +114 -0
- package/dist/core/skills/ManifestGenerator.d.ts.map +1 -0
- package/dist/core/skills/ManifestGenerator.js +449 -0
- package/dist/core/skills/ManifestGenerator.js.map +1 -0
- package/dist/core/skills/index.d.ts +9 -0
- package/dist/core/skills/index.d.ts.map +1 -0
- package/dist/core/skills/index.js +24 -0
- package/dist/core/skills/index.js.map +1 -0
- package/dist/mcp/handlers/chaos/chaos-inject-failure.d.ts +5 -0
- package/dist/mcp/handlers/chaos/chaos-inject-failure.d.ts.map +1 -1
- package/dist/mcp/handlers/chaos/chaos-inject-failure.js +36 -2
- package/dist/mcp/handlers/chaos/chaos-inject-failure.js.map +1 -1
- package/dist/mcp/handlers/chaos/chaos-inject-latency.d.ts +5 -0
- package/dist/mcp/handlers/chaos/chaos-inject-latency.d.ts.map +1 -1
- package/dist/mcp/handlers/chaos/chaos-inject-latency.js +36 -2
- package/dist/mcp/handlers/chaos/chaos-inject-latency.js.map +1 -1
- package/dist/mcp/server.d.ts +9 -9
- package/dist/mcp/server.d.ts.map +1 -1
- package/dist/mcp/server.js +1 -2
- package/dist/mcp/server.js.map +1 -1
- package/dist/types/qx.d.ts +113 -7
- package/dist/types/qx.d.ts.map +1 -1
- package/dist/types/qx.js.map +1 -1
- package/dist/visualization/api/RestEndpoints.js +1 -1
- package/dist/visualization/api/RestEndpoints.js.map +1 -1
- package/package.json +15 -54
|
@@ -1,836 +1,230 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: test-automation-strategy
|
|
3
|
-
description:
|
|
3
|
+
description: "Design and implement effective test automation with proper pyramid, patterns, and CI/CD integration. Use when building automation frameworks or improving test efficiency."
|
|
4
|
+
category: testing-methodologies
|
|
5
|
+
priority: high
|
|
6
|
+
tokenEstimate: 1000
|
|
7
|
+
agents: [qe-test-generator, qe-test-executor, qe-coverage-analyzer, qe-flaky-test-hunter, qe-regression-risk-analyzer]
|
|
8
|
+
implementation_status: optimized
|
|
9
|
+
optimization_version: 1.0
|
|
10
|
+
last_optimized: 2025-12-02
|
|
11
|
+
dependencies: []
|
|
12
|
+
quick_reference_card: true
|
|
13
|
+
tags: [automation, test-pyramid, page-object, first-principles, ci-cd, flaky-tests]
|
|
4
14
|
---
|
|
5
15
|
|
|
6
16
|
# Test Automation Strategy
|
|
7
17
|
|
|
8
|
-
|
|
18
|
+
<default_to_action>
|
|
19
|
+
When designing or improving test automation:
|
|
20
|
+
1. FOLLOW test pyramid: 70% unit, 20% integration, 10% E2E
|
|
21
|
+
2. APPLY F.I.R.S.T. principles: Fast, Isolated, Repeatable, Self-validating, Timely
|
|
22
|
+
3. USE patterns: Page Object Model, Builder pattern, Factory pattern
|
|
23
|
+
4. INTEGRATE in CI/CD: Every commit runs tests, fail fast, clear feedback
|
|
24
|
+
5. MANAGE flaky tests: Quarantine, fix, or delete - never ignore
|
|
25
|
+
|
|
26
|
+
**Quick Anti-Pattern Detection:**
|
|
27
|
+
- Ice cream cone (many E2E, few unit) → Invert to pyramid
|
|
28
|
+
- Slow tests (> 10 min suite) → Parallelize, mock external deps
|
|
29
|
+
- Flaky tests → Fix timing, isolate data, or quarantine
|
|
30
|
+
- Test duplication → Share fixtures, use page objects
|
|
31
|
+
- Brittle selectors → Use data-testid, semantic locators
|
|
32
|
+
|
|
33
|
+
**Critical Success Factors:**
|
|
34
|
+
- Fast feedback is the goal (< 10 min full suite)
|
|
35
|
+
- Automation supports testing, doesn't replace judgment
|
|
36
|
+
- Invest in test infrastructure like production code
|
|
37
|
+
</default_to_action>
|
|
38
|
+
|
|
39
|
+
## Quick Reference Card
|
|
40
|
+
|
|
41
|
+
### When to Use
|
|
42
|
+
- Building new automation framework
|
|
43
|
+
- Improving existing test efficiency
|
|
44
|
+
- Reducing flaky test burden
|
|
45
|
+
- Optimizing CI/CD pipeline speed
|
|
46
|
+
|
|
47
|
+
### Test Pyramid
|
|
48
|
+
| Layer | % | Speed | Isolation | Examples |
|
|
49
|
+
|-------|---|-------|-----------|----------|
|
|
50
|
+
| **Unit** | 70% | < 1ms | Complete | Pure functions, logic |
|
|
51
|
+
| **Integration** | 20% | < 1s | Partial | API, database |
|
|
52
|
+
| **E2E** | 10% | < 30s | None | User journeys |
|
|
53
|
+
|
|
54
|
+
### F.I.R.S.T. Principles
|
|
55
|
+
| Principle | Meaning | How |
|
|
56
|
+
|-----------|---------|-----|
|
|
57
|
+
| **F**ast | Quick execution | Mock external deps |
|
|
58
|
+
| **I**solated | No shared state | Fresh fixtures per test |
|
|
59
|
+
| **R**epeatable | Same result every time | No random data |
|
|
60
|
+
| **S**elf-validating | Clear pass/fail | Assert, don't print |
|
|
61
|
+
| **T**imely | Written with code | TDD, not after |
|
|
62
|
+
|
|
63
|
+
### Anti-Patterns
|
|
64
|
+
| Problem | Symptom | Fix |
|
|
65
|
+
|---------|---------|-----|
|
|
66
|
+
| Ice cream cone | 80% E2E, 10% unit | Invert pyramid |
|
|
67
|
+
| Slow suite | 30+ min CI | Parallelize, prune |
|
|
68
|
+
| Flaky tests | Random failures | Quarantine, fix timing |
|
|
69
|
+
| Coupled tests | Order-dependent | Isolate data |
|
|
70
|
+
| Brittle selectors | Break on CSS change | Use data-testid |
|
|
9
71
|
|
|
10
|
-
|
|
11
|
-
|
|
12
|
-
Automation is not the goal. Automation is a tool to get faster feedback and free humans for skilled testing work.
|
|
13
|
-
|
|
14
|
-
## When to Automate
|
|
15
|
-
|
|
16
|
-
### Automation Makes Sense When
|
|
17
|
-
|
|
18
|
-
**Repetitive and Stable**
|
|
19
|
-
- Same test executed frequently
|
|
20
|
-
- Test steps won't change much
|
|
21
|
-
- Cost of automation < cost of manual execution over time
|
|
22
|
-
|
|
23
|
-
**Fast Feedback Needed**
|
|
24
|
-
- Tests in CI pipeline
|
|
25
|
-
- Regression safety net
|
|
26
|
-
- Sanity checks before deployment
|
|
27
|
-
|
|
28
|
-
**Difficult to Test Manually**
|
|
29
|
-
- Performance testing (1000 concurrent users)
|
|
30
|
-
- Load testing over hours/days
|
|
31
|
-
- Cross-browser testing (20 browsers)
|
|
32
|
-
- API testing (hundreds of endpoints)
|
|
33
|
-
|
|
34
|
-
**Precise Verification Required**
|
|
35
|
-
- Complex calculations
|
|
36
|
-
- Data integrity checks
|
|
37
|
-
- Exact timing requirements
|
|
38
|
-
|
|
39
|
-
### Automation Doesn't Make Sense When
|
|
40
|
-
|
|
41
|
-
**Rapidly Changing**
|
|
42
|
-
- Feature in active development
|
|
43
|
-
- Requirements unclear
|
|
44
|
-
- UI redesign in progress
|
|
45
|
-
- Cost of maintenance > value
|
|
46
|
-
|
|
47
|
-
**Complex Setup**
|
|
48
|
-
- Requires extensive test data
|
|
49
|
-
- Multiple systems/dependencies
|
|
50
|
-
- Environment fragile
|
|
51
|
-
- Setup time >> execution time
|
|
52
|
-
|
|
53
|
-
**Requires Human Judgment**
|
|
54
|
-
- Usability testing
|
|
55
|
-
- Visual design validation
|
|
56
|
-
- Exploratory testing
|
|
57
|
-
- Accessibility evaluation (partly automatable)
|
|
58
|
-
|
|
59
|
-
**One-Time or Rare**
|
|
60
|
-
- Ad-hoc investigation
|
|
61
|
-
- Production debugging
|
|
62
|
-
- One-time migration
|
|
63
|
-
- Annual report generation
|
|
64
|
-
|
|
65
|
-
**Low Risk, Low Value**
|
|
66
|
-
- Admin feature used monthly
|
|
67
|
-
- Edge case that never occurs
|
|
68
|
-
- Deprecated feature
|
|
69
|
-
|
|
70
|
-
## The Test Automation Pyramid
|
|
71
|
-
|
|
72
|
-
```
|
|
73
|
-
/\
|
|
74
|
-
/ \ E2E/UI Tests
|
|
75
|
-
/ \ (Few)
|
|
76
|
-
/------\
|
|
77
|
-
/ \ Integration Tests
|
|
78
|
-
/ \ (Some)
|
|
79
|
-
/------------\
|
|
80
|
-
/ \ Unit Tests
|
|
81
|
-
/________________\ (Many)
|
|
82
|
-
```
|
|
83
|
-
|
|
84
|
-
### Unit Tests (70% of automated tests)
|
|
85
|
-
|
|
86
|
-
**What:** Test individual functions/methods in isolation
|
|
87
|
-
|
|
88
|
-
**Benefits:**
|
|
89
|
-
- Fast (milliseconds)
|
|
90
|
-
- Reliable
|
|
91
|
-
- Pinpoint failures
|
|
92
|
-
- Enable fearless refactoring
|
|
93
|
-
|
|
94
|
-
**Tools:** Jest, JUnit, pytest, RSpec
|
|
95
|
-
|
|
96
|
-
**Example:**
|
|
97
|
-
```javascript
|
|
98
|
-
test('calculateTax returns correct amount', () => {
|
|
99
|
-
expect(calculateTax(100, 0.10)).toBe(10);
|
|
100
|
-
expect(calculateTax(0, 0.10)).toBe(0);
|
|
101
|
-
expect(calculateTax(100, 0)).toBe(0);
|
|
102
|
-
});
|
|
103
|
-
```
|
|
104
|
-
|
|
105
|
-
### Integration Tests (20% of automated tests)
|
|
106
|
-
|
|
107
|
-
**What:** Test multiple components working together
|
|
108
|
-
|
|
109
|
-
**Examples:**
|
|
110
|
-
- API endpoint + database
|
|
111
|
-
- Service layer + external API
|
|
112
|
-
- Module A + Module B interaction
|
|
113
|
-
|
|
114
|
-
**Benefits:**
|
|
115
|
-
- Test realistic interactions
|
|
116
|
-
- Catch integration issues
|
|
117
|
-
- Still relatively fast
|
|
118
|
-
|
|
119
|
-
**Tools:** Supertest, RestAssured, Testcontainers
|
|
120
|
-
|
|
121
|
-
**Example:**
|
|
122
|
-
```javascript
|
|
123
|
-
test('POST /orders creates order in database', async () => {
|
|
124
|
-
const response = await request(app)
|
|
125
|
-
.post('/orders')
|
|
126
|
-
.send({ userId: 123, items: [...] });
|
|
127
|
-
|
|
128
|
-
expect(response.status).toBe(201);
|
|
129
|
-
|
|
130
|
-
const order = await db.orders.findOne({ id: response.body.id });
|
|
131
|
-
expect(order.userId).toBe(123);
|
|
132
|
-
});
|
|
133
|
-
```
|
|
134
|
-
|
|
135
|
-
### E2E/UI Tests (10% of automated tests)
|
|
136
|
-
|
|
137
|
-
**What:** Test complete user workflows through the UI
|
|
138
|
-
|
|
139
|
-
**Benefits:**
|
|
140
|
-
- Test real user experience
|
|
141
|
-
- Catch UI/UX issues
|
|
142
|
-
- Verify complete flows
|
|
143
|
-
|
|
144
|
-
**Costs:**
|
|
145
|
-
- Slow (seconds to minutes)
|
|
146
|
-
- Flaky (timing, network, rendering)
|
|
147
|
-
- Expensive to maintain
|
|
148
|
-
- Hard to debug
|
|
149
|
-
|
|
150
|
-
**Tools:** Playwright, Cypress, Selenium
|
|
151
|
-
|
|
152
|
-
**Example:**
|
|
153
|
-
```javascript
|
|
154
|
-
test('user can complete checkout', async () => {
|
|
155
|
-
await page.goto('/products');
|
|
156
|
-
await page.click('[data-testid="add-to-cart"]');
|
|
157
|
-
await page.click('[data-testid="checkout"]');
|
|
158
|
-
await page.fill('[name="cardNumber"]', '4242424242424242');
|
|
159
|
-
await page.click('[data-testid="place-order"]');
|
|
160
|
-
|
|
161
|
-
await expect(page.locator('.success-message')).toBeVisible();
|
|
162
|
-
});
|
|
163
|
-
```
|
|
164
|
-
|
|
165
|
-
**Keep E2E tests to critical happy paths only.**
|
|
166
|
-
|
|
167
|
-
## Anti-Patterns to Avoid
|
|
168
|
-
|
|
169
|
-
### 🚫 Ice Cream Cone (Inverted Pyramid)
|
|
170
|
-
|
|
171
|
-
```
|
|
172
|
-
/________________\ E2E Tests (Many) ❌
|
|
173
|
-
\ / Integration Tests (Some)
|
|
174
|
-
\ / Unit Tests (Few)
|
|
175
|
-
\ /
|
|
176
|
-
\________/
|
|
177
|
-
```
|
|
178
|
-
|
|
179
|
-
**Problem:** Slow, flaky, expensive test suite
|
|
180
|
-
**Fix:** Write more unit tests, fewer E2E tests
|
|
181
|
-
|
|
182
|
-
### 🚫 Test All the Things
|
|
183
|
-
|
|
184
|
-
**Problem:** Automating everything because "automation is good"
|
|
185
|
-
|
|
186
|
-
**Reality:**
|
|
187
|
-
- Some tests cost more than they provide value
|
|
188
|
-
- Maintenance burden grows unsustainably
|
|
189
|
-
- Team drowns in test maintenance
|
|
190
|
-
|
|
191
|
-
**Fix:** Automate strategically based on value
|
|
192
|
-
|
|
193
|
-
### 🚫 Testing Through UI Only
|
|
194
|
-
|
|
195
|
-
**Problem:** All tests go through browser UI
|
|
196
|
-
|
|
197
|
-
**Issues:**
|
|
198
|
-
- Slow execution
|
|
199
|
-
- Flaky tests
|
|
200
|
-
- Hard to test edge cases
|
|
201
|
-
- Expensive maintenance
|
|
202
|
-
|
|
203
|
-
**Fix:** Test at appropriate level
|
|
204
|
-
- Business logic → unit tests
|
|
205
|
-
- API contracts → integration tests
|
|
206
|
-
- Critical flows → E2E tests
|
|
207
|
-
|
|
208
|
-
### 🚫 Record and Playback
|
|
209
|
-
|
|
210
|
-
**Problem:** Using record/playback tools without understanding
|
|
211
|
-
|
|
212
|
-
**Issues:**
|
|
213
|
-
- Fragile tests (break on minor UI changes)
|
|
214
|
-
- Unmaintainable
|
|
215
|
-
- No code review possible
|
|
216
|
-
- Vendor lock-in
|
|
217
|
-
|
|
218
|
-
**Fix:** Write code-based tests with good selectors
|
|
219
|
-
|
|
220
|
-
### 🚫 No Test Data Strategy
|
|
221
|
-
|
|
222
|
-
**Problem:** Tests depend on specific database state
|
|
223
|
-
|
|
224
|
-
**Issues:**
|
|
225
|
-
- Tests fail randomly
|
|
226
|
-
- Can't run tests in parallel
|
|
227
|
-
- Hard to reproduce failures
|
|
228
|
-
- Environment-dependent
|
|
229
|
-
|
|
230
|
-
**Fix:**
|
|
231
|
-
- Generate test data per test
|
|
232
|
-
- Use database transactions
|
|
233
|
-
- Isolate test data
|
|
234
|
-
- Clean up after tests
|
|
235
|
-
|
|
236
|
-
### 🚫 Sleep-Based Waits
|
|
237
|
-
|
|
238
|
-
```javascript
|
|
239
|
-
// BAD
|
|
240
|
-
await page.click('button');
|
|
241
|
-
await sleep(5000); // Hope 5 seconds is enough
|
|
242
|
-
await page.click('next-button');
|
|
243
|
-
```
|
|
244
|
-
|
|
245
|
-
**Problem:** Slow (always waits full time) and flaky (sometimes not enough)
|
|
246
|
-
|
|
247
|
-
**Fix:** Use explicit waits
|
|
248
|
-
```javascript
|
|
249
|
-
// GOOD
|
|
250
|
-
await page.click('button');
|
|
251
|
-
await page.waitForSelector('next-button', { state: 'visible' });
|
|
252
|
-
await page.click('next-button');
|
|
253
|
-
```
|
|
254
|
-
|
|
255
|
-
### 🚫 God Tests
|
|
256
|
-
|
|
257
|
-
**Problem:** One test that tests everything (1000+ lines)
|
|
258
|
-
|
|
259
|
-
**Issues:**
|
|
260
|
-
- Hard to understand
|
|
261
|
-
- Hard to debug when fails
|
|
262
|
-
- Slow
|
|
263
|
-
- Change in any feature breaks test
|
|
264
|
-
|
|
265
|
-
**Fix:** Small, focused tests - one concept per test
|
|
266
|
-
|
|
267
|
-
## Building Maintainable Tests
|
|
268
|
-
|
|
269
|
-
### Good Test Characteristics (F.I.R.S.T.)
|
|
270
|
-
|
|
271
|
-
**Fast**
|
|
272
|
-
- Unit tests < 100ms
|
|
273
|
-
- Integration tests < 1s
|
|
274
|
-
- E2E tests < 30s
|
|
275
|
-
- Total suite < 10 minutes
|
|
276
|
-
|
|
277
|
-
**Independent**
|
|
278
|
-
- Tests don't depend on each other
|
|
279
|
-
- Can run in any order
|
|
280
|
-
- Can run in parallel
|
|
281
|
-
|
|
282
|
-
**Repeatable**
|
|
283
|
-
- Same result every time
|
|
284
|
-
- No flaky tests
|
|
285
|
-
- Works in any environment
|
|
286
|
-
|
|
287
|
-
**Self-Validating**
|
|
288
|
-
- Clear pass/fail
|
|
289
|
-
- No manual verification
|
|
290
|
-
- Assertions meaningful
|
|
291
|
-
|
|
292
|
-
**Timely**
|
|
293
|
-
- Written with code (TDD)
|
|
294
|
-
- Run frequently
|
|
295
|
-
- Fast feedback
|
|
296
|
-
|
|
297
|
-
### Page Object Model (for UI tests)
|
|
298
|
-
|
|
299
|
-
**Bad: Direct selectors everywhere**
|
|
300
|
-
```javascript
|
|
301
|
-
test('login', async () => {
|
|
302
|
-
await page.fill('#username', 'user@test.com');
|
|
303
|
-
await page.fill('#password', 'pass123');
|
|
304
|
-
await page.click('button[type="submit"]');
|
|
305
|
-
});
|
|
72
|
+
---
|
|
306
73
|
|
|
307
|
-
|
|
308
|
-
await page.fill('#username', 'user@test.com'); // Duplicated
|
|
309
|
-
await page.fill('#password', 'pass123');
|
|
310
|
-
await page.click('button[type="submit"]');
|
|
311
|
-
await page.click('a[href="/profile"]');
|
|
312
|
-
});
|
|
313
|
-
```
|
|
74
|
+
## Page Object Model
|
|
314
75
|
|
|
315
|
-
**Good: Page Object Model**
|
|
316
76
|
```javascript
|
|
77
|
+
// pages/LoginPage.js
|
|
317
78
|
class LoginPage {
|
|
318
79
|
constructor(page) {
|
|
319
80
|
this.page = page;
|
|
320
|
-
this.
|
|
321
|
-
this.passwordInput = '
|
|
322
|
-
this.submitButton = '
|
|
81
|
+
this.emailInput = '[data-testid="email"]';
|
|
82
|
+
this.passwordInput = '[data-testid="password"]';
|
|
83
|
+
this.submitButton = '[data-testid="submit"]';
|
|
84
|
+
this.errorMessage = '[data-testid="error"]';
|
|
323
85
|
}
|
|
324
|
-
|
|
325
|
-
async login(
|
|
326
|
-
await this.page.fill(this.
|
|
86
|
+
|
|
87
|
+
async login(email, password) {
|
|
88
|
+
await this.page.fill(this.emailInput, email);
|
|
327
89
|
await this.page.fill(this.passwordInput, password);
|
|
328
90
|
await this.page.click(this.submitButton);
|
|
329
91
|
}
|
|
330
|
-
}
|
|
331
|
-
|
|
332
|
-
test('login', async () => {
|
|
333
|
-
const loginPage = new LoginPage(page);
|
|
334
|
-
await loginPage.login('user@test.com', 'pass123');
|
|
335
|
-
});
|
|
336
|
-
|
|
337
|
-
test('profile', async () => {
|
|
338
|
-
const loginPage = new LoginPage(page);
|
|
339
|
-
await loginPage.login('user@test.com', 'pass123'); // Reusable
|
|
340
|
-
await page.click('a[href="/profile"]');
|
|
341
|
-
});
|
|
342
|
-
```
|
|
343
92
|
|
|
344
|
-
|
|
345
|
-
|
|
346
|
-
- Reusable code
|
|
347
|
-
- More readable tests
|
|
348
|
-
- Easier maintenance
|
|
349
|
-
|
|
350
|
-
### Good Selectors
|
|
351
|
-
|
|
352
|
-
**Priority order:**
|
|
353
|
-
1. **data-testid** (best for testing)
|
|
354
|
-
```html
|
|
355
|
-
<button data-testid="checkout-btn">Checkout</button>
|
|
356
|
-
```
|
|
357
|
-
```javascript
|
|
358
|
-
await page.click('[data-testid="checkout-btn"]');
|
|
359
|
-
```
|
|
360
|
-
|
|
361
|
-
2. **Semantic HTML** (accessible)
|
|
362
|
-
```javascript
|
|
363
|
-
await page.click('button[aria-label="Submit form"]');
|
|
364
|
-
await page.getByRole('button', { name: 'Submit' });
|
|
365
|
-
```
|
|
366
|
-
|
|
367
|
-
3. **Stable attributes** (id, name)
|
|
368
|
-
```javascript
|
|
369
|
-
await page.click('#submit-button');
|
|
370
|
-
```
|
|
371
|
-
|
|
372
|
-
4. **Avoid:** CSS classes (styling changes break tests), XPath (fragile)
|
|
373
|
-
|
|
374
|
-
### Test Data Management
|
|
375
|
-
|
|
376
|
-
**Pattern 1: Test Data Builders**
|
|
377
|
-
```javascript
|
|
378
|
-
class OrderBuilder {
|
|
379
|
-
constructor() {
|
|
380
|
-
this.order = {
|
|
381
|
-
userId: 123,
|
|
382
|
-
items: [],
|
|
383
|
-
status: 'pending'
|
|
384
|
-
};
|
|
385
|
-
}
|
|
386
|
-
|
|
387
|
-
withItems(items) {
|
|
388
|
-
this.order.items = items;
|
|
389
|
-
return this;
|
|
390
|
-
}
|
|
391
|
-
|
|
392
|
-
withStatus(status) {
|
|
393
|
-
this.order.status = status;
|
|
394
|
-
return this;
|
|
395
|
-
}
|
|
396
|
-
|
|
397
|
-
build() {
|
|
398
|
-
return this.order;
|
|
93
|
+
async getError() {
|
|
94
|
+
return this.page.textContent(this.errorMessage);
|
|
399
95
|
}
|
|
400
96
|
}
|
|
401
97
|
|
|
402
|
-
//
|
|
403
|
-
|
|
404
|
-
|
|
405
|
-
.
|
|
406
|
-
.
|
|
407
|
-
```
|
|
408
|
-
|
|
409
|
-
**Pattern 2: Fixtures/Factories**
|
|
410
|
-
```javascript
|
|
411
|
-
// fixtures.js
|
|
412
|
-
export const validUser = {
|
|
413
|
-
email: 'test@example.com',
|
|
414
|
-
password: 'ValidPass123!',
|
|
415
|
-
role: 'customer'
|
|
416
|
-
};
|
|
417
|
-
|
|
418
|
-
export const adminUser = {
|
|
419
|
-
...validUser,
|
|
420
|
-
role: 'admin'
|
|
421
|
-
};
|
|
422
|
-
|
|
423
|
-
// test
|
|
424
|
-
import { validUser } from './fixtures';
|
|
425
|
-
await createUser(validUser);
|
|
426
|
-
```
|
|
427
|
-
|
|
428
|
-
**Pattern 3: Database Seeding**
|
|
429
|
-
```javascript
|
|
430
|
-
beforeEach(async () => {
|
|
431
|
-
await db.seed([
|
|
432
|
-
{ table: 'users', data: testUsers },
|
|
433
|
-
{ table: 'orders', data: testOrders }
|
|
434
|
-
]);
|
|
435
|
-
});
|
|
436
|
-
|
|
437
|
-
afterEach(async () => {
|
|
438
|
-
await db.clean(); // Reset to clean state
|
|
439
|
-
});
|
|
440
|
-
```
|
|
441
|
-
|
|
442
|
-
## CI/CD Integration
|
|
443
|
-
|
|
444
|
-
### Pipeline Structure
|
|
445
|
-
|
|
446
|
-
```
|
|
447
|
-
Commit → Fast Tests → Slower Tests → Deploy
|
|
448
|
-
(< 5 min) (< 15 min)
|
|
449
|
-
|
|
450
|
-
Fast Tests:
|
|
451
|
-
- Unit tests
|
|
452
|
-
- Linting
|
|
453
|
-
- Type checking
|
|
454
|
-
|
|
455
|
-
Slower Tests:
|
|
456
|
-
- Integration tests
|
|
457
|
-
- E2E critical paths
|
|
458
|
-
- Security scans
|
|
459
|
-
|
|
460
|
-
Deploy:
|
|
461
|
-
- Staging (auto)
|
|
462
|
-
- Production (manual or auto with feature flags)
|
|
463
|
-
```
|
|
464
|
-
|
|
465
|
-
### Parallel Execution
|
|
466
|
-
|
|
467
|
-
**Benefits:**
|
|
468
|
-
- Faster feedback
|
|
469
|
-
- Better resource utilization
|
|
470
|
-
|
|
471
|
-
**Strategies:**
|
|
472
|
-
```javascript
|
|
473
|
-
// Run test files in parallel
|
|
474
|
-
// jest.config.js
|
|
475
|
-
module.exports = {
|
|
476
|
-
maxWorkers: 4, // 4 parallel workers
|
|
477
|
-
};
|
|
478
|
-
|
|
479
|
-
// Playwright
|
|
480
|
-
npx playwright test --workers=4
|
|
481
|
-
|
|
482
|
-
// Split by test file
|
|
483
|
-
CI Matrix:
|
|
484
|
-
- Worker 1: Unit tests (auth)
|
|
485
|
-
- Worker 2: Unit tests (orders)
|
|
486
|
-
- Worker 3: Integration tests
|
|
487
|
-
- Worker 4: E2E tests
|
|
488
|
-
```
|
|
489
|
-
|
|
490
|
-
### Flaky Test Management
|
|
491
|
-
|
|
492
|
-
**When test fails:**
|
|
493
|
-
1. Does it fail consistently? → Real bug
|
|
494
|
-
2. Fails sometimes? → Flaky test
|
|
495
|
-
|
|
496
|
-
**Handling flaky tests:**
|
|
497
|
-
```javascript
|
|
498
|
-
// WRONG: Retry until passes
|
|
499
|
-
test.retry(5); // Hiding the problem
|
|
500
|
-
|
|
501
|
-
// RIGHT: Fix the flakiness
|
|
502
|
-
// - Add proper waits
|
|
503
|
-
// - Fix race conditions
|
|
504
|
-
// - Isolate test data
|
|
505
|
-
// - Improve selectors
|
|
506
|
-
```
|
|
507
|
-
|
|
508
|
-
**Short-term mitigation:**
|
|
509
|
-
```javascript
|
|
510
|
-
// Quarantine flaky tests
|
|
511
|
-
test.skip('flaky test', () => {
|
|
512
|
-
// TODO: Fix flakiness (TICKET-123)
|
|
513
|
-
});
|
|
514
|
-
```
|
|
515
|
-
|
|
516
|
-
## Measuring Automation Success
|
|
517
|
-
|
|
518
|
-
### Metrics That Matter
|
|
519
|
-
|
|
520
|
-
**Coverage (with context):**
|
|
521
|
-
- Not just % code coverage
|
|
522
|
-
- Risk coverage (high-risk areas tested?)
|
|
523
|
-
- Feature coverage (user flows tested?)
|
|
524
|
-
|
|
525
|
-
**Feedback Speed:**
|
|
526
|
-
- Time to run full suite
|
|
527
|
-
- Time to first failure
|
|
528
|
-
- Time from commit to deploy
|
|
529
|
-
|
|
530
|
-
**Reliability:**
|
|
531
|
-
- Flaky test rate (<2% acceptable)
|
|
532
|
-
- False positive rate
|
|
533
|
-
- Test maintenance time
|
|
534
|
-
|
|
535
|
-
**Value:**
|
|
536
|
-
- Bugs caught by automation
|
|
537
|
-
- Regressions prevented
|
|
538
|
-
- Time saved vs manual testing
|
|
539
|
-
|
|
540
|
-
### Metrics That Don't Matter Much
|
|
541
|
-
|
|
542
|
-
- ❌ Number of automated tests
|
|
543
|
-
- ❌ Lines of test code
|
|
544
|
-
- ❌ 100% code coverage (diminishing returns)
|
|
545
|
-
|
|
546
|
-
## Test Automation Strategy Document
|
|
547
|
-
|
|
548
|
-
```markdown
|
|
549
|
-
## Test Automation Strategy - [Project Name]
|
|
550
|
-
|
|
551
|
-
### Goals
|
|
552
|
-
- 80% of regression testing automated
|
|
553
|
-
- Test suite runs in < 10 minutes
|
|
554
|
-
- < 2% flaky test rate
|
|
555
|
-
- Catch critical bugs before production
|
|
556
|
-
|
|
557
|
-
### Test Pyramid Distribution
|
|
558
|
-
- Unit: 70% (target: 2000 tests, <5 min)
|
|
559
|
-
- Integration: 20% (target: 400 tests, <3 min)
|
|
560
|
-
- E2E: 10% (target: 50 tests, <2 min)
|
|
561
|
-
|
|
562
|
-
### What We Automate
|
|
563
|
-
✅ Critical user flows (checkout, payment, signup)
|
|
564
|
-
✅ Regression-prone areas (historical bug zones)
|
|
565
|
-
✅ API contracts and integrations
|
|
566
|
-
✅ Security checks (injection, XSS)
|
|
567
|
-
✅ Performance benchmarks
|
|
568
|
-
|
|
569
|
-
### What We Don't Automate
|
|
570
|
-
❌ Visual design (manual review)
|
|
571
|
-
❌ Usability testing (requires human judgment)
|
|
572
|
-
❌ Exploratory testing (human investigation)
|
|
573
|
-
❌ One-time migrations
|
|
574
|
-
❌ Frequently changing features (manual until stable)
|
|
575
|
-
|
|
576
|
-
### Tools
|
|
577
|
-
- Unit: Jest
|
|
578
|
-
- Integration: Supertest
|
|
579
|
-
- E2E: Playwright
|
|
580
|
-
- CI: GitHub Actions
|
|
581
|
-
- Test Data: Factory functions + database seeding
|
|
582
|
-
|
|
583
|
-
### Maintenance
|
|
584
|
-
- Review flaky tests weekly
|
|
585
|
-
- Update tests when features change
|
|
586
|
-
- Delete tests for removed features
|
|
587
|
-
- Refactor tests quarterly
|
|
588
|
-
|
|
589
|
-
### Success Criteria
|
|
590
|
-
- Deploy to production daily with confidence
|
|
591
|
-
- Catch 90% of bugs before production
|
|
592
|
-
- Test suite stays fast (<10 min)
|
|
593
|
-
```
|
|
594
|
-
|
|
595
|
-
## Growing Automation Gradually
|
|
596
|
-
|
|
597
|
-
### Phase 1: Foundation (Month 1-2)
|
|
598
|
-
- Set up CI pipeline
|
|
599
|
-
- Write unit tests for new code
|
|
600
|
-
- Automate smoke tests
|
|
601
|
-
- Target: 40% unit test coverage of critical code
|
|
602
|
-
|
|
603
|
-
### Phase 2: Core Flows (Month 3-4)
|
|
604
|
-
- Automate critical happy paths (E2E)
|
|
605
|
-
- Add integration tests for APIs
|
|
606
|
-
- Implement page object model
|
|
607
|
-
- Target: 5-10 stable E2E tests
|
|
608
|
-
|
|
609
|
-
### Phase 3: Expansion (Month 5-6)
|
|
610
|
-
- Increase unit test coverage to 60%
|
|
611
|
-
- Add edge case integration tests
|
|
612
|
-
- Automate top 10 bug-prone areas
|
|
613
|
-
- Target: Fast feedback (<10 min CI run)
|
|
614
|
-
|
|
615
|
-
### Phase 4: Optimization (Month 7+)
|
|
616
|
-
- Parallelize test execution
|
|
617
|
-
- Refactor duplicate test code
|
|
618
|
-
- Fix all flaky tests
|
|
619
|
-
- Target: Deploy multiple times daily
|
|
620
|
-
|
|
621
|
-
## Using with QE Agents
|
|
622
|
-
|
|
623
|
-
### Agent-Driven Test Pyramid
|
|
624
|
-
|
|
625
|
-
**qe-test-generator** builds the pyramid intelligently:
|
|
626
|
-
```typescript
|
|
627
|
-
// Generate unit tests (70% of pyramid)
|
|
628
|
-
await agent.generateTests({
|
|
629
|
-
level: 'unit',
|
|
630
|
-
target: 'src/services/PaymentService.ts',
|
|
631
|
-
coverage: 'comprehensive'
|
|
632
|
-
});
|
|
633
|
-
// → Fast, isolated tests for business logic
|
|
634
|
-
|
|
635
|
-
// Generate integration tests (20% of pyramid)
|
|
636
|
-
await agent.generateTests({
|
|
637
|
-
level: 'integration',
|
|
638
|
-
target: 'src/api/orders',
|
|
639
|
-
focus: 'database-interactions'
|
|
640
|
-
});
|
|
641
|
-
// → Tests for component integration
|
|
642
|
-
|
|
643
|
-
// Generate E2E tests (10% of pyramid)
|
|
644
|
-
await agent.generateTests({
|
|
645
|
-
level: 'e2e',
|
|
646
|
-
flows: ['checkout', 'payment'],
|
|
647
|
-
priority: 'critical-paths-only'
|
|
98
|
+
// Test uses page object
|
|
99
|
+
test('shows error for invalid credentials', async ({ page }) => {
|
|
100
|
+
const loginPage = new LoginPage(page);
|
|
101
|
+
await loginPage.login('bad@email.com', 'wrong');
|
|
102
|
+
expect(await loginPage.getError()).toBe('Invalid credentials');
|
|
648
103
|
});
|
|
649
|
-
// → Minimal UI tests for happy paths
|
|
650
104
|
```
|
|
651
105
|
|
|
652
|
-
|
|
653
|
-
|
|
654
|
-
**qe-regression-risk-analyzer** optimizes what to automate:
|
|
655
|
-
```typescript
|
|
656
|
-
// Analyze which tests provide most value
|
|
657
|
-
const analysis = await agent.analyzeAutomationValue({
|
|
658
|
-
candidates: allManualTests,
|
|
659
|
-
criteria: ['repetition', 'stability', 'risk', 'execution-time']
|
|
660
|
-
});
|
|
661
|
-
|
|
662
|
-
// Returns prioritized list
|
|
663
|
-
// High value: Frequent + stable + high-risk + fast
|
|
664
|
-
// Low value: Rare + changing + low-risk + slow
|
|
665
|
-
```
|
|
106
|
+
---
|
|
666
107
|
|
|
667
|
-
|
|
108
|
+
## CI/CD Integration
|
|
668
109
|
|
|
669
110
|
```yaml
|
|
670
|
-
|
|
671
|
-
name: QE Agent Pipeline
|
|
672
|
-
|
|
111
|
+
name: Test Pipeline
|
|
673
112
|
on: [push, pull_request]
|
|
674
113
|
|
|
675
114
|
jobs:
|
|
676
|
-
|
|
115
|
+
unit-tests:
|
|
677
116
|
runs-on: ubuntu-latest
|
|
678
117
|
steps:
|
|
679
|
-
|
|
680
|
-
-
|
|
681
|
-
|
|
682
|
-
|
|
683
|
-
|
|
684
|
-
|
|
685
|
-
|
|
118
|
+
- uses: actions/checkout@v4
|
|
119
|
+
- run: npm ci
|
|
120
|
+
- run: npm run test:unit -- --coverage
|
|
121
|
+
timeout-minutes: 5
|
|
122
|
+
- uses: codecov/codecov-action@v3
|
|
123
|
+
|
|
124
|
+
integration-tests:
|
|
125
|
+
needs: unit-tests
|
|
126
|
+
runs-on: ubuntu-latest
|
|
127
|
+
services:
|
|
128
|
+
postgres:
|
|
129
|
+
image: postgres:15
|
|
686
130
|
steps:
|
|
687
|
-
|
|
688
|
-
|
|
689
|
-
|
|
690
|
-
|
|
691
|
-
|
|
692
|
-
|
|
693
|
-
- name: Coverage Analysis
|
|
694
|
-
run: aqe agent run qe-coverage-analyzer --threshold 80
|
|
695
|
-
# → Identifies gaps in real-time
|
|
696
|
-
|
|
697
|
-
deployment-gate:
|
|
698
|
-
needs: comprehensive-check
|
|
131
|
+
- run: npm run test:integration
|
|
132
|
+
timeout-minutes: 10
|
|
133
|
+
|
|
134
|
+
e2e-tests:
|
|
135
|
+
needs: integration-tests
|
|
136
|
+
runs-on: ubuntu-latest
|
|
699
137
|
steps:
|
|
700
|
-
|
|
701
|
-
|
|
702
|
-
run: aqe agent run qe-test-executor --level e2e --critical-only
|
|
703
|
-
|
|
704
|
-
# Quality gate decision
|
|
705
|
-
- name: Quality Gate
|
|
706
|
-
run: aqe agent run qe-quality-gate
|
|
707
|
-
# → GO/NO-GO decision
|
|
138
|
+
- run: npx playwright test
|
|
139
|
+
timeout-minutes: 15
|
|
708
140
|
```
|
|
709
141
|
|
|
710
|
-
|
|
142
|
+
---
|
|
711
143
|
|
|
712
|
-
|
|
713
|
-
```typescript
|
|
714
|
-
// Detect flakiness patterns
|
|
715
|
-
await agent.huntFlakyTests({
|
|
716
|
-
suite: 'all',
|
|
717
|
-
runs: 100,
|
|
718
|
-
statisticalConfidence: 0.95
|
|
719
|
-
});
|
|
720
|
-
// → Identifies tests with <95% pass rate
|
|
144
|
+
## Flaky Test Management
|
|
721
145
|
|
|
722
|
-
|
|
723
|
-
|
|
724
|
-
|
|
725
|
-
fix
|
|
146
|
+
```typescript
|
|
147
|
+
// Quarantine flaky tests
|
|
148
|
+
describe.skip('Quarantined - INC-123', () => {
|
|
149
|
+
test('flaky test awaiting fix', () => { /* ... */ });
|
|
726
150
|
});
|
|
727
|
-
// → Converts sleep() to waitFor()
|
|
728
151
|
|
|
729
|
-
//
|
|
730
|
-
await
|
|
731
|
-
|
|
732
|
-
|
|
733
|
-
|
|
152
|
+
// Agent-assisted stabilization
|
|
153
|
+
await Task("Fix Flaky Tests", {
|
|
154
|
+
tests: quarantinedTests,
|
|
155
|
+
analysis: ['timing-issues', 'data-isolation', 'race-conditions'],
|
|
156
|
+
strategies: ['add-waits', 'isolate-fixtures', 'mock-externals']
|
|
157
|
+
}, "qe-flaky-test-hunter");
|
|
734
158
|
```
|
|
735
159
|
|
|
736
|
-
|
|
160
|
+
---
|
|
737
161
|
|
|
738
|
-
|
|
739
|
-
```typescript
|
|
740
|
-
// Generate isolated test data per test
|
|
741
|
-
const testData = await agent.generateTestData({
|
|
742
|
-
schema: 'users',
|
|
743
|
-
count: 100,
|
|
744
|
-
realistic: true,
|
|
745
|
-
isolation: 'per-test' // Each test gets unique data
|
|
746
|
-
});
|
|
162
|
+
## Agent-Assisted Automation
|
|
747
163
|
|
|
748
|
-
|
|
749
|
-
|
|
750
|
-
|
|
751
|
-
|
|
752
|
-
}
|
|
164
|
+
```typescript
|
|
165
|
+
// Generate tests following pyramid
|
|
166
|
+
await Task("Generate Test Suite", {
|
|
167
|
+
sourceCode: 'src/',
|
|
168
|
+
pyramid: { unit: 70, integration: 20, e2e: 10 },
|
|
169
|
+
patterns: ['page-object', 'builder', 'factory'],
|
|
170
|
+
framework: 'jest'
|
|
171
|
+
}, "qe-test-generator");
|
|
172
|
+
|
|
173
|
+
// Optimize test execution
|
|
174
|
+
await Task("Optimize Suite", {
|
|
175
|
+
algorithm: 'johnson-lindenstrauss',
|
|
176
|
+
targetReduction: 0.3,
|
|
177
|
+
maintainCoverage: 0.95
|
|
178
|
+
}, "qe-regression-risk-analyzer");
|
|
179
|
+
|
|
180
|
+
// Analyze flaky patterns
|
|
181
|
+
await Task("Flaky Analysis", {
|
|
182
|
+
testHistory: 'last-30-days',
|
|
183
|
+
detectPatterns: ['timing', 'data', 'environment'],
|
|
184
|
+
recommend: 'stabilization-strategy'
|
|
185
|
+
}, "qe-flaky-test-hunter");
|
|
753
186
|
```
|
|
754
187
|
|
|
755
|
-
|
|
188
|
+
---
|
|
189
|
+
|
|
190
|
+
## Agent Coordination Hints
|
|
756
191
|
|
|
192
|
+
### Memory Namespace
|
|
193
|
+
```
|
|
194
|
+
aqe/automation/
|
|
195
|
+
├── test-pyramid/* - Coverage by layer
|
|
196
|
+
├── page-objects/* - Shared page objects
|
|
197
|
+
├── flaky-registry/* - Quarantined tests
|
|
198
|
+
└── execution-metrics/* - Suite performance data
|
|
199
|
+
```
|
|
200
|
+
|
|
201
|
+
### Fleet Coordination
|
|
757
202
|
```typescript
|
|
758
|
-
// Coordinate multiple agents for automation strategy
|
|
759
203
|
const automationFleet = await FleetManager.coordinate({
|
|
760
204
|
strategy: 'test-automation',
|
|
761
205
|
agents: [
|
|
762
|
-
'qe-test-generator',
|
|
763
|
-
'qe-test-executor',
|
|
764
|
-
'qe-coverage-analyzer',
|
|
765
|
-
'qe-flaky-test-hunter',
|
|
766
|
-
'qe-regression-risk-analyzer'
|
|
206
|
+
'qe-test-generator', // Generate pyramid-compliant tests
|
|
207
|
+
'qe-test-executor', // Parallel execution
|
|
208
|
+
'qe-coverage-analyzer', // Coverage gaps
|
|
209
|
+
'qe-flaky-test-hunter', // Flaky detection
|
|
210
|
+
'qe-regression-risk-analyzer' // Smart selection
|
|
767
211
|
],
|
|
768
|
-
topology: '
|
|
769
|
-
});
|
|
770
|
-
|
|
771
|
-
// Execute full automation workflow
|
|
772
|
-
await automationFleet.execute({
|
|
773
|
-
scope: 'payment-module',
|
|
774
|
-
pyramidBalance: { unit: 0.7, integration: 0.2, e2e: 0.1 }
|
|
212
|
+
topology: 'hierarchical'
|
|
775
213
|
});
|
|
776
214
|
```
|
|
777
215
|
|
|
778
|
-
### Strategic Automation Decisions with Agent Insights
|
|
779
|
-
|
|
780
|
-
```typescript
|
|
781
|
-
// Agent helps decide what to automate
|
|
782
|
-
const recommendation = await qe-quality-analyzer.recommendAutomation({
|
|
783
|
-
feature: 'checkout-flow',
|
|
784
|
-
currentCoverage: 'manual-only',
|
|
785
|
-
executionFrequency: 'daily',
|
|
786
|
-
stability: 'stable',
|
|
787
|
-
risk: 'high'
|
|
788
|
-
});
|
|
789
|
-
|
|
790
|
-
// Returns:
|
|
791
|
-
// {
|
|
792
|
-
// shouldAutomate: true,
|
|
793
|
-
// level: 'integration', // Skip E2E, test via API
|
|
794
|
-
// priority: 'high',
|
|
795
|
-
// estimatedROI: '15 hours saved/month',
|
|
796
|
-
// maintainanceCost: 'low'
|
|
797
|
-
// }
|
|
798
|
-
```
|
|
799
|
-
|
|
800
216
|
---
|
|
801
217
|
|
|
802
218
|
## Related Skills
|
|
803
|
-
|
|
804
|
-
|
|
805
|
-
- [
|
|
806
|
-
- [
|
|
807
|
-
|
|
808
|
-
**Development Practices:**
|
|
809
|
-
- [tdd-london-chicago](../tdd-london-chicago/) - TDD drives automation at unit level
|
|
810
|
-
- [xp-practices](../xp-practices/) - CI/CD integration with automation
|
|
811
|
-
|
|
812
|
-
**Testing Specializations:**
|
|
813
|
-
- [api-testing-patterns](../api-testing-patterns/) - API automation strategies
|
|
814
|
-
- [performance-testing](../performance-testing/) - Performance automation
|
|
815
|
-
- [security-testing](../security-testing/) - Security scan automation
|
|
816
|
-
- [exploratory-testing-advanced](../exploratory-testing-advanced/) - Balance with manual exploration
|
|
817
|
-
|
|
818
|
-
**Communication:**
|
|
819
|
-
- [quality-metrics](../quality-metrics/) - Measure automation effectiveness
|
|
219
|
+
- [tdd-london-chicago](../tdd-london-chicago/) - TDD for unit tests
|
|
220
|
+
- [api-testing-patterns](../api-testing-patterns/) - Integration patterns
|
|
221
|
+
- [cicd-pipeline-qe-orchestrator](../cicd-pipeline-qe-orchestrator/) - Pipeline integration
|
|
222
|
+
- [shift-left-testing](../shift-left-testing/) - Early automation
|
|
820
223
|
|
|
821
224
|
---
|
|
822
225
|
|
|
823
226
|
## Remember
|
|
824
227
|
|
|
825
|
-
**
|
|
826
|
-
|
|
827
|
-
The goal is confident, frequent deployments of high-quality software. Automation enables that by:
|
|
828
|
-
- Giving fast feedback
|
|
829
|
-
- Catching regressions
|
|
830
|
-
- Freeing humans for skilled testing
|
|
831
|
-
|
|
832
|
-
Don't automate for automation's sake. Automate strategically.
|
|
833
|
-
|
|
834
|
-
**Good automation amplifies good testing. Bad automation wastes everyone's time.**
|
|
228
|
+
**Pyramid: 70% unit, 20% integration, 10% E2E.** F.I.R.S.T. principles for every test. Page Object Model for E2E. Parallelize for speed. Quarantine flaky tests - never ignore them. Treat test code like production code.
|
|
835
229
|
|
|
836
|
-
**With Agents
|
|
230
|
+
**With Agents:** Agents generate pyramid-compliant tests, detect flaky patterns, optimize execution time, and maintain test infrastructure. Use agents to scale automation quality.
|