agentic-qe 2.0.0 → 2.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.claude/agents/qx-partner.md +17 -4
- package/.claude/skills/accessibility-testing/SKILL.md +144 -692
- package/.claude/skills/agentic-quality-engineering/SKILL.md +176 -529
- package/.claude/skills/api-testing-patterns/SKILL.md +180 -560
- package/.claude/skills/brutal-honesty-review/SKILL.md +113 -603
- package/.claude/skills/bug-reporting-excellence/SKILL.md +116 -517
- package/.claude/skills/chaos-engineering-resilience/SKILL.md +127 -72
- package/.claude/skills/cicd-pipeline-qe-orchestrator/SKILL.md +209 -404
- package/.claude/skills/code-review-quality/SKILL.md +158 -608
- package/.claude/skills/compatibility-testing/SKILL.md +148 -38
- package/.claude/skills/compliance-testing/SKILL.md +132 -63
- package/.claude/skills/consultancy-practices/SKILL.md +114 -446
- package/.claude/skills/context-driven-testing/SKILL.md +117 -381
- package/.claude/skills/contract-testing/SKILL.md +176 -141
- package/.claude/skills/database-testing/SKILL.md +137 -130
- package/.claude/skills/exploratory-testing-advanced/SKILL.md +160 -629
- package/.claude/skills/holistic-testing-pact/SKILL.md +140 -188
- package/.claude/skills/localization-testing/SKILL.md +145 -33
- package/.claude/skills/mobile-testing/SKILL.md +132 -448
- package/.claude/skills/mutation-testing/SKILL.md +147 -41
- package/.claude/skills/performance-testing/SKILL.md +200 -546
- package/.claude/skills/quality-metrics/SKILL.md +164 -519
- package/.claude/skills/refactoring-patterns/SKILL.md +132 -699
- package/.claude/skills/regression-testing/SKILL.md +120 -926
- package/.claude/skills/risk-based-testing/SKILL.md +157 -660
- package/.claude/skills/security-testing/SKILL.md +199 -538
- package/.claude/skills/sherlock-review/SKILL.md +163 -699
- package/.claude/skills/shift-left-testing/SKILL.md +161 -465
- package/.claude/skills/shift-right-testing/SKILL.md +161 -519
- package/.claude/skills/six-thinking-hats/SKILL.md +175 -1110
- package/.claude/skills/skills-manifest.json +71 -20
- package/.claude/skills/tdd-london-chicago/SKILL.md +131 -448
- package/.claude/skills/technical-writing/SKILL.md +103 -154
- package/.claude/skills/test-automation-strategy/SKILL.md +166 -772
- package/.claude/skills/test-data-management/SKILL.md +126 -910
- package/.claude/skills/test-design-techniques/SKILL.md +179 -89
- package/.claude/skills/test-environment-management/SKILL.md +136 -91
- package/.claude/skills/test-reporting-analytics/SKILL.md +169 -92
- package/.claude/skills/testability-scoring/SKILL.md +172 -538
- package/.claude/skills/testability-scoring/scripts/generate-html-report.js +0 -0
- package/.claude/skills/visual-testing-advanced/SKILL.md +155 -78
- package/.claude/skills/xp-practices/SKILL.md +151 -587
- package/CHANGELOG.md +48 -0
- package/README.md +23 -16
- package/dist/agents/QXPartnerAgent.d.ts +8 -1
- package/dist/agents/QXPartnerAgent.d.ts.map +1 -1
- package/dist/agents/QXPartnerAgent.js +1174 -112
- package/dist/agents/QXPartnerAgent.js.map +1 -1
- package/dist/agents/lifecycle/AgentLifecycleManager.d.ts.map +1 -1
- package/dist/agents/lifecycle/AgentLifecycleManager.js +34 -31
- package/dist/agents/lifecycle/AgentLifecycleManager.js.map +1 -1
- package/dist/cli/commands/init-claude-md-template.d.ts.map +1 -1
- package/dist/cli/commands/init-claude-md-template.js +14 -0
- package/dist/cli/commands/init-claude-md-template.js.map +1 -1
- package/dist/core/SwarmCoordinator.d.ts +180 -0
- package/dist/core/SwarmCoordinator.d.ts.map +1 -0
- package/dist/core/SwarmCoordinator.js +473 -0
- package/dist/core/SwarmCoordinator.js.map +1 -0
- package/dist/core/metrics/MetricsAggregator.d.ts +228 -0
- package/dist/core/metrics/MetricsAggregator.d.ts.map +1 -0
- package/dist/core/metrics/MetricsAggregator.js +482 -0
- package/dist/core/metrics/MetricsAggregator.js.map +1 -0
- package/dist/core/metrics/index.d.ts +5 -0
- package/dist/core/metrics/index.d.ts.map +1 -0
- package/dist/core/metrics/index.js +11 -0
- package/dist/core/metrics/index.js.map +1 -0
- package/dist/core/optimization/SwarmOptimizer.d.ts +5 -0
- package/dist/core/optimization/SwarmOptimizer.d.ts.map +1 -1
- package/dist/core/optimization/SwarmOptimizer.js +17 -0
- package/dist/core/optimization/SwarmOptimizer.js.map +1 -1
- package/dist/core/orchestration/AdaptiveScheduler.d.ts +190 -0
- package/dist/core/orchestration/AdaptiveScheduler.d.ts.map +1 -0
- package/dist/core/orchestration/AdaptiveScheduler.js +460 -0
- package/dist/core/orchestration/AdaptiveScheduler.js.map +1 -0
- package/dist/core/orchestration/WorkflowOrchestrator.d.ts +13 -0
- package/dist/core/orchestration/WorkflowOrchestrator.d.ts.map +1 -1
- package/dist/core/orchestration/WorkflowOrchestrator.js +32 -0
- package/dist/core/orchestration/WorkflowOrchestrator.js.map +1 -1
- package/dist/core/recovery/CircuitBreaker.d.ts +176 -0
- package/dist/core/recovery/CircuitBreaker.d.ts.map +1 -0
- package/dist/core/recovery/CircuitBreaker.js +382 -0
- package/dist/core/recovery/CircuitBreaker.js.map +1 -0
- package/dist/core/recovery/RecoveryOrchestrator.d.ts +186 -0
- package/dist/core/recovery/RecoveryOrchestrator.d.ts.map +1 -0
- package/dist/core/recovery/RecoveryOrchestrator.js +476 -0
- package/dist/core/recovery/RecoveryOrchestrator.js.map +1 -0
- package/dist/core/recovery/RetryStrategy.d.ts +127 -0
- package/dist/core/recovery/RetryStrategy.d.ts.map +1 -0
- package/dist/core/recovery/RetryStrategy.js +314 -0
- package/dist/core/recovery/RetryStrategy.js.map +1 -0
- package/dist/core/recovery/index.d.ts +8 -0
- package/dist/core/recovery/index.d.ts.map +1 -0
- package/dist/core/recovery/index.js +27 -0
- package/dist/core/recovery/index.js.map +1 -0
- package/dist/core/skills/DependencyResolver.d.ts +99 -0
- package/dist/core/skills/DependencyResolver.d.ts.map +1 -0
- package/dist/core/skills/DependencyResolver.js +260 -0
- package/dist/core/skills/DependencyResolver.js.map +1 -0
- package/dist/core/skills/ManifestGenerator.d.ts +114 -0
- package/dist/core/skills/ManifestGenerator.d.ts.map +1 -0
- package/dist/core/skills/ManifestGenerator.js +449 -0
- package/dist/core/skills/ManifestGenerator.js.map +1 -0
- package/dist/core/skills/index.d.ts +9 -0
- package/dist/core/skills/index.d.ts.map +1 -0
- package/dist/core/skills/index.js +24 -0
- package/dist/core/skills/index.js.map +1 -0
- package/dist/mcp/server.d.ts +9 -9
- package/dist/mcp/server.d.ts.map +1 -1
- package/dist/mcp/server.js +1 -2
- package/dist/mcp/server.js.map +1 -1
- package/dist/types/qx.d.ts +39 -7
- package/dist/types/qx.d.ts.map +1 -1
- package/dist/types/qx.js.map +1 -1
- package/dist/visualization/api/RestEndpoints.js +1 -1
- package/dist/visualization/api/RestEndpoints.js.map +1 -1
- package/package.json +13 -55
|
@@ -1,325 +1,91 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: regression-testing
|
|
3
|
-
description: Strategic regression testing with test selection, impact analysis, and continuous regression management. Use when verifying fixes don't break existing functionality, planning regression suites, or optimizing test execution for faster feedback.
|
|
3
|
+
description: "Strategic regression testing with test selection, impact analysis, and continuous regression management. Use when verifying fixes don't break existing functionality, planning regression suites, or optimizing test execution for faster feedback."
|
|
4
|
+
category: specialized-testing
|
|
5
|
+
priority: high
|
|
6
|
+
tokenEstimate: 1000
|
|
7
|
+
agents: [qe-regression-risk-analyzer, qe-test-executor, qe-coverage-analyzer]
|
|
8
|
+
implementation_status: optimized
|
|
9
|
+
optimization_version: 1.0
|
|
10
|
+
last_optimized: 2025-12-02
|
|
11
|
+
dependencies: []
|
|
12
|
+
quick_reference_card: true
|
|
13
|
+
tags: [regression, test-selection, impact-analysis, ci-cd, change-based, risk-based]
|
|
4
14
|
---
|
|
5
15
|
|
|
6
16
|
# Regression Testing
|
|
7
17
|
|
|
8
|
-
|
|
9
|
-
|
|
10
|
-
|
|
11
|
-
|
|
12
|
-
|
|
13
|
-
|
|
14
|
-
|
|
15
|
-
|
|
16
|
-
**Regression:**
|
|
17
|
-
|
|
18
|
-
|
|
19
|
-
-
|
|
20
|
-
|
|
21
|
-
|
|
22
|
-
-
|
|
23
|
-
-
|
|
24
|
-
|
|
25
|
-
|
|
26
|
-
|
|
27
|
-
##
|
|
28
|
-
|
|
29
|
-
###
|
|
30
|
-
|
|
31
|
-
**When:** No changes to code, just re-running existing tests
|
|
32
|
-
|
|
33
|
-
**Use case:**
|
|
34
|
-
- Environment change (new database version)
|
|
35
|
-
- Configuration change
|
|
36
|
-
- Infrastructure update
|
|
37
|
-
- Verifying test stability
|
|
38
|
-
|
|
39
|
-
**Strategy:** Re-run full existing test suite
|
|
40
|
-
|
|
41
|
-
**Example:**
|
|
42
|
-
```bash
|
|
43
|
-
# After upgrading Node.js from 18 to 20
|
|
44
|
-
npm test # Run all tests to ensure nothing broke
|
|
45
|
-
```
|
|
46
|
-
|
|
47
|
-
### 2. Progressive Regression (New Features)
|
|
48
|
-
|
|
49
|
-
**When:** New features added, existing tests still valid
|
|
50
|
-
|
|
51
|
-
**Strategy:**
|
|
52
|
-
- Run existing regression suite
|
|
53
|
-
- Add new tests for new features
|
|
54
|
-
- Focus on integration points with existing code
|
|
55
|
-
|
|
56
|
-
**Example:**
|
|
57
|
-
```
|
|
58
|
-
New Feature: Social login
|
|
59
|
-
Regression Focus:
|
|
60
|
-
✓ Existing email login still works
|
|
61
|
-
✓ User profile creation unchanged
|
|
62
|
-
✓ Session management compatible
|
|
63
|
-
✓ New social login tests added
|
|
64
|
-
```
|
|
65
|
-
|
|
66
|
-
### 3. Selective Regression (Targeted Testing)
|
|
67
|
-
|
|
68
|
-
**When:** Specific modules changed, test only impacted areas
|
|
69
|
-
|
|
70
|
-
**Strategy:**
|
|
71
|
-
- Analyze code changes
|
|
72
|
-
- Identify dependent modules
|
|
73
|
-
- Run tests for changed + dependent code
|
|
74
|
-
- Skip unrelated tests
|
|
75
|
-
|
|
76
|
-
**Benefits:**
|
|
77
|
-
- Faster feedback (minutes vs hours)
|
|
78
|
-
- Resource efficient
|
|
79
|
-
- Scales to large codebases
|
|
80
|
-
|
|
81
|
-
**Example:**
|
|
82
|
-
```typescript
|
|
83
|
-
// Payment module changed
|
|
84
|
-
// Run tests for:
|
|
85
|
-
- payment.test.ts ✓
|
|
86
|
-
- checkout.test.ts ✓ (depends on payment)
|
|
87
|
-
- order-confirmation.test.ts ✓ (depends on payment)
|
|
88
|
-
// Skip unrelated:
|
|
89
|
-
- user-profile.test.ts ✗ (no dependency)
|
|
90
|
-
- search.test.ts ✗ (no dependency)
|
|
91
|
-
```
|
|
92
|
-
|
|
93
|
-
### 4. Complete Regression (Full Suite)
|
|
94
|
-
|
|
95
|
-
**When:**
|
|
96
|
-
- Major refactoring
|
|
18
|
+
<default_to_action>
|
|
19
|
+
When verifying changes don't break existing functionality:
|
|
20
|
+
1. ANALYZE what changed (git diff, impact analysis)
|
|
21
|
+
2. SELECT tests based on change + risk (not everything)
|
|
22
|
+
3. RUN in priority order (smoke → selective → full)
|
|
23
|
+
4. OPTIMIZE execution (parallel, sharding)
|
|
24
|
+
5. MONITOR suite health (flakiness, execution time)
|
|
25
|
+
|
|
26
|
+
**Quick Regression Strategy:**
|
|
27
|
+
- Per-commit: Smoke + changed code tests (5-10 min)
|
|
28
|
+
- Nightly: Extended regression (30-60 min)
|
|
29
|
+
- Pre-release: Full regression (2-4 hours)
|
|
30
|
+
|
|
31
|
+
**Critical Success Factors:**
|
|
32
|
+
- Smart selection catches 90% of regressions in 10% of time
|
|
33
|
+
- Flaky tests waste more time than they save
|
|
34
|
+
- Every production bug becomes a regression test
|
|
35
|
+
</default_to_action>
|
|
36
|
+
|
|
37
|
+
## Quick Reference Card
|
|
38
|
+
|
|
39
|
+
### When to Use
|
|
40
|
+
- After any code change
|
|
97
41
|
- Before release
|
|
98
|
-
- After
|
|
99
|
-
-
|
|
100
|
-
|
|
101
|
-
|
|
102
|
-
|
|
103
|
-
|
|
104
|
-
|
|
105
|
-
|
|
106
|
-
|
|
107
|
-
|
|
108
|
-
|
|
109
|
-
|
|
110
|
-
|
|
111
|
-
|
|
112
|
-
**
|
|
113
|
-
-
|
|
114
|
-
|
|
115
|
-
-
|
|
116
|
-
- Skip low-priority tests
|
|
117
|
-
|
|
118
|
-
**Example:**
|
|
119
|
-
```
|
|
120
|
-
High Priority (always run):
|
|
121
|
-
✓ Login/authentication
|
|
122
|
-
✓ Payment processing
|
|
123
|
-
✓ Data integrity checks
|
|
124
|
-
|
|
125
|
-
Medium Priority (run if time):
|
|
126
|
-
✓ User profile management
|
|
127
|
-
✓ Search functionality
|
|
128
|
-
|
|
129
|
-
Low Priority (skip in quick regression):
|
|
130
|
-
✗ Admin panel features
|
|
131
|
-
✗ Reporting dashboards
|
|
132
|
-
```
|
|
133
|
-
|
|
134
|
-
## Test Selection Strategies
|
|
42
|
+
- After dependency updates
|
|
43
|
+
- After environment changes
|
|
44
|
+
|
|
45
|
+
### Regression Types
|
|
46
|
+
| Type | When | Scope |
|
|
47
|
+
|------|------|-------|
|
|
48
|
+
| **Corrective** | No code change | Full suite |
|
|
49
|
+
| **Progressive** | New features | Existing + new |
|
|
50
|
+
| **Selective** | Specific changes | Changed + dependent |
|
|
51
|
+
| **Complete** | Major refactor | Everything |
|
|
52
|
+
|
|
53
|
+
### Test Selection Strategies
|
|
54
|
+
| Strategy | How | Reduction |
|
|
55
|
+
|----------|-----|-----------|
|
|
56
|
+
| **Change-based** | Git diff analysis | 70-90% |
|
|
57
|
+
| **Risk-based** | Priority by impact | 50-70% |
|
|
58
|
+
| **Historical** | Frequently failing | 40-60% |
|
|
59
|
+
| **Time-budget** | Fixed time window | Variable |
|
|
135
60
|
|
|
136
|
-
|
|
137
|
-
|
|
138
|
-
**Analyze what changed, test accordingly**
|
|
139
|
-
|
|
140
|
-
```typescript
|
|
141
|
-
// PR changes:
|
|
142
|
-
- src/services/payment.ts
|
|
143
|
-
- src/utils/currency.ts
|
|
144
|
-
|
|
145
|
-
// Select tests covering:
|
|
146
|
-
- payment.test.ts ✓
|
|
147
|
-
- currency.test.ts ✓
|
|
148
|
-
- integration/checkout.test.ts ✓ (uses payment)
|
|
149
|
-
- e2e/purchase-flow.test.ts ✓ (exercises payment)
|
|
150
|
-
```
|
|
151
|
-
|
|
152
|
-
**Tools:**
|
|
153
|
-
- Git diff analysis
|
|
154
|
-
- Code coverage mapping
|
|
155
|
-
- Dependency graphs
|
|
156
|
-
|
|
157
|
-
**Benefits:**
|
|
158
|
-
- Fast feedback (5-10 min vs 2 hours)
|
|
159
|
-
- 70-90% defect detection
|
|
160
|
-
- Scalable to large suites
|
|
161
|
-
|
|
162
|
-
### Strategy 2: Risk-Based Selection
|
|
163
|
-
|
|
164
|
-
**Prioritize based on failure risk and impact**
|
|
165
|
-
|
|
166
|
-
```typescript
|
|
167
|
-
Risk Score = Probability of Failure × Impact
|
|
168
|
-
|
|
169
|
-
High Risk:
|
|
170
|
-
- Payment processing (critical + complex)
|
|
171
|
-
- Authentication (critical + frequently changed)
|
|
172
|
-
- Data migration (high impact)
|
|
173
|
-
|
|
174
|
-
Medium Risk:
|
|
175
|
-
- User profile (moderate impact)
|
|
176
|
-
- Search (stable code)
|
|
177
|
-
|
|
178
|
-
Low Risk:
|
|
179
|
-
- Help documentation (low impact + stable)
|
|
180
|
-
```
|
|
181
|
-
|
|
182
|
-
### Strategy 3: Historical Failure Analysis
|
|
183
|
-
|
|
184
|
-
**Test what breaks frequently**
|
|
185
|
-
|
|
186
|
-
```typescript
|
|
187
|
-
// Track test failures over last 30 days
|
|
188
|
-
const flakyTests = [
|
|
189
|
-
'checkout.test.ts - 15 failures',
|
|
190
|
-
'auth.test.ts - 12 failures',
|
|
191
|
-
'api-integration.test.ts - 8 failures'
|
|
192
|
-
];
|
|
193
|
-
|
|
194
|
-
// Always include in regression
|
|
195
|
-
```
|
|
196
|
-
|
|
197
|
-
### Strategy 4: Code Coverage-Based
|
|
198
|
-
|
|
199
|
-
**Test code with poor existing coverage**
|
|
200
|
-
|
|
201
|
-
```typescript
|
|
202
|
-
// Coverage analysis
|
|
203
|
-
payment.ts: 45% coverage → High priority for regression
|
|
204
|
-
checkout.ts: 85% coverage → Medium priority
|
|
205
|
-
utils.ts: 95% coverage → Low priority
|
|
206
|
-
|
|
207
|
-
// Focus regression on under-tested code
|
|
208
|
-
```
|
|
209
|
-
|
|
210
|
-
### Strategy 5: Time-Budget Selection
|
|
61
|
+
---
|
|
211
62
|
|
|
212
|
-
|
|
63
|
+
## Change-Based Test Selection
|
|
213
64
|
|
|
214
65
|
```typescript
|
|
215
|
-
//
|
|
216
|
-
|
|
217
|
-
|
|
218
|
-
{ test: 'payment-flows', time: '3 min', value: 'high' },
|
|
219
|
-
{ test: 'auth-flows', time: '2 min', value: 'high' },
|
|
220
|
-
{ test: 'search', time: '2 min', value: 'medium' },
|
|
221
|
-
{ test: 'profiles', time: '3 min', value: 'medium' }
|
|
222
|
-
];
|
|
66
|
+
// Analyze changed files and select impacted tests
|
|
67
|
+
function selectTests(changedFiles: string[]): string[] {
|
|
68
|
+
const testsToRun = new Set<string>();
|
|
223
69
|
|
|
224
|
-
|
|
225
|
-
//
|
|
226
|
-
|
|
70
|
+
for (const file of changedFiles) {
|
|
71
|
+
// Direct tests
|
|
72
|
+
testsToRun.add(`${file.replace('.ts', '.test.ts')}`);
|
|
227
73
|
|
|
228
|
-
|
|
229
|
-
|
|
230
|
-
|
|
231
|
-
|
|
232
|
-
Start with smoke tests - does basic functionality work?
|
|
233
|
-
|
|
234
|
-
```javascript
|
|
235
|
-
// Smoke test suite (5-10 min)
|
|
236
|
-
describe('Smoke Tests', () => {
|
|
237
|
-
test('App starts without errors', () => {
|
|
238
|
-
expect(app.isRunning()).toBe(true);
|
|
239
|
-
});
|
|
240
|
-
|
|
241
|
-
test('Database connection works', () => {
|
|
242
|
-
expect(db.isConnected()).toBe(true);
|
|
243
|
-
});
|
|
244
|
-
|
|
245
|
-
test('Critical API endpoints respond', () => {
|
|
246
|
-
expect(api.health()).toBe('OK');
|
|
247
|
-
});
|
|
248
|
-
|
|
249
|
-
test('User can login', () => {
|
|
250
|
-
login('test@example.com', 'password');
|
|
251
|
-
expect(session.isActive()).toBe(true);
|
|
252
|
-
});
|
|
253
|
-
});
|
|
254
|
-
```
|
|
74
|
+
// Dependent tests (via coverage mapping)
|
|
75
|
+
const dependentTests = testCoverage[file] || [];
|
|
76
|
+
dependentTests.forEach(t => testsToRun.add(t));
|
|
77
|
+
}
|
|
255
78
|
|
|
256
|
-
|
|
257
|
-
|
|
258
|
-
Cover main user workflows end-to-end.
|
|
259
|
-
|
|
260
|
-
```javascript
|
|
261
|
-
// Happy path suite (20-30 min)
|
|
262
|
-
describe('Core User Journeys', () => {
|
|
263
|
-
test('User can sign up and verify email', async () => {
|
|
264
|
-
await signup('new@example.com', 'SecurePass123!');
|
|
265
|
-
const email = await getLatestEmail('new@example.com');
|
|
266
|
-
await verifyEmail(email.verificationLink);
|
|
267
|
-
expect(user.isVerified()).toBe(true);
|
|
268
|
-
});
|
|
269
|
-
|
|
270
|
-
test('User can complete purchase', async () => {
|
|
271
|
-
await login();
|
|
272
|
-
await addToCart(product);
|
|
273
|
-
await checkout();
|
|
274
|
-
await submitPayment(validCard);
|
|
275
|
-
expect(order.status()).toBe('completed');
|
|
276
|
-
});
|
|
277
|
-
});
|
|
278
|
-
```
|
|
279
|
-
|
|
280
|
-
### Phase 3: Add Edge Cases and Error Scenarios
|
|
281
|
-
|
|
282
|
-
```javascript
|
|
283
|
-
// Edge case suite (30-45 min)
|
|
284
|
-
describe('Edge Cases', () => {
|
|
285
|
-
test('Handles expired credit card', async () => {
|
|
286
|
-
await checkout();
|
|
287
|
-
await submitPayment(expiredCard);
|
|
288
|
-
expect(error.message()).toContain('Card expired');
|
|
289
|
-
});
|
|
79
|
+
return Array.from(testsToRun);
|
|
80
|
+
}
|
|
290
81
|
|
|
291
|
-
|
|
292
|
-
|
|
293
|
-
checkout(user1),
|
|
294
|
-
checkout(user1) // Same user, same time
|
|
295
|
-
];
|
|
296
|
-
const results = await Promise.all(promises);
|
|
297
|
-
expect(results.filter(r => r.success).length).toBe(1);
|
|
298
|
-
});
|
|
299
|
-
});
|
|
82
|
+
// Example: payment.ts changed
|
|
83
|
+
// Runs: payment.test.ts, checkout.integration.test.ts, e2e/purchase.test.ts
|
|
300
84
|
```
|
|
301
85
|
|
|
302
|
-
|
|
303
|
-
|
|
304
|
-
```javascript
|
|
305
|
-
// Integration suite (45-60 min)
|
|
306
|
-
describe('System Integration', () => {
|
|
307
|
-
test('Payment gateway integration works', async () => {
|
|
308
|
-
const result = await paymentGateway.charge(card, amount);
|
|
309
|
-
expect(result.status).toBe('succeeded');
|
|
310
|
-
expect(db.transaction).toHaveBeenRecorded();
|
|
311
|
-
expect(email.receipt).toHaveBeenSent();
|
|
312
|
-
});
|
|
313
|
-
|
|
314
|
-
test('Inventory sync with warehouse', async () => {
|
|
315
|
-
await purchaseProduct(product);
|
|
316
|
-
const inventory = await warehouse.checkStock(product.id);
|
|
317
|
-
expect(inventory.quantity).toBe(originalQuantity - 1);
|
|
318
|
-
});
|
|
319
|
-
});
|
|
320
|
-
```
|
|
86
|
+
---
|
|
321
87
|
|
|
322
|
-
|
|
88
|
+
## Regression Suite Pyramid
|
|
323
89
|
|
|
324
90
|
```
|
|
325
91
|
/\
|
|
@@ -327,213 +93,24 @@ describe('System Integration', () => {
|
|
|
327
93
|
/ \ - All tests (2-4 hours)
|
|
328
94
|
/------\
|
|
329
95
|
/ \ Extended Regression (nightly)
|
|
330
|
-
/ \ -
|
|
96
|
+
/ \ - Unit + integration + critical E2E (30-60 min)
|
|
331
97
|
/------------\
|
|
332
98
|
/ \ Quick Regression (per commit)
|
|
333
|
-
/________________\ - Changed code
|
|
334
|
-
```
|
|
335
|
-
|
|
336
|
-
## Test Impact Analysis
|
|
337
|
-
|
|
338
|
-
### Mapping Tests to Code
|
|
339
|
-
|
|
340
|
-
**Build dependency graph:**
|
|
341
|
-
|
|
342
|
-
```typescript
|
|
343
|
-
// Track which tests cover which code
|
|
344
|
-
const testCoverage = {
|
|
345
|
-
'payment.ts': [
|
|
346
|
-
'payment.test.ts',
|
|
347
|
-
'checkout.integration.test.ts',
|
|
348
|
-
'e2e/purchase.test.ts'
|
|
349
|
-
],
|
|
350
|
-
'user.ts': [
|
|
351
|
-
'user.test.ts',
|
|
352
|
-
'auth.integration.test.ts',
|
|
353
|
-
'e2e/signup.test.ts'
|
|
354
|
-
]
|
|
355
|
-
};
|
|
356
|
-
|
|
357
|
-
// When payment.ts changes, run all related tests
|
|
358
|
-
function selectTests(changedFiles) {
|
|
359
|
-
const testsToRun = new Set();
|
|
360
|
-
changedFiles.forEach(file => {
|
|
361
|
-
testCoverage[file]?.forEach(test => testsToRun.add(test));
|
|
362
|
-
});
|
|
363
|
-
return Array.from(testsToRun);
|
|
364
|
-
}
|
|
365
|
-
```
|
|
366
|
-
|
|
367
|
-
### Transitive Dependencies
|
|
368
|
-
|
|
369
|
-
**Account for indirect dependencies:**
|
|
370
|
-
|
|
371
|
-
```typescript
|
|
372
|
-
// Direct dependency
|
|
373
|
-
payment.ts → uses → currency.ts
|
|
374
|
-
|
|
375
|
-
// Transitive dependency
|
|
376
|
-
checkout.ts → uses → payment.ts → uses → currency.ts
|
|
377
|
-
|
|
378
|
-
// When currency.ts changes, test:
|
|
379
|
-
- currency.test.ts (direct)
|
|
380
|
-
- payment.test.ts (direct dependency on currency)
|
|
381
|
-
- checkout.test.ts (transitive dependency)
|
|
382
|
-
```
|
|
383
|
-
|
|
384
|
-
### Static Analysis for Test Selection
|
|
385
|
-
|
|
386
|
-
```typescript
|
|
387
|
-
// Analyze imports to build dependency graph
|
|
388
|
-
import ts from 'typescript';
|
|
389
|
-
|
|
390
|
-
function findDependencies(sourceFile: string): string[] {
|
|
391
|
-
const program = ts.createProgram([sourceFile], {});
|
|
392
|
-
const checker = program.getTypeChecker();
|
|
393
|
-
|
|
394
|
-
// Extract all imports
|
|
395
|
-
const dependencies = [];
|
|
396
|
-
const sourceFileObj = program.getSourceFile(sourceFile);
|
|
397
|
-
|
|
398
|
-
ts.forEachChild(sourceFileObj, node => {
|
|
399
|
-
if (ts.isImportDeclaration(node)) {
|
|
400
|
-
dependencies.push(node.moduleSpecifier.text);
|
|
401
|
-
}
|
|
402
|
-
});
|
|
403
|
-
|
|
404
|
-
return dependencies;
|
|
405
|
-
}
|
|
406
|
-
```
|
|
407
|
-
|
|
408
|
-
## Regression Test Optimization
|
|
409
|
-
|
|
410
|
-
### Technique 1: Test Parallelization
|
|
411
|
-
|
|
412
|
-
**Run tests concurrently for faster feedback**
|
|
413
|
-
|
|
414
|
-
```javascript
|
|
415
|
-
// Sequential: 60 min
|
|
416
|
-
test1(); // 20 min
|
|
417
|
-
test2(); // 20 min
|
|
418
|
-
test3(); // 20 min
|
|
419
|
-
|
|
420
|
-
// Parallel: 20 min
|
|
421
|
-
Promise.all([
|
|
422
|
-
test1(), // 20 min
|
|
423
|
-
test2(), // 20 min
|
|
424
|
-
test3() // 20 min
|
|
425
|
-
]);
|
|
426
|
-
|
|
427
|
-
// Jest configuration
|
|
428
|
-
module.exports = {
|
|
429
|
-
maxWorkers: '50%', // Use half CPU cores
|
|
430
|
-
testTimeout: 30000
|
|
431
|
-
};
|
|
432
|
-
```
|
|
433
|
-
|
|
434
|
-
### Technique 2: Test Sharding
|
|
435
|
-
|
|
436
|
-
**Distribute tests across multiple machines**
|
|
437
|
-
|
|
438
|
-
```yaml
|
|
439
|
-
# CI pipeline with 4 workers
|
|
440
|
-
jobs:
|
|
441
|
-
test:
|
|
442
|
-
strategy:
|
|
443
|
-
matrix:
|
|
444
|
-
shard: [1, 2, 3, 4]
|
|
445
|
-
steps:
|
|
446
|
-
- run: npm test -- --shard=${{ matrix.shard }}/4
|
|
447
|
-
|
|
448
|
-
# Each worker runs 25% of tests
|
|
449
|
-
# Total time: 60 min / 4 = 15 min
|
|
450
|
-
```
|
|
451
|
-
|
|
452
|
-
### Technique 3: Incremental Testing
|
|
453
|
-
|
|
454
|
-
**Test only what changed since last run**
|
|
455
|
-
|
|
456
|
-
```typescript
|
|
457
|
-
// Track test results
|
|
458
|
-
const lastRun = {
|
|
459
|
-
timestamp: '2025-10-24T10:00:00Z',
|
|
460
|
-
passed: ['test1', 'test2', 'test3'],
|
|
461
|
-
failed: []
|
|
462
|
-
};
|
|
463
|
-
|
|
464
|
-
// Current run
|
|
465
|
-
const currentChanges = ['payment.ts', 'checkout.ts'];
|
|
466
|
-
|
|
467
|
-
// Run:
|
|
468
|
-
// 1. Tests for changed code (payment, checkout)
|
|
469
|
-
// 2. Tests that failed last time (if any)
|
|
470
|
-
// Skip tests that passed last time for unchanged code
|
|
471
|
-
```
|
|
472
|
-
|
|
473
|
-
### Technique 4: Smoke Test Fast Fail
|
|
474
|
-
|
|
475
|
-
**Run fastest, most critical tests first**
|
|
476
|
-
|
|
477
|
-
```yaml
|
|
478
|
-
# CI Pipeline
|
|
479
|
-
stages:
|
|
480
|
-
- smoke-test: # 2 min
|
|
481
|
-
- critical-paths
|
|
482
|
-
- fail-fast: true # Stop if smoke fails
|
|
483
|
-
|
|
484
|
-
- quick-regression: # 10 min
|
|
485
|
-
- changed-code-tests
|
|
486
|
-
- fail-fast: true
|
|
487
|
-
|
|
488
|
-
- full-regression: # 60 min
|
|
489
|
-
- all-tests
|
|
490
|
-
- fail-fast: false # Run all to find all issues
|
|
491
|
-
```
|
|
492
|
-
|
|
493
|
-
### Technique 5: Test Flakiness Removal
|
|
494
|
-
|
|
495
|
-
**Eliminate unreliable tests**
|
|
496
|
-
|
|
497
|
-
```typescript
|
|
498
|
-
// Track test stability over 100 runs
|
|
499
|
-
const testStability = {
|
|
500
|
-
'reliable-test': { runs: 100, passes: 100, passRate: 1.00 },
|
|
501
|
-
'flaky-test': { runs: 100, passes: 87, passRate: 0.87 },
|
|
502
|
-
'unstable-test': { runs: 100, passes: 64, passRate: 0.64 }
|
|
503
|
-
};
|
|
504
|
-
|
|
505
|
-
// Strategy:
|
|
506
|
-
// passRate >= 0.98 → Keep in regression suite
|
|
507
|
-
// passRate 0.90-0.98 → Fix flakiness
|
|
508
|
-
// passRate < 0.90 → Quarantine until fixed
|
|
509
|
-
|
|
510
|
-
// Flaky tests waste time and reduce confidence
|
|
99
|
+
/________________\ - Changed code + smoke tests (5-10 min)
|
|
511
100
|
```
|
|
512
101
|
|
|
513
|
-
|
|
102
|
+
---
|
|
514
103
|
|
|
515
|
-
|
|
104
|
+
## CI/CD Integration
|
|
516
105
|
|
|
517
106
|
```yaml
|
|
518
107
|
# .github/workflows/regression.yml
|
|
519
|
-
name: Regression Testing
|
|
520
|
-
|
|
521
|
-
on:
|
|
522
|
-
pull_request:
|
|
523
|
-
branches: [main]
|
|
524
|
-
push:
|
|
525
|
-
branches: [main]
|
|
526
|
-
schedule:
|
|
527
|
-
- cron: '0 2 * * *' # Nightly at 2 AM
|
|
528
|
-
|
|
529
108
|
jobs:
|
|
530
109
|
quick-regression:
|
|
531
|
-
name: Quick Regression (per PR)
|
|
532
110
|
runs-on: ubuntu-latest
|
|
533
111
|
timeout-minutes: 15
|
|
534
112
|
steps:
|
|
535
|
-
-
|
|
536
|
-
- name: Changed file analysis
|
|
113
|
+
- name: Analyze changes
|
|
537
114
|
id: changes
|
|
538
115
|
uses: dorny/paths-filter@v2
|
|
539
116
|
with:
|
|
@@ -544,490 +121,107 @@ jobs:
|
|
|
544
121
|
- 'src/auth/**'
|
|
545
122
|
|
|
546
123
|
- name: Run affected tests
|
|
547
|
-
run:
|
|
548
|
-
npm run test:payment
|
|
549
|
-
npm run test:auth
|
|
550
|
-
if: steps.changes.outputs.payment == 'true' || steps.changes.outputs.auth == 'true'
|
|
124
|
+
run: npm run test:affected
|
|
551
125
|
|
|
552
126
|
- name: Smoke tests (always)
|
|
553
127
|
run: npm run test:smoke
|
|
554
128
|
|
|
555
129
|
nightly-regression:
|
|
556
|
-
name: Nightly Full Regression
|
|
557
|
-
runs-on: ubuntu-latest
|
|
558
|
-
timeout-minutes: 120
|
|
559
130
|
if: github.event_name == 'schedule'
|
|
131
|
+
timeout-minutes: 120
|
|
560
132
|
steps:
|
|
561
|
-
-
|
|
562
|
-
run: npm test
|
|
563
|
-
|
|
564
|
-
- name: Coverage report
|
|
565
|
-
run: npm run coverage
|
|
566
|
-
|
|
567
|
-
- name: Upload results
|
|
568
|
-
uses: actions/upload-artifact@v3
|
|
569
|
-
with:
|
|
570
|
-
name: regression-results
|
|
571
|
-
path: test-results/
|
|
572
|
-
```
|
|
573
|
-
|
|
574
|
-
### Pre-Commit Regression
|
|
575
|
-
|
|
576
|
-
**Local regression before committing**
|
|
577
|
-
|
|
578
|
-
```bash
|
|
579
|
-
# .git/hooks/pre-commit
|
|
580
|
-
#!/bin/bash
|
|
581
|
-
|
|
582
|
-
echo "Running quick regression tests..."
|
|
583
|
-
|
|
584
|
-
# Get staged files
|
|
585
|
-
STAGED_FILES=$(git diff --cached --name-only --diff-filter=ACM | grep ".ts$")
|
|
586
|
-
|
|
587
|
-
if [ -z "$STAGED_FILES" ]; then
|
|
588
|
-
echo "No TypeScript files changed, skipping tests"
|
|
589
|
-
exit 0
|
|
590
|
-
fi
|
|
591
|
-
|
|
592
|
-
# Run tests for changed files
|
|
593
|
-
npm run test:changed -- $STAGED_FILES
|
|
594
|
-
|
|
595
|
-
if [ $? -ne 0 ]; then
|
|
596
|
-
echo "❌ Regression tests failed. Commit aborted."
|
|
597
|
-
exit 1
|
|
598
|
-
fi
|
|
599
|
-
|
|
600
|
-
echo "✅ Regression tests passed"
|
|
601
|
-
exit 0
|
|
602
|
-
```
|
|
603
|
-
|
|
604
|
-
### Pre-Deployment Regression
|
|
605
|
-
|
|
606
|
-
**Final validation before production**
|
|
607
|
-
|
|
608
|
-
```yaml
|
|
609
|
-
# Deployment pipeline
|
|
610
|
-
deploy:
|
|
611
|
-
steps:
|
|
612
|
-
- name: Production smoke tests
|
|
613
|
-
run: npm run test:smoke:production
|
|
614
|
-
env:
|
|
615
|
-
API_URL: https://staging.example.com
|
|
616
|
-
|
|
617
|
-
- name: Critical path regression
|
|
618
|
-
run: npm run test:critical-paths
|
|
619
|
-
|
|
620
|
-
- name: Security regression
|
|
621
|
-
run: npm run test:security
|
|
622
|
-
|
|
623
|
-
- name: Performance baseline check
|
|
624
|
-
run: npm run test:performance
|
|
625
|
-
|
|
626
|
-
- name: Deploy to production
|
|
627
|
-
if: success()
|
|
628
|
-
run: ./deploy.sh production
|
|
133
|
+
- run: npm test -- --coverage
|
|
629
134
|
```
|
|
630
135
|
|
|
631
|
-
|
|
632
|
-
|
|
633
|
-
### Anti-Pattern: Test Rot
|
|
136
|
+
---
|
|
634
137
|
|
|
635
|
-
|
|
138
|
+
## Optimization Techniques
|
|
636
139
|
|
|
637
140
|
```javascript
|
|
638
|
-
//
|
|
639
|
-
|
|
640
|
-
|
|
641
|
-
|
|
642
|
-
});
|
|
643
|
-
|
|
644
|
-
// GOOD: Remove obsolete tests
|
|
645
|
-
// Delete test when feature is removed
|
|
646
|
-
```
|
|
647
|
-
|
|
648
|
-
### Strategy: Regular Test Review
|
|
649
|
-
|
|
650
|
-
```markdown
|
|
651
|
-
## Quarterly Test Audit
|
|
652
|
-
|
|
653
|
-
Review each test:
|
|
654
|
-
- [ ] Is feature still in product?
|
|
655
|
-
- [ ] Does test provide value?
|
|
656
|
-
- [ ] Is test reliable (not flaky)?
|
|
657
|
-
- [ ] Is test documented/understandable?
|
|
658
|
-
- [ ] Execution time acceptable?
|
|
659
|
-
|
|
660
|
-
Actions:
|
|
661
|
-
- Delete: Obsolete tests
|
|
662
|
-
- Fix: Flaky tests
|
|
663
|
-
- Optimize: Slow tests
|
|
664
|
-
- Document: Unclear tests
|
|
665
|
-
- Consolidate: Duplicate tests
|
|
666
|
-
```
|
|
667
|
-
|
|
668
|
-
### Strategy: Test Suite Metrics
|
|
669
|
-
|
|
670
|
-
```typescript
|
|
671
|
-
interface TestSuiteHealth {
|
|
672
|
-
totalTests: number;
|
|
673
|
-
executionTime: string;
|
|
674
|
-
passRate: number;
|
|
675
|
-
flakiness: number;
|
|
676
|
-
coverage: number;
|
|
677
|
-
lastUpdated: Date;
|
|
678
|
-
}
|
|
679
|
-
|
|
680
|
-
const regressionHealth: TestSuiteHealth = {
|
|
681
|
-
totalTests: 1247,
|
|
682
|
-
executionTime: '18 min',
|
|
683
|
-
passRate: 0.987, // 98.7% pass rate
|
|
684
|
-
flakiness: 0.013, // 1.3% flaky tests (target: <2%)
|
|
685
|
-
coverage: 0.82, // 82% code coverage
|
|
686
|
-
lastUpdated: new Date('2025-10-24')
|
|
141
|
+
// 1. Parallel execution
|
|
142
|
+
module.exports = {
|
|
143
|
+
maxWorkers: '50%', // Use half CPU cores
|
|
144
|
+
testTimeout: 30000
|
|
687
145
|
};
|
|
688
146
|
|
|
689
|
-
//
|
|
690
|
-
//
|
|
691
|
-
// Trend: flakiness ↑ → need stabilization
|
|
692
|
-
// Trend: coverage ↓ → add tests for new code
|
|
693
|
-
```
|
|
147
|
+
// 2. Sharding across CI workers
|
|
148
|
+
// npm test -- --shard=1/4
|
|
694
149
|
|
|
695
|
-
|
|
150
|
+
// 3. Incremental testing (only changed since last run)
|
|
151
|
+
// Track last run state, skip passing unchanged tests
|
|
696
152
|
|
|
697
|
-
|
|
698
|
-
// tests
|
|
699
|
-
/**
|
|
700
|
-
* @owner team-payments
|
|
701
|
-
* @created 2025-06-15
|
|
702
|
-
* @criticality high
|
|
703
|
-
* @executionTime 45s
|
|
704
|
-
* @lastReview 2025-09-20
|
|
705
|
-
*
|
|
706
|
-
* Tests the complete checkout flow including:
|
|
707
|
-
* - Cart validation
|
|
708
|
-
* - Payment processing
|
|
709
|
-
* - Order confirmation
|
|
710
|
-
* - Email receipt
|
|
711
|
-
*/
|
|
712
|
-
|
|
713
|
-
// Benefits:
|
|
714
|
-
// - Clear responsibility for maintenance
|
|
715
|
-
// - Context for why test exists
|
|
716
|
-
// - Trigger for regular review
|
|
717
|
-
```
|
|
718
|
-
|
|
719
|
-
## Regression Testing Best Practices
|
|
720
|
-
|
|
721
|
-
### ✅ DO: Automate Regression Tests
|
|
722
|
-
|
|
723
|
-
**Manual regression doesn't scale**
|
|
724
|
-
|
|
725
|
-
```typescript
|
|
726
|
-
// Automated regression runs on every PR
|
|
727
|
-
// Catches issues in minutes, not days
|
|
153
|
+
// 4. Fast-fail on smoke
|
|
154
|
+
// Run critical tests first, abort if they fail
|
|
728
155
|
```
|
|
729
156
|
|
|
730
|
-
|
|
731
|
-
|
|
732
|
-
```javascript
|
|
733
|
-
// BAD
|
|
734
|
-
test('test1', () => { ... });
|
|
735
|
-
|
|
736
|
-
// GOOD
|
|
737
|
-
test('checkout fails gracefully when payment gateway is down', () => {
|
|
738
|
-
mockPaymentGateway.simulateOutage();
|
|
739
|
-
const result = await checkout(cart);
|
|
740
|
-
expect(result.status).toBe('payment-failed');
|
|
741
|
-
expect(result.userMessage).toContain('payment service unavailable');
|
|
742
|
-
});
|
|
743
|
-
```
|
|
744
|
-
|
|
745
|
-
### ✅ DO: Keep Tests Independent
|
|
746
|
-
|
|
747
|
-
```javascript
|
|
748
|
-
// BAD: Tests depend on execution order
|
|
749
|
-
test('create user', () => {
|
|
750
|
-
user = createUser('test@example.com');
|
|
751
|
-
});
|
|
752
|
-
|
|
753
|
-
test('user can login', () => {
|
|
754
|
-
login(user); // Fails if previous test didn't run
|
|
755
|
-
});
|
|
756
|
-
|
|
757
|
-
// GOOD: Each test independent
|
|
758
|
-
test('user can login', () => {
|
|
759
|
-
const user = createUser('test@example.com'); // Setup in test
|
|
760
|
-
login(user);
|
|
761
|
-
expect(session.isActive()).toBe(true);
|
|
762
|
-
});
|
|
763
|
-
```
|
|
764
|
-
|
|
765
|
-
### ✅ DO: Use Setup/Teardown
|
|
766
|
-
|
|
767
|
-
```javascript
|
|
768
|
-
describe('Payment Tests', () => {
|
|
769
|
-
beforeEach(async () => {
|
|
770
|
-
// Clean slate for each test
|
|
771
|
-
await db.clean();
|
|
772
|
-
await db.seed(testData);
|
|
773
|
-
});
|
|
774
|
-
|
|
775
|
-
afterEach(async () => {
|
|
776
|
-
// Cleanup
|
|
777
|
-
await db.clean();
|
|
778
|
-
});
|
|
779
|
-
|
|
780
|
-
test('payment succeeds with valid card', async () => {
|
|
781
|
-
// Test has clean environment
|
|
782
|
-
});
|
|
783
|
-
});
|
|
784
|
-
```
|
|
785
|
-
|
|
786
|
-
### ❌ DON'T: Test Everything, Every Time
|
|
787
|
-
|
|
788
|
-
```typescript
|
|
789
|
-
// DON'T run 4-hour full regression on every commit
|
|
790
|
-
// DO run smart selection based on changes
|
|
791
|
-
```
|
|
792
|
-
|
|
793
|
-
### ❌ DON'T: Ignore Flaky Tests
|
|
794
|
-
|
|
795
|
-
```typescript
|
|
796
|
-
// DON'T keep flaky tests and ignore failures
|
|
797
|
-
// DO fix or quarantine flaky tests immediately
|
|
798
|
-
```
|
|
799
|
-
|
|
800
|
-
### ❌ DON'T: Duplicate Test Coverage
|
|
801
|
-
|
|
802
|
-
```typescript
|
|
803
|
-
// DON'T test same thing at multiple levels
|
|
804
|
-
// Unit test: currency conversion logic
|
|
805
|
-
// Integration test: currency conversion in payment
|
|
806
|
-
// E2E test: full purchase with currency conversion
|
|
807
|
-
|
|
808
|
-
// DO test each concern at appropriate level
|
|
809
|
-
// Unit: Conversion logic
|
|
810
|
-
// Integration: Payment service uses conversion correctly
|
|
811
|
-
// E2E: Happy path only (one currency conversion example)
|
|
812
|
-
```
|
|
813
|
-
|
|
814
|
-
## Using with QE Agents
|
|
815
|
-
|
|
816
|
-
### Intelligent Test Selection
|
|
157
|
+
---
|
|
817
158
|
|
|
818
|
-
|
|
159
|
+
## Agent-Driven Regression
|
|
819
160
|
|
|
820
161
|
```typescript
|
|
821
|
-
//
|
|
822
|
-
|
|
162
|
+
// Smart test selection
|
|
163
|
+
await Task("Regression Analysis", {
|
|
823
164
|
pr: 1234,
|
|
824
165
|
strategy: 'change-based-with-risk',
|
|
825
|
-
|
|
826
|
-
});
|
|
166
|
+
timeBudget: '15min'
|
|
167
|
+
}, "qe-regression-risk-analyzer");
|
|
827
168
|
|
|
828
169
|
// Returns:
|
|
829
170
|
// {
|
|
830
171
|
// mustRun: ['payment.test.ts', 'checkout.integration.test.ts'],
|
|
831
|
-
// shouldRun: ['order.test.ts'
|
|
172
|
+
// shouldRun: ['order.test.ts'],
|
|
832
173
|
// canSkip: ['profile.test.ts', 'search.test.ts'],
|
|
833
174
|
// estimatedTime: '12 min',
|
|
834
175
|
// riskCoverage: 0.94
|
|
835
176
|
// }
|
|
836
|
-
```
|
|
837
|
-
|
|
838
|
-
### Automated Test Generation for Regression
|
|
839
177
|
|
|
840
|
-
|
|
841
|
-
|
|
842
|
-
|
|
843
|
-
// Bug found in production
|
|
844
|
-
const bug = {
|
|
845
|
-
id: 'BUG-567',
|
|
846
|
-
description: 'Checkout fails when user has > 100 items in cart',
|
|
847
|
-
severity: 'high'
|
|
848
|
-
};
|
|
849
|
-
|
|
850
|
-
// Agent generates regression test
|
|
851
|
-
await agent.generateRegressionTest({
|
|
852
|
-
bug: bug,
|
|
178
|
+
// Generate regression test from production bug
|
|
179
|
+
await Task("Bug Regression Test", {
|
|
180
|
+
bug: { id: 'BUG-567', description: 'Checkout fails > 100 items' },
|
|
853
181
|
preventRecurrence: true
|
|
854
|
-
});
|
|
855
|
-
|
|
856
|
-
// Creates:
|
|
857
|
-
// tests/regression/BUG-567-large-cart.test.ts
|
|
858
|
-
test('checkout succeeds with 100+ items in cart', async () => {
|
|
859
|
-
const cart = generateCart(150); // Edge case that failed
|
|
860
|
-
const result = await checkout(cart);
|
|
861
|
-
expect(result.status).toBe('success');
|
|
862
|
-
});
|
|
182
|
+
}, "qe-test-generator");
|
|
863
183
|
```
|
|
864
184
|
|
|
865
|
-
|
|
866
|
-
|
|
867
|
-
**qe-quality-analyzer** tracks regression suite health:
|
|
868
|
-
|
|
869
|
-
```typescript
|
|
870
|
-
// Monitor regression suite metrics
|
|
871
|
-
const health = await agent.analyzeRegressionHealth({
|
|
872
|
-
suite: 'main-regression',
|
|
873
|
-
period: '30d'
|
|
874
|
-
});
|
|
875
|
-
|
|
876
|
-
// Returns insights:
|
|
877
|
-
// {
|
|
878
|
-
// executionTimeTrend: 'increasing', // ⚠️ Getting slower
|
|
879
|
-
// flakinessRate: 0.03, // ⚠️ Above 2% threshold
|
|
880
|
-
// coverageTrend: 'stable', // ✅ Maintaining coverage
|
|
881
|
-
// recommendations: [
|
|
882
|
-
// 'Parallelize slow tests',
|
|
883
|
-
// 'Fix 12 flaky tests',
|
|
884
|
-
// 'Remove 5 obsolete tests'
|
|
885
|
-
// ]
|
|
886
|
-
// }
|
|
887
|
-
```
|
|
888
|
-
|
|
889
|
-
### Regression Testing in CI/CD
|
|
185
|
+
---
|
|
890
186
|
|
|
891
|
-
|
|
187
|
+
## Agent Coordination Hints
|
|
892
188
|
|
|
893
|
-
|
|
894
|
-
// Execute smart regression in CI pipeline
|
|
895
|
-
await agent.executeRegression({
|
|
896
|
-
trigger: 'pull-request',
|
|
897
|
-
strategy: 'selective',
|
|
898
|
-
changedFiles: ['src/payment/stripe.ts', 'src/models/order.ts'],
|
|
899
|
-
parallel: true,
|
|
900
|
-
failFast: false
|
|
901
|
-
});
|
|
902
|
-
|
|
903
|
-
// Execution plan:
|
|
904
|
-
// 1. Analyze changed files
|
|
905
|
-
// 2. Select impacted tests (change-based)
|
|
906
|
-
// 3. Add critical path tests (risk-based)
|
|
907
|
-
// 4. Run in parallel across 4 workers
|
|
908
|
-
// 5. Report results with coverage metrics
|
|
189
|
+
### Memory Namespace
|
|
909
190
|
```
|
|
910
|
-
|
|
911
|
-
|
|
912
|
-
|
|
913
|
-
|
|
914
|
-
|
|
915
|
-
```typescript
|
|
916
|
-
// Detect visual changes after code update
|
|
917
|
-
await agent.visualRegression({
|
|
918
|
-
baseline: 'main-branch',
|
|
919
|
-
current: 'feature-branch',
|
|
920
|
-
pages: ['checkout', 'product-detail', 'cart'],
|
|
921
|
-
threshold: 0.01 // 1% pixel difference tolerance
|
|
922
|
-
});
|
|
923
|
-
|
|
924
|
-
// Returns:
|
|
925
|
-
// {
|
|
926
|
-
// changed: ['checkout-page'],
|
|
927
|
-
// diff: {
|
|
928
|
-
// 'checkout-page': {
|
|
929
|
-
// pixelDifference: 234,
|
|
930
|
-
// percentageChange: 0.023,
|
|
931
|
-
// screenshot: 'diff-checkout.png',
|
|
932
|
-
// verdict: 'review-required'
|
|
933
|
-
// }
|
|
934
|
-
// }
|
|
935
|
-
// }
|
|
191
|
+
aqe/regression-testing/
|
|
192
|
+
├── test-selection/* - Impact analysis results
|
|
193
|
+
├── suite-health/* - Flakiness, timing trends
|
|
194
|
+
├── coverage-maps/* - Test-to-code mapping
|
|
195
|
+
└── bug-regressions/* - Tests from production bugs
|
|
936
196
|
```
|
|
937
197
|
|
|
938
|
-
### Fleet
|
|
939
|
-
|
|
198
|
+
### Fleet Coordination
|
|
940
199
|
```typescript
|
|
941
|
-
// Multiple agents collaborate on regression testing
|
|
942
200
|
const regressionFleet = await FleetManager.coordinate({
|
|
943
201
|
strategy: 'comprehensive-regression',
|
|
944
202
|
agents: [
|
|
945
|
-
'qe-regression-risk-analyzer', //
|
|
946
|
-
'qe-test-
|
|
947
|
-
'qe-
|
|
948
|
-
'qe-
|
|
949
|
-
'qe-visual-tester', // 5. Check visual regressions
|
|
950
|
-
'qe-quality-gate' // 6. Make GO/NO-GO decision
|
|
203
|
+
'qe-regression-risk-analyzer', // Analyze changes, select tests
|
|
204
|
+
'qe-test-executor', // Execute selected tests
|
|
205
|
+
'qe-coverage-analyzer', // Analyze coverage gaps
|
|
206
|
+
'qe-quality-gate' // Go/no-go decision
|
|
951
207
|
],
|
|
952
208
|
topology: 'sequential'
|
|
953
209
|
});
|
|
954
|
-
|
|
955
|
-
// Execute full regression workflow
|
|
956
|
-
const result = await regressionFleet.execute({
|
|
957
|
-
pr: 1234,
|
|
958
|
-
release: 'v3.2.0',
|
|
959
|
-
riskTolerance: 'low'
|
|
960
|
-
});
|
|
961
|
-
|
|
962
|
-
// Returns comprehensive regression report
|
|
963
|
-
```
|
|
964
|
-
|
|
965
|
-
### Production Regression Detection
|
|
966
|
-
|
|
967
|
-
**qe-production-intelligence** detects regressions in production:
|
|
968
|
-
|
|
969
|
-
```typescript
|
|
970
|
-
// Monitor production for regression signals
|
|
971
|
-
await agent.detectProductionRegression({
|
|
972
|
-
baseline: 'v3.1.0',
|
|
973
|
-
current: 'v3.2.0',
|
|
974
|
-
metrics: ['error-rate', 'latency', 'conversion'],
|
|
975
|
-
alertThreshold: 0.1 // 10% degradation
|
|
976
|
-
});
|
|
977
|
-
|
|
978
|
-
// Alerts if:
|
|
979
|
-
// - Error rate increases > 10%
|
|
980
|
-
// - Latency degrades > 10%
|
|
981
|
-
// - Conversion drops > 10%
|
|
982
|
-
|
|
983
|
-
// Enables fast rollback before major impact
|
|
984
210
|
```
|
|
985
211
|
|
|
986
212
|
---
|
|
987
213
|
|
|
988
214
|
## Related Skills
|
|
989
|
-
|
|
990
|
-
|
|
991
|
-
- [
|
|
992
|
-
- [holistic-testing-pact](../holistic-testing-pact/) - Regression across test quadrants
|
|
993
|
-
- [risk-based-testing](../risk-based-testing/) - Risk-based regression prioritization
|
|
994
|
-
|
|
995
|
-
**Testing Approaches:**
|
|
996
|
-
- [test-automation-strategy](../test-automation-strategy/) - Building automation pyramid with regression
|
|
997
|
-
- [api-testing-patterns](../api-testing-patterns/) - API regression testing
|
|
998
|
-
- [exploratory-testing-advanced](../exploratory-testing-advanced/) - Exploratory regression sessions
|
|
999
|
-
- [performance-testing](../performance-testing/) - Performance regression testing
|
|
1000
|
-
- [security-testing](../security-testing/) - Security regression scans
|
|
1001
|
-
|
|
1002
|
-
**Development Practices:**
|
|
1003
|
-
- [tdd-london-chicago](../tdd-london-chicago/) - Test-first creates regression safety
|
|
1004
|
-
- [xp-practices](../xp-practices/) - Continuous integration with regression
|
|
1005
|
-
- [refactoring-patterns](../refactoring-patterns/) - Safe refactoring with regression coverage
|
|
1006
|
-
|
|
1007
|
-
**Communication:**
|
|
1008
|
-
- [quality-metrics](../quality-metrics/) - Regression test effectiveness metrics
|
|
1009
|
-
- [bug-reporting-excellence](../bug-reporting-excellence/) - Regression test creation from bugs
|
|
215
|
+
- [risk-based-testing](../risk-based-testing/) - Risk-based prioritization
|
|
216
|
+
- [test-automation-strategy](../test-automation-strategy/) - Automation pyramid
|
|
217
|
+
- [continuous-testing-shift-left](../continuous-testing-shift-left/) - CI/CD integration
|
|
1010
218
|
|
|
1011
219
|
---
|
|
1012
220
|
|
|
1013
221
|
## Remember
|
|
1014
222
|
|
|
1015
|
-
**Regression testing is insurance against change.**
|
|
1016
|
-
|
|
1017
|
-
Every code change is a risk. Regression testing mitigates that risk by:
|
|
1018
|
-
- Verifying existing functionality still works
|
|
1019
|
-
- Catching unintended side effects early
|
|
1020
|
-
- Building confidence for continuous deployment
|
|
1021
|
-
- Enabling safe refactoring and evolution
|
|
1022
|
-
|
|
1023
|
-
**Good regression testing is strategic, not exhaustive.**
|
|
1024
|
-
|
|
1025
|
-
You cannot test everything, every time. Smart regression testing:
|
|
1026
|
-
- Selects tests based on changes and risk
|
|
1027
|
-
- Runs fast enough for continuous feedback
|
|
1028
|
-
- Maintains reliability (no flaky tests)
|
|
1029
|
-
- Evolves with the product
|
|
223
|
+
**Regression testing is insurance against change.** Every code change is a risk. Smart regression testing mitigates that risk by testing what matters based on what changed.
|
|
1030
224
|
|
|
1031
|
-
**
|
|
225
|
+
**Good regression testing is strategic, not exhaustive.** You cannot test everything, every time. Select based on changes, risk, and time budget.
|
|
1032
226
|
|
|
1033
|
-
**With Agents
|
|
227
|
+
**With Agents:** `qe-regression-risk-analyzer` provides intelligent test selection achieving 90% defect detection in 10% of execution time. Agents generate regression tests from production bugs automatically.
|