agentic-team-templates 0.5.0 → 0.6.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -0,0 +1,1134 @@
1
+ # Testing Development Guide
2
+
3
+ Comprehensive guidelines for building world-class test suites that provide confidence, enable rapid delivery, and catch defects early.
4
+
5
+ ---
6
+
7
+ ## Overview
8
+
9
+ This guide applies to:
10
+ - Unit testing
11
+ - Integration testing
12
+ - End-to-end testing
13
+ - Performance testing
14
+ - Contract testing
15
+ - Property-based testing
16
+ - Mutation testing
17
+
18
+ ### Core Philosophy
19
+
20
+ **Tests are a first-class deliverable.** They are not an afterthought, a checkbox, or technical debt to be addressed "later." A feature without tests is incomplete.
21
+
22
+ ### Key Principles
23
+
24
+ 1. **Test Behavior, Not Implementation** - Tests should verify what the system does, not how it does it
25
+ 2. **Testing Trophy Over Pyramid** - Prioritize integration tests for maximum confidence
26
+ 3. **Tests as Documentation** - Tests are executable specifications
27
+ 4. **Fast Feedback Loops** - Tests must run quickly to be useful
28
+ 5. **Deterministic Results** - Same inputs must produce same outputs, always
29
+
30
+ ### Testing Trophy Distribution
31
+
32
+ ```
33
+ ┌───────────────────┐
34
+ │ End-to-End │ ~10%
35
+ │ (Critical Paths)│
36
+ ├───────────────────┤
37
+ │ │
38
+ │ Integration │ ~60%
39
+ │ Tests │
40
+ │ │
41
+ ├───────────────────┤
42
+ │ Unit Tests │ ~20%
43
+ ├───────────────────┤
44
+ │ Static Analysis │ ~10%
45
+ └───────────────────┘
46
+ ```
47
+
48
+ ---
49
+
50
+ ## TDD Methodology
51
+
52
+ ### Red-Green-Refactor Cycle
53
+
54
+ ```
55
+ ┌─────────────────────────────────────────────────────────────┐
56
+ │ │
57
+ │ ┌─────────┐ ┌─────────┐ ┌─────────────┐ │
58
+ │ │ RED │ ───► │ GREEN │ ───► │ REFACTOR │ ───┐ │
59
+ │ │ (Write │ │ (Make │ │ (Improve │ │ │
60
+ │ │ failing│ │ it │ │ code │ │ │
61
+ │ │ test) │ │ pass) │ │ quality) │ │ │
62
+ │ └─────────┘ └─────────┘ └─────────────┘ │ │
63
+ │ ▲ │ │
64
+ │ └────────────────────────────────────────────────┘ │
65
+ │ │
66
+ └─────────────────────────────────────────────────────────────┘
67
+ ```
68
+
69
+ ### TDD Best Practices
70
+
71
+ 1. **Write the test first** - Forces you to think about the interface before implementation
72
+ 2. **Keep cycles short** - Each cycle should be 2-10 minutes
73
+ 3. **One assertion per test** - Each test verifies one behavior
74
+ 4. **Never refactor on red** - Only refactor when tests pass
75
+ 5. **Commit after each green** - Small, atomic commits
76
+
77
+ ### When TDD Adds Value
78
+
79
+ - Complex business logic
80
+ - Long-term projects with stable requirements
81
+ - Systems with many integrations
82
+ - Code that handles edge cases
83
+ - Security-critical code
84
+
85
+ ### When to Skip TDD
86
+
87
+ - Rapid prototypes (but throw them away)
88
+ - Exploratory spikes
89
+ - One-off scripts
90
+
91
+ ---
92
+
93
+ ## Test Types
94
+
95
+ ### Static Analysis (~10% of effort)
96
+
97
+ Catch errors before runtime.
98
+
99
+ ```ts
100
+ // TypeScript strict mode catches nullability issues
101
+ function greet(name: string): string {
102
+ return `Hello, ${name}!`;
103
+ }
104
+
105
+ greet(null); // Error: Argument of type 'null' is not assignable
106
+ ```
107
+
108
+ **Tools:**
109
+ - TypeScript (strict mode)
110
+ - ESLint with strict rules
111
+ - Prettier for formatting
112
+ - Biome for fast linting
113
+
114
+ ### Unit Tests (~20% of effort)
115
+
116
+ Test pure functions and isolated logic.
117
+
118
+ ```ts
119
+ // Pure function - perfect for unit testing
120
+ import { describe, it, expect } from 'vitest';
121
+
122
+ describe('calculateTax', () => {
123
+ it('calculates 10% tax for standard rate', () => {
124
+ expect(calculateTax(100, 'standard')).toBe(110);
125
+ });
126
+
127
+ it('calculates 0% tax for exempt items', () => {
128
+ expect(calculateTax(100, 'exempt')).toBe(100);
129
+ });
130
+
131
+ it('handles zero amount', () => {
132
+ expect(calculateTax(0, 'standard')).toBe(0);
133
+ });
134
+
135
+ it('throws for negative amount', () => {
136
+ expect(() => calculateTax(-100, 'standard')).toThrow('Amount must be positive');
137
+ });
138
+ });
139
+ ```
140
+
141
+ **Best Practices:**
142
+ - Test edge cases (zero, empty, null, boundaries)
143
+ - Test error conditions explicitly
144
+ - Use parameterized tests for variations
145
+ - Keep tests focused and fast
146
+
147
+ ### Integration Tests (~60% of effort)
148
+
149
+ Test components working together. This is where you get the most value.
150
+
151
+ ```ts
152
+ // Integration test with real database
153
+ import { describe, it, expect, beforeAll, afterAll, beforeEach } from 'vitest';
154
+ import { db } from '../lib/db';
155
+ import { userService } from '../services/userService';
156
+
157
+ describe('UserService', () => {
158
+ beforeAll(async () => {
159
+ await db.$connect();
160
+ });
161
+
162
+ afterAll(async () => {
163
+ await db.$disconnect();
164
+ });
165
+
166
+ beforeEach(async () => {
167
+ await db.user.deleteMany();
168
+ });
169
+
170
+ it('creates user with hashed password', async () => {
171
+ const user = await userService.create({
172
+ email: 'test@example.com',
173
+ password: 'securePassword123',
174
+ });
175
+
176
+ expect(user.id).toBeDefined();
177
+ expect(user.email).toBe('test@example.com');
178
+ expect(user.password).not.toBe('securePassword123'); // Hashed
179
+ });
180
+
181
+ it('prevents duplicate emails', async () => {
182
+ await userService.create({ email: 'test@example.com', password: 'pass1' });
183
+
184
+ await expect(
185
+ userService.create({ email: 'test@example.com', password: 'pass2' })
186
+ ).rejects.toThrow('Email already exists');
187
+ });
188
+
189
+ it('finds user by email', async () => {
190
+ const created = await userService.create({
191
+ email: 'test@example.com',
192
+ password: 'pass',
193
+ });
194
+
195
+ const found = await userService.findByEmail('test@example.com');
196
+
197
+ expect(found?.id).toBe(created.id);
198
+ });
199
+ });
200
+ ```
201
+
202
+ **What to Test:**
203
+ - Service layer with real repositories
204
+ - API routes with real database
205
+ - Message handlers with real queues
206
+ - Multiple components interacting
207
+
208
+ ### End-to-End Tests (~10% of effort)
209
+
210
+ Test critical user journeys only.
211
+
212
+ ```ts
213
+ // Playwright E2E test
214
+ import { test, expect } from '@playwright/test';
215
+
216
+ test.describe('Checkout Flow', () => {
217
+ test('user can complete purchase', async ({ page }) => {
218
+ // Arrange
219
+ await page.goto('/products');
220
+
221
+ // Add to cart
222
+ await page.click('[data-testid="product-1"] >> text=Add to Cart');
223
+ await expect(page.locator('[data-testid="cart-count"]')).toHaveText('1');
224
+
225
+ // Go to checkout
226
+ await page.click('text=Checkout');
227
+ await expect(page).toHaveURL('/checkout');
228
+
229
+ // Fill shipping info
230
+ await page.fill('[name="address"]', '123 Main St');
231
+ await page.fill('[name="city"]', 'Anytown');
232
+ await page.fill('[name="zip"]', '12345');
233
+
234
+ // Complete purchase
235
+ await page.click('text=Place Order');
236
+
237
+ // Verify success
238
+ await expect(page.locator('h1')).toHaveText('Order Confirmed');
239
+ await expect(page.locator('[data-testid="order-number"]')).toBeVisible();
240
+ });
241
+ });
242
+ ```
243
+
244
+ **What to Test:**
245
+ - Critical revenue paths (checkout, signup)
246
+ - Authentication flows
247
+ - Core user journeys
248
+ - Cross-browser compatibility (sparse)
249
+
250
+ **What NOT to Test:**
251
+ - Every UI permutation
252
+ - Form validation (use integration tests)
253
+ - Edge cases (use unit/integration tests)
254
+
255
+ ---
256
+
257
+ ## Test Design Patterns
258
+
259
+ ### Arrange-Act-Assert (AAA)
260
+
261
+ ```ts
262
+ it('calculates order total with discount', () => {
263
+ // Arrange - Set up test data
264
+ const items = [
265
+ { price: 100, quantity: 2 },
266
+ { price: 50, quantity: 1 },
267
+ ];
268
+ const discount = { type: 'percentage', value: 10 };
269
+
270
+ // Act - Execute the code under test
271
+ const total = calculateOrderTotal(items, discount);
272
+
273
+ // Assert - Verify the result
274
+ expect(total).toBe(225); // (200 + 50) * 0.9
275
+ });
276
+ ```
277
+
278
+ ### Given-When-Then (BDD)
279
+
280
+ ```ts
281
+ describe('Shopping Cart', () => {
282
+ describe('given items in cart', () => {
283
+ describe('when removing an item', () => {
284
+ it('then cart count decreases', () => {
285
+ const cart = createCart([item1, item2]);
286
+
287
+ cart.remove(item1.id);
288
+
289
+ expect(cart.count).toBe(1);
290
+ });
291
+ });
292
+ });
293
+ });
294
+ ```
295
+
296
+ ### Test Factories
297
+
298
+ ```ts
299
+ // factories/user.ts
300
+ import { faker } from '@faker-js/faker';
301
+
302
+ interface UserFactoryOptions {
303
+ email?: string;
304
+ role?: 'user' | 'admin';
305
+ verified?: boolean;
306
+ }
307
+
308
+ export const createTestUser = (overrides: UserFactoryOptions = {}) => ({
309
+ id: faker.string.uuid(),
310
+ email: overrides.email ?? faker.internet.email(),
311
+ name: faker.person.fullName(),
312
+ role: overrides.role ?? 'user',
313
+ verified: overrides.verified ?? true,
314
+ createdAt: faker.date.past(),
315
+ });
316
+
317
+ // Usage
318
+ const admin = createTestUser({ role: 'admin' });
319
+ const unverified = createTestUser({ verified: false });
320
+ ```
321
+
322
+ ### Test Builders
323
+
324
+ ```ts
325
+ // builders/OrderBuilder.ts
326
+ export class OrderBuilder {
327
+ private order: Partial<Order> = {};
328
+
329
+ withId(id: string) {
330
+ this.order.id = id;
331
+ return this;
332
+ }
333
+
334
+ withItems(items: OrderItem[]) {
335
+ this.order.items = items;
336
+ return this;
337
+ }
338
+
339
+ withStatus(status: OrderStatus) {
340
+ this.order.status = status;
341
+ return this;
342
+ }
343
+
344
+ withDiscount(discount: Discount) {
345
+ this.order.discount = discount;
346
+ return this;
347
+ }
348
+
349
+ build(): Order {
350
+ return {
351
+ id: this.order.id ?? faker.string.uuid(),
352
+ items: this.order.items ?? [],
353
+ status: this.order.status ?? 'pending',
354
+ discount: this.order.discount,
355
+ createdAt: new Date(),
356
+ } as Order;
357
+ }
358
+ }
359
+
360
+ // Usage
361
+ const order = new OrderBuilder()
362
+ .withStatus('completed')
363
+ .withItems([itemFactory()])
364
+ .withDiscount({ type: 'fixed', value: 10 })
365
+ .build();
366
+ ```
367
+
368
+ ### Custom Matchers
369
+
370
+ ```ts
371
+ // test/matchers.ts
372
+ import { expect } from 'vitest';
373
+
374
+ expect.extend({
375
+ toBeValidEmail(received: string) {
376
+ const emailRegex = /^[^\s@]+@[^\s@]+\.[^\s@]+$/;
377
+ const pass = emailRegex.test(received);
378
+ return {
379
+ pass,
380
+ message: () =>
381
+ pass
382
+ ? `Expected ${received} not to be a valid email`
383
+ : `Expected ${received} to be a valid email`,
384
+ };
385
+ },
386
+
387
+ toBeWithinRange(received: number, floor: number, ceiling: number) {
388
+ const pass = received >= floor && received <= ceiling;
389
+ return {
390
+ pass,
391
+ message: () =>
392
+ pass
393
+ ? `Expected ${received} not to be within range ${floor} - ${ceiling}`
394
+ : `Expected ${received} to be within range ${floor} - ${ceiling}`,
395
+ };
396
+ },
397
+ });
398
+
399
+ // Usage
400
+ expect(user.email).toBeValidEmail();
401
+ expect(score).toBeWithinRange(0, 100);
402
+ ```
403
+
404
+ ---
405
+
406
+ ## Test Data Management
407
+
408
+ ### Factories with Faker
409
+
410
+ ```ts
411
+ // factories/index.ts
412
+ import { faker } from '@faker-js/faker';
413
+
414
+ // Seed for deterministic tests
415
+ faker.seed(12345);
416
+
417
+ export const factories = {
418
+ user: (overrides = {}) => ({
419
+ id: faker.string.uuid(),
420
+ email: faker.internet.email(),
421
+ name: faker.person.fullName(),
422
+ createdAt: faker.date.past(),
423
+ ...overrides,
424
+ }),
425
+
426
+ product: (overrides = {}) => ({
427
+ id: faker.string.uuid(),
428
+ name: faker.commerce.productName(),
429
+ price: parseFloat(faker.commerce.price()),
430
+ sku: faker.string.alphanumeric(10),
431
+ ...overrides,
432
+ }),
433
+
434
+ order: (overrides = {}) => ({
435
+ id: faker.string.uuid(),
436
+ userId: faker.string.uuid(),
437
+ items: [],
438
+ total: 0,
439
+ status: 'pending',
440
+ createdAt: new Date(),
441
+ ...overrides,
442
+ }),
443
+ };
444
+ ```
445
+
446
+ ### Database Fixtures
447
+
448
+ ```ts
449
+ // fixtures/setup.ts
450
+ import { db } from '../lib/db';
451
+ import { factories } from './factories';
452
+
453
+ export async function seedTestDatabase() {
454
+ // Clear all data
455
+ await db.$transaction([
456
+ db.orderItem.deleteMany(),
457
+ db.order.deleteMany(),
458
+ db.product.deleteMany(),
459
+ db.user.deleteMany(),
460
+ ]);
461
+
462
+ // Seed users
463
+ const users = await Promise.all([
464
+ db.user.create({ data: factories.user({ email: 'admin@test.com', role: 'admin' }) }),
465
+ db.user.create({ data: factories.user({ email: 'user@test.com', role: 'user' }) }),
466
+ ]);
467
+
468
+ // Seed products
469
+ const products = await Promise.all(
470
+ Array.from({ length: 10 }, () =>
471
+ db.product.create({ data: factories.product() })
472
+ )
473
+ );
474
+
475
+ return { users, products };
476
+ }
477
+
478
+ export async function cleanupTestDatabase() {
479
+ await db.$transaction([
480
+ db.orderItem.deleteMany(),
481
+ db.order.deleteMany(),
482
+ db.product.deleteMany(),
483
+ db.user.deleteMany(),
484
+ ]);
485
+ }
486
+ ```
487
+
488
+ ### Snapshot Testing (Use Sparingly)
489
+
490
+ ```ts
491
+ // Only for stable, complex output
492
+ it('renders user profile correctly', () => {
493
+ const profile = renderProfile(testUser);
494
+ expect(profile).toMatchSnapshot();
495
+ });
496
+
497
+ // Inline snapshots for small outputs
498
+ it('formats currency correctly', () => {
499
+ expect(formatCurrency(1234.56, 'USD')).toMatchInlineSnapshot(`"$1,234.56"`);
500
+ });
501
+ ```
502
+
503
+ ---
504
+
505
+ ## Quality Metrics
506
+
507
+ ### Coverage Targets (Meaningful, Not Maximum)
508
+
509
+ | Metric | Target | Why |
510
+ |--------|--------|-----|
511
+ | Line Coverage | 80%+ | Baseline hygiene |
512
+ | Branch Coverage | 75%+ | Decision paths tested |
513
+ | Function Coverage | 90%+ | All public APIs tested |
514
+ | **Mutation Score** | 70%+ | Tests actually catch bugs |
515
+
516
+ ### Mutation Testing
517
+
518
+ ```ts
519
+ // stryker.conf.json
520
+ {
521
+ "mutate": ["src/**/*.ts", "!src/**/*.test.ts"],
522
+ "testRunner": "vitest",
523
+ "reporters": ["clear-text", "html"],
524
+ "thresholds": {
525
+ "high": 80,
526
+ "low": 60,
527
+ "break": 50
528
+ }
529
+ }
530
+ ```
531
+
532
+ Mutation testing modifies your code and checks if tests catch the change:
533
+
534
+ ```ts
535
+ // Original
536
+ function isAdult(age: number): boolean {
537
+ return age >= 18;
538
+ }
539
+
540
+ // Mutant 1: Change >= to >
541
+ return age > 18; // Should be caught by test for age 18
542
+
543
+ // Mutant 2: Change 18 to 17
544
+ return age >= 17; // Should be caught by boundary test
545
+ ```
546
+
547
+ ### Quality Gates
548
+
549
+ ```yaml
550
+ # .github/workflows/quality-gates.yml
551
+ quality-gates:
552
+ runs-on: ubuntu-latest
553
+ steps:
554
+ - name: Run Tests
555
+ run: npm test -- --coverage
556
+
557
+ - name: Check Coverage
558
+ run: |
559
+ COVERAGE=$(cat coverage/coverage-summary.json | jq '.total.lines.pct')
560
+ if (( $(echo "$COVERAGE < 80" | bc -l) )); then
561
+ echo "Coverage $COVERAGE% is below 80% threshold"
562
+ exit 1
563
+ fi
564
+
565
+ - name: Run Mutation Tests
566
+ run: npx stryker run
567
+
568
+ - name: Check Mutation Score
569
+ run: |
570
+ SCORE=$(cat reports/mutation/mutation-score.json | jq '.mutationScore')
571
+ if (( $(echo "$SCORE < 70" | bc -l) )); then
572
+ echo "Mutation score $SCORE% is below 70% threshold"
573
+ exit 1
574
+ fi
575
+ ```
576
+
577
+ ---
578
+
579
+ ## Performance Testing
580
+
581
+ ### Load Testing with k6
582
+
583
+ ```js
584
+ // load-tests/checkout.js
585
+ import http from 'k6/http';
586
+ import { check, sleep } from 'k6';
587
+
588
+ export const options = {
589
+ stages: [
590
+ { duration: '2m', target: 100 }, // Ramp up
591
+ { duration: '5m', target: 100 }, // Stay at 100 users
592
+ { duration: '2m', target: 200 }, // Ramp up more
593
+ { duration: '5m', target: 200 }, // Stay at 200 users
594
+ { duration: '2m', target: 0 }, // Ramp down
595
+ ],
596
+ thresholds: {
597
+ http_req_duration: ['p(95)<500'], // 95% of requests under 500ms
598
+ http_req_failed: ['rate<0.01'], // Less than 1% errors
599
+ },
600
+ };
601
+
602
+ export default function () {
603
+ // Get product
604
+ const productRes = http.get('http://api.example.com/products/1');
605
+ check(productRes, { 'product status 200': (r) => r.status === 200 });
606
+
607
+ // Add to cart
608
+ const cartRes = http.post(
609
+ 'http://api.example.com/cart',
610
+ JSON.stringify({ productId: 1, quantity: 1 }),
611
+ { headers: { 'Content-Type': 'application/json' } }
612
+ );
613
+ check(cartRes, { 'cart status 200': (r) => r.status === 200 });
614
+
615
+ sleep(1); // Think time
616
+ }
617
+ ```
618
+
619
+ ### Performance Test Types
620
+
621
+ | Type | Purpose | Duration |
622
+ |------|---------|----------|
623
+ | Smoke | Verify system works | 1-2 min |
624
+ | Load | Normal expected load | 10-30 min |
625
+ | Stress | Beyond normal load | 30-60 min |
626
+ | Spike | Sudden load increase | 10-20 min |
627
+ | Soak | Extended period | 4-24 hours |
628
+
629
+ ---
630
+
631
+ ## Advanced Techniques
632
+
633
+ ### Property-Based Testing
634
+
635
+ Test properties that should hold for all inputs.
636
+
637
+ ```ts
638
+ import { fc } from '@fast-check/vitest';
639
+ import { describe, it, expect } from 'vitest';
640
+
641
+ describe('sort function', () => {
642
+ it.prop([fc.array(fc.integer())])('maintains array length', (arr) => {
643
+ expect(sort(arr).length).toBe(arr.length);
644
+ });
645
+
646
+ it.prop([fc.array(fc.integer())])('is idempotent', (arr) => {
647
+ expect(sort(sort(arr))).toEqual(sort(arr));
648
+ });
649
+
650
+ it.prop([fc.array(fc.integer())])('produces sorted output', (arr) => {
651
+ const sorted = sort(arr);
652
+ for (let i = 1; i < sorted.length; i++) {
653
+ expect(sorted[i]).toBeGreaterThanOrEqual(sorted[i - 1]);
654
+ }
655
+ });
656
+ });
657
+ ```
658
+
659
+ ### Contract Testing (Pact)
660
+
661
+ ```ts
662
+ // consumer.pact.spec.ts
663
+ import { PactV3 } from '@pact-foundation/pact';
664
+ import { UserApiClient } from './userApiClient';
665
+
666
+ const provider = new PactV3({
667
+ consumer: 'OrderService',
668
+ provider: 'UserService',
669
+ });
670
+
671
+ describe('User API Contract', () => {
672
+ it('gets user by ID', async () => {
673
+ await provider
674
+ .given('user 123 exists')
675
+ .uponReceiving('a request for user 123')
676
+ .withRequest({
677
+ method: 'GET',
678
+ path: '/users/123',
679
+ })
680
+ .willRespondWith({
681
+ status: 200,
682
+ body: {
683
+ id: '123',
684
+ name: 'John Doe',
685
+ email: 'john@example.com',
686
+ },
687
+ });
688
+
689
+ await provider.executeTest(async (mockServer) => {
690
+ const client = new UserApiClient(mockServer.url);
691
+ const user = await client.getUser('123');
692
+
693
+ expect(user.id).toBe('123');
694
+ expect(user.name).toBe('John Doe');
695
+ });
696
+ });
697
+ });
698
+ ```
699
+
700
+ ### Chaos Testing Integration
701
+
702
+ ```ts
703
+ // chaos/network-failure.test.ts
704
+ import { describe, it, expect } from 'vitest';
705
+ import { createChaosProxy } from './chaosProxy';
706
+ import { orderService } from '../services/orderService';
707
+
708
+ describe('Order Service Resilience', () => {
709
+ it('handles payment gateway timeout gracefully', async () => {
710
+ const chaos = createChaosProxy('payment-gateway');
711
+
712
+ // Inject 5 second delay
713
+ chaos.injectLatency(5000);
714
+
715
+ try {
716
+ const result = await orderService.createOrder({
717
+ items: [{ productId: '1', quantity: 1 }],
718
+ userId: 'user-1',
719
+ });
720
+
721
+ // Should timeout and return pending status
722
+ expect(result.status).toBe('pending');
723
+ expect(result.paymentStatus).toBe('timeout');
724
+ } finally {
725
+ chaos.restore();
726
+ }
727
+ });
728
+
729
+ it('retries on transient failures', async () => {
730
+ const chaos = createChaosProxy('inventory-service');
731
+
732
+ // Fail first 2 requests, then succeed
733
+ chaos.injectFailures({ count: 2, status: 503 });
734
+
735
+ try {
736
+ const result = await orderService.checkInventory('product-1');
737
+
738
+ expect(result.available).toBe(true);
739
+ expect(chaos.getRequestCount()).toBe(3); // 2 failures + 1 success
740
+ } finally {
741
+ chaos.restore();
742
+ }
743
+ });
744
+ });
745
+ ```
746
+
747
+ ---
748
+
749
+ ## Flaky Test Prevention
750
+
751
+ ### Root Causes and Solutions
752
+
753
+ | Cause | Solution |
754
+ |-------|----------|
755
+ | Timing dependencies | Use explicit waits, not sleep |
756
+ | Shared state | Isolate test data, reset between tests |
757
+ | Order dependencies | Each test must be independent |
758
+ | External services | Mock or use containers |
759
+ | Race conditions | Avoid async assumptions |
760
+ | Time-based logic | Mock Date/time functions |
761
+
762
+ ### Deterministic Testing
763
+
764
+ ```ts
765
+ // Bad: Non-deterministic
766
+ it('creates user with timestamp', () => {
767
+ const user = createUser({ name: 'Test' });
768
+ expect(user.createdAt).toBeDefined(); // Will vary
769
+ });
770
+
771
+ // Good: Deterministic
772
+ it('creates user with timestamp', () => {
773
+ vi.useFakeTimers();
774
+ vi.setSystemTime(new Date('2025-01-01'));
775
+
776
+ const user = createUser({ name: 'Test' });
777
+
778
+ expect(user.createdAt).toEqual(new Date('2025-01-01'));
779
+
780
+ vi.useRealTimers();
781
+ });
782
+ ```
783
+
784
+ ### Explicit Waits
785
+
786
+ ```ts
787
+ // Bad: Arbitrary sleep
788
+ await page.click('button');
789
+ await page.waitForTimeout(2000);
790
+ expect(await page.textContent('h1')).toBe('Success');
791
+
792
+ // Good: Wait for condition
793
+ await page.click('button');
794
+ await expect(page.locator('h1')).toHaveText('Success', { timeout: 5000 });
795
+ ```
796
+
797
+ ### Test Isolation
798
+
799
+ ```ts
800
+ // vitest.config.ts
801
+ export default {
802
+ test: {
803
+ isolate: true, // Isolate test files
804
+ sequence: {
805
+ shuffle: true, // Randomize order to catch dependencies
806
+ },
807
+ pool: 'forks', // True isolation between tests
808
+ },
809
+ };
810
+ ```
811
+
812
+ ---
813
+
814
+ ## CI/CD Integration
815
+
816
+ ### Test Pipeline
817
+
818
+ ```yaml
819
+ # .github/workflows/test.yml
820
+ name: Test Pipeline
821
+
822
+ on:
823
+ push:
824
+ branches: [main]
825
+ pull_request:
826
+
827
+ jobs:
828
+ static-analysis:
829
+ runs-on: ubuntu-latest
830
+ steps:
831
+ - uses: actions/checkout@v4
832
+ - uses: actions/setup-node@v4
833
+ with:
834
+ node-version: '20'
835
+ cache: 'npm'
836
+
837
+ - run: npm ci
838
+ - run: npm run type-check
839
+ - run: npm run lint
840
+
841
+ unit-tests:
842
+ runs-on: ubuntu-latest
843
+ steps:
844
+ - uses: actions/checkout@v4
845
+ - uses: actions/setup-node@v4
846
+ with:
847
+ node-version: '20'
848
+ cache: 'npm'
849
+
850
+ - run: npm ci
851
+ - run: npm test -- --reporter=junit --outputFile=test-results.xml
852
+
853
+ - uses: actions/upload-artifact@v4
854
+ with:
855
+ name: test-results
856
+ path: test-results.xml
857
+
858
+ integration-tests:
859
+ runs-on: ubuntu-latest
860
+ services:
861
+ postgres:
862
+ image: postgres:15
863
+ env:
864
+ POSTGRES_PASSWORD: test
865
+ options: >-
866
+ --health-cmd pg_isready
867
+ --health-interval 10s
868
+ --health-timeout 5s
869
+ --health-retries 5
870
+ ports:
871
+ - 5432:5432
872
+
873
+ steps:
874
+ - uses: actions/checkout@v4
875
+ - uses: actions/setup-node@v4
876
+ with:
877
+ node-version: '20'
878
+ cache: 'npm'
879
+
880
+ - run: npm ci
881
+ - run: npm run db:migrate
882
+ env:
883
+ DATABASE_URL: postgres://postgres:test@localhost:5432/test
884
+ - run: npm run test:integration
885
+ env:
886
+ DATABASE_URL: postgres://postgres:test@localhost:5432/test
887
+
888
+ e2e-tests:
889
+ runs-on: ubuntu-latest
890
+ steps:
891
+ - uses: actions/checkout@v4
892
+ - uses: actions/setup-node@v4
893
+ with:
894
+ node-version: '20'
895
+ cache: 'npm'
896
+
897
+ - run: npm ci
898
+ - run: npx playwright install --with-deps
899
+ - run: npm run build
900
+ - run: npm run test:e2e
901
+
902
+ - uses: actions/upload-artifact@v4
903
+ if: failure()
904
+ with:
905
+ name: playwright-report
906
+ path: playwright-report/
907
+
908
+ performance-tests:
909
+ runs-on: ubuntu-latest
910
+ if: github.ref == 'refs/heads/main'
911
+ steps:
912
+ - uses: actions/checkout@v4
913
+ - uses: grafana/k6-action@v0.3.1
914
+ with:
915
+ filename: load-tests/smoke.js
916
+ flags: --out json=results.json
917
+
918
+ - uses: actions/upload-artifact@v4
919
+ with:
920
+ name: k6-results
921
+ path: results.json
922
+ ```
923
+
924
+ ### Test Reporting
925
+
926
+ ```ts
927
+ // vitest.config.ts
928
+ export default {
929
+ test: {
930
+ reporters: ['default', 'junit', 'html'],
931
+ outputFile: {
932
+ junit: 'test-results/junit.xml',
933
+ html: 'test-results/report.html',
934
+ },
935
+ coverage: {
936
+ reporter: ['text', 'json', 'html'],
937
+ reportsDirectory: 'coverage',
938
+ },
939
+ },
940
+ };
941
+ ```
942
+
943
+ ---
944
+
945
+ ## Project Structure
946
+
947
+ ```
948
+ src/
949
+ ├── services/
950
+ │ ├── userService.ts
951
+ │ └── userService.test.ts # Co-located unit tests
952
+ ├── repositories/
953
+ │ ├── userRepository.ts
954
+ │ └── userRepository.test.ts # Integration tests
955
+ └── routes/
956
+ ├── users.ts
957
+ └── users.test.ts # API tests
958
+
959
+ tests/
960
+ ├── setup.ts # Global test setup
961
+ ├── factories/ # Test data factories
962
+ │ ├── index.ts
963
+ │ ├── user.ts
964
+ │ └── order.ts
965
+ ├── fixtures/ # Static test data
966
+ │ └── products.json
967
+ ├── mocks/ # Manual mocks
968
+ │ └── emailService.ts
969
+ ├── e2e/ # Playwright tests
970
+ │ ├── checkout.spec.ts
971
+ │ └── auth.spec.ts
972
+ ├── load/ # k6 performance tests
973
+ │ ├── smoke.js
974
+ │ └── stress.js
975
+ └── contracts/ # Pact contract tests
976
+ └── userApi.pact.spec.ts
977
+ ```
978
+
979
+ ---
980
+
981
+ ## Definition of Done
982
+
983
+ A test suite is complete when:
984
+
985
+ - [ ] All critical paths have integration tests
986
+ - [ ] Pure functions have unit tests
987
+ - [ ] Edge cases are explicitly tested
988
+ - [ ] Error conditions are tested
989
+ - [ ] Tests are deterministic (pass 100% on re-run)
990
+ - [ ] No flaky tests in CI (quarantine if found)
991
+ - [ ] Coverage meets thresholds (80%+ lines, 70%+ mutation)
992
+ - [ ] Performance baselines established
993
+ - [ ] Contract tests for external dependencies
994
+ - [ ] Tests run in under 5 minutes (unit + integration)
995
+ - [ ] E2E tests cover critical revenue paths
996
+ - [ ] Test documentation is up to date
997
+
998
+ ---
999
+
1000
+ ## Anti-Patterns to Avoid
1001
+
1002
+ ### 1. Testing Implementation Details
1003
+
1004
+ ```ts
1005
+ // Bad: Testing internals
1006
+ it('calls _processData internally', () => {
1007
+ const spy = vi.spyOn(service, '_processData');
1008
+ service.handleRequest(data);
1009
+ expect(spy).toHaveBeenCalled();
1010
+ });
1011
+
1012
+ // Good: Testing behavior
1013
+ it('returns processed result', () => {
1014
+ const result = service.handleRequest(data);
1015
+ expect(result.processed).toBe(true);
1016
+ });
1017
+ ```
1018
+
1019
+ ### 2. Excessive Mocking
1020
+
1021
+ ```ts
1022
+ // Bad: Mock everything
1023
+ const mockDb = vi.fn();
1024
+ const mockCache = vi.fn();
1025
+ const mockLogger = vi.fn();
1026
+ const mockConfig = vi.fn();
1027
+
1028
+ // Good: Use real dependencies in integration tests
1029
+ // Only mock external services you don't control
1030
+ ```
1031
+
1032
+ ### 3. Shared Mutable State
1033
+
1034
+ ```ts
1035
+ // Bad: Shared state between tests
1036
+ let user: User;
1037
+
1038
+ beforeAll(() => {
1039
+ user = createUser();
1040
+ });
1041
+
1042
+ it('test 1', () => {
1043
+ user.name = 'Changed'; // Mutates shared state
1044
+ });
1045
+
1046
+ it('test 2', () => {
1047
+ expect(user.name).toBe('Original'); // FAILS!
1048
+ });
1049
+
1050
+ // Good: Create fresh state per test
1051
+ beforeEach(() => {
1052
+ user = createUser();
1053
+ });
1054
+ ```
1055
+
1056
+ ### 4. Testing Everything with E2E
1057
+
1058
+ ```ts
1059
+ // Bad: Using E2E for form validation
1060
+ test('email validation', async ({ page }) => {
1061
+ await page.fill('[name=email]', 'invalid');
1062
+ await expect(page.locator('.error')).toBeVisible();
1063
+ });
1064
+
1065
+ // Good: Unit test the validation logic
1066
+ it('rejects invalid email', () => {
1067
+ expect(validateEmail('invalid')).toBe(false);
1068
+ });
1069
+ ```
1070
+
1071
+ ### 5. Ignoring Flaky Tests
1072
+
1073
+ ```ts
1074
+ // Bad: Skip and forget
1075
+ it.skip('sometimes fails', () => { ... });
1076
+
1077
+ // Good: Fix or quarantine with tracking
1078
+ it.todo('fix: race condition in async handler - JIRA-123');
1079
+ ```
1080
+
1081
+ ---
1082
+
1083
+ ## Quick Reference
1084
+
1085
+ ### Vitest Commands
1086
+
1087
+ ```bash
1088
+ # Run all tests
1089
+ npm test
1090
+
1091
+ # Run with coverage
1092
+ npm test -- --coverage
1093
+
1094
+ # Run specific file
1095
+ npm test -- src/services/user.test.ts
1096
+
1097
+ # Run in watch mode
1098
+ npm test -- --watch
1099
+
1100
+ # Run with UI
1101
+ npm test -- --ui
1102
+ ```
1103
+
1104
+ ### Playwright Commands
1105
+
1106
+ ```bash
1107
+ # Run E2E tests
1108
+ npx playwright test
1109
+
1110
+ # Run with UI
1111
+ npx playwright test --ui
1112
+
1113
+ # Run specific test
1114
+ npx playwright test checkout.spec.ts
1115
+
1116
+ # Debug mode
1117
+ npx playwright test --debug
1118
+
1119
+ # Generate tests
1120
+ npx playwright codegen localhost:3000
1121
+ ```
1122
+
1123
+ ### k6 Commands
1124
+
1125
+ ```bash
1126
+ # Run load test
1127
+ k6 run load-tests/smoke.js
1128
+
1129
+ # Run with more VUs
1130
+ k6 run --vus 100 --duration 5m load-tests/stress.js
1131
+
1132
+ # Output to cloud
1133
+ k6 run --out cloud load-tests/smoke.js
1134
+ ```