@assertrx/dino 1.0.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +293 -0
- package/extensions/dino.ts +70 -0
- package/extensions/mabl-mcp.ts +306 -0
- package/extensions/playwright-mcp.ts +142 -0
- package/package.json +47 -0
- package/prompts/create-pr.md +110 -0
- package/prompts/generate-bdd-from-url.md +65 -0
- package/prompts/generate-pom-from-url-cli.md +48 -0
- package/prompts/generate-pom-from-url.md +45 -0
- package/prompts/generate-spec-from-feature.md +64 -0
- package/prompts/heal-playwright-failures.md +106 -0
- package/prompts/setup-playwright.md +81 -0
- package/scripts/start-dino.cmd +8 -0
- package/scripts/start-dino.mjs +226 -0
- package/scripts/start-dino.ps1 +9 -0
- package/scripts/start-dino.sh +10 -0
- package/skills/acli/SKILL.md +292 -0
- package/skills/bdd-gherkin/SKILL.md +552 -0
- package/skills/build-review-interface/SKILL.md +96 -0
- package/skills/error-analysis/SKILL.md +164 -0
- package/skills/eval-audit/SKILL.md +183 -0
- package/skills/evaluate-rag/SKILL.md +177 -0
- package/skills/generate-synthetic-data/SKILL.md +131 -0
- package/skills/mabl-mcp/SKILL.md +79 -0
- package/skills/playwright-cli/SKILL.md +246 -0
- package/skills/playwright-cli/advanced-workflows.md +679 -0
- package/skills/playwright-cli/core-commands.md +375 -0
- package/skills/playwright-cli/device-emulation.md +475 -0
- package/skills/playwright-cli/request-mocking.md +373 -0
- package/skills/playwright-cli/running-custom-code.md +553 -0
- package/skills/playwright-cli/screenshots-and-media.md +457 -0
- package/skills/playwright-cli/session-management.md +425 -0
- package/skills/playwright-cli/storage-and-auth.md +487 -0
- package/skills/playwright-cli/test-generation.md +336 -0
- package/skills/playwright-cli/tracing-and-debugging.md +501 -0
- package/skills/playwright-core/SKILL.md +101 -0
- package/skills/playwright-core/accessibility.md +1517 -0
- package/skills/playwright-core/angular.md +1086 -0
- package/skills/playwright-core/api-testing.md +1623 -0
- package/skills/playwright-core/assertions-and-waiting.md +703 -0
- package/skills/playwright-core/auth-flows.md +1081 -0
- package/skills/playwright-core/authentication.md +1409 -0
- package/skills/playwright-core/browser-apis.md +673 -0
- package/skills/playwright-core/browser-extensions.md +319 -0
- package/skills/playwright-core/canvas-and-webgl.md +494 -0
- package/skills/playwright-core/clock-and-time-mocking.md +427 -0
- package/skills/playwright-core/common-pitfalls.md +1318 -0
- package/skills/playwright-core/component-testing.md +1179 -0
- package/skills/playwright-core/configuration.md +731 -0
- package/skills/playwright-core/crud-testing.md +945 -0
- package/skills/playwright-core/debugging.md +827 -0
- package/skills/playwright-core/drag-and-drop.md +992 -0
- package/skills/playwright-core/electron-testing.md +622 -0
- package/skills/playwright-core/error-and-edge-cases.md +1181 -0
- package/skills/playwright-core/error-index.md +1903 -0
- package/skills/playwright-core/file-operations.md +749 -0
- package/skills/playwright-core/file-upload-download.md +1012 -0
- package/skills/playwright-core/fixtures-and-hooks.md +1072 -0
- package/skills/playwright-core/flaky-tests.md +883 -0
- package/skills/playwright-core/forms-and-validation.md +1056 -0
- package/skills/playwright-core/i18n-and-localization.md +622 -0
- package/skills/playwright-core/iframes-and-shadow-dom.md +488 -0
- package/skills/playwright-core/locator-strategy.md +588 -0
- package/skills/playwright-core/locators.md +743 -0
- package/skills/playwright-core/mobile-and-responsive.md +1669 -0
- package/skills/playwright-core/multi-context-and-popups.md +581 -0
- package/skills/playwright-core/multi-user-and-collaboration.md +477 -0
- package/skills/playwright-core/network-mocking.md +1395 -0
- package/skills/playwright-core/nextjs.md +1013 -0
- package/skills/playwright-core/performance-testing.md +608 -0
- package/skills/playwright-core/react.md +1082 -0
- package/skills/playwright-core/search-and-filter.md +1366 -0
- package/skills/playwright-core/security-testing.md +624 -0
- package/skills/playwright-core/service-workers-and-pwa.md +524 -0
- package/skills/playwright-core/test-architecture.md +569 -0
- package/skills/playwright-core/test-data-management.md +1209 -0
- package/skills/playwright-core/test-organization.md +947 -0
- package/skills/playwright-core/third-party-integrations.md +754 -0
- package/skills/playwright-core/trace-analysis.md +161 -0
- package/skills/playwright-core/visual-regression.md +1027 -0
- package/skills/playwright-core/vue.md +1162 -0
- package/skills/playwright-core/websockets-and-realtime.md +623 -0
- package/skills/playwright-core/when-to-mock.md +827 -0
- package/skills/playwright-pom/SKILL.md +94 -0
- package/skills/playwright-pom/page-object-model.md +921 -0
- package/skills/playwright-pom/pom-vs-fixtures-vs-helpers.md +937 -0
- package/skills/validate-evaluator/SKILL.md +215 -0
- package/skills/write-judge-prompt/SKILL.md +144 -0
- package/themes/dino.json +66 -0
|
@@ -0,0 +1,827 @@
|
|
|
1
|
+
# When to Mock vs Use Real Services
|
|
2
|
+
|
|
3
|
+
> **When to use**: When deciding whether to mock API calls, intercept network requests, or hit real services in your Playwright tests.
|
|
4
|
+
> **Prerequisites**: [core/locators.md](locators.md), [core/assertions-and-waiting.md](assertions-and-waiting.md)
|
|
5
|
+
|
|
6
|
+
## Quick Answer
|
|
7
|
+
|
|
8
|
+
**Mock at the boundary, test your stack end-to-end.** Mock third-party services you do not own (Stripe, SendGrid, OAuth providers, analytics). Never mock your own frontend-to-backend communication. Your tests should prove that YOUR code works, not that third-party APIs are up.
|
|
9
|
+
|
|
10
|
+
## Decision Flowchart
|
|
11
|
+
|
|
12
|
+
```
|
|
13
|
+
Is this service part of YOUR codebase (your API, your backend)?
|
|
14
|
+
├── YES → Do NOT mock. Test the real integration.
|
|
15
|
+
│ ├── Is it slow? → Optimize the service, not the test.
|
|
16
|
+
│ └── Is it flaky? → Fix the service. Flaky infra is a bug.
|
|
17
|
+
└── NO → It's a third-party service.
|
|
18
|
+
├── Is it free, fast, and reliable? (rare)
|
|
19
|
+
│ └── Consider real in CI. Mock if rate-limited.
|
|
20
|
+
├── Is it paid per call? (Stripe, Twilio, SendGrid)
|
|
21
|
+
│ └── ALWAYS mock.
|
|
22
|
+
├── Is it rate-limited? (OAuth, social APIs)
|
|
23
|
+
│ └── ALWAYS mock.
|
|
24
|
+
├── Is it slow or unreliable?
|
|
25
|
+
│ └── ALWAYS mock.
|
|
26
|
+
└── Is it a complex multi-step flow? (OAuth redirect dance)
|
|
27
|
+
└── Mock with HAR recording. Update periodically.
|
|
28
|
+
```
|
|
29
|
+
|
|
30
|
+
## Decision Matrix
|
|
31
|
+
|
|
32
|
+
| Scenario | Mock? | Why | Strategy |
|
|
33
|
+
|---|---|---|---|
|
|
34
|
+
| Your own REST/GraphQL API | Never | This IS the integration you are testing | Hit real API against staging or local dev |
|
|
35
|
+
| Your database (through your API) | Never | Data round-trips are the whole point of E2E | Seed via API or fixtures, never mock DB |
|
|
36
|
+
| Authentication (your auth system) | Mostly no | Auth bugs are critical; test the real flow | Use `storageState` to skip login in most tests, but keep a few real login tests |
|
|
37
|
+
| Stripe / payment gateway | Always | Costs money, rate-limited, flaky in CI | `route.fulfill()` with expected responses |
|
|
38
|
+
| SendGrid / email service | Always | Side effects (real emails), no UI to assert | Mock the API call, verify request payload |
|
|
39
|
+
| OAuth providers (Google, GitHub) | Always | Redirect-heavy, rate-limited, CAPTCHAs | Mock token exchange, test your callback handler |
|
|
40
|
+
| Analytics (Segment, Mixpanel) | Always | Fire-and-forget, no UI impact, slows tests | `route.abort()` or `route.fulfill()` |
|
|
41
|
+
| Maps / geocoding APIs | Always | Rate-limited, paid, slow | Mock with static responses |
|
|
42
|
+
| Feature flags (LaunchDarkly, etc.) | Usually | Control test conditions deterministically | Mock to force specific flag states |
|
|
43
|
+
| CDN / static assets | Never | Already fast, part of your infra | Let them load normally |
|
|
44
|
+
| Flaky external dependency | CI: mock, local: real | Keeps CI green, catches real issues locally | Conditional mocking based on environment |
|
|
45
|
+
| Slow external dependency | Dev: mock, nightly: real | Fast feedback in dev, full integration in nightly | Separate test projects in config |
|
|
46
|
+
|
|
47
|
+
## Mocking Strategies
|
|
48
|
+
|
|
49
|
+
### Full Mock (route.fulfill)
|
|
50
|
+
|
|
51
|
+
**Use when**: You want to completely replace a third-party API response. The most common mocking strategy.
|
|
52
|
+
|
|
53
|
+
**TypeScript**
|
|
54
|
+
```typescript
|
|
55
|
+
import { test, expect } from '@playwright/test';
|
|
56
|
+
|
|
57
|
+
test('checkout flow with mocked payment API', async ({ page }) => {
|
|
58
|
+
// Mock Stripe payment intent creation
|
|
59
|
+
await page.route('**/api/create-payment-intent', (route) => {
|
|
60
|
+
route.fulfill({
|
|
61
|
+
status: 200,
|
|
62
|
+
contentType: 'application/json',
|
|
63
|
+
body: JSON.stringify({
|
|
64
|
+
clientSecret: 'pi_mock_secret_123',
|
|
65
|
+
amount: 9900,
|
|
66
|
+
currency: 'usd',
|
|
67
|
+
}),
|
|
68
|
+
});
|
|
69
|
+
});
|
|
70
|
+
|
|
71
|
+
// Mock Stripe confirmation
|
|
72
|
+
await page.route('**/api/confirm-payment', (route) => {
|
|
73
|
+
route.fulfill({
|
|
74
|
+
status: 200,
|
|
75
|
+
contentType: 'application/json',
|
|
76
|
+
body: JSON.stringify({
|
|
77
|
+
status: 'succeeded',
|
|
78
|
+
receiptUrl: 'https://receipt.example.com/123',
|
|
79
|
+
}),
|
|
80
|
+
});
|
|
81
|
+
});
|
|
82
|
+
|
|
83
|
+
await page.goto('/checkout');
|
|
84
|
+
await page.getByRole('button', { name: 'Pay $99.00' }).click();
|
|
85
|
+
await expect(page.getByText('Payment successful')).toBeVisible();
|
|
86
|
+
});
|
|
87
|
+
|
|
88
|
+
test('handle payment failure gracefully', async ({ page }) => {
|
|
89
|
+
// Mock a declined card response
|
|
90
|
+
await page.route('**/api/confirm-payment', (route) => {
|
|
91
|
+
route.fulfill({
|
|
92
|
+
status: 402,
|
|
93
|
+
contentType: 'application/json',
|
|
94
|
+
body: JSON.stringify({
|
|
95
|
+
error: { code: 'card_declined', message: 'Your card was declined.' },
|
|
96
|
+
}),
|
|
97
|
+
});
|
|
98
|
+
});
|
|
99
|
+
|
|
100
|
+
await page.goto('/checkout');
|
|
101
|
+
await page.getByRole('button', { name: 'Pay $99.00' }).click();
|
|
102
|
+
await expect(page.getByRole('alert')).toContainText('card was declined');
|
|
103
|
+
});
|
|
104
|
+
```
|
|
105
|
+
|
|
106
|
+
**JavaScript**
|
|
107
|
+
```javascript
|
|
108
|
+
const { test, expect } = require('@playwright/test');
|
|
109
|
+
|
|
110
|
+
test('checkout flow with mocked payment API', async ({ page }) => {
|
|
111
|
+
await page.route('**/api/create-payment-intent', (route) => {
|
|
112
|
+
route.fulfill({
|
|
113
|
+
status: 200,
|
|
114
|
+
contentType: 'application/json',
|
|
115
|
+
body: JSON.stringify({
|
|
116
|
+
clientSecret: 'pi_mock_secret_123',
|
|
117
|
+
amount: 9900,
|
|
118
|
+
currency: 'usd',
|
|
119
|
+
}),
|
|
120
|
+
});
|
|
121
|
+
});
|
|
122
|
+
|
|
123
|
+
await page.route('**/api/confirm-payment', (route) => {
|
|
124
|
+
route.fulfill({
|
|
125
|
+
status: 200,
|
|
126
|
+
contentType: 'application/json',
|
|
127
|
+
body: JSON.stringify({
|
|
128
|
+
status: 'succeeded',
|
|
129
|
+
receiptUrl: 'https://receipt.example.com/123',
|
|
130
|
+
}),
|
|
131
|
+
});
|
|
132
|
+
});
|
|
133
|
+
|
|
134
|
+
await page.goto('/checkout');
|
|
135
|
+
await page.getByRole('button', { name: 'Pay $99.00' }).click();
|
|
136
|
+
await expect(page.getByText('Payment successful')).toBeVisible();
|
|
137
|
+
});
|
|
138
|
+
|
|
139
|
+
test('handle payment failure gracefully', async ({ page }) => {
|
|
140
|
+
await page.route('**/api/confirm-payment', (route) => {
|
|
141
|
+
route.fulfill({
|
|
142
|
+
status: 402,
|
|
143
|
+
contentType: 'application/json',
|
|
144
|
+
body: JSON.stringify({
|
|
145
|
+
error: { code: 'card_declined', message: 'Your card was declined.' },
|
|
146
|
+
}),
|
|
147
|
+
});
|
|
148
|
+
});
|
|
149
|
+
|
|
150
|
+
await page.goto('/checkout');
|
|
151
|
+
await page.getByRole('button', { name: 'Pay $99.00' }).click();
|
|
152
|
+
await expect(page.getByRole('alert')).toContainText('card was declined');
|
|
153
|
+
});
|
|
154
|
+
```
|
|
155
|
+
|
|
156
|
+
### Partial Mock (Modify Responses)
|
|
157
|
+
|
|
158
|
+
**Use when**: You want the real API call to happen but need to tweak the response -- injecting error states, adding edge-case data, or overriding a single field.
|
|
159
|
+
|
|
160
|
+
**TypeScript**
|
|
161
|
+
```typescript
|
|
162
|
+
import { test, expect } from '@playwright/test';
|
|
163
|
+
|
|
164
|
+
test('show warning when inventory is low', async ({ page }) => {
|
|
165
|
+
// Let the real API call go through, but override the stock count
|
|
166
|
+
await page.route('**/api/products/*', async (route) => {
|
|
167
|
+
const response = await route.fetch(); // forward to real server
|
|
168
|
+
const body = await response.json();
|
|
169
|
+
|
|
170
|
+
// Modify just the field we care about
|
|
171
|
+
body.stockCount = 2;
|
|
172
|
+
body.lowStockWarning = true;
|
|
173
|
+
|
|
174
|
+
await route.fulfill({
|
|
175
|
+
response, // preserve headers, status
|
|
176
|
+
body: JSON.stringify(body),
|
|
177
|
+
});
|
|
178
|
+
});
|
|
179
|
+
|
|
180
|
+
await page.goto('/products/running-shoes');
|
|
181
|
+
await expect(page.getByText('Only 2 left in stock')).toBeVisible();
|
|
182
|
+
await expect(page.getByRole('button', { name: 'Add to cart' })).toBeEnabled();
|
|
183
|
+
});
|
|
184
|
+
|
|
185
|
+
test('inject additional items into a real API response', async ({ page }) => {
|
|
186
|
+
await page.route('**/api/notifications', async (route) => {
|
|
187
|
+
const response = await route.fetch();
|
|
188
|
+
const body = await response.json();
|
|
189
|
+
|
|
190
|
+
// Append a test notification to whatever real data comes back
|
|
191
|
+
body.notifications.push({
|
|
192
|
+
id: 'test-notif',
|
|
193
|
+
message: 'Your export is ready',
|
|
194
|
+
type: 'success',
|
|
195
|
+
read: false,
|
|
196
|
+
});
|
|
197
|
+
|
|
198
|
+
await route.fulfill({
|
|
199
|
+
response,
|
|
200
|
+
body: JSON.stringify(body),
|
|
201
|
+
});
|
|
202
|
+
});
|
|
203
|
+
|
|
204
|
+
await page.goto('/dashboard');
|
|
205
|
+
await expect(page.getByText('Your export is ready')).toBeVisible();
|
|
206
|
+
});
|
|
207
|
+
```
|
|
208
|
+
|
|
209
|
+
**JavaScript**
|
|
210
|
+
```javascript
|
|
211
|
+
const { test, expect } = require('@playwright/test');
|
|
212
|
+
|
|
213
|
+
test('show warning when inventory is low', async ({ page }) => {
|
|
214
|
+
await page.route('**/api/products/*', async (route) => {
|
|
215
|
+
const response = await route.fetch();
|
|
216
|
+
const body = await response.json();
|
|
217
|
+
|
|
218
|
+
body.stockCount = 2;
|
|
219
|
+
body.lowStockWarning = true;
|
|
220
|
+
|
|
221
|
+
await route.fulfill({
|
|
222
|
+
response,
|
|
223
|
+
body: JSON.stringify(body),
|
|
224
|
+
});
|
|
225
|
+
});
|
|
226
|
+
|
|
227
|
+
await page.goto('/products/running-shoes');
|
|
228
|
+
await expect(page.getByText('Only 2 left in stock')).toBeVisible();
|
|
229
|
+
await expect(page.getByRole('button', { name: 'Add to cart' })).toBeEnabled();
|
|
230
|
+
});
|
|
231
|
+
|
|
232
|
+
test('inject additional items into a real API response', async ({ page }) => {
|
|
233
|
+
await page.route('**/api/notifications', async (route) => {
|
|
234
|
+
const response = await route.fetch();
|
|
235
|
+
const body = await response.json();
|
|
236
|
+
|
|
237
|
+
body.notifications.push({
|
|
238
|
+
id: 'test-notif',
|
|
239
|
+
message: 'Your export is ready',
|
|
240
|
+
type: 'success',
|
|
241
|
+
read: false,
|
|
242
|
+
});
|
|
243
|
+
|
|
244
|
+
await route.fulfill({
|
|
245
|
+
response,
|
|
246
|
+
body: JSON.stringify(body),
|
|
247
|
+
});
|
|
248
|
+
});
|
|
249
|
+
|
|
250
|
+
await page.goto('/dashboard');
|
|
251
|
+
await expect(page.getByText('Your export is ready')).toBeVisible();
|
|
252
|
+
});
|
|
253
|
+
```
|
|
254
|
+
|
|
255
|
+
### Record and Replay (HAR Files)
|
|
256
|
+
|
|
257
|
+
**Use when**: Complex API sequences with many endpoints (OAuth flows, multi-step wizards, dashboard data loading). Record once from a real session, replay deterministically. Update the recording periodically so mocks do not drift from reality.
|
|
258
|
+
|
|
259
|
+
**Recording a HAR file:**
|
|
260
|
+
|
|
261
|
+
**TypeScript**
|
|
262
|
+
```typescript
|
|
263
|
+
import { test } from '@playwright/test';
|
|
264
|
+
|
|
265
|
+
// Record HAR — run this once, then commit the .har file
|
|
266
|
+
test('record API traffic for dashboard', async ({ page }) => {
|
|
267
|
+
await page.routeFromHAR('tests/fixtures/dashboard.har', {
|
|
268
|
+
url: '**/api/**',
|
|
269
|
+
update: true, // record mode: forwards requests and saves responses
|
|
270
|
+
});
|
|
271
|
+
|
|
272
|
+
await page.goto('/dashboard');
|
|
273
|
+
// Interact with the page to capture all relevant API calls
|
|
274
|
+
await page.getByRole('tab', { name: 'Analytics' }).click();
|
|
275
|
+
await page.getByRole('tab', { name: 'Users' }).click();
|
|
276
|
+
await page.getByRole('button', { name: 'Load more' }).click();
|
|
277
|
+
|
|
278
|
+
// HAR file is saved automatically when the page closes
|
|
279
|
+
});
|
|
280
|
+
```
|
|
281
|
+
|
|
282
|
+
**Replaying a HAR file:**
|
|
283
|
+
|
|
284
|
+
**TypeScript**
|
|
285
|
+
```typescript
|
|
286
|
+
import { test, expect } from '@playwright/test';
|
|
287
|
+
|
|
288
|
+
test('dashboard loads with recorded data', async ({ page }) => {
|
|
289
|
+
// Replay mode: serves responses from the HAR file
|
|
290
|
+
await page.routeFromHAR('tests/fixtures/dashboard.har', {
|
|
291
|
+
url: '**/api/**',
|
|
292
|
+
update: false, // replay mode (default)
|
|
293
|
+
});
|
|
294
|
+
|
|
295
|
+
await page.goto('/dashboard');
|
|
296
|
+
await expect(page.getByRole('heading', { name: 'Analytics' })).toBeVisible();
|
|
297
|
+
await expect(page.getByTestId('revenue-chart')).toBeVisible();
|
|
298
|
+
});
|
|
299
|
+
```
|
|
300
|
+
|
|
301
|
+
**JavaScript**
|
|
302
|
+
```javascript
|
|
303
|
+
const { test, expect } = require('@playwright/test');
|
|
304
|
+
|
|
305
|
+
// Record
|
|
306
|
+
test('record API traffic for dashboard', async ({ page }) => {
|
|
307
|
+
await page.routeFromHAR('tests/fixtures/dashboard.har', {
|
|
308
|
+
url: '**/api/**',
|
|
309
|
+
update: true,
|
|
310
|
+
});
|
|
311
|
+
|
|
312
|
+
await page.goto('/dashboard');
|
|
313
|
+
await page.getByRole('tab', { name: 'Analytics' }).click();
|
|
314
|
+
await page.getByRole('tab', { name: 'Users' }).click();
|
|
315
|
+
await page.getByRole('button', { name: 'Load more' }).click();
|
|
316
|
+
});
|
|
317
|
+
|
|
318
|
+
// Replay
|
|
319
|
+
test('dashboard loads with recorded data', async ({ page }) => {
|
|
320
|
+
await page.routeFromHAR('tests/fixtures/dashboard.har', {
|
|
321
|
+
url: '**/api/**',
|
|
322
|
+
update: false,
|
|
323
|
+
});
|
|
324
|
+
|
|
325
|
+
await page.goto('/dashboard');
|
|
326
|
+
await expect(page.getByRole('heading', { name: 'Analytics' })).toBeVisible();
|
|
327
|
+
await expect(page.getByTestId('revenue-chart')).toBeVisible();
|
|
328
|
+
});
|
|
329
|
+
```
|
|
330
|
+
|
|
331
|
+
**HAR maintenance workflow:**
|
|
332
|
+
1. Record HAR files against a known-good staging environment.
|
|
333
|
+
2. Commit `.har` files to version control (they are JSON, diffable).
|
|
334
|
+
3. Re-record monthly or when APIs change. Add a CI reminder or calendar event.
|
|
335
|
+
4. Use `update: true` in a dedicated test file to refresh recordings.
|
|
336
|
+
5. Scope HAR to specific URL patterns (`url: '**/api/v2/**'`) so unrelated requests still hit real servers.
|
|
337
|
+
|
|
338
|
+
### Blocking Unwanted Requests
|
|
339
|
+
|
|
340
|
+
**Use when**: Third-party scripts (analytics, ads, chat widgets) slow down tests and add no value. Block them outright.
|
|
341
|
+
|
|
342
|
+
**TypeScript**
|
|
343
|
+
```typescript
|
|
344
|
+
import { test, expect } from '@playwright/test';
|
|
345
|
+
|
|
346
|
+
test.beforeEach(async ({ page }) => {
|
|
347
|
+
// Block analytics and tracking — they slow tests and add no coverage
|
|
348
|
+
await page.route('**/{google-analytics,segment,hotjar,intercom}.{com,io}/**', (route) => {
|
|
349
|
+
route.abort();
|
|
350
|
+
});
|
|
351
|
+
|
|
352
|
+
// Block all image requests in tests that don't need them
|
|
353
|
+
// await page.route('**/*.{png,jpg,jpeg,gif,svg,webp}', (route) => route.abort());
|
|
354
|
+
});
|
|
355
|
+
|
|
356
|
+
test('page loads fast without third-party scripts', async ({ page }) => {
|
|
357
|
+
await page.goto('/dashboard');
|
|
358
|
+
await expect(page.getByRole('heading', { name: 'Dashboard' })).toBeVisible();
|
|
359
|
+
});
|
|
360
|
+
```
|
|
361
|
+
|
|
362
|
+
**JavaScript**
|
|
363
|
+
```javascript
|
|
364
|
+
const { test, expect } = require('@playwright/test');
|
|
365
|
+
|
|
366
|
+
test.beforeEach(async ({ page }) => {
|
|
367
|
+
await page.route('**/{google-analytics,segment,hotjar,intercom}.{com,io}/**', (route) => {
|
|
368
|
+
route.abort();
|
|
369
|
+
});
|
|
370
|
+
});
|
|
371
|
+
|
|
372
|
+
test('page loads fast without third-party scripts', async ({ page }) => {
|
|
373
|
+
await page.goto('/dashboard');
|
|
374
|
+
await expect(page.getByRole('heading', { name: 'Dashboard' })).toBeVisible();
|
|
375
|
+
});
|
|
376
|
+
```
|
|
377
|
+
|
|
378
|
+
## Real Service Strategies
|
|
379
|
+
|
|
380
|
+
### Against Staging Environment
|
|
381
|
+
|
|
382
|
+
**Use when**: You have a shared staging environment that mirrors production. Best for integration confidence.
|
|
383
|
+
|
|
384
|
+
**TypeScript**
|
|
385
|
+
```typescript
|
|
386
|
+
// playwright.config.ts
|
|
387
|
+
import { defineConfig } from '@playwright/test';
|
|
388
|
+
|
|
389
|
+
export default defineConfig({
|
|
390
|
+
use: {
|
|
391
|
+
baseURL: process.env.CI
|
|
392
|
+
? 'https://staging.yourapp.com'
|
|
393
|
+
: 'http://localhost:3000',
|
|
394
|
+
},
|
|
395
|
+
projects: [
|
|
396
|
+
{
|
|
397
|
+
name: 'integration',
|
|
398
|
+
testMatch: '**/*.integration.spec.ts',
|
|
399
|
+
use: { baseURL: 'https://staging.yourapp.com' },
|
|
400
|
+
},
|
|
401
|
+
{
|
|
402
|
+
name: 'e2e',
|
|
403
|
+
testMatch: '**/*.e2e.spec.ts',
|
|
404
|
+
use: { baseURL: 'http://localhost:3000' },
|
|
405
|
+
},
|
|
406
|
+
],
|
|
407
|
+
});
|
|
408
|
+
```
|
|
409
|
+
|
|
410
|
+
**JavaScript**
|
|
411
|
+
```javascript
|
|
412
|
+
// playwright.config.js
|
|
413
|
+
const { defineConfig } = require('@playwright/test');
|
|
414
|
+
|
|
415
|
+
module.exports = defineConfig({
|
|
416
|
+
use: {
|
|
417
|
+
baseURL: process.env.CI
|
|
418
|
+
? 'https://staging.yourapp.com'
|
|
419
|
+
: 'http://localhost:3000',
|
|
420
|
+
},
|
|
421
|
+
projects: [
|
|
422
|
+
{
|
|
423
|
+
name: 'integration',
|
|
424
|
+
testMatch: '**/*.integration.spec.js',
|
|
425
|
+
use: { baseURL: 'https://staging.yourapp.com' },
|
|
426
|
+
},
|
|
427
|
+
{
|
|
428
|
+
name: 'e2e',
|
|
429
|
+
testMatch: '**/*.e2e.spec.js',
|
|
430
|
+
use: { baseURL: 'http://localhost:3000' },
|
|
431
|
+
},
|
|
432
|
+
],
|
|
433
|
+
});
|
|
434
|
+
```
|
|
435
|
+
|
|
436
|
+
### Against Local Dev Server
|
|
437
|
+
|
|
438
|
+
**Use when**: Fastest feedback loop. Run your backend locally and test against it.
|
|
439
|
+
|
|
440
|
+
**TypeScript**
|
|
441
|
+
```typescript
|
|
442
|
+
// playwright.config.ts
|
|
443
|
+
import { defineConfig } from '@playwright/test';
|
|
444
|
+
|
|
445
|
+
export default defineConfig({
|
|
446
|
+
webServer: {
|
|
447
|
+
command: 'npm run dev',
|
|
448
|
+
url: 'http://localhost:3000',
|
|
449
|
+
reuseExistingServer: !process.env.CI, // start fresh in CI, reuse locally
|
|
450
|
+
timeout: 30_000,
|
|
451
|
+
},
|
|
452
|
+
use: {
|
|
453
|
+
baseURL: 'http://localhost:3000',
|
|
454
|
+
},
|
|
455
|
+
});
|
|
456
|
+
```
|
|
457
|
+
|
|
458
|
+
**JavaScript**
|
|
459
|
+
```javascript
|
|
460
|
+
// playwright.config.js
|
|
461
|
+
const { defineConfig } = require('@playwright/test');
|
|
462
|
+
|
|
463
|
+
module.exports = defineConfig({
|
|
464
|
+
webServer: {
|
|
465
|
+
command: 'npm run dev',
|
|
466
|
+
url: 'http://localhost:3000',
|
|
467
|
+
reuseExistingServer: !process.env.CI,
|
|
468
|
+
timeout: 30_000,
|
|
469
|
+
},
|
|
470
|
+
use: {
|
|
471
|
+
baseURL: 'http://localhost:3000',
|
|
472
|
+
},
|
|
473
|
+
});
|
|
474
|
+
```
|
|
475
|
+
|
|
476
|
+
### Against Test Containers
|
|
477
|
+
|
|
478
|
+
**Use when**: You need a fully isolated environment with databases, caches, and services. Best for reproducible CI runs.
|
|
479
|
+
|
|
480
|
+
**TypeScript**
|
|
481
|
+
```typescript
|
|
482
|
+
// playwright.config.ts
|
|
483
|
+
import { defineConfig } from '@playwright/test';
|
|
484
|
+
|
|
485
|
+
export default defineConfig({
|
|
486
|
+
webServer: {
|
|
487
|
+
command: 'docker compose -f docker-compose.test.yml up --wait',
|
|
488
|
+
url: 'http://localhost:3000/health',
|
|
489
|
+
reuseExistingServer: !process.env.CI,
|
|
490
|
+
timeout: 120_000, // containers take longer to start
|
|
491
|
+
},
|
|
492
|
+
use: {
|
|
493
|
+
baseURL: 'http://localhost:3000',
|
|
494
|
+
},
|
|
495
|
+
// Global teardown to stop containers
|
|
496
|
+
globalTeardown: './tests/global-teardown.ts',
|
|
497
|
+
});
|
|
498
|
+
```
|
|
499
|
+
|
|
500
|
+
```typescript
|
|
501
|
+
// tests/global-teardown.ts
|
|
502
|
+
import { execSync } from 'child_process';
|
|
503
|
+
|
|
504
|
+
export default function globalTeardown() {
|
|
505
|
+
if (process.env.CI) {
|
|
506
|
+
execSync('docker compose -f docker-compose.test.yml down -v');
|
|
507
|
+
}
|
|
508
|
+
}
|
|
509
|
+
```
|
|
510
|
+
|
|
511
|
+
**JavaScript**
|
|
512
|
+
```javascript
|
|
513
|
+
// playwright.config.js
|
|
514
|
+
const { defineConfig } = require('@playwright/test');
|
|
515
|
+
|
|
516
|
+
module.exports = defineConfig({
|
|
517
|
+
webServer: {
|
|
518
|
+
command: 'docker compose -f docker-compose.test.yml up --wait',
|
|
519
|
+
url: 'http://localhost:3000/health',
|
|
520
|
+
reuseExistingServer: !process.env.CI,
|
|
521
|
+
timeout: 120_000,
|
|
522
|
+
},
|
|
523
|
+
use: {
|
|
524
|
+
baseURL: 'http://localhost:3000',
|
|
525
|
+
},
|
|
526
|
+
globalTeardown: './tests/global-teardown.js',
|
|
527
|
+
});
|
|
528
|
+
```
|
|
529
|
+
|
|
530
|
+
```javascript
|
|
531
|
+
// tests/global-teardown.js
|
|
532
|
+
const { execSync } = require('child_process');
|
|
533
|
+
|
|
534
|
+
module.exports = function globalTeardown() {
|
|
535
|
+
if (process.env.CI) {
|
|
536
|
+
execSync('docker compose -f docker-compose.test.yml down -v');
|
|
537
|
+
}
|
|
538
|
+
};
|
|
539
|
+
```
|
|
540
|
+
|
|
541
|
+
## Hybrid Approach
|
|
542
|
+
|
|
543
|
+
The strongest test suites combine real and mocked services. The principle: **mock what you do not own, run what you do.**
|
|
544
|
+
|
|
545
|
+
### Fixture-Based Mock Control
|
|
546
|
+
|
|
547
|
+
Create a fixture that lets individual tests opt into mocking specific services while keeping everything else real.
|
|
548
|
+
|
|
549
|
+
**TypeScript**
|
|
550
|
+
```typescript
|
|
551
|
+
// tests/fixtures/mock-fixtures.ts
|
|
552
|
+
import { test as base } from '@playwright/test';
|
|
553
|
+
|
|
554
|
+
type MockOptions = {
|
|
555
|
+
mockPayments: boolean;
|
|
556
|
+
mockEmail: boolean;
|
|
557
|
+
mockAnalytics: boolean;
|
|
558
|
+
};
|
|
559
|
+
|
|
560
|
+
export const test = base.extend<MockOptions>({
|
|
561
|
+
mockPayments: [true, { option: true }], // default: mock payments
|
|
562
|
+
mockEmail: [true, { option: true }], // default: mock email
|
|
563
|
+
mockAnalytics: [true, { option: true }], // default: mock analytics
|
|
564
|
+
|
|
565
|
+
page: async ({ page, mockPayments, mockEmail, mockAnalytics }, use) => {
|
|
566
|
+
if (mockPayments) {
|
|
567
|
+
await page.route('**/api/payments/**', (route) => {
|
|
568
|
+
route.fulfill({
|
|
569
|
+
status: 200,
|
|
570
|
+
contentType: 'application/json',
|
|
571
|
+
body: JSON.stringify({ status: 'succeeded', id: 'pay_mock_123' }),
|
|
572
|
+
});
|
|
573
|
+
});
|
|
574
|
+
}
|
|
575
|
+
|
|
576
|
+
if (mockEmail) {
|
|
577
|
+
await page.route('**/api/send-email', (route) => {
|
|
578
|
+
route.fulfill({
|
|
579
|
+
status: 200,
|
|
580
|
+
contentType: 'application/json',
|
|
581
|
+
body: JSON.stringify({ messageId: 'msg_mock_456' }),
|
|
582
|
+
});
|
|
583
|
+
});
|
|
584
|
+
}
|
|
585
|
+
|
|
586
|
+
if (mockAnalytics) {
|
|
587
|
+
await page.route('**/{segment,google-analytics,mixpanel}.**/**', (route) => {
|
|
588
|
+
route.abort();
|
|
589
|
+
});
|
|
590
|
+
}
|
|
591
|
+
|
|
592
|
+
await use(page);
|
|
593
|
+
},
|
|
594
|
+
});
|
|
595
|
+
|
|
596
|
+
export { expect } from '@playwright/test';
|
|
597
|
+
```
|
|
598
|
+
|
|
599
|
+
```typescript
|
|
600
|
+
// tests/checkout.spec.ts
|
|
601
|
+
import { test, expect } from './fixtures/mock-fixtures';
|
|
602
|
+
|
|
603
|
+
// Uses defaults: payments mocked, email mocked, analytics blocked
|
|
604
|
+
test('checkout sends confirmation email', async ({ page }) => {
|
|
605
|
+
await page.goto('/checkout');
|
|
606
|
+
await page.getByRole('button', { name: 'Pay $99.00' }).click();
|
|
607
|
+
await expect(page.getByText('Confirmation email sent')).toBeVisible();
|
|
608
|
+
});
|
|
609
|
+
|
|
610
|
+
// Override: test with real payment API (nightly integration test)
|
|
611
|
+
test.describe('nightly integration', () => {
|
|
612
|
+
test.use({ mockPayments: false });
|
|
613
|
+
|
|
614
|
+
test('real payment flow against Stripe test mode', async ({ page }) => {
|
|
615
|
+
await page.goto('/checkout');
|
|
616
|
+
// This hits the real Stripe test API
|
|
617
|
+
await page.getByRole('button', { name: 'Pay $99.00' }).click();
|
|
618
|
+
await expect(page.getByText('Payment successful')).toBeVisible();
|
|
619
|
+
});
|
|
620
|
+
});
|
|
621
|
+
```
|
|
622
|
+
|
|
623
|
+
**JavaScript**
|
|
624
|
+
```javascript
|
|
625
|
+
// tests/fixtures/mock-fixtures.js
|
|
626
|
+
const { test: base } = require('@playwright/test');
|
|
627
|
+
|
|
628
|
+
const test = base.extend({
|
|
629
|
+
mockPayments: [true, { option: true }],
|
|
630
|
+
mockEmail: [true, { option: true }],
|
|
631
|
+
mockAnalytics: [true, { option: true }],
|
|
632
|
+
|
|
633
|
+
page: async ({ page, mockPayments, mockEmail, mockAnalytics }, use) => {
|
|
634
|
+
if (mockPayments) {
|
|
635
|
+
await page.route('**/api/payments/**', (route) => {
|
|
636
|
+
route.fulfill({
|
|
637
|
+
status: 200,
|
|
638
|
+
contentType: 'application/json',
|
|
639
|
+
body: JSON.stringify({ status: 'succeeded', id: 'pay_mock_123' }),
|
|
640
|
+
});
|
|
641
|
+
});
|
|
642
|
+
}
|
|
643
|
+
|
|
644
|
+
if (mockEmail) {
|
|
645
|
+
await page.route('**/api/send-email', (route) => {
|
|
646
|
+
route.fulfill({
|
|
647
|
+
status: 200,
|
|
648
|
+
contentType: 'application/json',
|
|
649
|
+
body: JSON.stringify({ messageId: 'msg_mock_456' }),
|
|
650
|
+
});
|
|
651
|
+
});
|
|
652
|
+
}
|
|
653
|
+
|
|
654
|
+
if (mockAnalytics) {
|
|
655
|
+
await page.route('**/{segment,google-analytics,mixpanel}.**/**', (route) => {
|
|
656
|
+
route.abort();
|
|
657
|
+
});
|
|
658
|
+
}
|
|
659
|
+
|
|
660
|
+
await use(page);
|
|
661
|
+
},
|
|
662
|
+
});
|
|
663
|
+
|
|
664
|
+
module.exports = { test, expect: require('@playwright/test').expect };
|
|
665
|
+
```
|
|
666
|
+
|
|
667
|
+
```javascript
|
|
668
|
+
// tests/checkout.spec.js
|
|
669
|
+
const { test, expect } = require('./fixtures/mock-fixtures');
|
|
670
|
+
|
|
671
|
+
test('checkout sends confirmation email', async ({ page }) => {
|
|
672
|
+
await page.goto('/checkout');
|
|
673
|
+
await page.getByRole('button', { name: 'Pay $99.00' }).click();
|
|
674
|
+
await expect(page.getByText('Confirmation email sent')).toBeVisible();
|
|
675
|
+
});
|
|
676
|
+
|
|
677
|
+
test.describe('nightly integration', () => {
|
|
678
|
+
test.use({ mockPayments: false });
|
|
679
|
+
|
|
680
|
+
test('real payment flow against Stripe test mode', async ({ page }) => {
|
|
681
|
+
await page.goto('/checkout');
|
|
682
|
+
await page.getByRole('button', { name: 'Pay $99.00' }).click();
|
|
683
|
+
await expect(page.getByText('Payment successful')).toBeVisible();
|
|
684
|
+
});
|
|
685
|
+
});
|
|
686
|
+
```
|
|
687
|
+
|
|
688
|
+
### Environment-Based Mocking
|
|
689
|
+
|
|
690
|
+
Split test projects by environment to run mocked tests in every CI push and full-integration tests nightly.
|
|
691
|
+
|
|
692
|
+
**TypeScript**
|
|
693
|
+
```typescript
|
|
694
|
+
// playwright.config.ts
|
|
695
|
+
import { defineConfig } from '@playwright/test';
|
|
696
|
+
|
|
697
|
+
export default defineConfig({
|
|
698
|
+
projects: [
|
|
699
|
+
{
|
|
700
|
+
name: 'fast-ci',
|
|
701
|
+
testMatch: '**/*.spec.ts',
|
|
702
|
+
use: {
|
|
703
|
+
baseURL: 'http://localhost:3000',
|
|
704
|
+
// All external services mocked via fixtures
|
|
705
|
+
},
|
|
706
|
+
},
|
|
707
|
+
{
|
|
708
|
+
name: 'nightly-integration',
|
|
709
|
+
testMatch: '**/*.integration.spec.ts',
|
|
710
|
+
use: {
|
|
711
|
+
baseURL: 'https://staging.yourapp.com',
|
|
712
|
+
// Real services, longer timeouts
|
|
713
|
+
},
|
|
714
|
+
timeout: 120_000,
|
|
715
|
+
},
|
|
716
|
+
],
|
|
717
|
+
});
|
|
718
|
+
```
|
|
719
|
+
|
|
720
|
+
**JavaScript**
|
|
721
|
+
```javascript
|
|
722
|
+
// playwright.config.js
|
|
723
|
+
const { defineConfig } = require('@playwright/test');
|
|
724
|
+
|
|
725
|
+
module.exports = defineConfig({
|
|
726
|
+
projects: [
|
|
727
|
+
{
|
|
728
|
+
name: 'fast-ci',
|
|
729
|
+
testMatch: '**/*.spec.js',
|
|
730
|
+
use: {
|
|
731
|
+
baseURL: 'http://localhost:3000',
|
|
732
|
+
},
|
|
733
|
+
},
|
|
734
|
+
{
|
|
735
|
+
name: 'nightly-integration',
|
|
736
|
+
testMatch: '**/*.integration.spec.js',
|
|
737
|
+
use: {
|
|
738
|
+
baseURL: 'https://staging.yourapp.com',
|
|
739
|
+
},
|
|
740
|
+
timeout: 120_000,
|
|
741
|
+
},
|
|
742
|
+
],
|
|
743
|
+
});
|
|
744
|
+
```
|
|
745
|
+
|
|
746
|
+
### Verifying Mock Accuracy
|
|
747
|
+
|
|
748
|
+
Mock responses drift from real APIs over time. Guard against this.
|
|
749
|
+
|
|
750
|
+
**TypeScript**
|
|
751
|
+
```typescript
|
|
752
|
+
import { test, expect } from '@playwright/test';
|
|
753
|
+
|
|
754
|
+
// Run this weekly or when APIs change — validates that mocks match reality
|
|
755
|
+
test.describe('mock contract validation', () => {
|
|
756
|
+
test('payment mock matches real Stripe test API shape', async ({ request }) => {
|
|
757
|
+
// Hit the real API
|
|
758
|
+
const realResponse = await request.post('/api/create-payment-intent', {
|
|
759
|
+
data: { amount: 9900, currency: 'usd' },
|
|
760
|
+
});
|
|
761
|
+
const realBody = await realResponse.json();
|
|
762
|
+
|
|
763
|
+
// Verify mock has the same shape
|
|
764
|
+
const mockBody = {
|
|
765
|
+
clientSecret: 'pi_mock_secret_123',
|
|
766
|
+
amount: 9900,
|
|
767
|
+
currency: 'usd',
|
|
768
|
+
};
|
|
769
|
+
|
|
770
|
+
// Same keys exist in both
|
|
771
|
+
expect(Object.keys(mockBody).sort()).toEqual(Object.keys(realBody).sort());
|
|
772
|
+
|
|
773
|
+
// Same types for each key
|
|
774
|
+
for (const key of Object.keys(mockBody)) {
|
|
775
|
+
expect(typeof mockBody[key]).toBe(typeof realBody[key]);
|
|
776
|
+
}
|
|
777
|
+
});
|
|
778
|
+
});
|
|
779
|
+
```
|
|
780
|
+
|
|
781
|
+
**JavaScript**
|
|
782
|
+
```javascript
|
|
783
|
+
const { test, expect } = require('@playwright/test');
|
|
784
|
+
|
|
785
|
+
test.describe('mock contract validation', () => {
|
|
786
|
+
test('payment mock matches real Stripe test API shape', async ({ request }) => {
|
|
787
|
+
const realResponse = await request.post('/api/create-payment-intent', {
|
|
788
|
+
data: { amount: 9900, currency: 'usd' },
|
|
789
|
+
});
|
|
790
|
+
const realBody = await realResponse.json();
|
|
791
|
+
|
|
792
|
+
const mockBody = {
|
|
793
|
+
clientSecret: 'pi_mock_secret_123',
|
|
794
|
+
amount: 9900,
|
|
795
|
+
currency: 'usd',
|
|
796
|
+
};
|
|
797
|
+
|
|
798
|
+
expect(Object.keys(mockBody).sort()).toEqual(Object.keys(realBody).sort());
|
|
799
|
+
|
|
800
|
+
for (const key of Object.keys(mockBody)) {
|
|
801
|
+
expect(typeof mockBody[key]).toBe(typeof realBody[key]);
|
|
802
|
+
}
|
|
803
|
+
});
|
|
804
|
+
});
|
|
805
|
+
```
|
|
806
|
+
|
|
807
|
+
## Anti-Patterns
|
|
808
|
+
|
|
809
|
+
| Don't Do This | Problem | Do This Instead |
|
|
810
|
+
|---|---|---|
|
|
811
|
+
| Mock your own API (`page.route('**/api/users', ...)` when you own the `/api/users` endpoint) | You are testing a fiction. Your frontend and backend may be completely incompatible. | Hit your real API. Mock only third-party services behind your API. |
|
|
812
|
+
| Mock everything for speed | Tests pass, app breaks. You have zero integration coverage. | Mock only external boundaries. Optimize your own services for test speed. |
|
|
813
|
+
| Never mock anything | Tests are slow, flaky, and fail when Stripe has an outage. You test third-party uptime, not your code. | Mock third-party services. Your CI should not depend on someone else's infrastructure. |
|
|
814
|
+
| Use outdated mocks that do not match the real API | Mock returns `{ status: "ok" }` but real API returns `{ status: "success", data: {...} }`. Tests pass, production breaks. | Run contract validation tests periodically. Re-record HAR files monthly. |
|
|
815
|
+
| Mock at the wrong layer (intercepting your own frontend HTTP client) | Bypasses request/response serialization, headers, error handling. | Mock at the network level with `page.route()`. This tests your full HTTP client code. |
|
|
816
|
+
| Copy-paste mock responses across dozens of test files | One API change requires updating 40 files. Mocks diverge. | Centralize mocks in fixtures or helper files. Single source of truth. |
|
|
817
|
+
| Mock with `page.evaluate()` to stub `fetch`/`XMLHttpRequest` | Fragile, does not survive navigation, misses service workers. | Use `page.route()` which intercepts at the network layer. |
|
|
818
|
+
| Block all network requests and whitelist | Extremely brittle. Every new API endpoint requires a whitelist update. Tests break on any backend change. | Allow all traffic by default. Selectively mock only the third-party services you need to. |
|
|
819
|
+
|
|
820
|
+
## Related
|
|
821
|
+
|
|
822
|
+
- [core/network-mocking.md](network-mocking.md) -- detailed network interception patterns and API
|
|
823
|
+
- [core/api-testing.md](api-testing.md) -- testing your API directly with `request` context
|
|
824
|
+
- [core/authentication.md](authentication.md) -- when to mock auth vs test real login flows
|
|
825
|
+
- [core/fixtures-and-hooks.md](fixtures-and-hooks.md) -- building reusable mock fixtures
|
|
826
|
+
- [core/configuration.md](configuration.md) -- `webServer`, `baseURL`, and project configuration
|
|
827
|
+
- [ci/ci-github-actions.md](../ci/ci-github-actions.md) -- CI setup for different test tiers
|