@assertrx/dino 1.0.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +293 -0
- package/extensions/dino.ts +70 -0
- package/extensions/mabl-mcp.ts +306 -0
- package/extensions/playwright-mcp.ts +142 -0
- package/package.json +47 -0
- package/prompts/create-pr.md +110 -0
- package/prompts/generate-bdd-from-url.md +65 -0
- package/prompts/generate-pom-from-url-cli.md +48 -0
- package/prompts/generate-pom-from-url.md +45 -0
- package/prompts/generate-spec-from-feature.md +64 -0
- package/prompts/heal-playwright-failures.md +106 -0
- package/prompts/setup-playwright.md +81 -0
- package/scripts/start-dino.cmd +8 -0
- package/scripts/start-dino.mjs +226 -0
- package/scripts/start-dino.ps1 +9 -0
- package/scripts/start-dino.sh +10 -0
- package/skills/acli/SKILL.md +292 -0
- package/skills/bdd-gherkin/SKILL.md +552 -0
- package/skills/build-review-interface/SKILL.md +96 -0
- package/skills/error-analysis/SKILL.md +164 -0
- package/skills/eval-audit/SKILL.md +183 -0
- package/skills/evaluate-rag/SKILL.md +177 -0
- package/skills/generate-synthetic-data/SKILL.md +131 -0
- package/skills/mabl-mcp/SKILL.md +79 -0
- package/skills/playwright-cli/SKILL.md +246 -0
- package/skills/playwright-cli/advanced-workflows.md +679 -0
- package/skills/playwright-cli/core-commands.md +375 -0
- package/skills/playwright-cli/device-emulation.md +475 -0
- package/skills/playwright-cli/request-mocking.md +373 -0
- package/skills/playwright-cli/running-custom-code.md +553 -0
- package/skills/playwright-cli/screenshots-and-media.md +457 -0
- package/skills/playwright-cli/session-management.md +425 -0
- package/skills/playwright-cli/storage-and-auth.md +487 -0
- package/skills/playwright-cli/test-generation.md +336 -0
- package/skills/playwright-cli/tracing-and-debugging.md +501 -0
- package/skills/playwright-core/SKILL.md +101 -0
- package/skills/playwright-core/accessibility.md +1517 -0
- package/skills/playwright-core/angular.md +1086 -0
- package/skills/playwright-core/api-testing.md +1623 -0
- package/skills/playwright-core/assertions-and-waiting.md +703 -0
- package/skills/playwright-core/auth-flows.md +1081 -0
- package/skills/playwright-core/authentication.md +1409 -0
- package/skills/playwright-core/browser-apis.md +673 -0
- package/skills/playwright-core/browser-extensions.md +319 -0
- package/skills/playwright-core/canvas-and-webgl.md +494 -0
- package/skills/playwright-core/clock-and-time-mocking.md +427 -0
- package/skills/playwright-core/common-pitfalls.md +1318 -0
- package/skills/playwright-core/component-testing.md +1179 -0
- package/skills/playwright-core/configuration.md +731 -0
- package/skills/playwright-core/crud-testing.md +945 -0
- package/skills/playwright-core/debugging.md +827 -0
- package/skills/playwright-core/drag-and-drop.md +992 -0
- package/skills/playwright-core/electron-testing.md +622 -0
- package/skills/playwright-core/error-and-edge-cases.md +1181 -0
- package/skills/playwright-core/error-index.md +1903 -0
- package/skills/playwright-core/file-operations.md +749 -0
- package/skills/playwright-core/file-upload-download.md +1012 -0
- package/skills/playwright-core/fixtures-and-hooks.md +1072 -0
- package/skills/playwright-core/flaky-tests.md +883 -0
- package/skills/playwright-core/forms-and-validation.md +1056 -0
- package/skills/playwright-core/i18n-and-localization.md +622 -0
- package/skills/playwright-core/iframes-and-shadow-dom.md +488 -0
- package/skills/playwright-core/locator-strategy.md +588 -0
- package/skills/playwright-core/locators.md +743 -0
- package/skills/playwright-core/mobile-and-responsive.md +1669 -0
- package/skills/playwright-core/multi-context-and-popups.md +581 -0
- package/skills/playwright-core/multi-user-and-collaboration.md +477 -0
- package/skills/playwright-core/network-mocking.md +1395 -0
- package/skills/playwright-core/nextjs.md +1013 -0
- package/skills/playwright-core/performance-testing.md +608 -0
- package/skills/playwright-core/react.md +1082 -0
- package/skills/playwright-core/search-and-filter.md +1366 -0
- package/skills/playwright-core/security-testing.md +624 -0
- package/skills/playwright-core/service-workers-and-pwa.md +524 -0
- package/skills/playwright-core/test-architecture.md +569 -0
- package/skills/playwright-core/test-data-management.md +1209 -0
- package/skills/playwright-core/test-organization.md +947 -0
- package/skills/playwright-core/third-party-integrations.md +754 -0
- package/skills/playwright-core/trace-analysis.md +161 -0
- package/skills/playwright-core/visual-regression.md +1027 -0
- package/skills/playwright-core/vue.md +1162 -0
- package/skills/playwright-core/websockets-and-realtime.md +623 -0
- package/skills/playwright-core/when-to-mock.md +827 -0
- package/skills/playwright-pom/SKILL.md +94 -0
- package/skills/playwright-pom/page-object-model.md +921 -0
- package/skills/playwright-pom/pom-vs-fixtures-vs-helpers.md +937 -0
- package/skills/validate-evaluator/SKILL.md +215 -0
- package/skills/write-judge-prompt/SKILL.md +144 -0
- package/themes/dino.json +66 -0
|
@@ -0,0 +1,1027 @@
|
|
|
1
|
+
# Visual Regression Testing
|
|
2
|
+
|
|
3
|
+
> **When to use**: Catching unintended visual changes -- layout shifts, style regressions, broken responsive designs, theme corruption -- that functional assertions miss. Visual tests answer "does it still look right?" after code changes.
|
|
4
|
+
> **Prerequisites**: [core/configuration.md](configuration.md) for project setup, [core/assertions-and-waiting.md](assertions-and-waiting.md) for assertion basics.
|
|
5
|
+
|
|
6
|
+
## Quick Reference
|
|
7
|
+
|
|
8
|
+
```typescript
|
|
9
|
+
// Page screenshot -- compare entire viewport
|
|
10
|
+
await expect(page).toHaveScreenshot();
|
|
11
|
+
|
|
12
|
+
// Element screenshot -- compare a specific component
|
|
13
|
+
await expect(page.getByTestId('pricing-card')).toHaveScreenshot();
|
|
14
|
+
|
|
15
|
+
// Named snapshot -- explicit file name
|
|
16
|
+
await expect(page).toHaveScreenshot('homepage-hero.png');
|
|
17
|
+
|
|
18
|
+
// With threshold -- allow minor pixel differences
|
|
19
|
+
await expect(page).toHaveScreenshot({ maxDiffPixelRatio: 0.01 });
|
|
20
|
+
|
|
21
|
+
// Mask dynamic content -- hide timestamps, avatars, ads
|
|
22
|
+
await expect(page).toHaveScreenshot({
|
|
23
|
+
mask: [page.getByTestId('timestamp'), page.getByRole('img', { name: 'Avatar' })],
|
|
24
|
+
});
|
|
25
|
+
|
|
26
|
+
// Disable animations -- prevent flaky diffs from CSS transitions
|
|
27
|
+
await expect(page).toHaveScreenshot({ animations: 'disabled' });
|
|
28
|
+
|
|
29
|
+
// Update baselines (CLI)
|
|
30
|
+
npx playwright test --update-snapshots
|
|
31
|
+
```
|
|
32
|
+
|
|
33
|
+
## Patterns
|
|
34
|
+
|
|
35
|
+
### 1. Screenshot Comparison Basics
|
|
36
|
+
|
|
37
|
+
**Use when**: Verifying that a page or component renders correctly after code changes. Best for pages with stable layouts -- landing pages, dashboards, settings panels.
|
|
38
|
+
**Avoid when**: The page is highly dynamic with real-time data, live feeds, or content that changes on every load. Use functional assertions instead.
|
|
39
|
+
|
|
40
|
+
Playwright compares a screenshot taken during the test against a stored baseline (golden) image. On first run, the baseline is created. On subsequent runs, differences cause the test to fail with a visual diff report.
|
|
41
|
+
|
|
42
|
+
**TypeScript**
|
|
43
|
+
```typescript
|
|
44
|
+
import { test, expect } from '@playwright/test';
|
|
45
|
+
|
|
46
|
+
test('homepage renders correctly', async ({ page }) => {
|
|
47
|
+
await page.goto('/');
|
|
48
|
+
|
|
49
|
+
// Full page screenshot comparison
|
|
50
|
+
await expect(page).toHaveScreenshot('homepage.png');
|
|
51
|
+
});
|
|
52
|
+
|
|
53
|
+
test('pricing card matches design', async ({ page }) => {
|
|
54
|
+
await page.goto('/pricing');
|
|
55
|
+
|
|
56
|
+
// Element-level screenshot -- scoped to a single component
|
|
57
|
+
const card = page.getByTestId('pro-plan-card');
|
|
58
|
+
await expect(card).toHaveScreenshot('pro-plan-card.png');
|
|
59
|
+
});
|
|
60
|
+
```
|
|
61
|
+
|
|
62
|
+
**JavaScript**
|
|
63
|
+
```javascript
|
|
64
|
+
const { test, expect } = require('@playwright/test');
|
|
65
|
+
|
|
66
|
+
test('homepage renders correctly', async ({ page }) => {
|
|
67
|
+
await page.goto('/');
|
|
68
|
+
|
|
69
|
+
await expect(page).toHaveScreenshot('homepage.png');
|
|
70
|
+
});
|
|
71
|
+
|
|
72
|
+
test('pricing card matches design', async ({ page }) => {
|
|
73
|
+
await page.goto('/pricing');
|
|
74
|
+
|
|
75
|
+
const card = page.getByTestId('pro-plan-card');
|
|
76
|
+
await expect(card).toHaveScreenshot('pro-plan-card.png');
|
|
77
|
+
});
|
|
78
|
+
```
|
|
79
|
+
|
|
80
|
+
Snapshots are stored alongside tests by default in a folder named `<test-file-name>-snapshots/`. The folder structure includes the project name and platform:
|
|
81
|
+
|
|
82
|
+
```
|
|
83
|
+
tests/
|
|
84
|
+
homepage.spec.ts
|
|
85
|
+
homepage.spec.ts-snapshots/
|
|
86
|
+
homepage-chromium-linux.png
|
|
87
|
+
homepage-chromium-darwin.png
|
|
88
|
+
```
|
|
89
|
+
|
|
90
|
+
### 2. Configuring Thresholds
|
|
91
|
+
|
|
92
|
+
**Use when**: Your UI has minor rendering differences between runs -- anti-aliasing, font hinting, sub-pixel rendering. Thresholds prevent false failures from pixel-level noise.
|
|
93
|
+
**Avoid when**: You want pixel-perfect comparisons (design-system component libraries, icon rendering). Keep thresholds at zero.
|
|
94
|
+
|
|
95
|
+
Three knobs control comparison sensitivity:
|
|
96
|
+
|
|
97
|
+
| Option | What it controls | Good default |
|
|
98
|
+
|---|---|---|
|
|
99
|
+
| `maxDiffPixels` | Absolute number of pixels that can differ | `100` for full pages, `10` for components |
|
|
100
|
+
| `maxDiffPixelRatio` | Fraction of total pixels that can differ (0-1) | `0.01` (1%) for full pages |
|
|
101
|
+
| `threshold` | Per-pixel color difference tolerance (0-1) | `0.2` for most UIs, `0.1` for design systems |
|
|
102
|
+
|
|
103
|
+
**TypeScript**
|
|
104
|
+
```typescript
|
|
105
|
+
import { test, expect } from '@playwright/test';
|
|
106
|
+
|
|
107
|
+
test('dashboard allows minor rendering variance', async ({ page }) => {
|
|
108
|
+
await page.goto('/dashboard');
|
|
109
|
+
|
|
110
|
+
// Allow up to 1% of pixels to differ
|
|
111
|
+
await expect(page).toHaveScreenshot('dashboard.png', {
|
|
112
|
+
maxDiffPixelRatio: 0.01,
|
|
113
|
+
});
|
|
114
|
+
});
|
|
115
|
+
|
|
116
|
+
test('icon renders pixel-perfect', async ({ page }) => {
|
|
117
|
+
await page.goto('/icons');
|
|
118
|
+
|
|
119
|
+
// Strict: zero tolerance
|
|
120
|
+
await expect(page.getByTestId('logo')).toHaveScreenshot('logo.png', {
|
|
121
|
+
maxDiffPixels: 0,
|
|
122
|
+
threshold: 0,
|
|
123
|
+
});
|
|
124
|
+
});
|
|
125
|
+
|
|
126
|
+
test('chart allows anti-aliasing differences', async ({ page }) => {
|
|
127
|
+
await page.goto('/analytics');
|
|
128
|
+
|
|
129
|
+
// Per-pixel color threshold + absolute pixel count cap
|
|
130
|
+
await expect(page.getByTestId('revenue-chart')).toHaveScreenshot('revenue-chart.png', {
|
|
131
|
+
threshold: 0.3,
|
|
132
|
+
maxDiffPixels: 200,
|
|
133
|
+
});
|
|
134
|
+
});
|
|
135
|
+
```
|
|
136
|
+
|
|
137
|
+
**JavaScript**
|
|
138
|
+
```javascript
|
|
139
|
+
const { test, expect } = require('@playwright/test');
|
|
140
|
+
|
|
141
|
+
test('dashboard allows minor rendering variance', async ({ page }) => {
|
|
142
|
+
await page.goto('/dashboard');
|
|
143
|
+
|
|
144
|
+
await expect(page).toHaveScreenshot('dashboard.png', {
|
|
145
|
+
maxDiffPixelRatio: 0.01,
|
|
146
|
+
});
|
|
147
|
+
});
|
|
148
|
+
|
|
149
|
+
test('icon renders pixel-perfect', async ({ page }) => {
|
|
150
|
+
await page.goto('/icons');
|
|
151
|
+
|
|
152
|
+
await expect(page.getByTestId('logo')).toHaveScreenshot('logo.png', {
|
|
153
|
+
maxDiffPixels: 0,
|
|
154
|
+
threshold: 0,
|
|
155
|
+
});
|
|
156
|
+
});
|
|
157
|
+
|
|
158
|
+
test('chart allows anti-aliasing differences', async ({ page }) => {
|
|
159
|
+
await page.goto('/analytics');
|
|
160
|
+
|
|
161
|
+
await expect(page.getByTestId('revenue-chart')).toHaveScreenshot('revenue-chart.png', {
|
|
162
|
+
threshold: 0.3,
|
|
163
|
+
maxDiffPixels: 200,
|
|
164
|
+
});
|
|
165
|
+
});
|
|
166
|
+
```
|
|
167
|
+
|
|
168
|
+
**Global thresholds** in `playwright.config.ts`:
|
|
169
|
+
|
|
170
|
+
```typescript
|
|
171
|
+
// playwright.config.ts
|
|
172
|
+
import { defineConfig } from '@playwright/test';
|
|
173
|
+
|
|
174
|
+
export default defineConfig({
|
|
175
|
+
expect: {
|
|
176
|
+
toHaveScreenshot: {
|
|
177
|
+
maxDiffPixelRatio: 0.01,
|
|
178
|
+
threshold: 0.2,
|
|
179
|
+
animations: 'disabled',
|
|
180
|
+
},
|
|
181
|
+
},
|
|
182
|
+
});
|
|
183
|
+
```
|
|
184
|
+
|
|
185
|
+
### 3. Full Page vs Element Screenshots
|
|
186
|
+
|
|
187
|
+
**Use when**: Deciding scope. Full page catches layout shifts and spacing regressions. Element screenshots isolate components and are more stable.
|
|
188
|
+
**Avoid when**: Neither -- use both strategically.
|
|
189
|
+
|
|
190
|
+
**TypeScript**
|
|
191
|
+
```typescript
|
|
192
|
+
import { test, expect } from '@playwright/test';
|
|
193
|
+
|
|
194
|
+
test('full page screenshot catches layout shifts', async ({ page }) => {
|
|
195
|
+
await page.goto('/');
|
|
196
|
+
|
|
197
|
+
// Captures the visible viewport
|
|
198
|
+
await expect(page).toHaveScreenshot('homepage-viewport.png');
|
|
199
|
+
|
|
200
|
+
// Captures the entire scrollable page
|
|
201
|
+
await expect(page).toHaveScreenshot('homepage-full.png', {
|
|
202
|
+
fullPage: true,
|
|
203
|
+
});
|
|
204
|
+
});
|
|
205
|
+
|
|
206
|
+
test('element screenshot isolates component changes', async ({ page }) => {
|
|
207
|
+
await page.goto('/pricing');
|
|
208
|
+
|
|
209
|
+
// Only the pricing table -- immune to header/footer changes
|
|
210
|
+
await expect(page.getByRole('table')).toHaveScreenshot('pricing-table.png');
|
|
211
|
+
|
|
212
|
+
// A specific card within the page
|
|
213
|
+
await expect(page.getByTestId('enterprise-card')).toHaveScreenshot('enterprise-card.png');
|
|
214
|
+
});
|
|
215
|
+
```
|
|
216
|
+
|
|
217
|
+
**JavaScript**
|
|
218
|
+
```javascript
|
|
219
|
+
const { test, expect } = require('@playwright/test');
|
|
220
|
+
|
|
221
|
+
test('full page screenshot catches layout shifts', async ({ page }) => {
|
|
222
|
+
await page.goto('/');
|
|
223
|
+
|
|
224
|
+
await expect(page).toHaveScreenshot('homepage-viewport.png');
|
|
225
|
+
|
|
226
|
+
await expect(page).toHaveScreenshot('homepage-full.png', {
|
|
227
|
+
fullPage: true,
|
|
228
|
+
});
|
|
229
|
+
});
|
|
230
|
+
|
|
231
|
+
test('element screenshot isolates component changes', async ({ page }) => {
|
|
232
|
+
await page.goto('/pricing');
|
|
233
|
+
|
|
234
|
+
await expect(page.getByRole('table')).toHaveScreenshot('pricing-table.png');
|
|
235
|
+
await expect(page.getByTestId('enterprise-card')).toHaveScreenshot('enterprise-card.png');
|
|
236
|
+
});
|
|
237
|
+
```
|
|
238
|
+
|
|
239
|
+
**Rule of thumb**: Use element screenshots for components that change independently. Use full page screenshots for key landing pages and layouts where spacing between sections matters.
|
|
240
|
+
|
|
241
|
+
### 4. Masking Dynamic Content
|
|
242
|
+
|
|
243
|
+
**Use when**: The page contains content that changes between test runs -- timestamps, user avatars, ad slots, relative dates ("3 minutes ago"), random hero images, A/B test variants.
|
|
244
|
+
**Avoid when**: All content is deterministic. Masking adds maintenance overhead.
|
|
245
|
+
|
|
246
|
+
The `mask` option overlays a solid-colored box over specified locators before taking the screenshot. The masked region is excluded from comparison.
|
|
247
|
+
|
|
248
|
+
**TypeScript**
|
|
249
|
+
```typescript
|
|
250
|
+
import { test, expect } from '@playwright/test';
|
|
251
|
+
|
|
252
|
+
test('dashboard with masked dynamic content', async ({ page }) => {
|
|
253
|
+
await page.goto('/dashboard');
|
|
254
|
+
|
|
255
|
+
await expect(page).toHaveScreenshot('dashboard.png', {
|
|
256
|
+
mask: [
|
|
257
|
+
page.getByTestId('current-time'),
|
|
258
|
+
page.getByTestId('user-avatar'),
|
|
259
|
+
page.getByTestId('live-visitor-count'),
|
|
260
|
+
page.locator('.ad-banner'),
|
|
261
|
+
],
|
|
262
|
+
maskColor: '#FF00FF', // visible magenta -- makes it obvious what's masked in reviews
|
|
263
|
+
});
|
|
264
|
+
});
|
|
265
|
+
|
|
266
|
+
test('feed page with relative timestamps', async ({ page }) => {
|
|
267
|
+
await page.goto('/feed');
|
|
268
|
+
|
|
269
|
+
// Mask all relative time elements at once
|
|
270
|
+
await expect(page).toHaveScreenshot('feed.png', {
|
|
271
|
+
mask: [page.locator('time[datetime]')],
|
|
272
|
+
});
|
|
273
|
+
});
|
|
274
|
+
|
|
275
|
+
test('profile page with user-generated content', async ({ page }) => {
|
|
276
|
+
await page.goto('/profile/test-user');
|
|
277
|
+
|
|
278
|
+
await expect(page).toHaveScreenshot('profile.png', {
|
|
279
|
+
mask: [
|
|
280
|
+
page.getByRole('img', { name: 'Profile photo' }),
|
|
281
|
+
page.getByTestId('last-login'),
|
|
282
|
+
page.getByTestId('member-since'),
|
|
283
|
+
],
|
|
284
|
+
});
|
|
285
|
+
});
|
|
286
|
+
```
|
|
287
|
+
|
|
288
|
+
**JavaScript**
|
|
289
|
+
```javascript
|
|
290
|
+
const { test, expect } = require('@playwright/test');
|
|
291
|
+
|
|
292
|
+
test('dashboard with masked dynamic content', async ({ page }) => {
|
|
293
|
+
await page.goto('/dashboard');
|
|
294
|
+
|
|
295
|
+
await expect(page).toHaveScreenshot('dashboard.png', {
|
|
296
|
+
mask: [
|
|
297
|
+
page.getByTestId('current-time'),
|
|
298
|
+
page.getByTestId('user-avatar'),
|
|
299
|
+
page.getByTestId('live-visitor-count'),
|
|
300
|
+
page.locator('.ad-banner'),
|
|
301
|
+
],
|
|
302
|
+
maskColor: '#FF00FF',
|
|
303
|
+
});
|
|
304
|
+
});
|
|
305
|
+
|
|
306
|
+
test('feed page with relative timestamps', async ({ page }) => {
|
|
307
|
+
await page.goto('/feed');
|
|
308
|
+
|
|
309
|
+
await expect(page).toHaveScreenshot('feed.png', {
|
|
310
|
+
mask: [page.locator('time[datetime]')],
|
|
311
|
+
});
|
|
312
|
+
});
|
|
313
|
+
|
|
314
|
+
test('profile page with user-generated content', async ({ page }) => {
|
|
315
|
+
await page.goto('/profile/test-user');
|
|
316
|
+
|
|
317
|
+
await expect(page).toHaveScreenshot('profile.png', {
|
|
318
|
+
mask: [
|
|
319
|
+
page.getByRole('img', { name: 'Profile photo' }),
|
|
320
|
+
page.getByTestId('last-login'),
|
|
321
|
+
page.getByTestId('member-since'),
|
|
322
|
+
],
|
|
323
|
+
});
|
|
324
|
+
});
|
|
325
|
+
```
|
|
326
|
+
|
|
327
|
+
**Alternative: freeze dynamic content with JavaScript**
|
|
328
|
+
|
|
329
|
+
When masking is not sufficient (e.g., content affects layout), inject JavaScript to freeze the content:
|
|
330
|
+
|
|
331
|
+
```typescript
|
|
332
|
+
test('freeze clock before screenshot', async ({ page }) => {
|
|
333
|
+
await page.goto('/dashboard');
|
|
334
|
+
|
|
335
|
+
// Replace all dynamic timestamps with a fixed value
|
|
336
|
+
await page.evaluate(() => {
|
|
337
|
+
document.querySelectorAll('[data-testid="timestamp"]').forEach((el) => {
|
|
338
|
+
el.textContent = 'Jan 1, 2025 12:00 PM';
|
|
339
|
+
});
|
|
340
|
+
});
|
|
341
|
+
|
|
342
|
+
await expect(page).toHaveScreenshot('dashboard-frozen.png');
|
|
343
|
+
});
|
|
344
|
+
```
|
|
345
|
+
|
|
346
|
+
### 5. Animations Handling
|
|
347
|
+
|
|
348
|
+
**Use when**: Always. CSS animations and transitions are the number one cause of flaky visual diffs. A screenshot captured mid-animation will never match the baseline.
|
|
349
|
+
**Avoid when**: You are explicitly testing the animation itself (rare).
|
|
350
|
+
|
|
351
|
+
**TypeScript**
|
|
352
|
+
```typescript
|
|
353
|
+
import { test, expect } from '@playwright/test';
|
|
354
|
+
|
|
355
|
+
test('page renders without animation interference', async ({ page }) => {
|
|
356
|
+
await page.goto('/');
|
|
357
|
+
|
|
358
|
+
// Disables CSS animations and transitions before screenshotting
|
|
359
|
+
await expect(page).toHaveScreenshot('homepage.png', {
|
|
360
|
+
animations: 'disabled',
|
|
361
|
+
});
|
|
362
|
+
});
|
|
363
|
+
```
|
|
364
|
+
|
|
365
|
+
**JavaScript**
|
|
366
|
+
```javascript
|
|
367
|
+
const { test, expect } = require('@playwright/test');
|
|
368
|
+
|
|
369
|
+
test('page renders without animation interference', async ({ page }) => {
|
|
370
|
+
await page.goto('/');
|
|
371
|
+
|
|
372
|
+
await expect(page).toHaveScreenshot('homepage.png', {
|
|
373
|
+
animations: 'disabled',
|
|
374
|
+
});
|
|
375
|
+
});
|
|
376
|
+
```
|
|
377
|
+
|
|
378
|
+
**Set globally** -- this should be the default for every project:
|
|
379
|
+
|
|
380
|
+
```typescript
|
|
381
|
+
// playwright.config.ts
|
|
382
|
+
import { defineConfig } from '@playwright/test';
|
|
383
|
+
|
|
384
|
+
export default defineConfig({
|
|
385
|
+
expect: {
|
|
386
|
+
toHaveScreenshot: {
|
|
387
|
+
animations: 'disabled',
|
|
388
|
+
},
|
|
389
|
+
},
|
|
390
|
+
});
|
|
391
|
+
```
|
|
392
|
+
|
|
393
|
+
When `animations: 'disabled'` is set, Playwright:
|
|
394
|
+
1. Injects a stylesheet that sets `* { animation-duration: 0s !important; transition-duration: 0s !important; }`.
|
|
395
|
+
2. Waits for any currently running animations to finish (forces them to their end state).
|
|
396
|
+
3. Takes the screenshot.
|
|
397
|
+
|
|
398
|
+
For JavaScript-driven animations (requestAnimationFrame, GSAP, Framer Motion), you may also need to wait for stability:
|
|
399
|
+
|
|
400
|
+
```typescript
|
|
401
|
+
test('page with JS animations', async ({ page }) => {
|
|
402
|
+
await page.goto('/animated-landing');
|
|
403
|
+
|
|
404
|
+
// Wait for the hero animation to settle
|
|
405
|
+
await page.getByTestId('hero-section').waitFor({ state: 'visible' });
|
|
406
|
+
await page.waitForTimeout(500); // last resort for JS animations -- use sparingly
|
|
407
|
+
|
|
408
|
+
await expect(page).toHaveScreenshot('landing.png', {
|
|
409
|
+
animations: 'disabled',
|
|
410
|
+
});
|
|
411
|
+
});
|
|
412
|
+
```
|
|
413
|
+
|
|
414
|
+
### 6. Updating Snapshots
|
|
415
|
+
|
|
416
|
+
**Use when**: You have intentionally changed the UI and need to update the baselines. A design refresh, rebrand, new feature, or layout change.
|
|
417
|
+
**Avoid when**: The diff is unexpected -- investigate the cause before blindly updating.
|
|
418
|
+
|
|
419
|
+
```bash
|
|
420
|
+
# Update all snapshots
|
|
421
|
+
npx playwright test --update-snapshots
|
|
422
|
+
|
|
423
|
+
# Update snapshots for a specific test file
|
|
424
|
+
npx playwright test tests/homepage.spec.ts --update-snapshots
|
|
425
|
+
|
|
426
|
+
# Update snapshots for a specific project (browser)
|
|
427
|
+
npx playwright test --project=chromium --update-snapshots
|
|
428
|
+
```
|
|
429
|
+
|
|
430
|
+
**Workflow for reviewing snapshot changes:**
|
|
431
|
+
|
|
432
|
+
1. Run tests and observe failures in the HTML report:
|
|
433
|
+
```bash
|
|
434
|
+
npx playwright test
|
|
435
|
+
npx playwright show-report
|
|
436
|
+
```
|
|
437
|
+
The report shows the expected image, actual image, and a diff image side-by-side.
|
|
438
|
+
|
|
439
|
+
2. If the changes are intentional, update the snapshots:
|
|
440
|
+
```bash
|
|
441
|
+
npx playwright test --update-snapshots
|
|
442
|
+
```
|
|
443
|
+
|
|
444
|
+
3. Review the updated snapshots in your diff tool before committing:
|
|
445
|
+
```bash
|
|
446
|
+
git diff --name-only # see which snapshot files changed
|
|
447
|
+
```
|
|
448
|
+
|
|
449
|
+
4. Commit the updated snapshots with a clear message explaining why they changed.
|
|
450
|
+
|
|
451
|
+
**Tip**: Never run `--update-snapshots` in CI. Always update locally, review the diffs, and commit the new baselines.
|
|
452
|
+
|
|
453
|
+
**TypeScript -- helper for controlled updates**
|
|
454
|
+
```typescript
|
|
455
|
+
import { test, expect } from '@playwright/test';
|
|
456
|
+
|
|
457
|
+
// Tag visual tests so you can update them selectively
|
|
458
|
+
test('homepage visual @visual', async ({ page }) => {
|
|
459
|
+
await page.goto('/');
|
|
460
|
+
await expect(page).toHaveScreenshot('homepage.png', {
|
|
461
|
+
animations: 'disabled',
|
|
462
|
+
});
|
|
463
|
+
});
|
|
464
|
+
|
|
465
|
+
test('pricing visual @visual', async ({ page }) => {
|
|
466
|
+
await page.goto('/pricing');
|
|
467
|
+
await expect(page).toHaveScreenshot('pricing.png', {
|
|
468
|
+
animations: 'disabled',
|
|
469
|
+
});
|
|
470
|
+
});
|
|
471
|
+
```
|
|
472
|
+
|
|
473
|
+
```bash
|
|
474
|
+
# Update only visual tests
|
|
475
|
+
npx playwright test --grep @visual --update-snapshots
|
|
476
|
+
```
|
|
477
|
+
|
|
478
|
+
### 7. Cross-Browser Visual Testing
|
|
479
|
+
|
|
480
|
+
**Use when**: Your users span Chrome, Firefox, and Safari and you need to verify rendering consistency per browser.
|
|
481
|
+
**Avoid when**: Early stage projects targeting a single browser -- visual tests across three browsers triple the maintenance burden.
|
|
482
|
+
|
|
483
|
+
Playwright automatically separates snapshots by project name. Each browser gets its own baseline file. This is correct behavior -- browsers render fonts, shadows, and anti-aliasing differently.
|
|
484
|
+
|
|
485
|
+
**TypeScript**
|
|
486
|
+
```typescript
|
|
487
|
+
// playwright.config.ts
|
|
488
|
+
import { defineConfig, devices } from '@playwright/test';
|
|
489
|
+
|
|
490
|
+
export default defineConfig({
|
|
491
|
+
testDir: './tests',
|
|
492
|
+
expect: {
|
|
493
|
+
toHaveScreenshot: {
|
|
494
|
+
animations: 'disabled',
|
|
495
|
+
maxDiffPixelRatio: 0.01,
|
|
496
|
+
},
|
|
497
|
+
},
|
|
498
|
+
projects: [
|
|
499
|
+
{
|
|
500
|
+
name: 'chromium',
|
|
501
|
+
use: { ...devices['Desktop Chrome'] },
|
|
502
|
+
},
|
|
503
|
+
{
|
|
504
|
+
name: 'firefox',
|
|
505
|
+
use: { ...devices['Desktop Firefox'] },
|
|
506
|
+
},
|
|
507
|
+
{
|
|
508
|
+
name: 'webkit',
|
|
509
|
+
use: { ...devices['Desktop Safari'] },
|
|
510
|
+
},
|
|
511
|
+
],
|
|
512
|
+
});
|
|
513
|
+
```
|
|
514
|
+
|
|
515
|
+
```typescript
|
|
516
|
+
// tests/homepage.spec.ts
|
|
517
|
+
import { test, expect } from '@playwright/test';
|
|
518
|
+
|
|
519
|
+
test('homepage renders correctly per browser', async ({ page }) => {
|
|
520
|
+
await page.goto('/');
|
|
521
|
+
|
|
522
|
+
// Playwright creates separate baselines per project:
|
|
523
|
+
// homepage.spec.ts-snapshots/homepage-chromium-linux.png
|
|
524
|
+
// homepage.spec.ts-snapshots/homepage-firefox-linux.png
|
|
525
|
+
// homepage.spec.ts-snapshots/homepage-webkit-linux.png
|
|
526
|
+
await expect(page).toHaveScreenshot('homepage.png', {
|
|
527
|
+
animations: 'disabled',
|
|
528
|
+
});
|
|
529
|
+
});
|
|
530
|
+
```
|
|
531
|
+
|
|
532
|
+
**JavaScript**
|
|
533
|
+
```javascript
|
|
534
|
+
const { test, expect } = require('@playwright/test');
|
|
535
|
+
|
|
536
|
+
test('homepage renders correctly per browser', async ({ page }) => {
|
|
537
|
+
await page.goto('/');
|
|
538
|
+
|
|
539
|
+
await expect(page).toHaveScreenshot('homepage.png', {
|
|
540
|
+
animations: 'disabled',
|
|
541
|
+
});
|
|
542
|
+
});
|
|
543
|
+
```
|
|
544
|
+
|
|
545
|
+
**Strategy for managing cross-browser snapshots**: Run visual tests in a single browser (Chromium on Linux in CI) to minimize snapshot count and only add other browsers when you have actual cross-browser rendering bugs. You can scope visual tests to one project:
|
|
546
|
+
|
|
547
|
+
```typescript
|
|
548
|
+
// playwright.config.ts
|
|
549
|
+
import { defineConfig, devices } from '@playwright/test';
|
|
550
|
+
|
|
551
|
+
export default defineConfig({
|
|
552
|
+
projects: [
|
|
553
|
+
{
|
|
554
|
+
name: 'visual',
|
|
555
|
+
testMatch: '**/*.visual.spec.ts',
|
|
556
|
+
use: { ...devices['Desktop Chrome'] },
|
|
557
|
+
},
|
|
558
|
+
{
|
|
559
|
+
name: 'chromium',
|
|
560
|
+
testIgnore: '**/*.visual.spec.ts',
|
|
561
|
+
use: { ...devices['Desktop Chrome'] },
|
|
562
|
+
},
|
|
563
|
+
{
|
|
564
|
+
name: 'firefox',
|
|
565
|
+
testIgnore: '**/*.visual.spec.ts',
|
|
566
|
+
use: { ...devices['Desktop Firefox'] },
|
|
567
|
+
},
|
|
568
|
+
],
|
|
569
|
+
});
|
|
570
|
+
```
|
|
571
|
+
|
|
572
|
+
### 8. Responsive Visual Testing
|
|
573
|
+
|
|
574
|
+
**Use when**: Your application has responsive breakpoints and you need to verify layouts at different viewport sizes.
|
|
575
|
+
**Avoid when**: The page has a single fixed-width layout.
|
|
576
|
+
|
|
577
|
+
**TypeScript**
|
|
578
|
+
```typescript
|
|
579
|
+
import { test, expect } from '@playwright/test';
|
|
580
|
+
|
|
581
|
+
const viewports = [
|
|
582
|
+
{ name: 'mobile', width: 375, height: 812 },
|
|
583
|
+
{ name: 'tablet', width: 768, height: 1024 },
|
|
584
|
+
{ name: 'desktop', width: 1440, height: 900 },
|
|
585
|
+
];
|
|
586
|
+
|
|
587
|
+
for (const viewport of viewports) {
|
|
588
|
+
test(`homepage at ${viewport.name} (${viewport.width}x${viewport.height})`, async ({ page }) => {
|
|
589
|
+
await page.setViewportSize({ width: viewport.width, height: viewport.height });
|
|
590
|
+
await page.goto('/');
|
|
591
|
+
|
|
592
|
+
await expect(page).toHaveScreenshot(`homepage-${viewport.name}.png`, {
|
|
593
|
+
animations: 'disabled',
|
|
594
|
+
fullPage: true,
|
|
595
|
+
});
|
|
596
|
+
});
|
|
597
|
+
}
|
|
598
|
+
```
|
|
599
|
+
|
|
600
|
+
**JavaScript**
|
|
601
|
+
```javascript
|
|
602
|
+
const { test, expect } = require('@playwright/test');
|
|
603
|
+
|
|
604
|
+
const viewports = [
|
|
605
|
+
{ name: 'mobile', width: 375, height: 812 },
|
|
606
|
+
{ name: 'tablet', width: 768, height: 1024 },
|
|
607
|
+
{ name: 'desktop', width: 1440, height: 900 },
|
|
608
|
+
];
|
|
609
|
+
|
|
610
|
+
for (const viewport of viewports) {
|
|
611
|
+
test(`homepage at ${viewport.name} (${viewport.width}x${viewport.height})`, async ({ page }) => {
|
|
612
|
+
await page.setViewportSize({ width: viewport.width, height: viewport.height });
|
|
613
|
+
await page.goto('/');
|
|
614
|
+
|
|
615
|
+
await expect(page).toHaveScreenshot(`homepage-${viewport.name}.png`, {
|
|
616
|
+
animations: 'disabled',
|
|
617
|
+
fullPage: true,
|
|
618
|
+
});
|
|
619
|
+
});
|
|
620
|
+
}
|
|
621
|
+
```
|
|
622
|
+
|
|
623
|
+
**Alternative: use Playwright projects for responsive testing.**
|
|
624
|
+
|
|
625
|
+
```typescript
|
|
626
|
+
// playwright.config.ts
|
|
627
|
+
import { defineConfig, devices } from '@playwright/test';
|
|
628
|
+
|
|
629
|
+
export default defineConfig({
|
|
630
|
+
projects: [
|
|
631
|
+
{
|
|
632
|
+
name: 'desktop',
|
|
633
|
+
testMatch: '**/*.visual.spec.ts',
|
|
634
|
+
use: {
|
|
635
|
+
...devices['Desktop Chrome'],
|
|
636
|
+
viewport: { width: 1440, height: 900 },
|
|
637
|
+
},
|
|
638
|
+
},
|
|
639
|
+
{
|
|
640
|
+
name: 'tablet',
|
|
641
|
+
testMatch: '**/*.visual.spec.ts',
|
|
642
|
+
use: {
|
|
643
|
+
...devices['iPad (gen 7)'],
|
|
644
|
+
},
|
|
645
|
+
},
|
|
646
|
+
{
|
|
647
|
+
name: 'mobile',
|
|
648
|
+
testMatch: '**/*.visual.spec.ts',
|
|
649
|
+
use: {
|
|
650
|
+
...devices['iPhone 14'],
|
|
651
|
+
},
|
|
652
|
+
},
|
|
653
|
+
],
|
|
654
|
+
});
|
|
655
|
+
```
|
|
656
|
+
|
|
657
|
+
This approach gives you separate snapshot folders per device and runs them in parallel.
|
|
658
|
+
|
|
659
|
+
### 9. CI Setup for Visual Tests
|
|
660
|
+
|
|
661
|
+
**Use when**: Running visual regression tests in CI. The critical requirement is consistent rendering -- the same test must produce the same screenshot every time.
|
|
662
|
+
**Avoid when**: Never skip this. Visual tests without CI consistency are worthless.
|
|
663
|
+
|
|
664
|
+
**The problem**: Font rendering, anti-aliasing, and sub-pixel rendering differ across operating systems. A snapshot taken on macOS will not match one taken on Linux. This is the most common source of visual test pain.
|
|
665
|
+
|
|
666
|
+
**The solution**: Run visual tests inside Docker using the official Playwright container. Generate and update snapshots from the same container.
|
|
667
|
+
|
|
668
|
+
**GitHub Actions with Docker**
|
|
669
|
+
|
|
670
|
+
```yaml
|
|
671
|
+
# .github/workflows/visual-tests.yml
|
|
672
|
+
name: Visual Regression Tests
|
|
673
|
+
on: [push, pull_request]
|
|
674
|
+
|
|
675
|
+
jobs:
|
|
676
|
+
visual-tests:
|
|
677
|
+
runs-on: ubuntu-latest
|
|
678
|
+
container:
|
|
679
|
+
image: mcr.microsoft.com/playwright:v1.50.0-noble
|
|
680
|
+
steps:
|
|
681
|
+
- uses: actions/checkout@v4
|
|
682
|
+
|
|
683
|
+
- uses: actions/setup-node@v4
|
|
684
|
+
with:
|
|
685
|
+
node-version: 20
|
|
686
|
+
cache: npm
|
|
687
|
+
|
|
688
|
+
- run: npm ci
|
|
689
|
+
|
|
690
|
+
- name: Run visual tests
|
|
691
|
+
run: npx playwright test --project=visual
|
|
692
|
+
env:
|
|
693
|
+
HOME: /root # required for Playwright in Docker
|
|
694
|
+
|
|
695
|
+
- uses: actions/upload-artifact@v4
|
|
696
|
+
if: failure()
|
|
697
|
+
with:
|
|
698
|
+
name: visual-test-report
|
|
699
|
+
path: playwright-report/
|
|
700
|
+
retention-days: 14
|
|
701
|
+
```
|
|
702
|
+
|
|
703
|
+
**Updating snapshots locally using Docker** -- so your local baselines match CI:
|
|
704
|
+
|
|
705
|
+
```bash
|
|
706
|
+
# Generate/update snapshots using the same container as CI
|
|
707
|
+
docker run --rm -v $(pwd):/work -w /work \
|
|
708
|
+
mcr.microsoft.com/playwright:v1.50.0-noble \
|
|
709
|
+
npx playwright test --update-snapshots --project=visual
|
|
710
|
+
```
|
|
711
|
+
|
|
712
|
+
**Add a script to `package.json`** for convenience:
|
|
713
|
+
|
|
714
|
+
```json
|
|
715
|
+
{
|
|
716
|
+
"scripts": {
|
|
717
|
+
"test:visual": "npx playwright test --project=visual",
|
|
718
|
+
"test:visual:update": "docker run --rm -v $(pwd):/work -w /work mcr.microsoft.com/playwright:v1.50.0-noble npx playwright test --update-snapshots --project=visual"
|
|
719
|
+
}
|
|
720
|
+
}
|
|
721
|
+
```
|
|
722
|
+
|
|
723
|
+
**Configure snapshots for Linux only** in your config:
|
|
724
|
+
|
|
725
|
+
```typescript
|
|
726
|
+
// playwright.config.ts
|
|
727
|
+
import { defineConfig, devices } from '@playwright/test';
|
|
728
|
+
|
|
729
|
+
export default defineConfig({
|
|
730
|
+
snapshotPathTemplate: '{testDir}/{testFileDir}/{testFileName}-snapshots/{arg}{-projectName}{ext}',
|
|
731
|
+
// Omits {-snapshotSuffix} which includes the platform name (linux, darwin, win32).
|
|
732
|
+
// This means snapshots are platform-agnostic -- you MUST generate them in Docker.
|
|
733
|
+
projects: [
|
|
734
|
+
{
|
|
735
|
+
name: 'visual',
|
|
736
|
+
testMatch: '**/*.visual.spec.ts',
|
|
737
|
+
use: { ...devices['Desktop Chrome'] },
|
|
738
|
+
},
|
|
739
|
+
],
|
|
740
|
+
});
|
|
741
|
+
```
|
|
742
|
+
|
|
743
|
+
### 10. Component Visual Testing
|
|
744
|
+
|
|
745
|
+
**Use when**: Testing individual UI components in isolation -- buttons, cards, forms, modals. Faster than full-page screenshots, more stable, and easier to maintain.
|
|
746
|
+
**Avoid when**: You need to verify interactions between multiple components or page-level layout.
|
|
747
|
+
|
|
748
|
+
**TypeScript -- using Playwright component testing**
|
|
749
|
+
```typescript
|
|
750
|
+
// tests/components/button.visual.spec.ts
|
|
751
|
+
import { test, expect } from '@playwright/test';
|
|
752
|
+
|
|
753
|
+
test.describe('Button component visual states', () => {
|
|
754
|
+
test('primary button', async ({ page }) => {
|
|
755
|
+
await page.goto('/storybook/iframe.html?id=button--primary');
|
|
756
|
+
const button = page.getByRole('button');
|
|
757
|
+
await expect(button).toHaveScreenshot('button-primary.png', {
|
|
758
|
+
animations: 'disabled',
|
|
759
|
+
});
|
|
760
|
+
});
|
|
761
|
+
|
|
762
|
+
test('primary button hover', async ({ page }) => {
|
|
763
|
+
await page.goto('/storybook/iframe.html?id=button--primary');
|
|
764
|
+
const button = page.getByRole('button');
|
|
765
|
+
await button.hover();
|
|
766
|
+
await expect(button).toHaveScreenshot('button-primary-hover.png', {
|
|
767
|
+
animations: 'disabled',
|
|
768
|
+
});
|
|
769
|
+
});
|
|
770
|
+
|
|
771
|
+
test('primary button disabled', async ({ page }) => {
|
|
772
|
+
await page.goto('/storybook/iframe.html?id=button--primary-disabled');
|
|
773
|
+
const button = page.getByRole('button');
|
|
774
|
+
await expect(button).toHaveScreenshot('button-primary-disabled.png', {
|
|
775
|
+
animations: 'disabled',
|
|
776
|
+
});
|
|
777
|
+
});
|
|
778
|
+
|
|
779
|
+
test('button sizes', async ({ page }) => {
|
|
780
|
+
for (const size of ['small', 'medium', 'large']) {
|
|
781
|
+
await page.goto(`/storybook/iframe.html?id=button--${size}`);
|
|
782
|
+
const button = page.getByRole('button');
|
|
783
|
+
await expect(button).toHaveScreenshot(`button-${size}.png`, {
|
|
784
|
+
animations: 'disabled',
|
|
785
|
+
});
|
|
786
|
+
}
|
|
787
|
+
});
|
|
788
|
+
});
|
|
789
|
+
```
|
|
790
|
+
|
|
791
|
+
**JavaScript**
|
|
792
|
+
```javascript
|
|
793
|
+
const { test, expect } = require('@playwright/test');
|
|
794
|
+
|
|
795
|
+
test.describe('Button component visual states', () => {
|
|
796
|
+
test('primary button', async ({ page }) => {
|
|
797
|
+
await page.goto('/storybook/iframe.html?id=button--primary');
|
|
798
|
+
const button = page.getByRole('button');
|
|
799
|
+
await expect(button).toHaveScreenshot('button-primary.png', {
|
|
800
|
+
animations: 'disabled',
|
|
801
|
+
});
|
|
802
|
+
});
|
|
803
|
+
|
|
804
|
+
test('primary button hover', async ({ page }) => {
|
|
805
|
+
await page.goto('/storybook/iframe.html?id=button--primary');
|
|
806
|
+
const button = page.getByRole('button');
|
|
807
|
+
await button.hover();
|
|
808
|
+
await expect(button).toHaveScreenshot('button-primary-hover.png', {
|
|
809
|
+
animations: 'disabled',
|
|
810
|
+
});
|
|
811
|
+
});
|
|
812
|
+
|
|
813
|
+
test('primary button disabled', async ({ page }) => {
|
|
814
|
+
await page.goto('/storybook/iframe.html?id=button--primary-disabled');
|
|
815
|
+
const button = page.getByRole('button');
|
|
816
|
+
await expect(button).toHaveScreenshot('button-primary-disabled.png', {
|
|
817
|
+
animations: 'disabled',
|
|
818
|
+
});
|
|
819
|
+
});
|
|
820
|
+
|
|
821
|
+
test('button sizes', async ({ page }) => {
|
|
822
|
+
for (const size of ['small', 'medium', 'large']) {
|
|
823
|
+
await page.goto(`/storybook/iframe.html?id=button--${size}`);
|
|
824
|
+
const button = page.getByRole('button');
|
|
825
|
+
await expect(button).toHaveScreenshot(`button-${size}.png`, {
|
|
826
|
+
animations: 'disabled',
|
|
827
|
+
});
|
|
828
|
+
}
|
|
829
|
+
});
|
|
830
|
+
});
|
|
831
|
+
```
|
|
832
|
+
|
|
833
|
+
**Using a dedicated visual test page** instead of Storybook:
|
|
834
|
+
|
|
835
|
+
```typescript
|
|
836
|
+
// tests/components/card.visual.spec.ts
|
|
837
|
+
import { test, expect } from '@playwright/test';
|
|
838
|
+
|
|
839
|
+
test.describe('Card component', () => {
|
|
840
|
+
test.beforeEach(async ({ page }) => {
|
|
841
|
+
// Navigate to a page that renders the component in isolation
|
|
842
|
+
await page.goto('/test-harness/card');
|
|
843
|
+
});
|
|
844
|
+
|
|
845
|
+
test('default state', async ({ page }) => {
|
|
846
|
+
await expect(page.getByTestId('card')).toHaveScreenshot('card-default.png', {
|
|
847
|
+
animations: 'disabled',
|
|
848
|
+
});
|
|
849
|
+
});
|
|
850
|
+
|
|
851
|
+
test('with long content truncates correctly', async ({ page }) => {
|
|
852
|
+
await page.goto('/test-harness/card?content=long');
|
|
853
|
+
await expect(page.getByTestId('card')).toHaveScreenshot('card-long-content.png', {
|
|
854
|
+
animations: 'disabled',
|
|
855
|
+
});
|
|
856
|
+
});
|
|
857
|
+
|
|
858
|
+
test('error state', async ({ page }) => {
|
|
859
|
+
await page.goto('/test-harness/card?state=error');
|
|
860
|
+
await expect(page.getByTestId('card')).toHaveScreenshot('card-error.png', {
|
|
861
|
+
animations: 'disabled',
|
|
862
|
+
});
|
|
863
|
+
});
|
|
864
|
+
});
|
|
865
|
+
```
|
|
866
|
+
|
|
867
|
+
### 11. Asserting Pseudo-Element Styles Instead of a Screenshot (Playwright 1.60+)
|
|
868
|
+
|
|
869
|
+
**Use when**: You only care about one or two computed styles on a `::before` / `::after` pseudo-element (an icon glyph, a required asterisk, a status dot color). A targeted CSS assertion is faster and far less brittle than a pixel diff.
|
|
870
|
+
**Avoid when**: The thing you're verifying is genuinely visual (layout, gradients, shadows). Keep using `toHaveScreenshot()` there.
|
|
871
|
+
|
|
872
|
+
Playwright 1.60's `pseudo` option on `toHaveCSS()` lets you check pseudo-element styles without a screenshot baseline to maintain:
|
|
873
|
+
|
|
874
|
+
```typescript
|
|
875
|
+
import { test, expect } from '@playwright/test';
|
|
876
|
+
|
|
877
|
+
test('status badge renders a green dot via ::before', async ({ page }) => {
|
|
878
|
+
await page.goto('/orders');
|
|
879
|
+
|
|
880
|
+
const badge = page.getByTestId('status-online');
|
|
881
|
+
// No snapshot file to update — assert the computed style directly
|
|
882
|
+
await expect(badge).toHaveCSS('background-color', 'rgb(22, 163, 74)', { pseudo: '::before' });
|
|
883
|
+
});
|
|
884
|
+
```
|
|
885
|
+
|
|
886
|
+
This complements visual snapshots: use it to lock down specific pseudo-element styling that would otherwise force a full screenshot. See [core/assertions-and-waiting.md](assertions-and-waiting.md#asserting-pseudo-element-styles-tohavecss-pseudo-playwright-160).
|
|
887
|
+
|
|
888
|
+
## Decision Guide
|
|
889
|
+
|
|
890
|
+
| Scenario | Recommended Approach | Why |
|
|
891
|
+
|---|---|---|
|
|
892
|
+
| Key landing pages, marketing site | Full page screenshot, `fullPage: true` | Catches layout shifts, spacing, and overall visual harmony |
|
|
893
|
+
| Individual UI components (buttons, cards, modals) | Element screenshot on the component | Isolated, fast, stable -- immune to unrelated page changes |
|
|
894
|
+
| Page with dynamic content (timestamps, live data) | Full page + `mask` on dynamic elements | Covers layout while ignoring volatile content |
|
|
895
|
+
| Design system component library | Element screenshot per variant, zero threshold | Pixel-perfect enforcement for shared components |
|
|
896
|
+
| Responsive layout verification | Screenshot per viewport (loop or projects) | Catches breakpoint bugs at mobile/tablet/desktop |
|
|
897
|
+
| Cross-browser rendering consistency | Separate snapshots per browser project | Browsers render fonts and shadows differently |
|
|
898
|
+
| CI pipeline | Docker container (Playwright image), Linux-only snapshots | Consistent rendering, no OS-dependent diffs |
|
|
899
|
+
| Pixel threshold: design system | `threshold: 0`, `maxDiffPixels: 0` | Zero tolerance for component library |
|
|
900
|
+
| Pixel threshold: content pages | `maxDiffPixelRatio: 0.01`, `threshold: 0.2` | Allows minor anti-aliasing variance |
|
|
901
|
+
| Pixel threshold: charts and graphs | `maxDiffPixels: 200`, `threshold: 0.3` | Anti-aliasing on curves varies across runs |
|
|
902
|
+
| Visual tests add value | Stable pages, design systems, post-refactor verification | Clear baseline, predictable content |
|
|
903
|
+
| Visual tests are noise | Highly dynamic pages, real-time dashboards, A/B test pages | Content changes on every load, diffs are meaningless |
|
|
904
|
+
|
|
905
|
+
## Anti-Patterns
|
|
906
|
+
|
|
907
|
+
| Don't Do This | Problem | Do This Instead |
|
|
908
|
+
|---|---|---|
|
|
909
|
+
| Visual testing every page in the app | Massive snapshot maintenance, constant false failures, team ignores results | Pick 5-10 key pages and critical components. Quality over quantity. |
|
|
910
|
+
| Not masking dynamic content (timestamps, avatars, counters) | Screenshots differ on every run. Tests are permanently flaky. | Use `mask` option for all dynamic elements. Audit pages for volatility before adding visual tests. |
|
|
911
|
+
| Running visual tests across macOS, Linux, and Windows | Font rendering differs per OS. Snapshots never match cross-platform. | Standardize on Linux via Docker. Generate and run snapshots in the same Playwright Docker container. |
|
|
912
|
+
| Not using Docker in CI for visual tests | CI runner OS updates, font changes, or library upgrades silently shift rendering. | Pin a specific Playwright Docker image version. Update intentionally. |
|
|
913
|
+
| Updating snapshots blindly with `--update-snapshots` | Accepts unintentional regressions. The baseline becomes wrong. | Always review the diff in the HTML report first. Understand why the snapshot changed. |
|
|
914
|
+
| Skipping `animations: 'disabled'` | CSS transitions and keyframe animations create random diffs. One of the top causes of flaky visual tests. | Set `animations: 'disabled'` globally in config. |
|
|
915
|
+
| Using visual tests instead of functional assertions | Screenshot diffs do not tell you *what* broke -- just that *something* looks different. Slow to debug. | Use functional assertions (`toHaveText`, `toBeVisible`) for behavior. Visual tests complement, never replace. |
|
|
916
|
+
| Committing snapshots generated on different platforms | The repo has macOS snapshots from dev A, Linux snapshots from dev B. Tests fail for everyone. | All team members generate snapshots using the same Docker container. Add a `test:visual:update` script. |
|
|
917
|
+
| Setting threshold too high (e.g., `maxDiffPixelRatio: 0.1`) | 10% of pixels can change and the test still passes. Defeats the purpose. | Start with `0.01` (1%) and adjust per-test if needed. |
|
|
918
|
+
| Full page screenshots on pages with infinite scroll or lazy loading | Page height is nondeterministic. Screenshots vary by load timing. | Use element screenshots on the above-the-fold content, or scroll to a deterministic state first. |
|
|
919
|
+
|
|
920
|
+
## Troubleshooting
|
|
921
|
+
|
|
922
|
+
### "Screenshot comparison failed" on first CI run after local development
|
|
923
|
+
|
|
924
|
+
**Cause**: Snapshots were generated on macOS (or Windows) locally. CI runs on Linux. Font rendering differs between operating systems, so every pixel comparison fails.
|
|
925
|
+
|
|
926
|
+
**Fix**: Generate snapshots using Docker locally so they match CI:
|
|
927
|
+
|
|
928
|
+
```bash
|
|
929
|
+
docker run --rm -v $(pwd):/work -w /work \
|
|
930
|
+
mcr.microsoft.com/playwright:v1.50.0-noble \
|
|
931
|
+
npx playwright test --update-snapshots --project=visual
|
|
932
|
+
```
|
|
933
|
+
|
|
934
|
+
Commit the Linux-generated snapshots. Never commit macOS-generated snapshots if CI runs on Linux.
|
|
935
|
+
|
|
936
|
+
### "Expected screenshot to match but X pixels differ"
|
|
937
|
+
|
|
938
|
+
**Cause**: Minor rendering differences from anti-aliasing, font hinting, or sub-pixel rendering. Common with text-heavy pages and curved shapes (charts, rounded corners).
|
|
939
|
+
|
|
940
|
+
**Fix**: Add a small tolerance:
|
|
941
|
+
|
|
942
|
+
```typescript
|
|
943
|
+
await expect(page).toHaveScreenshot('page.png', {
|
|
944
|
+
maxDiffPixelRatio: 0.01, // allow 1% variance
|
|
945
|
+
threshold: 0.2, // per-pixel color tolerance
|
|
946
|
+
});
|
|
947
|
+
```
|
|
948
|
+
|
|
949
|
+
If the diff is larger, check the HTML report. Look at the diff image to determine if the change is a real regression or rendering noise.
|
|
950
|
+
|
|
951
|
+
### Visual tests pass locally but fail in CI (even with Docker)
|
|
952
|
+
|
|
953
|
+
**Cause**: Different Playwright versions locally vs CI. The Docker image pins a specific Playwright version. If your local `@playwright/test` is newer, the rendering engine may differ.
|
|
954
|
+
|
|
955
|
+
**Fix**: Ensure the Playwright version in `package.json` matches the Docker image tag:
|
|
956
|
+
|
|
957
|
+
```json
|
|
958
|
+
{
|
|
959
|
+
"devDependencies": {
|
|
960
|
+
"@playwright/test": "1.50.0"
|
|
961
|
+
}
|
|
962
|
+
}
|
|
963
|
+
```
|
|
964
|
+
|
|
965
|
+
```yaml
|
|
966
|
+
# CI config
|
|
967
|
+
container:
|
|
968
|
+
image: mcr.microsoft.com/playwright:v1.50.0-noble # must match
|
|
969
|
+
```
|
|
970
|
+
|
|
971
|
+
### Animations cause random diff failures
|
|
972
|
+
|
|
973
|
+
**Cause**: CSS animations or transitions captured mid-frame. The screenshot is taken at a non-deterministic point in the animation timeline.
|
|
974
|
+
|
|
975
|
+
**Fix**: Set `animations: 'disabled'` globally:
|
|
976
|
+
|
|
977
|
+
```typescript
|
|
978
|
+
// playwright.config.ts
|
|
979
|
+
export default defineConfig({
|
|
980
|
+
expect: {
|
|
981
|
+
toHaveScreenshot: {
|
|
982
|
+
animations: 'disabled',
|
|
983
|
+
},
|
|
984
|
+
},
|
|
985
|
+
});
|
|
986
|
+
```
|
|
987
|
+
|
|
988
|
+
For JavaScript-driven animations, wait for a stable state before screenshotting. As a last resort, use `page.waitForTimeout(300)` after the animation trigger.
|
|
989
|
+
|
|
990
|
+
### Snapshot file names conflict between tests
|
|
991
|
+
|
|
992
|
+
**Cause**: Two tests in different files use the same screenshot name (`'homepage.png'`) without unique file paths.
|
|
993
|
+
|
|
994
|
+
**Fix**: Playwright includes the test file name in the snapshot path by default. If you still have conflicts, use explicit unique names:
|
|
995
|
+
|
|
996
|
+
```typescript
|
|
997
|
+
await expect(page).toHaveScreenshot('auth-homepage.png'); // in auth.spec.ts
|
|
998
|
+
await expect(page).toHaveScreenshot('public-homepage.png'); // in public.spec.ts
|
|
999
|
+
```
|
|
1000
|
+
|
|
1001
|
+
Or customize the snapshot path template in config:
|
|
1002
|
+
|
|
1003
|
+
```typescript
|
|
1004
|
+
export default defineConfig({
|
|
1005
|
+
snapshotPathTemplate: '{testDir}/{testFileDir}/{testFileName}-snapshots/{arg}{-projectName}{ext}',
|
|
1006
|
+
});
|
|
1007
|
+
```
|
|
1008
|
+
|
|
1009
|
+
### Too many snapshot files to maintain
|
|
1010
|
+
|
|
1011
|
+
**Cause**: Visual tests for every page, every browser, every viewport. The snapshot count explodes.
|
|
1012
|
+
|
|
1013
|
+
**Fix**: Be selective. Visual test only pages where visual regressions are high-risk:
|
|
1014
|
+
- Landing pages and marketing pages
|
|
1015
|
+
- Design system components
|
|
1016
|
+
- Complex layouts (dashboards, data tables)
|
|
1017
|
+
- Pages after a major refactor
|
|
1018
|
+
|
|
1019
|
+
Skip pages where functional assertions already cover the key elements. A `toHaveText` and `toBeVisible` check is often enough.
|
|
1020
|
+
|
|
1021
|
+
## Related
|
|
1022
|
+
|
|
1023
|
+
- [core/assertions-and-waiting.md](assertions-and-waiting.md) -- `toHaveScreenshot()` is a web-first assertion and follows the same retry semantics
|
|
1024
|
+
- [core/configuration.md](configuration.md) -- global screenshot config, project setup, `snapshotPathTemplate`
|
|
1025
|
+
- [core/mobile-and-responsive.md](mobile-and-responsive.md) -- responsive viewport testing patterns
|
|
1026
|
+
- [ci/docker-and-containers.md](../ci/docker-and-containers.md) -- Docker setup for consistent rendering
|
|
1027
|
+
- [ci/ci-github-actions.md](../ci/ci-github-actions.md) -- CI pipeline configuration for visual tests
|