opencode-skills-collection 1.0.186 → 1.0.187
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/bundled-skills/.antigravity-install-manifest.json +5 -1
- package/bundled-skills/3d-web-experience/SKILL.md +152 -37
- package/bundled-skills/agent-evaluation/SKILL.md +1088 -26
- package/bundled-skills/agent-memory-systems/SKILL.md +1037 -25
- package/bundled-skills/agent-tool-builder/SKILL.md +668 -16
- package/bundled-skills/ai-agents-architect/SKILL.md +271 -31
- package/bundled-skills/ai-product/SKILL.md +716 -26
- package/bundled-skills/ai-wrapper-product/SKILL.md +450 -44
- package/bundled-skills/algolia-search/SKILL.md +867 -15
- package/bundled-skills/autonomous-agents/SKILL.md +1033 -26
- package/bundled-skills/aws-serverless/SKILL.md +1046 -35
- package/bundled-skills/azure-functions/SKILL.md +1318 -19
- package/bundled-skills/browser-automation/SKILL.md +1065 -28
- package/bundled-skills/browser-extension-builder/SKILL.md +159 -32
- package/bundled-skills/bullmq-specialist/SKILL.md +347 -16
- package/bundled-skills/clerk-auth/SKILL.md +796 -15
- package/bundled-skills/computer-use-agents/SKILL.md +1870 -28
- package/bundled-skills/context-window-management/SKILL.md +271 -18
- package/bundled-skills/conversation-memory/SKILL.md +453 -24
- package/bundled-skills/crewai/SKILL.md +252 -46
- package/bundled-skills/discord-bot-architect/SKILL.md +1207 -34
- package/bundled-skills/docs/integrations/jetski-cortex.md +3 -3
- package/bundled-skills/docs/integrations/jetski-gemini-loader/README.md +1 -1
- package/bundled-skills/docs/maintainers/repo-growth-seo.md +3 -3
- package/bundled-skills/docs/maintainers/skills-update-guide.md +1 -1
- package/bundled-skills/docs/users/bundles.md +1 -1
- package/bundled-skills/docs/users/claude-code-skills.md +1 -1
- package/bundled-skills/docs/users/gemini-cli-skills.md +1 -1
- package/bundled-skills/docs/users/getting-started.md +1 -1
- package/bundled-skills/docs/users/kiro-integration.md +1 -1
- package/bundled-skills/docs/users/usage.md +4 -4
- package/bundled-skills/docs/users/visual-guide.md +4 -4
- package/bundled-skills/email-systems/SKILL.md +646 -26
- package/bundled-skills/faf-expert/SKILL.md +221 -0
- package/bundled-skills/faf-wizard/SKILL.md +252 -0
- package/bundled-skills/file-uploads/SKILL.md +212 -11
- package/bundled-skills/firebase/SKILL.md +646 -16
- package/bundled-skills/gcp-cloud-run/SKILL.md +1117 -32
- package/bundled-skills/graphql/SKILL.md +1026 -27
- package/bundled-skills/hubspot-integration/SKILL.md +804 -19
- package/bundled-skills/idea-darwin/SKILL.md +120 -0
- package/bundled-skills/inngest/SKILL.md +431 -16
- package/bundled-skills/interactive-portfolio/SKILL.md +342 -44
- package/bundled-skills/langfuse/SKILL.md +296 -41
- package/bundled-skills/langgraph/SKILL.md +259 -50
- package/bundled-skills/micro-saas-launcher/SKILL.md +343 -44
- package/bundled-skills/neon-postgres/SKILL.md +572 -15
- package/bundled-skills/nextjs-supabase-auth/SKILL.md +269 -21
- package/bundled-skills/notion-template-business/SKILL.md +371 -44
- package/bundled-skills/personal-tool-builder/SKILL.md +537 -44
- package/bundled-skills/plaid-fintech/SKILL.md +825 -19
- package/bundled-skills/prompt-caching/SKILL.md +438 -25
- package/bundled-skills/rag-engineer/SKILL.md +271 -29
- package/bundled-skills/salesforce-development/SKILL.md +912 -19
- package/bundled-skills/satori/SKILL.md +54 -0
- package/bundled-skills/scroll-experience/SKILL.md +381 -44
- package/bundled-skills/segment-cdp/SKILL.md +817 -19
- package/bundled-skills/shopify-apps/SKILL.md +1475 -19
- package/bundled-skills/slack-bot-builder/SKILL.md +1162 -28
- package/bundled-skills/telegram-bot-builder/SKILL.md +152 -37
- package/bundled-skills/telegram-mini-app/SKILL.md +445 -44
- package/bundled-skills/trigger-dev/SKILL.md +916 -27
- package/bundled-skills/twilio-communications/SKILL.md +1310 -28
- package/bundled-skills/upstash-qstash/SKILL.md +898 -27
- package/bundled-skills/vercel-deployment/SKILL.md +637 -39
- package/bundled-skills/viral-generator-builder/SKILL.md +132 -37
- package/bundled-skills/voice-agents/SKILL.md +937 -27
- package/bundled-skills/voice-ai-development/SKILL.md +375 -46
- package/bundled-skills/workflow-automation/SKILL.md +982 -29
- package/bundled-skills/zapier-make-patterns/SKILL.md +772 -27
- package/package.json +1 -1
|
@@ -1,24 +1,37 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: browser-automation
|
|
3
|
-
description:
|
|
3
|
+
description: Browser automation powers web testing, scraping, and AI agent
|
|
4
|
+
interactions. The difference between a flaky script and a reliable system
|
|
5
|
+
comes down to understanding selectors, waiting strategies, and anti-detection
|
|
6
|
+
patterns.
|
|
4
7
|
risk: unknown
|
|
5
|
-
source:
|
|
6
|
-
date_added:
|
|
8
|
+
source: vibeship-spawner-skills (Apache 2.0)
|
|
9
|
+
date_added: 2026-02-27
|
|
7
10
|
---
|
|
8
11
|
|
|
9
12
|
# Browser Automation
|
|
10
13
|
|
|
11
|
-
|
|
12
|
-
|
|
13
|
-
|
|
14
|
-
when each tool shines.
|
|
14
|
+
Browser automation powers web testing, scraping, and AI agent interactions.
|
|
15
|
+
The difference between a flaky script and a reliable system comes down to
|
|
16
|
+
understanding selectors, waiting strategies, and anti-detection patterns.
|
|
15
17
|
|
|
16
|
-
|
|
17
|
-
|
|
18
|
-
|
|
19
|
-
|
|
18
|
+
This skill covers Playwright (recommended) and Puppeteer, with patterns for
|
|
19
|
+
testing, scraping, and agentic browser control. Key insight: Playwright won
|
|
20
|
+
the framework war. Unless you need Puppeteer's stealth ecosystem or are
|
|
21
|
+
Chrome-only, Playwright is the better choice in 2025.
|
|
20
22
|
|
|
21
|
-
|
|
23
|
+
Critical distinction: Testing automation (predictable apps you control) vs
|
|
24
|
+
scraping/agent automation (unpredictable sites that fight back). Different
|
|
25
|
+
problems, different solutions.
|
|
26
|
+
|
|
27
|
+
## Principles
|
|
28
|
+
|
|
29
|
+
- Use user-facing locators (getByRole, getByText) over CSS/XPath
|
|
30
|
+
- Never add manual waits - Playwright's auto-wait handles it
|
|
31
|
+
- Each test/task should be fully isolated with fresh context
|
|
32
|
+
- Screenshots and traces are your debugging lifeline
|
|
33
|
+
- Headless for CI, headed for debugging
|
|
34
|
+
- Anti-detection is cat-and-mouse - stay current or get blocked
|
|
22
35
|
|
|
23
36
|
## Capabilities
|
|
24
37
|
|
|
@@ -32,44 +45,1068 @@ For scraping, yo
|
|
|
32
45
|
- ui-automation
|
|
33
46
|
- selenium-alternatives
|
|
34
47
|
|
|
48
|
+
## Scope
|
|
49
|
+
|
|
50
|
+
- api-testing → backend
|
|
51
|
+
- load-testing → performance-thinker
|
|
52
|
+
- accessibility-testing → accessibility-specialist
|
|
53
|
+
- visual-regression-testing → ui-design
|
|
54
|
+
|
|
55
|
+
## Tooling
|
|
56
|
+
|
|
57
|
+
### Frameworks
|
|
58
|
+
|
|
59
|
+
- Playwright - When: Default choice - cross-browser, auto-waiting, best DX Note: 96% success rate, 4.5s avg execution, Microsoft-backed
|
|
60
|
+
- Puppeteer - When: Chrome-only, need stealth plugins, existing codebase Note: 75% success rate at scale, but best stealth ecosystem
|
|
61
|
+
- Selenium - When: Legacy systems, specific language bindings Note: Slower, more verbose, but widest browser support
|
|
62
|
+
|
|
63
|
+
### Stealth_tools
|
|
64
|
+
|
|
65
|
+
- puppeteer-extra-plugin-stealth - When: Need to bypass bot detection with Puppeteer Note: Gold standard for anti-detection
|
|
66
|
+
- playwright-extra - When: Stealth plugins for Playwright Note: Port of puppeteer-extra ecosystem
|
|
67
|
+
- undetected-chromedriver - When: Selenium anti-detection Note: Dynamic bypass of detection
|
|
68
|
+
|
|
69
|
+
### Cloud_browsers
|
|
70
|
+
|
|
71
|
+
- Browserbase - When: Managed headless infrastructure Note: Built-in stealth mode, session management
|
|
72
|
+
- BrowserStack - When: Cross-browser testing at scale Note: Real devices, CI integration
|
|
73
|
+
|
|
35
74
|
## Patterns
|
|
36
75
|
|
|
37
76
|
### Test Isolation Pattern
|
|
38
77
|
|
|
39
78
|
Each test runs in complete isolation with fresh state
|
|
40
79
|
|
|
80
|
+
**When to use**: Testing, any automation that needs reproducibility
|
|
81
|
+
|
|
82
|
+
# TEST ISOLATION:
|
|
83
|
+
|
|
84
|
+
"""
|
|
85
|
+
Each test gets its own:
|
|
86
|
+
- Browser context (cookies, storage)
|
|
87
|
+
- Fresh page
|
|
88
|
+
- Clean state
|
|
89
|
+
"""
|
|
90
|
+
|
|
91
|
+
## Playwright Test Example
|
|
92
|
+
"""
|
|
93
|
+
import { test, expect } from '@playwright/test';
|
|
94
|
+
|
|
95
|
+
// Each test runs in isolated browser context
|
|
96
|
+
test('user can add item to cart', async ({ page }) => {
|
|
97
|
+
// Fresh context - no cookies, no storage from other tests
|
|
98
|
+
await page.goto('/products');
|
|
99
|
+
await page.getByRole('button', { name: 'Add to Cart' }).click();
|
|
100
|
+
await expect(page.getByTestId('cart-count')).toHaveText('1');
|
|
101
|
+
});
|
|
102
|
+
|
|
103
|
+
test('user can remove item from cart', async ({ page }) => {
|
|
104
|
+
// Completely isolated - cart is empty
|
|
105
|
+
await page.goto('/cart');
|
|
106
|
+
await expect(page.getByText('Your cart is empty')).toBeVisible();
|
|
107
|
+
});
|
|
108
|
+
"""
|
|
109
|
+
|
|
110
|
+
## Shared Authentication Pattern
|
|
111
|
+
"""
|
|
112
|
+
// Save auth state once, reuse across tests
|
|
113
|
+
// setup.ts
|
|
114
|
+
import { test as setup } from '@playwright/test';
|
|
115
|
+
|
|
116
|
+
setup('authenticate', async ({ page }) => {
|
|
117
|
+
await page.goto('/login');
|
|
118
|
+
await page.getByLabel('Email').fill('user@example.com');
|
|
119
|
+
await page.getByLabel('Password').fill('password');
|
|
120
|
+
await page.getByRole('button', { name: 'Sign in' }).click();
|
|
121
|
+
|
|
122
|
+
// Wait for auth to complete
|
|
123
|
+
await page.waitForURL('/dashboard');
|
|
124
|
+
|
|
125
|
+
// Save authentication state
|
|
126
|
+
await page.context().storageState({
|
|
127
|
+
path: './playwright/.auth/user.json'
|
|
128
|
+
});
|
|
129
|
+
});
|
|
130
|
+
|
|
131
|
+
// playwright.config.ts
|
|
132
|
+
export default defineConfig({
|
|
133
|
+
projects: [
|
|
134
|
+
{ name: 'setup', testMatch: /.*\.setup\.ts/ },
|
|
135
|
+
{
|
|
136
|
+
name: 'tests',
|
|
137
|
+
dependencies: ['setup'],
|
|
138
|
+
use: {
|
|
139
|
+
storageState: './playwright/.auth/user.json',
|
|
140
|
+
},
|
|
141
|
+
},
|
|
142
|
+
],
|
|
143
|
+
});
|
|
144
|
+
"""
|
|
145
|
+
|
|
41
146
|
### User-Facing Locator Pattern
|
|
42
147
|
|
|
43
148
|
Select elements the way users see them
|
|
44
149
|
|
|
150
|
+
**When to use**: Always - the default approach for selectors
|
|
151
|
+
|
|
152
|
+
# USER-FACING LOCATORS:
|
|
153
|
+
|
|
154
|
+
"""
|
|
155
|
+
Priority order:
|
|
156
|
+
1. getByRole - Best: matches accessibility tree
|
|
157
|
+
2. getByText - Good: matches visible content
|
|
158
|
+
3. getByLabel - Good: matches form labels
|
|
159
|
+
4. getByTestId - Fallback: explicit test contracts
|
|
160
|
+
5. CSS/XPath - Last resort: fragile, avoid
|
|
161
|
+
"""
|
|
162
|
+
|
|
163
|
+
## Good Examples (User-Facing)
|
|
164
|
+
"""
|
|
165
|
+
// By role - THE BEST CHOICE
|
|
166
|
+
await page.getByRole('button', { name: 'Submit' }).click();
|
|
167
|
+
await page.getByRole('link', { name: 'Sign up' }).click();
|
|
168
|
+
await page.getByRole('heading', { name: 'Dashboard' }).isVisible();
|
|
169
|
+
await page.getByRole('textbox', { name: 'Search' }).fill('query');
|
|
170
|
+
|
|
171
|
+
// By text content
|
|
172
|
+
await page.getByText('Welcome back').isVisible();
|
|
173
|
+
await page.getByText(/Order #\d+/).click(); // Regex supported
|
|
174
|
+
|
|
175
|
+
// By label (forms)
|
|
176
|
+
await page.getByLabel('Email address').fill('user@example.com');
|
|
177
|
+
await page.getByLabel('Password').fill('secret');
|
|
178
|
+
|
|
179
|
+
// By placeholder
|
|
180
|
+
await page.getByPlaceholder('Search...').fill('query');
|
|
181
|
+
|
|
182
|
+
// By test ID (when no user-facing option works)
|
|
183
|
+
await page.getByTestId('submit-button').click();
|
|
184
|
+
"""
|
|
185
|
+
|
|
186
|
+
## Bad Examples (Fragile)
|
|
187
|
+
"""
|
|
188
|
+
// DON'T - CSS selectors tied to structure
|
|
189
|
+
await page.locator('.btn-primary.submit-form').click();
|
|
190
|
+
await page.locator('#header > div > button:nth-child(2)').click();
|
|
191
|
+
|
|
192
|
+
// DON'T - XPath tied to structure
|
|
193
|
+
await page.locator('//div[@class="form"]/button[1]').click();
|
|
194
|
+
|
|
195
|
+
// DON'T - Auto-generated selectors
|
|
196
|
+
await page.locator('[data-v-12345]').click();
|
|
197
|
+
"""
|
|
198
|
+
|
|
199
|
+
## Filtering and Chaining
|
|
200
|
+
"""
|
|
201
|
+
// Filter by containing text
|
|
202
|
+
await page.getByRole('listitem')
|
|
203
|
+
.filter({ hasText: 'Product A' })
|
|
204
|
+
.getByRole('button', { name: 'Add to cart' })
|
|
205
|
+
.click();
|
|
206
|
+
|
|
207
|
+
// Filter by NOT containing
|
|
208
|
+
await page.getByRole('listitem')
|
|
209
|
+
.filter({ hasNotText: 'Sold out' })
|
|
210
|
+
.first()
|
|
211
|
+
.click();
|
|
212
|
+
|
|
213
|
+
// Chain locators
|
|
214
|
+
const row = page.getByRole('row', { name: 'John Doe' });
|
|
215
|
+
await row.getByRole('button', { name: 'Edit' }).click();
|
|
216
|
+
"""
|
|
217
|
+
|
|
45
218
|
### Auto-Wait Pattern
|
|
46
219
|
|
|
47
220
|
Let Playwright wait automatically, never add manual waits
|
|
48
221
|
|
|
49
|
-
|
|
222
|
+
**When to use**: Always with Playwright
|
|
223
|
+
|
|
224
|
+
# AUTO-WAIT PATTERN:
|
|
225
|
+
|
|
226
|
+
"""
|
|
227
|
+
Playwright waits automatically for:
|
|
228
|
+
- Element to be attached to DOM
|
|
229
|
+
- Element to be visible
|
|
230
|
+
- Element to be stable (not animating)
|
|
231
|
+
- Element to receive events
|
|
232
|
+
- Element to be enabled
|
|
233
|
+
|
|
234
|
+
NEVER add manual waits!
|
|
235
|
+
"""
|
|
236
|
+
|
|
237
|
+
## Wrong - Manual Waits
|
|
238
|
+
"""
|
|
239
|
+
// DON'T DO THIS
|
|
240
|
+
await page.goto('/dashboard');
|
|
241
|
+
await page.waitForTimeout(2000); // NO! Arbitrary wait
|
|
242
|
+
await page.click('.submit-button');
|
|
243
|
+
|
|
244
|
+
// DON'T DO THIS
|
|
245
|
+
await page.waitForSelector('.loading-spinner', { state: 'hidden' });
|
|
246
|
+
await page.waitForTimeout(500); // "Just to be safe" - NO!
|
|
247
|
+
"""
|
|
248
|
+
|
|
249
|
+
## Correct - Let Auto-Wait Work
|
|
250
|
+
"""
|
|
251
|
+
// Auto-waits for button to be clickable
|
|
252
|
+
await page.getByRole('button', { name: 'Submit' }).click();
|
|
253
|
+
|
|
254
|
+
// Auto-waits for text to appear
|
|
255
|
+
await expect(page.getByText('Success!')).toBeVisible();
|
|
256
|
+
|
|
257
|
+
// Auto-waits for navigation to complete
|
|
258
|
+
await page.goto('/dashboard');
|
|
259
|
+
// Page is ready - no manual wait needed
|
|
260
|
+
"""
|
|
261
|
+
|
|
262
|
+
## When You DO Need to Wait
|
|
263
|
+
"""
|
|
264
|
+
// Wait for specific network request
|
|
265
|
+
const responsePromise = page.waitForResponse(
|
|
266
|
+
response => response.url().includes('/api/data')
|
|
267
|
+
);
|
|
268
|
+
await page.getByRole('button', { name: 'Load' }).click();
|
|
269
|
+
const response = await responsePromise;
|
|
270
|
+
|
|
271
|
+
// Wait for URL change
|
|
272
|
+
await Promise.all([
|
|
273
|
+
page.waitForURL('**/dashboard'),
|
|
274
|
+
page.getByRole('button', { name: 'Login' }).click(),
|
|
275
|
+
]);
|
|
276
|
+
|
|
277
|
+
// Wait for download
|
|
278
|
+
const downloadPromise = page.waitForEvent('download');
|
|
279
|
+
await page.getByText('Export CSV').click();
|
|
280
|
+
const download = await downloadPromise;
|
|
281
|
+
"""
|
|
282
|
+
|
|
283
|
+
### Stealth Browser Pattern
|
|
284
|
+
|
|
285
|
+
Avoid bot detection for scraping
|
|
286
|
+
|
|
287
|
+
**When to use**: Scraping sites with anti-bot protection
|
|
288
|
+
|
|
289
|
+
# STEALTH BROWSER PATTERN:
|
|
290
|
+
|
|
291
|
+
"""
|
|
292
|
+
Bot detection checks for:
|
|
293
|
+
- navigator.webdriver property
|
|
294
|
+
- Chrome DevTools protocol artifacts
|
|
295
|
+
- Browser fingerprint inconsistencies
|
|
296
|
+
- Behavioral patterns (perfect timing, no mouse movement)
|
|
297
|
+
- Headless indicators
|
|
298
|
+
"""
|
|
299
|
+
|
|
300
|
+
## Puppeteer Stealth (Best Anti-Detection)
|
|
301
|
+
"""
|
|
302
|
+
import puppeteer from 'puppeteer-extra';
|
|
303
|
+
import StealthPlugin from 'puppeteer-extra-plugin-stealth';
|
|
304
|
+
|
|
305
|
+
puppeteer.use(StealthPlugin());
|
|
306
|
+
|
|
307
|
+
const browser = await puppeteer.launch({
|
|
308
|
+
headless: 'new',
|
|
309
|
+
args: [
|
|
310
|
+
'--no-sandbox',
|
|
311
|
+
'--disable-setuid-sandbox',
|
|
312
|
+
'--disable-blink-features=AutomationControlled',
|
|
313
|
+
],
|
|
314
|
+
});
|
|
315
|
+
|
|
316
|
+
const page = await browser.newPage();
|
|
317
|
+
|
|
318
|
+
// Set realistic viewport
|
|
319
|
+
await page.setViewport({ width: 1920, height: 1080 });
|
|
320
|
+
|
|
321
|
+
// Realistic user agent
|
|
322
|
+
await page.setUserAgent(
|
|
323
|
+
'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 ' +
|
|
324
|
+
'(KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36'
|
|
325
|
+
);
|
|
326
|
+
|
|
327
|
+
// Navigate with human-like behavior
|
|
328
|
+
await page.goto('https://target-site.com', {
|
|
329
|
+
waitUntil: 'networkidle0',
|
|
330
|
+
});
|
|
331
|
+
"""
|
|
332
|
+
|
|
333
|
+
## Playwright Stealth
|
|
334
|
+
"""
|
|
335
|
+
import { chromium } from 'playwright-extra';
|
|
336
|
+
import stealth from 'puppeteer-extra-plugin-stealth';
|
|
337
|
+
|
|
338
|
+
chromium.use(stealth());
|
|
339
|
+
|
|
340
|
+
const browser = await chromium.launch({ headless: true });
|
|
341
|
+
const context = await browser.newContext({
|
|
342
|
+
viewport: { width: 1920, height: 1080 },
|
|
343
|
+
userAgent: 'Mozilla/5.0 ...',
|
|
344
|
+
locale: 'en-US',
|
|
345
|
+
timezoneId: 'America/New_York',
|
|
346
|
+
});
|
|
347
|
+
"""
|
|
348
|
+
|
|
349
|
+
## Human-Like Behavior
|
|
350
|
+
"""
|
|
351
|
+
// Random delays between actions
|
|
352
|
+
const randomDelay = (min: number, max: number) =>
|
|
353
|
+
new Promise(r => setTimeout(r, Math.random() * (max - min) + min));
|
|
354
|
+
|
|
355
|
+
await page.goto(url);
|
|
356
|
+
await randomDelay(500, 1500);
|
|
357
|
+
|
|
358
|
+
// Mouse movement before click
|
|
359
|
+
const button = await page.$('button.submit');
|
|
360
|
+
const box = await button.boundingBox();
|
|
361
|
+
await page.mouse.move(
|
|
362
|
+
box.x + box.width / 2,
|
|
363
|
+
box.y + box.height / 2,
|
|
364
|
+
{ steps: 10 } // Move in steps like a human
|
|
365
|
+
);
|
|
366
|
+
await randomDelay(100, 300);
|
|
367
|
+
await button.click();
|
|
368
|
+
|
|
369
|
+
// Scroll naturally
|
|
370
|
+
await page.evaluate(() => {
|
|
371
|
+
window.scrollBy({
|
|
372
|
+
top: 300 + Math.random() * 200,
|
|
373
|
+
behavior: 'smooth'
|
|
374
|
+
});
|
|
375
|
+
});
|
|
376
|
+
"""
|
|
377
|
+
|
|
378
|
+
### Error Recovery Pattern
|
|
379
|
+
|
|
380
|
+
Handle failures gracefully with screenshots and retries
|
|
381
|
+
|
|
382
|
+
**When to use**: Any production automation
|
|
383
|
+
|
|
384
|
+
# ERROR RECOVERY PATTERN:
|
|
385
|
+
|
|
386
|
+
## Automatic Screenshot on Failure
|
|
387
|
+
"""
|
|
388
|
+
// playwright.config.ts
|
|
389
|
+
export default defineConfig({
|
|
390
|
+
use: {
|
|
391
|
+
screenshot: 'only-on-failure',
|
|
392
|
+
trace: 'retain-on-failure',
|
|
393
|
+
video: 'retain-on-failure',
|
|
394
|
+
},
|
|
395
|
+
retries: 2, // Retry failed tests
|
|
396
|
+
});
|
|
397
|
+
"""
|
|
398
|
+
|
|
399
|
+
## Try-Catch with Debug Info
|
|
400
|
+
"""
|
|
401
|
+
async function scrapeProduct(page: Page, url: string) {
|
|
402
|
+
try {
|
|
403
|
+
await page.goto(url, { timeout: 30000 });
|
|
404
|
+
|
|
405
|
+
const title = await page.getByRole('heading', { level: 1 }).textContent();
|
|
406
|
+
const price = await page.getByTestId('price').textContent();
|
|
407
|
+
|
|
408
|
+
return { title, price, success: true };
|
|
409
|
+
|
|
410
|
+
} catch (error) {
|
|
411
|
+
// Capture debug info
|
|
412
|
+
const screenshot = await page.screenshot({
|
|
413
|
+
path: `errors/${Date.now()}-error.png`,
|
|
414
|
+
fullPage: true
|
|
415
|
+
});
|
|
416
|
+
|
|
417
|
+
const html = await page.content();
|
|
418
|
+
await fs.writeFile(`errors/${Date.now()}-page.html`, html);
|
|
419
|
+
|
|
420
|
+
console.error({
|
|
421
|
+
url,
|
|
422
|
+
error: error.message,
|
|
423
|
+
currentUrl: page.url(),
|
|
424
|
+
});
|
|
425
|
+
|
|
426
|
+
return { success: false, error: error.message };
|
|
427
|
+
}
|
|
428
|
+
}
|
|
429
|
+
"""
|
|
430
|
+
|
|
431
|
+
## Retry with Exponential Backoff
|
|
432
|
+
"""
|
|
433
|
+
async function withRetry<T>(
|
|
434
|
+
fn: () => Promise<T>,
|
|
435
|
+
maxRetries = 3,
|
|
436
|
+
baseDelay = 1000
|
|
437
|
+
): Promise<T> {
|
|
438
|
+
let lastError: Error;
|
|
439
|
+
|
|
440
|
+
for (let attempt = 0; attempt < maxRetries; attempt++) {
|
|
441
|
+
try {
|
|
442
|
+
return await fn();
|
|
443
|
+
} catch (error) {
|
|
444
|
+
lastError = error;
|
|
445
|
+
|
|
446
|
+
if (attempt < maxRetries - 1) {
|
|
447
|
+
const delay = baseDelay * Math.pow(2, attempt);
|
|
448
|
+
const jitter = delay * 0.1 * Math.random();
|
|
449
|
+
await new Promise(r => setTimeout(r, delay + jitter));
|
|
450
|
+
}
|
|
451
|
+
}
|
|
452
|
+
}
|
|
453
|
+
|
|
454
|
+
throw lastError;
|
|
455
|
+
}
|
|
456
|
+
|
|
457
|
+
// Usage
|
|
458
|
+
const result = await withRetry(
|
|
459
|
+
() => scrapeProduct(page, url),
|
|
460
|
+
3,
|
|
461
|
+
2000
|
|
462
|
+
);
|
|
463
|
+
"""
|
|
464
|
+
|
|
465
|
+
### Parallel Execution Pattern
|
|
466
|
+
|
|
467
|
+
Run tests/tasks in parallel for speed
|
|
468
|
+
|
|
469
|
+
**When to use**: Multiple independent pages or tests
|
|
470
|
+
|
|
471
|
+
# PARALLEL EXECUTION:
|
|
472
|
+
|
|
473
|
+
## Playwright Test Parallelization
|
|
474
|
+
"""
|
|
475
|
+
// playwright.config.ts
|
|
476
|
+
export default defineConfig({
|
|
477
|
+
fullyParallel: true,
|
|
478
|
+
workers: process.env.CI ? 4 : undefined, // CI: 4 workers, local: CPU-based
|
|
479
|
+
|
|
480
|
+
projects: [
|
|
481
|
+
{ name: 'chromium', use: { ...devices['Desktop Chrome'] } },
|
|
482
|
+
{ name: 'firefox', use: { ...devices['Desktop Firefox'] } },
|
|
483
|
+
{ name: 'webkit', use: { ...devices['Desktop Safari'] } },
|
|
484
|
+
],
|
|
485
|
+
});
|
|
486
|
+
"""
|
|
487
|
+
|
|
488
|
+
## Browser Contexts for Parallel Scraping
|
|
489
|
+
"""
|
|
490
|
+
const browser = await chromium.launch();
|
|
491
|
+
|
|
492
|
+
const urls = ['url1', 'url2', 'url3', 'url4', 'url5'];
|
|
493
|
+
|
|
494
|
+
// Create multiple contexts - each is isolated
|
|
495
|
+
const results = await Promise.all(
|
|
496
|
+
urls.map(async (url) => {
|
|
497
|
+
const context = await browser.newContext();
|
|
498
|
+
const page = await context.newPage();
|
|
499
|
+
|
|
500
|
+
try {
|
|
501
|
+
await page.goto(url);
|
|
502
|
+
const data = await extractData(page);
|
|
503
|
+
return { url, data, success: true };
|
|
504
|
+
} catch (error) {
|
|
505
|
+
return { url, error: error.message, success: false };
|
|
506
|
+
} finally {
|
|
507
|
+
await context.close();
|
|
508
|
+
}
|
|
509
|
+
})
|
|
510
|
+
);
|
|
511
|
+
|
|
512
|
+
await browser.close();
|
|
513
|
+
"""
|
|
514
|
+
|
|
515
|
+
## Rate-Limited Parallel Processing
|
|
516
|
+
"""
|
|
517
|
+
import pLimit from 'p-limit';
|
|
518
|
+
|
|
519
|
+
const limit = pLimit(5); // Max 5 concurrent
|
|
520
|
+
|
|
521
|
+
const results = await Promise.all(
|
|
522
|
+
urls.map(url => limit(async () => {
|
|
523
|
+
const context = await browser.newContext();
|
|
524
|
+
const page = await context.newPage();
|
|
525
|
+
|
|
526
|
+
// Random delay between requests
|
|
527
|
+
await new Promise(r => setTimeout(r, Math.random() * 2000));
|
|
528
|
+
|
|
529
|
+
try {
|
|
530
|
+
return await scrapePage(page, url);
|
|
531
|
+
} finally {
|
|
532
|
+
await context.close();
|
|
533
|
+
}
|
|
534
|
+
}))
|
|
535
|
+
);
|
|
536
|
+
"""
|
|
537
|
+
|
|
538
|
+
### Network Interception Pattern
|
|
539
|
+
|
|
540
|
+
Mock, block, or modify network requests
|
|
541
|
+
|
|
542
|
+
**When to use**: Testing, blocking ads/analytics, modifying responses
|
|
543
|
+
|
|
544
|
+
# NETWORK INTERCEPTION:
|
|
545
|
+
|
|
546
|
+
## Block Unnecessary Resources
|
|
547
|
+
"""
|
|
548
|
+
await page.route('**/*', (route) => {
|
|
549
|
+
const url = route.request().url();
|
|
550
|
+
const resourceType = route.request().resourceType();
|
|
551
|
+
|
|
552
|
+
// Block images, fonts, analytics for faster scraping
|
|
553
|
+
if (['image', 'font', 'media'].includes(resourceType)) {
|
|
554
|
+
return route.abort();
|
|
555
|
+
}
|
|
556
|
+
|
|
557
|
+
// Block tracking/analytics
|
|
558
|
+
if (url.includes('google-analytics') ||
|
|
559
|
+
url.includes('facebook.com/tr')) {
|
|
560
|
+
return route.abort();
|
|
561
|
+
}
|
|
562
|
+
|
|
563
|
+
return route.continue();
|
|
564
|
+
});
|
|
565
|
+
"""
|
|
566
|
+
|
|
567
|
+
## Mock API Responses (Testing)
|
|
568
|
+
"""
|
|
569
|
+
await page.route('**/api/products', async (route) => {
|
|
570
|
+
await route.fulfill({
|
|
571
|
+
status: 200,
|
|
572
|
+
contentType: 'application/json',
|
|
573
|
+
body: JSON.stringify([
|
|
574
|
+
{ id: 1, name: 'Mock Product', price: 99.99 },
|
|
575
|
+
]),
|
|
576
|
+
});
|
|
577
|
+
});
|
|
578
|
+
|
|
579
|
+
// Now page will receive mocked data
|
|
580
|
+
await page.goto('/products');
|
|
581
|
+
"""
|
|
582
|
+
|
|
583
|
+
## Capture API Responses
|
|
584
|
+
"""
|
|
585
|
+
const apiResponses: any[] = [];
|
|
586
|
+
|
|
587
|
+
page.on('response', async (response) => {
|
|
588
|
+
if (response.url().includes('/api/')) {
|
|
589
|
+
const data = await response.json().catch(() => null);
|
|
590
|
+
apiResponses.push({
|
|
591
|
+
url: response.url(),
|
|
592
|
+
status: response.status(),
|
|
593
|
+
data,
|
|
594
|
+
});
|
|
595
|
+
}
|
|
596
|
+
});
|
|
597
|
+
|
|
598
|
+
await page.goto('/dashboard');
|
|
599
|
+
// apiResponses now contains all API calls
|
|
600
|
+
"""
|
|
601
|
+
|
|
602
|
+
## Sharp Edges
|
|
603
|
+
|
|
604
|
+
### Using waitForTimeout Instead of Proper Waits
|
|
605
|
+
|
|
606
|
+
Severity: CRITICAL
|
|
607
|
+
|
|
608
|
+
Situation: Waiting for elements or page state
|
|
609
|
+
|
|
610
|
+
Symptoms:
|
|
611
|
+
Tests pass locally, fail in CI. Pass 9 times, fail on the 10th.
|
|
612
|
+
"Element not found" errors that seem random. Tests take 30+ seconds
|
|
613
|
+
when they should take 3.
|
|
614
|
+
|
|
615
|
+
Why this breaks:
|
|
616
|
+
waitForTimeout is a fixed delay. If the page loads in 500ms, you wait
|
|
617
|
+
2000ms anyway. If the page takes 2100ms (CI is slower), you fail.
|
|
618
|
+
There's no correct value - it's always either too short or too long.
|
|
619
|
+
|
|
620
|
+
Recommended fix:
|
|
621
|
+
|
|
622
|
+
# REMOVE all waitForTimeout calls
|
|
623
|
+
|
|
624
|
+
# WRONG:
|
|
625
|
+
await page.goto('/dashboard');
|
|
626
|
+
await page.waitForTimeout(2000); # Arbitrary!
|
|
627
|
+
await page.click('.submit');
|
|
628
|
+
|
|
629
|
+
# CORRECT - Auto-wait handles it:
|
|
630
|
+
await page.goto('/dashboard');
|
|
631
|
+
await page.getByRole('button', { name: 'Submit' }).click();
|
|
632
|
+
|
|
633
|
+
# If you need to wait for specific condition:
|
|
634
|
+
await expect(page.getByText('Dashboard')).toBeVisible();
|
|
635
|
+
await page.waitForURL('**/dashboard');
|
|
636
|
+
await page.waitForResponse(resp => resp.url().includes('/api/data'));
|
|
637
|
+
|
|
638
|
+
# For animations, wait for element to be stable:
|
|
639
|
+
await page.getByRole('button').click(); # Auto-waits for stable
|
|
640
|
+
|
|
641
|
+
# NEVER use setTimeout or waitForTimeout in production code
|
|
642
|
+
|
|
643
|
+
### CSS Selectors Tied to Styling Classes
|
|
644
|
+
|
|
645
|
+
Severity: HIGH
|
|
646
|
+
|
|
647
|
+
Situation: Selecting elements for interaction
|
|
648
|
+
|
|
649
|
+
Symptoms:
|
|
650
|
+
Tests break after CSS refactoring. Selectors like .btn-primary stop
|
|
651
|
+
working. Frontend redesign breaks all tests without changing behavior.
|
|
652
|
+
|
|
653
|
+
Why this breaks:
|
|
654
|
+
CSS class names are implementation details for styling, not semantic
|
|
655
|
+
meaning. When designers change from .btn-primary to .button--primary,
|
|
656
|
+
your tests break even though behavior is identical.
|
|
657
|
+
|
|
658
|
+
Recommended fix:
|
|
659
|
+
|
|
660
|
+
# Use user-facing locators instead:
|
|
661
|
+
|
|
662
|
+
# WRONG - Tied to CSS:
|
|
663
|
+
await page.locator('.btn-primary.submit-form').click();
|
|
664
|
+
await page.locator('#sidebar > div.menu > ul > li:nth-child(3)').click();
|
|
665
|
+
|
|
666
|
+
# CORRECT - User-facing:
|
|
667
|
+
await page.getByRole('button', { name: 'Submit' }).click();
|
|
668
|
+
await page.getByRole('menuitem', { name: 'Settings' }).click();
|
|
50
669
|
|
|
51
|
-
|
|
670
|
+
# If you must use CSS, use data-testid:
|
|
671
|
+
<button data-testid="submit-order">Submit</button>
|
|
52
672
|
|
|
53
|
-
|
|
673
|
+
await page.getByTestId('submit-order').click();
|
|
54
674
|
|
|
55
|
-
|
|
675
|
+
# Locator priority:
|
|
676
|
+
# 1. getByRole - matches accessibility
|
|
677
|
+
# 2. getByText - matches visible content
|
|
678
|
+
# 3. getByLabel - matches form labels
|
|
679
|
+
# 4. getByTestId - explicit test contract
|
|
680
|
+
# 5. CSS/XPath - last resort only
|
|
56
681
|
|
|
57
|
-
|
|
682
|
+
### navigator.webdriver Exposes Automation
|
|
58
683
|
|
|
59
|
-
|
|
60
|
-
|
|
61
|
-
|
|
62
|
-
|
|
63
|
-
|
|
64
|
-
|
|
65
|
-
|
|
66
|
-
|
|
67
|
-
|
|
68
|
-
|
|
684
|
+
Severity: HIGH
|
|
685
|
+
|
|
686
|
+
Situation: Scraping sites with bot detection
|
|
687
|
+
|
|
688
|
+
Symptoms:
|
|
689
|
+
Immediate 403 errors. CAPTCHA challenges. Empty pages. "Access Denied"
|
|
690
|
+
messages. Works for 1 request, then gets blocked.
|
|
691
|
+
|
|
692
|
+
Why this breaks:
|
|
693
|
+
By default, headless browsers set navigator.webdriver = true. This is
|
|
694
|
+
the first thing bot detection checks. It's a bright red flag that
|
|
695
|
+
says "I'm automated."
|
|
696
|
+
|
|
697
|
+
Recommended fix:
|
|
698
|
+
|
|
699
|
+
# Use stealth plugins:
|
|
700
|
+
|
|
701
|
+
## Puppeteer Stealth (best option):
|
|
702
|
+
import puppeteer from 'puppeteer-extra';
|
|
703
|
+
import StealthPlugin from 'puppeteer-extra-plugin-stealth';
|
|
704
|
+
|
|
705
|
+
puppeteer.use(StealthPlugin());
|
|
706
|
+
|
|
707
|
+
const browser = await puppeteer.launch({
|
|
708
|
+
headless: 'new',
|
|
709
|
+
args: ['--disable-blink-features=AutomationControlled'],
|
|
710
|
+
});
|
|
711
|
+
|
|
712
|
+
## Playwright Stealth:
|
|
713
|
+
import { chromium } from 'playwright-extra';
|
|
714
|
+
import stealth from 'puppeteer-extra-plugin-stealth';
|
|
715
|
+
|
|
716
|
+
chromium.use(stealth());
|
|
717
|
+
|
|
718
|
+
## Manual (partial):
|
|
719
|
+
await page.evaluateOnNewDocument(() => {
|
|
720
|
+
Object.defineProperty(navigator, 'webdriver', {
|
|
721
|
+
get: () => undefined,
|
|
722
|
+
});
|
|
723
|
+
});
|
|
724
|
+
|
|
725
|
+
# Note: This is cat-and-mouse. Detection evolves.
|
|
726
|
+
# For serious scraping, consider managed solutions like Browserbase.
|
|
727
|
+
|
|
728
|
+
### Tests Share State and Affect Each Other
|
|
729
|
+
|
|
730
|
+
Severity: HIGH
|
|
731
|
+
|
|
732
|
+
Situation: Running multiple tests in sequence
|
|
733
|
+
|
|
734
|
+
Symptoms:
|
|
735
|
+
Tests pass individually but fail when run together. Order matters -
|
|
736
|
+
test B fails if test A runs first. Random failures that "fix themselves"
|
|
737
|
+
on rerun.
|
|
738
|
+
|
|
739
|
+
Why this breaks:
|
|
740
|
+
Shared browser context means shared cookies, localStorage, and session
|
|
741
|
+
state. Test A logs in, test B expects logged-out state. Test A adds
|
|
742
|
+
item to cart, test B's cart count is wrong.
|
|
743
|
+
|
|
744
|
+
Recommended fix:
|
|
745
|
+
|
|
746
|
+
# Each test must be fully isolated:
|
|
747
|
+
|
|
748
|
+
## Playwright Test (automatic isolation):
|
|
749
|
+
test('first test', async ({ page }) => {
|
|
750
|
+
// Fresh context, fresh page
|
|
751
|
+
});
|
|
752
|
+
|
|
753
|
+
test('second test', async ({ page }) => {
|
|
754
|
+
// Completely isolated from first test
|
|
755
|
+
});
|
|
756
|
+
|
|
757
|
+
## Manual isolation:
|
|
758
|
+
const context = await browser.newContext(); // Fresh context
|
|
759
|
+
const page = await context.newPage();
|
|
760
|
+
// ... test code ...
|
|
761
|
+
await context.close(); // Clean up
|
|
762
|
+
|
|
763
|
+
## Shared authentication (the right way):
|
|
764
|
+
// 1. Save auth state to file
|
|
765
|
+
await context.storageState({ path: './auth.json' });
|
|
766
|
+
|
|
767
|
+
// 2. Reuse in other tests
|
|
768
|
+
const context = await browser.newContext({
|
|
769
|
+
storageState: './auth.json'
|
|
770
|
+
});
|
|
771
|
+
|
|
772
|
+
# Never modify global state in tests
|
|
773
|
+
# Never rely on previous test's actions
|
|
774
|
+
|
|
775
|
+
### No Trace Capture for CI Failures
|
|
776
|
+
|
|
777
|
+
Severity: MEDIUM
|
|
778
|
+
|
|
779
|
+
Situation: Debugging test failures in CI
|
|
780
|
+
|
|
781
|
+
Symptoms:
|
|
782
|
+
"Test failed in CI" with no useful information. Can't reproduce
|
|
783
|
+
locally. Screenshot shows page but not what went wrong. Guessing
|
|
784
|
+
at root cause.
|
|
785
|
+
|
|
786
|
+
Why this breaks:
|
|
787
|
+
CI runs headless on different hardware. Timing is different. Network
|
|
788
|
+
is different. Without traces, you can't see what actually happened -
|
|
789
|
+
the sequence of actions, network requests, console logs.
|
|
790
|
+
|
|
791
|
+
Recommended fix:
|
|
792
|
+
|
|
793
|
+
# Enable traces for failures:
|
|
794
|
+
|
|
795
|
+
## playwright.config.ts:
|
|
796
|
+
export default defineConfig({
|
|
797
|
+
use: {
|
|
798
|
+
trace: 'retain-on-failure', # Keep trace on failure
|
|
799
|
+
screenshot: 'only-on-failure', # Screenshot on failure
|
|
800
|
+
video: 'retain-on-failure', # Video on failure
|
|
801
|
+
},
|
|
802
|
+
outputDir: './test-results',
|
|
803
|
+
});
|
|
804
|
+
|
|
805
|
+
## View trace locally:
|
|
806
|
+
npx playwright show-trace test-results/path/to/trace.zip
|
|
807
|
+
|
|
808
|
+
## In CI, upload test-results as artifact:
|
|
809
|
+
# GitHub Actions:
|
|
810
|
+
- uses: actions/upload-artifact@v3
|
|
811
|
+
if: failure()
|
|
812
|
+
with:
|
|
813
|
+
name: playwright-traces
|
|
814
|
+
path: test-results/
|
|
815
|
+
|
|
816
|
+
# Trace shows:
|
|
817
|
+
# - Timeline of actions
|
|
818
|
+
# - Screenshots at each step
|
|
819
|
+
# - Network requests and responses
|
|
820
|
+
# - Console logs
|
|
821
|
+
# - DOM snapshots
|
|
822
|
+
|
|
823
|
+
### Tests Pass Headed but Fail Headless
|
|
824
|
+
|
|
825
|
+
Severity: MEDIUM
|
|
826
|
+
|
|
827
|
+
Situation: Running tests in headless mode for CI
|
|
828
|
+
|
|
829
|
+
Symptoms:
|
|
830
|
+
Works perfectly when you watch it. Fails mysteriously in CI.
|
|
831
|
+
"Element not visible" in headless but visible in headed mode.
|
|
832
|
+
|
|
833
|
+
Why this breaks:
|
|
834
|
+
Headless browsers have no display, which affects some CSS (visibility
|
|
835
|
+
calculations), viewport sizing, and font rendering. Some animations
|
|
836
|
+
behave differently. Popup windows may not work.
|
|
837
|
+
|
|
838
|
+
Recommended fix:
|
|
839
|
+
|
|
840
|
+
# Set consistent viewport:
|
|
841
|
+
const browser = await chromium.launch({
|
|
842
|
+
headless: true,
|
|
843
|
+
});
|
|
844
|
+
|
|
845
|
+
const context = await browser.newContext({
|
|
846
|
+
viewport: { width: 1280, height: 720 },
|
|
847
|
+
});
|
|
848
|
+
|
|
849
|
+
# Or in config:
|
|
850
|
+
export default defineConfig({
|
|
851
|
+
use: {
|
|
852
|
+
viewport: { width: 1280, height: 720 },
|
|
853
|
+
},
|
|
854
|
+
});
|
|
855
|
+
|
|
856
|
+
# Debug headless failures:
|
|
857
|
+
# 1. Run with headed mode locally
|
|
858
|
+
npx playwright test --headed
|
|
859
|
+
|
|
860
|
+
# 2. Slow down to watch
|
|
861
|
+
npx playwright test --headed --slowmo 100
|
|
862
|
+
|
|
863
|
+
# 3. Use trace viewer for CI failures
|
|
864
|
+
npx playwright show-trace trace.zip
|
|
865
|
+
|
|
866
|
+
# 4. For stubborn issues, screenshot at failure point:
|
|
867
|
+
await page.screenshot({ path: 'debug.png', fullPage: true });
|
|
868
|
+
|
|
869
|
+
### Getting Blocked by Rate Limiting
|
|
870
|
+
|
|
871
|
+
Severity: HIGH
|
|
872
|
+
|
|
873
|
+
Situation: Scraping multiple pages quickly
|
|
874
|
+
|
|
875
|
+
Symptoms:
|
|
876
|
+
Works for first 50 pages, then 429 errors. Suddenly all requests fail.
|
|
877
|
+
IP gets blocked. CAPTCHA starts appearing after successful requests.
|
|
878
|
+
|
|
879
|
+
Why this breaks:
|
|
880
|
+
Sites monitor request patterns. 100 requests per second from one IP
|
|
881
|
+
is obviously automated. Rate limits protect servers and catch scrapers.
|
|
882
|
+
|
|
883
|
+
Recommended fix:
|
|
884
|
+
|
|
885
|
+
# Add delays between requests:
|
|
886
|
+
|
|
887
|
+
const randomDelay = () =>
|
|
888
|
+
new Promise(r => setTimeout(r, 1000 + Math.random() * 2000));
|
|
889
|
+
|
|
890
|
+
for (const url of urls) {
|
|
891
|
+
await randomDelay(); // 1-3 second delay
|
|
892
|
+
await page.goto(url);
|
|
893
|
+
// ... scrape ...
|
|
894
|
+
}
|
|
895
|
+
|
|
896
|
+
# Use rotating proxies:
|
|
897
|
+
const proxies = ['http://proxy1:8080', 'http://proxy2:8080'];
|
|
898
|
+
let proxyIndex = 0;
|
|
899
|
+
|
|
900
|
+
const getNextProxy = () => proxies[proxyIndex++ % proxies.length];
|
|
901
|
+
|
|
902
|
+
const context = await browser.newContext({
|
|
903
|
+
proxy: { server: getNextProxy() },
|
|
904
|
+
});
|
|
905
|
+
|
|
906
|
+
# Limit concurrent requests:
|
|
907
|
+
import pLimit from 'p-limit';
|
|
908
|
+
const limit = pLimit(3); // Max 3 concurrent
|
|
909
|
+
|
|
910
|
+
await Promise.all(
|
|
911
|
+
urls.map(url => limit(() => scrapePage(url)))
|
|
912
|
+
);
|
|
913
|
+
|
|
914
|
+
# Rotate user agents:
|
|
915
|
+
const userAgents = [
|
|
916
|
+
'Mozilla/5.0 (Windows...',
|
|
917
|
+
'Mozilla/5.0 (Macintosh...',
|
|
918
|
+
];
|
|
919
|
+
|
|
920
|
+
await page.setExtraHTTPHeaders({
|
|
921
|
+
'User-Agent': userAgents[Math.floor(Math.random() * userAgents.length)]
|
|
922
|
+
});
|
|
923
|
+
|
|
924
|
+
### New Windows/Popups Not Handled
|
|
925
|
+
|
|
926
|
+
Severity: MEDIUM
|
|
927
|
+
|
|
928
|
+
Situation: Clicking links that open new windows
|
|
929
|
+
|
|
930
|
+
Symptoms:
|
|
931
|
+
Click button, nothing happens. Test hangs. "Window not found" errors.
|
|
932
|
+
Actions succeed but verification fails because you're on wrong page.
|
|
933
|
+
|
|
934
|
+
Why this breaks:
|
|
935
|
+
target="_blank" links open new windows. Your page reference still
|
|
936
|
+
points to the original page. The new window exists but you're not
|
|
937
|
+
listening for it.
|
|
938
|
+
|
|
939
|
+
Recommended fix:
|
|
940
|
+
|
|
941
|
+
# Wait for popup BEFORE triggering it:
|
|
942
|
+
|
|
943
|
+
## New window/tab:
|
|
944
|
+
const pagePromise = context.waitForEvent('page');
|
|
945
|
+
await page.getByRole('link', { name: 'Open in new tab' }).click();
|
|
946
|
+
const newPage = await pagePromise;
|
|
947
|
+
await newPage.waitForLoadState();
|
|
948
|
+
|
|
949
|
+
// Now interact with new page
|
|
950
|
+
await expect(newPage.getByRole('heading')).toBeVisible();
|
|
951
|
+
|
|
952
|
+
// Close when done
|
|
953
|
+
await newPage.close();
|
|
954
|
+
|
|
955
|
+
## Popup windows:
|
|
956
|
+
const popupPromise = page.waitForEvent('popup');
|
|
957
|
+
await page.getByRole('button', { name: 'Open popup' }).click();
|
|
958
|
+
const popup = await popupPromise;
|
|
959
|
+
await popup.waitForLoadState();
|
|
960
|
+
|
|
961
|
+
## Multiple windows:
|
|
962
|
+
const pages = context.pages(); // Get all open pages
|
|
963
|
+
|
|
964
|
+
### Can't Interact with Elements in iframes
|
|
965
|
+
|
|
966
|
+
Severity: MEDIUM
|
|
967
|
+
|
|
968
|
+
Situation: Page contains embedded iframes
|
|
969
|
+
|
|
970
|
+
Symptoms:
|
|
971
|
+
Element clearly visible but "not found". Selector works in DevTools
|
|
972
|
+
but not in Playwright. Parent page selectors work, iframe content
|
|
973
|
+
doesn't.
|
|
974
|
+
|
|
975
|
+
Why this breaks:
|
|
976
|
+
iframes are separate documents. page.locator only searches the main
|
|
977
|
+
frame. You need to explicitly get the iframe's frame to interact
|
|
978
|
+
with its contents.
|
|
979
|
+
|
|
980
|
+
Recommended fix:
|
|
981
|
+
|
|
982
|
+
# Get frame by name or selector:
|
|
983
|
+
|
|
984
|
+
## By frame name:
|
|
985
|
+
const frame = page.frame('payment-iframe');
|
|
986
|
+
await frame.getByRole('textbox', { name: 'Card number' }).fill('4242...');
|
|
987
|
+
|
|
988
|
+
## By selector:
|
|
989
|
+
const frame = page.frameLocator('iframe#payment');
|
|
990
|
+
await frame.getByRole('textbox', { name: 'Card number' }).fill('4242...');
|
|
991
|
+
|
|
992
|
+
## Nested iframes:
|
|
993
|
+
const outer = page.frameLocator('iframe#outer');
|
|
994
|
+
const inner = outer.frameLocator('iframe#inner');
|
|
995
|
+
await inner.getByRole('button').click();
|
|
996
|
+
|
|
997
|
+
## Wait for iframe to load:
|
|
998
|
+
await page.waitForSelector('iframe#payment');
|
|
999
|
+
const frame = page.frameLocator('iframe#payment');
|
|
1000
|
+
await frame.getByText('Secure Payment').waitFor();
|
|
1001
|
+
|
|
1002
|
+
## Validation Checks
|
|
1003
|
+
|
|
1004
|
+
### Using waitForTimeout
|
|
1005
|
+
|
|
1006
|
+
Severity: ERROR
|
|
1007
|
+
|
|
1008
|
+
waitForTimeout causes flaky tests and slow execution
|
|
1009
|
+
|
|
1010
|
+
Message: Using waitForTimeout - remove it. Playwright auto-waits for elements. Use waitForResponse, waitForURL, or assertions instead.
|
|
1011
|
+
|
|
1012
|
+
### Using setTimeout in Test Code
|
|
1013
|
+
|
|
1014
|
+
Severity: WARNING
|
|
1015
|
+
|
|
1016
|
+
setTimeout is unreliable for timing in tests
|
|
1017
|
+
|
|
1018
|
+
Message: Using setTimeout instead of Playwright waits. Replace with await expect(...).toBeVisible() or page.waitFor*.
|
|
1019
|
+
|
|
1020
|
+
### Custom Sleep Function
|
|
1021
|
+
|
|
1022
|
+
Severity: WARNING
|
|
1023
|
+
|
|
1024
|
+
Sleep functions indicate improper waiting strategy
|
|
1025
|
+
|
|
1026
|
+
Message: Custom sleep function detected. Use Playwright's built-in waiting mechanisms instead.
|
|
1027
|
+
|
|
1028
|
+
### CSS Class Selector Used
|
|
1029
|
+
|
|
1030
|
+
Severity: WARNING
|
|
1031
|
+
|
|
1032
|
+
CSS class selectors are fragile
|
|
1033
|
+
|
|
1034
|
+
Message: Using CSS class selector. Prefer getByRole, getByText, getByLabel, or getByTestId for more stable selectors.
|
|
1035
|
+
|
|
1036
|
+
### nth-child CSS Selector
|
|
1037
|
+
|
|
1038
|
+
Severity: WARNING
|
|
1039
|
+
|
|
1040
|
+
Position-based selectors are very fragile
|
|
1041
|
+
|
|
1042
|
+
Message: Using position-based selector. These break when DOM order changes. Use user-facing locators instead.
|
|
1043
|
+
|
|
1044
|
+
### XPath Selector Used
|
|
1045
|
+
|
|
1046
|
+
Severity: INFO
|
|
1047
|
+
|
|
1048
|
+
XPath should be last resort
|
|
1049
|
+
|
|
1050
|
+
Message: Using XPath selector. Consider getByRole, getByText first. XPath should be last resort for complex DOM traversal.
|
|
1051
|
+
|
|
1052
|
+
### Auto-Generated Selector
|
|
1053
|
+
|
|
1054
|
+
Severity: WARNING
|
|
1055
|
+
|
|
1056
|
+
Framework-generated selectors are extremely fragile
|
|
1057
|
+
|
|
1058
|
+
Message: Using auto-generated selector. These change on every build. Use data-testid instead.
|
|
1059
|
+
|
|
1060
|
+
### Puppeteer Without Stealth Plugin
|
|
1061
|
+
|
|
1062
|
+
Severity: INFO
|
|
1063
|
+
|
|
1064
|
+
Scraping without stealth is easily detected
|
|
1065
|
+
|
|
1066
|
+
Message: Using Puppeteer without stealth plugin. Consider puppeteer-extra-plugin-stealth for anti-detection.
|
|
1067
|
+
|
|
1068
|
+
### navigator.webdriver Not Hidden
|
|
1069
|
+
|
|
1070
|
+
Severity: INFO
|
|
1071
|
+
|
|
1072
|
+
navigator.webdriver exposes automation
|
|
1073
|
+
|
|
1074
|
+
Message: Launching browser without hiding automation flags. For scraping, add stealth measures.
|
|
1075
|
+
|
|
1076
|
+
### Scraping Loop Without Error Handling
|
|
1077
|
+
|
|
1078
|
+
Severity: WARNING
|
|
1079
|
+
|
|
1080
|
+
One failure shouldn't crash entire scrape
|
|
1081
|
+
|
|
1082
|
+
Message: Scraping loop without try/catch. One page failure will crash the entire scrape. Add error handling.
|
|
1083
|
+
|
|
1084
|
+
## Collaboration
|
|
1085
|
+
|
|
1086
|
+
### Delegation Triggers
|
|
1087
|
+
|
|
1088
|
+
- user needs full desktop control beyond browser -> computer-use-agents (Desktop automation for non-browser apps)
|
|
1089
|
+
- user needs API testing alongside browser tests -> backend (API integration and testing patterns)
|
|
1090
|
+
- user needs testing strategy -> test-architect (Overall test architecture decisions)
|
|
1091
|
+
- user needs visual regression testing -> ui-design (Visual comparison and design validation)
|
|
1092
|
+
- user needs browser automation in workflows -> workflow-automation (Durable execution for browser tasks)
|
|
1093
|
+
- user building browser tools for agents -> agent-tool-builder (Tool design patterns for LLM agents)
|
|
69
1094
|
|
|
70
1095
|
## Related Skills
|
|
71
1096
|
|
|
72
1097
|
Works well with: `agent-tool-builder`, `workflow-automation`, `computer-use-agents`, `test-architect`
|
|
73
1098
|
|
|
74
1099
|
## When to Use
|
|
75
|
-
|
|
1100
|
+
|
|
1101
|
+
- User mentions or implies: playwright
|
|
1102
|
+
- User mentions or implies: puppeteer
|
|
1103
|
+
- User mentions or implies: browser automation
|
|
1104
|
+
- User mentions or implies: headless
|
|
1105
|
+
- User mentions or implies: web scraping
|
|
1106
|
+
- User mentions or implies: e2e test
|
|
1107
|
+
- User mentions or implies: end-to-end
|
|
1108
|
+
- User mentions or implies: selenium
|
|
1109
|
+
- User mentions or implies: chromium
|
|
1110
|
+
- User mentions or implies: browser test
|
|
1111
|
+
- User mentions or implies: page.click
|
|
1112
|
+
- User mentions or implies: locator
|