openspec-playwright 0.1.73 → 0.1.75

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -16,7 +16,9 @@ metadata:
16
16
 
17
17
  ## Output
18
18
 
19
- - **Test file**: `tests/playwright/<name>.spec.ts` (e.g. `app-all.spec.ts` for "all")
19
+ - **Test file**: `tests/playwright/<name>.spec.ts`
20
+ - **Page Objects** (all mode): `tests/playwright/pages/<Route>Page.ts`
21
+ - **Auth setup**: `tests/playwright/auth.setup.ts` (if auth required)
20
22
  - **Auth setup**: `tests/playwright/auth.setup.ts` (if auth required)
21
23
  - **Report**: `openspec/reports/playwright-e2e-<name>-<timestamp>.md`
22
24
  - **Test plan**: `openspec/changes/<name>/specs/playwright/test-plan.md` (change mode only)
@@ -25,14 +27,14 @@ metadata:
25
27
 
26
28
  Two modes, same pipeline:
27
29
 
28
- | Mode | Command | Route source | Output |
29
- | ------ | ------------------ | ------------------------ | ----------------- |
30
- | Change | `/opsx:e2e <name>` | OpenSpec specs | `<name>.spec.ts` |
31
- | All | `/opsx:e2e all` | sitemap + homepage crawl | `app-all.spec.ts` |
30
+ | Mode | Command | Route source | Output |
31
+ | ------ | ------------------ | ------------------------ | ------------------------------- |
32
+ | Change | `/opsx:e2e <name>` | OpenSpec specs | `<name>.spec.ts` |
33
+ | All | `/opsx:e2e all` | sitemap + homepage crawl | `pages/*.ts` (Page Objects) |
32
34
 
33
35
  Both modes update `app-knowledge.md` and `app-exploration.md`. All `.spec.ts` files run together as regression suite.
34
36
 
35
- > **Role mapping**: Planner (Step 4–5) → test-plan.md; Generator (Step 6) → `.spec.ts` with verified selectors; Healer (Step 9) → repairs failures via MCP.
37
+ > **Role mapping**: Planner (Step 4–5) → test-plan.md; Generator (Step 6) → `.spec.ts` + Page Objects; Healer (Step 9) → repairs failures via MCP.
36
38
 
37
39
  ## Testing principles
38
40
 
@@ -71,7 +73,8 @@ Can this be tested through the UI?
71
73
 
72
74
  **"all" mode** (`/opsx:e2e all` — no OpenSpec needed):
73
75
 
74
- - Announce: "Mode: full app exploration"
76
+ - Announce: "Mode: full app exploration + Page Object discovery"
77
+ - **Goal**: Discover new routes, extract selectors, and build `pages/*.ts` Page Objects — accumulated asset for future Change tests
75
78
  - Discover routes via:
76
79
  1. Navigate to `${BASE_URL}/sitemap.xml` (if exists)
77
80
  2. Navigate to `${BASE_URL}/` → extract all links from snapshot
@@ -138,6 +141,9 @@ browser_navigate → browser_console_messages → browser_snapshot → browser_t
138
141
  | JS error in console | App runtime error | **STOP** — tell user: "Page has JS errors. Fix them, then re-run /opsx:e2e." |
139
142
  | HTTP 404 | Route not in app (metadata issue) | Continue — mark `⚠️ route not found` in app-exploration.md |
140
143
  | Auth required, no credentials | Missing auth setup | Continue — skip protected routes, explore login page |
144
+ | Suspicious network request | API returned 4xx/5xx | Continue — mark `⚠️ API error: <endpoint> returned <code>` in app-exploration.md |
145
+
146
+ **Network monitoring**: After navigating, use `browser_network_requests` to check for failed API calls. Failed requests (status ≥ 400) on a route indicate an API/backend issue — record in `app-exploration.md` for reference.
141
147
 
142
148
  **For guest routes** (no auth):
143
149
 
@@ -190,11 +196,14 @@ From `browser_snapshot` + `browser_evaluate`, identify these special elements pe
190
196
  | ------- | --------------- | ---------------------------------------------- | ------------------- |
191
197
  | `<canvas>` | `role="img"`, `tagName="CANVAS"` | `canvas.getContext('2d'/'webgl')`, `width`, `height` | High |
192
198
  | `<iframe>` | `role="iframe"`, `src` attribute | `frameLocator` available | High |
199
+ | CAPTCHA | `.g-recaptcha`, `.h-captcha`, `[data-sitekey]`, canvas+slider | recaptcha score via API (if configured) | High |
200
+ | OTP / SMS | 6-digit input, countdown timer | Check if dev bypass exists | High |
193
201
  | Shadow DOM | `role="generic"` with no children | Check `shadowRoot` via evaluate | Medium |
194
202
  | Rich text editor | `[contenteditable]`, `role="textbox"` | `innerHTML`, `getContent()` | Medium |
195
203
  | Video / Audio | `role="application"` or name contains "video"/"audio" | `evaluate` checks both `<video>` and `<audio>` tags | Medium |
196
- | Date picker | specific `data-testid` or class patterns | Click triggers evaluate value | Low (skip unless specs mention) |
204
+ | File upload | `<input type="file">` | `accept` attribute, `multiple` flag | Medium |
197
205
  | Drag-and-drop | drag events in JS | Simulate DnD via coordinate clicks | Low |
206
+ | Date picker | specific `data-testid` or class patterns | Click triggers → evaluate value | Low (skip unless specs mention) |
198
207
  | Infinite scroll | Dynamic row insertion | Count elements before/after scroll | Low |
199
208
  | WebSocket / SSE | No DOM signal | Check `browser_console_messages` for WS events | Low |
200
209
 
@@ -241,6 +250,29 @@ const isContentEditable = await browser_evaluate(() => {
241
250
  const el = document.querySelector('[contenteditable]');
242
251
  return !!el;
243
252
  });
253
+
254
+ // CAPTCHA — detect type
255
+ const captchaInfo = await browser_evaluate(() => {
256
+ const recaptcha = document.querySelector('.g-recaptcha, [data-sitekey]');
257
+ if (recaptcha) return { type: 'recaptcha', sitekey: recaptcha.getAttribute('data-sitekey') };
258
+ const hcaptcha = document.querySelector('.h-captcha');
259
+ if (hcaptcha) return { type: 'hcaptcha', sitekey: hcaptcha.getAttribute('data-sitekey') };
260
+ const turnstile = document.querySelector('[data-sitekey*="cloudflare"]');
261
+ if (turnstile) return { type: 'turnstile' };
262
+ const canvas = document.querySelector('canvas[class*="captcha"]');
263
+ if (canvas) return { type: 'canvas-captcha' };
264
+ const slider = document.querySelector('[class*="slider"], [class*="drag"]');
265
+ if (slider) return { type: 'slider-captcha' };
266
+ return null;
267
+ });
268
+
269
+ // OTP input — detect
270
+ const otpInfo = await browser_evaluate(() => {
271
+ const inputs = document.querySelectorAll('input');
272
+ const otpInputs = Array.from(inputs).filter(i => i.maxLength === 1 && i.type === 'text' || i.type === 'tel');
273
+ if (otpInputs.length >= 4) return { type: 'otp-sms', digits: otpInputs.length };
274
+ return null;
275
+ });
244
276
  ```
245
277
 
246
278
  Record findings in `app-exploration.md` → **Special Elements Detected** table.
@@ -297,7 +329,16 @@ Read `tests/playwright/app-knowledge.md` as context for cross-change patterns.
297
329
 
298
330
  ### 5. Generate test plan
299
331
 
300
- > **"all" mode: skip this step go directly to Step 6.**
332
+ > **"all" mode: skip this step.** No OpenSpec specs → no test-plan to generate. All mode skips test-plan verification — Page Objects are discovered incrementally from exploration, not from structured specs.
333
+
334
+ **All mode — brief confirmation before Step 6:**
335
+ ```
336
+ ## All Mode: Page Object Discovery
337
+ Discovered <N> routes (<M> guest, <K> protected)
338
+ Special elements: <element summary>
339
+ Ready to generate Page Objects for: <page-name>Page.ts, <page-name>Page.ts, ...
340
+ Reply **yes** to proceed, or tell me to exclude routes or adjust strategies.
341
+ ```
301
342
 
302
343
  **Change mode — prerequisite**: If `openspec/changes/<name>/specs/playwright/app-exploration.md` does not exist → **STOP**. Run Step 4 (explore application) before generating tests. Without real DOM data from exploration, selectors are guesses and tests will be fragile.
303
344
 
@@ -311,13 +352,62 @@ Template: `templates/test-plan.md`
311
352
 
312
353
  **Idempotency**: If test-plan.md exists → read and use, do NOT regenerate.
313
354
 
355
+ **⚠️ Human verification — STOP before generating code.**
356
+
357
+ After creating (or reading existing) test-plan.md, **stop and display the test plan summary** for user confirmation:
358
+
359
+ **Output format** — show the test plan in markdown directly in the conversation:
360
+
361
+ ````markdown
362
+ ## Test Plan Summary: `<change-name>`
363
+
364
+ **Auth**: required / not required | Roles: ...
365
+
366
+ ### Test Cases
367
+ - ✅ `<test-name>` — `<route>`, happy path
368
+ - ✅ `<test-name>` — `<route>`, error path: `<error condition>`
369
+
370
+ ### Special Elements
371
+ - ⚠️ **CAPTCHA** at `<route>` — strategy: `auth.setup bypass / skip / api-only`
372
+ - ⚠️ **Canvas/WebGL** at `<route>` — strategy: screenshot + dimensions
373
+ - ⚠️ **OTP** at `<route>` — strategy: test credentials / dev bypass
374
+ - ⚠️ **Iframe** at `<route>` — strategy: frameLocator + assert inner content
375
+ - ⚠️ **Video/Audio** at `<route>` — strategy: play() + assert !paused
376
+ - ⚠️ **File Upload** at `<route>` — strategy: setInputFiles + assert upload
377
+ - ⚠️ **Drag-and-Drop** at `<route>` — strategy: dragAndDrop or evaluate events
378
+ - ⚠️ **WebSocket/SSE** at `<route>` — strategy: waitForResponse + waitForFunction
379
+
380
+ ### Not Covered
381
+ - `<element or scenario not testable>`
382
+ ````
383
+
384
+ Then ask: "Does this coverage match your intent? Reply **yes** to proceed, or tell me what to add/change."
385
+
386
+ **Why this matters**: Step 5 is the last human-reviewable checkpoint before code generation. Once test code is written, fixes address *how* tests run, not *what* they verify. Reviewing the test plan takes seconds and catches logic errors that Healer cannot fix.
387
+
388
+ **Confirmation criteria**:
389
+ - All scenarios from OpenSpec specs are covered
390
+ - Special elements (Canvas, Iframe, Video, Audio, CAPTCHA, OTP, File Upload, Drag-drop, WebSocket) have correct automation strategy
391
+ - Auth states and roles are accurate
392
+ - Nothing important is missing
393
+
394
+ If the user requests changes → update test-plan.md → re-display summary → re-confirm → proceed.
395
+
314
396
  ### 6. Generate test file
315
397
 
316
- **"all" mode** `tests/playwright/app-all.spec.ts` (smoke regression):
398
+ **"all" mode**: Build and expand Page Objects for future Change tests.
399
+
400
+ **Prerequisite**: If `app-exploration.md` does not exist → **STOP**. Run Step 4 first. All mode explores routes via browser MCP to build exploration data.
317
401
 
318
- - For each discovered route: navigate → assert HTTP 200 → assert ready signal visible
319
- - No detailed assertions — just "this page loads without crashing"
320
- - This is a regression baselinecatches when existing pages break
402
+ For each discovered route:
403
+
404
+ 1. Read existing `pages/<Route>Page.ts` (if anyincremental, not overwrite)
405
+ 2. Navigate to route with correct auth state
406
+ 3. browser_snapshot to extract interactive elements (see 4.3 table)
407
+ 4. Write or update `pages/<Route>Page.ts` — extend with newly discovered elements
408
+ 5. Also write `tests/playwright/app-all.spec.ts` — smoke test (route loads without crash)
409
+
410
+ **Output priority**: Page Objects (`pages/*.ts`) are the primary asset. Smoke test is secondary. Existing Page Objects are never overwritten — only extended.
321
411
 
322
412
  **Change mode** → `tests/playwright/<name>.spec.ts` (functional):
323
413
 
@@ -399,9 +489,48 @@ test('video can be played', async ({ page }) => {
399
489
  const isPlaying = await video.evaluate((v: HTMLVideoElement) => !v.paused);
400
490
  expect(isPlaying).toBe(true);
401
491
  });
492
+
493
+ // Audio — playback state
494
+ test('audio can be played', async ({ page }) => {
495
+ await page.goto(`${BASE_URL}/<route>`);
496
+ const audio = page.locator('audio');
497
+ await expect(audio).toBeVisible();
498
+ await audio.evaluate((a: HTMLAudioElement) => { a.play(); });
499
+ const isPlaying = await audio.evaluate((a: HTMLAudioElement) => !a.paused);
500
+ expect(isPlaying).toBe(true);
501
+ });
402
502
  ```
403
503
 
404
- See `templates/test-plan.md` → **Special Element Test Cases** for full templates.
504
+ See `templates/test-plan.md` → **Special Element Test Cases** for full templates including Canvas, Video, Audio, Iframe, and Rich Text Editor.
505
+
506
+ **Test coverage — AI-opaque elements**: For CAPTCHA, OTP, slider CAPTCHA, file upload, and drag-drop — elements that Playwright cannot reliably automate:
507
+
508
+ 1. Mark the element in `app-exploration.md` → **Special Elements Detected** table with type and automation strategy
509
+ 2. Generate the test using the appropriate strategy from `templates/test-plan.md` → **AI-Opaque Elements** section:
510
+ - **CAPTCHA**: Bypass via `auth.setup.ts` storageState, or skip with `test.skip()`, or verify via API
511
+ - **OTP**: Use pre-verified test credentials (`E2E_OTP_CODE` env var), or development bypass flag
512
+ - **File upload**: Use `page.setInputFiles()` with fixture files
513
+ - **Drag-drop**: Use `page.dragAndDrop()` or `page.evaluate()` with custom event dispatching
514
+ 3. If the element is truly non-automatable, write `test.skip()` with a comment explaining why, and mark with `/handoff` for manual testing
515
+
516
+ **Test coverage — performance**: Verify Core Web Vitals metrics. If the app specifies performance targets, generate a test:
517
+
518
+ ```typescript
519
+ // Performance — Core Web Vitals
520
+ test('page loads within performance budget', async ({ page }) => {
521
+ await page.goto(`${BASE_URL}/<route>`);
522
+ await expect(page.getByRole('heading')).toBeVisible();
523
+ const timings = await page.evaluate(() => {
524
+ const nav = performance.getEntriesByType('navigation')[0] as PerformanceNavigationTiming;
525
+ return {
526
+ ttfb: nav.responseStart - nav.requestStart,
527
+ lcp: nav.loadEventEnd - nav.requestStart,
528
+ };
529
+ });
530
+ expect(timings.ttfb).toBeLessThan(500);
531
+ expect(timings.lcp).toBeLessThan(2500);
532
+ });
533
+ ```
405
534
 
406
535
  ```typescript
407
536
  // 🚫 Avoid for special elements:
@@ -425,7 +554,7 @@ expect(box.width).toBeGreaterThan(0);
425
554
  Read `tests/playwright/pages/BasePage.ts` for shared utilities:
426
555
  - `goto(path)` — navigation with configurable `waitUntil`
427
556
  - `byTestId(id)`, `byRole(role, opts)`, `byLabel(label)`, `byText(text)`, `byPlaceholder(text)` — selector helpers in priority order
428
- - `click(locator)`, `fill(locator, value)` — safe interactions with built-in `scrollIntoViewIfNeeded`
557
+ - `click(locator)`, `fill(locator, value)`, `fillAndVerify(locator, value)` — safe interactions; use `fillAndVerify` when the next action depends on the value being committed
429
558
  - `waitForToast(text?)`, `waitForLoad(spinnerSelector?)` — wait utilities
430
559
  - `reload()` — page reload with hydration
431
560
 
@@ -445,8 +574,8 @@ export class LoginPage extends BasePage {
445
574
 
446
575
  async login(user: string, pass: string) {
447
576
  await this.goto('/login');
448
- await this.usernameInput.fill(user);
449
- await this.passwordInput.fill(pass);
577
+ await this.fillAndVerify(this.usernameInput, user);
578
+ await this.fillAndVerify(this.passwordInput, pass);
450
579
  await this.submitBtn.click();
451
580
  }
452
581
  }
@@ -501,12 +630,13 @@ await app.click(app.byRole('button', { name: '提交' }));
501
630
  **Code examples — UI first:**
502
631
 
503
632
  ```typescript
504
- // ✅ UI 测试
505
- await page.goto(`${BASE_URL}/orders`);
506
- await page.getByRole("button", { name: "新建订单" }).click();
507
- await page.getByLabel("订单名称").fill("Test Order");
508
- await page.getByRole("button", { name: "提交" }).click();
509
- await expect(page.getByText("订单创建成功")).toBeVisible();
633
+ // ✅ UI 测试 — fill 后必须验证值,确保框架同步完成
634
+ const app = new AppPage(page);
635
+ await app.goto(`${BASE_URL}/orders`);
636
+ await app.click(app.byRole('button', { name: '新建订单' }));
637
+ await app.fillAndVerify(app.byLabel('订单名称'), 'Test Order');
638
+ await app.click(app.byRole('button', { name: '提交' }));
639
+ await expect(page.getByText('订单创建成功')).toBeVisible();
510
640
 
511
641
  // ✅ Error path
512
642
  await page.goto(`${BASE_URL}/orders`);
@@ -599,13 +729,12 @@ If tests fail → use Playwright MCP tools to inspect UI, fix selectors, re-run.
599
729
 
600
730
  **Healer MCP tools** (in order of use):
601
731
 
602
- <!-- MCP_VERSION: 0.0.70 -->
603
-
604
732
  | Tool | Purpose |
605
733
  | -------------------------- | ----------------------------------------------- |
606
734
  | `browser_navigate` | Go to the failing test's page |
607
735
  | `browser_snapshot` | Get page structure to find equivalent selectors |
608
736
  | `browser_console_messages` | Diagnose JS errors that may cause failures |
737
+ | `browser_network_requests` | Diagnose backend/API failures (4xx/5xx) |
609
738
  | `browser_take_screenshot` | Visually compare before/after fixes |
610
739
  | `browser_run_code` | Execute custom fix logic (optional) |
611
740
 
@@ -616,7 +745,7 @@ If tests fail → use Playwright MCP tools to inspect UI, fix selectors, re-run.
616
745
 
617
746
  | Failure type | Signal | Action |
618
747
  | ---------------------------- | --------------------------------------- | ----------------------------------------------------- |
619
- | **Network/backend** | `fetch failed`, `net::ERR`, 5xx | `browser_console_messages` → `test.skip()` |
748
+ | **Network/backend** | `fetch failed`, `net::ERR`, 4xx/5xx | `browser_network_requests` → `browser_console_messages` → `test.skip()` |
620
749
  | **Selector changed** | Element not found | `browser_snapshot` → fix selector → re-run |
621
750
  | **Assertion mismatch** | Wrong content/value | `browser_snapshot` → compare → fix assertion → re-run |
622
751
  | **Timing issue** | `waitFor`/`page.evaluate` timeout | Switch to `request` API or add `waitFor` → re-run |
@@ -656,7 +785,7 @@ Reference: `templates/report.md`
656
785
  | No specs / app-exploration.md missing (change mode) | **STOP** |
657
786
  | JS errors or HTTP 5xx during exploration | **STOP** |
658
787
  | Sitemap fails ("all" mode) | Continue with homepage links fallback |
659
- | File already exists (app-exploration, test-plan, app-all) | Read and use — never regenerate |
788
+ | File already exists (app-exploration, test-plan, app-all.spec.ts, Page Objects) | Read and use — never regenerate |
660
789
  | Test fails (backend) | `test.skip()` + report |
661
790
  | Test fails (selector/assertion) | Healer: snapshot → fix → re-run (≤3) |
662
791
  | 3 heals failed | `test.skip()` if app bug; report if unclear |
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "openspec-playwright",
3
- "version": "0.1.73",
3
+ "version": "0.1.75",
4
4
  "description": "OpenSpec + Playwright E2E verification setup tool for Claude Code",
5
5
  "type": "module",
6
6
  "bin": {
@@ -46,7 +46,9 @@ BASE_URL: <from env or seed.spec.ts>
46
46
  | | | | | |
47
47
  | | | | | |
48
48
 
49
- > **Special element type legend**: `canvas-2d` | `canvas-webgl` | `iframe` | `shadow-dom` | `contenteditable` | `video` | `audio` | `datepicker` | `drag-drop` | `infinite-scroll`
49
+ > **Special element type legend**: `canvas-2d` | `canvas-webgl` | `iframe` | `shadow-dom` | `contenteditable` | `video` | `audio` | `datepicker` | `drag-drop` | `infinite-scroll` | `file-upload` | `captcha-image` | `captcha-slider` | `captcha-3d` | `recaptcha` | `hcaptcha` | `turnstile` | `otp-sms` | `otp-totp` | `websocket` | `sse`
50
+ >
51
+ > **AI-opaque strategy**: CAPTCHA/OTP/slider-CAPTCHA — see `templates/test-plan.md` → **AI-Opaque Elements**
50
52
  >
51
53
  > **Test strategy**: See `templates/test-plan.md` → **Special Element Test Cases**
52
54
 
@@ -68,10 +68,11 @@ setup('authenticate via UI', async ({ page }) => {
68
68
 
69
69
  // Common selector patterns (uncomment the one that matches):
70
70
  await page.fill('[data-testid="username"]', username);
71
+ await expect(page.locator('[data-testid="username"]')).toHaveValue(username);
71
72
  // await page.fill('input[name="email"]', username);
72
- // await page.fill('input[type="email"]', username);
73
73
 
74
74
  await page.fill('[data-testid="password"]', password);
75
+ await expect(page.locator('[data-testid="password"]')).toHaveValue(password);
75
76
  // await page.fill('input[name="password"]', password);
76
77
 
77
78
  await page.click('[data-testid="login-button"]');
@@ -18,8 +18,8 @@ const BASE_URL = process.env.BASE_URL || 'http://localhost:3000';
18
18
  * get passwordInput() { return this.byLabel('密码'); }
19
19
  * get submitBtn() { return this.byRole('button', { name: '登录' }); }
20
20
  * async login(user: string, pass: string) {
21
- * await this.usernameInput.fill(user);
22
- * await this.passwordInput.fill(pass);
21
+ * await this.fillAndVerify(this.usernameInput, user);
22
+ * await this.fillAndVerify(this.passwordInput, pass);
23
23
  * await this.submitBtn.click();
24
24
  * }
25
25
  * }
@@ -82,23 +82,40 @@ export class BasePage {
82
82
  }
83
83
 
84
84
  /**
85
- * Fill with automatic scroll-into-view. Clears existing value first.
85
+ * Fill with automatic scroll-into-view + blur.
86
+ * blur() triggers framework (Vue/React) change events and reactive updates,
87
+ * ensuring form state syncs before the next action.
88
+ * For fields with debounced validation, use fillAndVerify() instead.
86
89
  */
87
90
  async fill(selector: Locator | string, value: string) {
88
91
  const el = typeof selector === 'string' ? this.page.locator(selector) : selector;
89
92
  await el.scrollIntoViewIfNeeded();
90
93
  await el.fill(value);
94
+ await el.blur();
91
95
  }
92
96
 
93
97
  /**
94
- * Type with character-by-character input. Triggers keydown/keyup events.
95
- * Use for editors and inputs that listen to keystroke events.
98
+ * Type with character-by-character input + blur. Same sync guarantee as fill().
96
99
  */
97
100
  async type(selector: Locator | string, text: string) {
98
101
  const el = typeof selector === 'string' ? this.page.locator(selector) : selector;
99
102
  await el.scrollIntoViewIfNeeded();
100
103
  await el.click();
101
104
  await this.page.keyboard.type(text);
105
+ await el.blur();
106
+ }
107
+
108
+ // ─── Verified interactions ─────────────────────────────────────────────────
109
+
110
+ /**
111
+ * Fill + verify: waits for the value to appear in the input.
112
+ * Use for fields with debounced validation, async handlers, or
113
+ * when the next action depends on the value being fully committed.
114
+ */
115
+ async fillAndVerify(selector: Locator | string, value: string) {
116
+ await this.fill(selector, value);
117
+ const el = typeof selector === 'string' ? this.page.locator(selector) : selector;
118
+ await expect(el).toHaveValue(value);
102
119
  }
103
120
 
104
121
  // ─── Wait utilities ─────────────────────────────────────────────────────────
@@ -127,9 +127,10 @@ test.describe('Environment validation', () => {
127
127
 
128
128
  // test.describe('Error handling', () => {
129
129
  // test('shows error message on invalid input', async ({ page }) => {
130
- // await page.goto(`${BASE_URL}/submit`);
131
- // await page.getByTestId('input').fill('');
132
- // await page.getByTestId('submit').click();
130
+ // const app = createPage(page);
131
+ // await app.goto(`${BASE_URL}/submit`);
132
+ // await app.fillAndVerify(app.byTestId('input'), '');
133
+ // await app.click(app.byTestId('submit'));
133
134
  // await expect(page.getByTestId('error')).toContainText('不能为空');
134
135
  // });
135
136
  // });
@@ -98,4 +98,156 @@ Generated from: `openspec/changes/<change-name>/specs/`
98
98
  2. Call `audio.play()` via `page.evaluate()`
99
99
  3. Assert `!audio.paused`
100
100
 
101
+ ## AI-Opaque Elements: Out of Scope for Automation
102
+
103
+ <!-- Elements that AI cannot reliably interact with. Mark in `app-exploration.md` → Special Elements table. -->
104
+
105
+ ### CAPTCHA — Human Verification Required
106
+
107
+ - **Route**: `/<page>`
108
+ - **Type**: `captcha-image | captcha-slider | captcha-3d | recaptcha | hcaptcha | turnstile`
109
+ - **Element**: `<canvas id="...">`, `.g-recaptcha`, `.h-captcha`, `[data-sitekey]`, slider track
110
+
111
+ **Automation strategy — choose one:**
112
+
113
+ #### Strategy A: Auth-Setup Bypass (Recommended)
114
+ Use pre-authenticated sessions from `auth.setup.ts` so CAPTCHA is bypassed before the test runs.
115
+
116
+ ```
117
+ 1. Store authenticated storageState in auth.setup.ts
118
+ 2. In test: use storageState to skip login
119
+ 3. Verify post-CAPTCHA state via API instead
120
+ 4. Assert: API response after CAPTCHA = expected state
121
+ ```
122
+
123
+ **Example assertions:**
124
+ - `API POST /submit-form` returns 200 + `success: true`
125
+ - UI redirects to `/dashboard` after CAPTCHA pass
126
+ - Database record created with correct values
127
+
128
+ #### Strategy B: Skip with /handoff
129
+ If CAPTCHA is the primary interaction under test, mark as non-automatable:
130
+
131
+ ```
132
+ test.skip('<feature> requires CAPTCHA', async ({ page }) => {
133
+ // Human reviewer must manually pass CAPTCHA
134
+ // Then run: npx playwright test --grep "<feature>"
135
+ });
136
+ ```
137
+
138
+ #### Strategy C: API Verification Only
139
+ Test the result of CAPTCHA-protected actions without automating the CAPTCHA:
140
+
141
+ ```
142
+ 1. Manually pass CAPTCHA once → capture resulting session/token
143
+ 2. Use that session in subsequent API calls
144
+ 3. Assert: API behavior matches CAPTCHA-passed state
145
+ ```
146
+
147
+ **Detected CAPTCHA type** (fill in based on `app-exploration.md`):
148
+ - Type: `<!-- image | slider | 3d-object | recaptcha | hcaptcha | turnstile -->`
149
+ - Bypass method: `<!-- auth.setup | skip | api-only -->`
150
+ - Verified by: `<!-- human-manual | api-response | db-state -->`
151
+
152
+ ---
153
+
154
+ ### OTP / SMS Verification — Test Account Bypass
155
+
156
+ - **Route**: `/<page>`
157
+ - **Type**: `otp-sms | otp-email | totp`
158
+ - **Element**: 6-digit input, countdown timer, resend button
159
+
160
+ **Automation strategy:**
161
+
162
+ #### Strategy A: Test Credentials (Recommended)
163
+ Use pre-verified test accounts with known OTP codes.
164
+
165
+ ```
166
+ 1. Set E2E_OTP_CODE=<valid-test-code> in credentials.yaml
167
+ 2. In test:
168
+ await page.getByRole('textbox').fill(process.env.E2E_OTP_CODE);
169
+ await page.getByRole('button', { name: 'Verify' }).click();
170
+ ```
171
+
172
+ #### Strategy B: Development Bypass Flag
173
+ If dev mode disables OTP, use that flag.
174
+
175
+ ```
176
+ 1. Set BASE_URL to dev environment with OTP disabled
177
+ 2. Proceed as normal authenticated user
178
+ ```
179
+
180
+ #### Strategy C: API-Only Verification
181
+ Bypass OTP UI entirely and test the protected endpoint directly.
182
+
183
+ ```
184
+ 1. Obtain valid token via API (skip OTP step)
185
+ 2. Use token in subsequent API calls
186
+ 3. Assert: protected endpoint returns expected data
187
+ ```
188
+
189
+ ---
190
+
191
+ ### File Upload — Complex Input
192
+
193
+ - **Route**: `/<page>`
194
+ - **Type**: `file-upload`
195
+ - **Element**: `<input type="file">`, drag-and-drop zone
196
+
197
+ **Test approach:**
198
+ ```
199
+ 1. Create test fixture file: `test fixtures/login-hero.png`
200
+ 2. Set `accept: '<mime-type>'` if specified
201
+ 3. Use `page.setInputFiles()` for reliable upload
202
+ 4. Assert: upload progress completes → success message or preview
203
+ ```
204
+
205
+ **Assertions:**
206
+ - Upload progress bar completes (if shown)
207
+ - File preview renders correctly
208
+ - API response contains uploaded file reference
209
+
210
+ ---
211
+
212
+ ### Drag-and-Drop — Custom Implementation
213
+
214
+ - **Route**: `/<page>`
215
+ - **Type**: `drag-drop`
216
+ - **Element**: `[draggable]`, `.drop-zone`, sortable list items
217
+
218
+ **Test approach:**
219
+ ```
220
+ 1. Identify drag handle and drop target via Playwright MCP snapshot
221
+ 2. Use `page.dragAndDrop()` or `locator.dragTo()`
222
+ 3. If custom implementation uses JS events, use `page.evaluate()` to dispatch events
223
+ 4. Assert: target state reflects the drag result
224
+ ```
225
+
226
+ **Note**: If the drag-drop uses complex physics (e.g., Kanban board, calendar), prefer `page.evaluate()` with custom event dispatching over `dragAndDrop()`.
227
+
228
+ ---
229
+
230
+ ### WebSocket / Real-Time — Timing Sensitive
231
+
232
+ - **Route**: `/<page>`
233
+ - **Type**: `websocket | sse | polling`
234
+ - **Element**: live data display, notification badge, live chart
235
+
236
+ **Test approach:**
237
+ ```
238
+ 1. Establish WebSocket/SSE connection via page
239
+ 2. Wait for specific message/event: `await page.waitForResponse(/<ws-endpoint>/)`
240
+ 3. Assert: DOM updates after message received
241
+ 4. Use `page.waitForFunction()` to poll for expected state
242
+ ```
243
+
244
+ **Assertions:**
245
+ - Live data counter increments after server push
246
+ - Notification badge shows correct count
247
+ - Chart redraws with new data points
248
+
249
+ **Note**: Increase test timeout for real-time tests. See `playwright.config.ts` → `timeout`.
250
+
251
+ ---
252
+
101
253
  > **Reference**: See `app-exploration.md` → **Special Elements Detected** table for per-route specifics.