openspec-playwright 0.1.63 → 0.1.65

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -0,0 +1,12 @@
1
+ <claude-mem-context>
2
+ # Recent Activity
3
+
4
+ <!-- This section is auto-generated by claude-mem. Edit content outside the tags. -->
5
+
6
+ ### Mar 27, 2026
7
+
8
+ | ID | Time | T | Title | Read |
9
+ | ----- | -------- | --- | ---------------------------------------------------------- | ---- |
10
+ | #5419 | 10:01 AM | 🟣 | Added Claude Code command opsx-e2e.md for E2E verification | ~213 |
11
+
12
+ </claude-mem-context>
@@ -0,0 +1,12 @@
1
+ <claude-mem-context>
2
+ # Recent Activity
3
+
4
+ <!-- This section is auto-generated by claude-mem. Edit content outside the tags. -->
5
+
6
+ ### Mar 27, 2026
7
+
8
+ | ID | Time | T | Title | Read |
9
+ | ----- | -------- | --- | ----------------------------------------------------- | ---- |
10
+ | #5434 | 10:15 AM | 🔄 | Organized OpenSpec command files under opsx namespace | ~145 |
11
+
12
+ </claude-mem-context>
@@ -0,0 +1,44 @@
1
+ Run Playwright E2E verification for an OpenSpec change.
2
+
3
+ ## Workflow
4
+
5
+ 1. **Validate environment**: Run the seed test to confirm your app is reachable.
6
+
7
+ ```bash
8
+ npx playwright test tests/playwright/seed.spec.ts --project=chromium
9
+ ```
10
+
11
+ If it fails, fix your BASE_URL or start the dev server first.
12
+
13
+ 2. **Select the change**: If no change name is provided, run `openspec list --json` and pick one. Then announce: "Using change: `<name>`".
14
+
15
+ 3. **Read specs**: Read all files from `openspec/changes/<name>/specs/*.md`.
16
+
17
+ 4. **Detect auth**: Check if specs mention login, protected routes, or session handling. See `tests/playwright/auth.setup.ts` for auth setup.
18
+
19
+ 5. **Generate test plan**: Create `openspec/changes/<name>/specs/playwright/test-plan.md` listing each test case with `@auth(required|none)` and `@role(...)` tags. Skip if already exists.
20
+
21
+ 6. **Generate tests**: Write `tests/playwright/<name>.spec.ts` from the test plan. Follow the patterns in `seed.spec.ts`:
22
+ - Prefer `data-testid`, fallback to `getByRole`, `getByLabel`, `getByText`
23
+ - Include happy path AND error/edge cases
24
+ - Never use conditional `if (isVisible())` — always use `expect().toBeVisible()` (false-pass anti-pattern)
25
+ - Use `browser.newContext()` for auth guard tests (fresh session, no cookies)
26
+
27
+ 7. **Run tests**:
28
+
29
+ ```bash
30
+ openspec-pw run <change-name>
31
+ ```
32
+
33
+ Or with role filtering:
34
+
35
+ ```bash
36
+ npx playwright test tests/playwright/<name>.spec.ts --grep "@<role>"
37
+ ```
38
+
39
+ 8. **Fix failures**: If tests fail, analyze the error:
40
+ - Network/backend error → `test.skip()` + report
41
+ - Selector changed → use Playwright MCP tools to find equivalent selectors, fix, re-run
42
+ - Auto-heal up to 3 attempts
43
+
44
+ 9. **Report**: Results are saved to `openspec/reports/playwright-e2e-<name>-<timestamp>.md`. Present the summary table to the user.
@@ -0,0 +1,51 @@
1
+ ---
2
+ name: "OPSX: E2E"
3
+ description: Run Playwright E2E verification for an OpenSpec change
4
+ category: OpenSpec
5
+ tags: [openspec, playwright, e2e, testing]
6
+ ---
7
+
8
+ Run Playwright E2E verification for an OpenSpec change.
9
+
10
+ ## Workflow
11
+
12
+ 1. **Validate environment**: Run the seed test to confirm your app is reachable.
13
+
14
+ ```bash
15
+ npx playwright test tests/playwright/seed.spec.ts --project=chromium
16
+ ```
17
+
18
+ If it fails, fix your BASE_URL or start the dev server first.
19
+
20
+ 2. **Select the change**: If no change name is provided, run `openspec list --json` and pick one. Then announce: "Using change: `<name>`".
21
+
22
+ 3. **Read specs**: Read all files from `openspec/changes/<name>/specs/*.md`.
23
+
24
+ 4. **Detect auth**: Check if specs mention login, protected routes, or session handling. See `tests/playwright/auth.setup.ts` for auth setup.
25
+
26
+ 5. **Generate test plan**: Create `openspec/changes/<name>/specs/playwright/test-plan.md` listing each test case with `@auth(required|none)` and `@role(...)` tags. Skip if already exists.
27
+
28
+ 6. **Generate tests**: Write `tests/playwright/<name>.spec.ts` from the test plan. Follow the patterns in `seed.spec.ts`:
29
+ - Prefer `data-testid`, fallback to `getByRole`, `getByLabel`, `getByText`
30
+ - Include happy path AND error/edge cases
31
+ - Never use conditional `if (isVisible())` — always use `expect().toBeVisible()` (false-pass anti-pattern)
32
+ - Use `browser.newContext()` for auth guard tests (fresh session, no cookies)
33
+
34
+ 7. **Run tests**:
35
+
36
+ ```bash
37
+ openspec-pw run <change-name>
38
+ ```
39
+
40
+ Or with role filtering:
41
+
42
+ ```bash
43
+ npx playwright test tests/playwright/<name>.spec.ts --grep "@<role>"
44
+ ```
45
+
46
+ 8. **Fix failures**: If tests fail, analyze the error:
47
+ - Network/backend error → `test.skip()` + report
48
+ - Selector changed → use Playwright MCP tools to find equivalent selectors, fix, re-run
49
+ - Auto-heal up to 3 attempts
50
+
51
+ 9. **Report**: Results are saved to `openspec/reports/playwright-e2e-<name>-<timestamp>.md`. Present the summary table to the user.
@@ -0,0 +1,12 @@
1
+ <claude-mem-context>
2
+ # Recent Activity
3
+
4
+ <!-- This section is auto-generated by claude-mem. Edit content outside the tags. -->
5
+
6
+ ### Mar 27, 2026
7
+
8
+ | ID | Time | T | Title | Read |
9
+ | ----- | ------- | --- | -------------------------------------------------- | ---- |
10
+ | #5372 | 9:44 AM | ✅ | Created skill directory structure for openspec-e2e | ~124 |
11
+
12
+ </claude-mem-context>
@@ -0,0 +1,14 @@
1
+ <claude-mem-context>
2
+ # Recent Activity
3
+
4
+ <!-- This section is auto-generated by claude-mem. Edit content outside the tags. -->
5
+
6
+ ### Mar 27, 2026
7
+
8
+ | ID | Time | T | Title | Read |
9
+ | ----- | -------- | --- | ----------------------------------------------------------------------- | ---- |
10
+ | #5511 | 11:22 AM | ✅ | Extended guardrails to protect auth files from overwriting | ~181 |
11
+ | #5510 | 11:21 AM | 🟣 | Enhanced SKILL.md with Playwright auth project configuration | ~228 |
12
+ | #5427 | 10:05 AM | ✅ | Committed style fixes aligning SKILL.md with OpenSpec template standard | ~281 |
13
+
14
+ </claude-mem-context>
@@ -0,0 +1,488 @@
1
+ ---
2
+ name: openspec-e2e
3
+ description: Run Playwright E2E verification for an OpenSpec change. Use when the user wants to validate that the implementation works end-to-end by running Playwright tests generated from the specs.
4
+ license: MIT
5
+ compatibility: Requires openspec CLI, Playwright (with browsers installed), and @playwright/mcp (globally installed via `claude mcp add playwright npx @playwright/mcp@latest`).
6
+ metadata:
7
+ author: openspec-playwright
8
+ version: "2.11"
9
+ ---
10
+
11
+ ## Input
12
+
13
+ - **Change name**: `/opsx:e2e <name>` or `/opsx:e2e all` (full app exploration, no OpenSpec needed)
14
+ - **Specs**: `openspec/changes/<name>/specs/*.md` (if change mode)
15
+ - **Credentials**: `E2E_USERNAME` + `E2E_PASSWORD` env vars
16
+
17
+ ## Output
18
+
19
+ - **Test file**: `tests/playwright/<name>.spec.ts` (e.g. `app-all.spec.ts` for "all")
20
+ - **Auth setup**: `tests/playwright/auth.setup.ts` (if auth required)
21
+ - **Report**: `openspec/reports/playwright-e2e-<name>-<timestamp>.md`
22
+ - **Test plan**: `openspec/changes/<name>/specs/playwright/test-plan.md` (change mode only)
23
+
24
+ ## Architecture
25
+
26
+ Two modes, same pipeline:
27
+
28
+ | Mode | Command | Route source | Output |
29
+ | ------ | ------------------ | ------------------------ | ----------------- |
30
+ | Change | `/opsx:e2e <name>` | OpenSpec specs | `<name>.spec.ts` |
31
+ | All | `/opsx:e2e all` | sitemap + homepage crawl | `app-all.spec.ts` |
32
+
33
+ Both modes update `app-knowledge.md` and `app-exploration.md`. All `.spec.ts` files run together as regression suite.
34
+
35
+ **Role mapping** (Playwright Test Agents terminology):
36
+
37
+ | Role | This SKILL | What it does |
38
+ | --------- | ---------- | ------------------------------------------------------------ |
39
+ | Planner | Step 4–5 | Explores app via Playwright MCP → produces test-plan.md |
40
+ | Generator | Step 6 | Transforms test-plan.md → `.spec.ts` with verified selectors |
41
+ | Healer | Step 9 | Executes tests, repairs failures via Playwright MCP |
42
+
43
+ ## Testing principles
44
+
45
+ **UI first** — Test every user flow through the browser UI. E2E validates that users can accomplish tasks in the real interface, not just that the backend responds correctly.
46
+
47
+ ```
48
+ 用户操作 → 浏览器 UI → 后端 → 数据库 → UI 反馈
49
+ ```
50
+
51
+ **API only as fallback** — Use `page.request` only when UI genuinely cannot cover the scenario:
52
+
53
+ - Triggering HTTP 5xx/4xx error responses (hard to reach via UI)
54
+ - Edge cases requiring pre-condition data that UI cannot set up
55
+ - Cases where Step 4 exploration confirmed no UI element exists
56
+
57
+ **Decision rule**:
58
+
59
+ ```
60
+ Can this be tested through the UI?
61
+ → Yes → page.getByRole/ByLabel/ByText + click/fill/type + assert UI
62
+ → No → record reason → use page.request
63
+ ```
64
+
65
+ **Never use API calls to replace routine UI flows.** If a test completes in < 200ms, it is almost certainly using `page.request` instead of real UI interactions.
66
+
67
+ ## Steps
68
+
69
+ ### 1. Select the change or mode
70
+
71
+ **Change mode** (`/opsx:e2e <name>`):
72
+
73
+ - Use provided name, or infer from context, or auto-select if only one exists
74
+ - If ambiguous → `openspec list --json` + AskUserQuestion
75
+ - Verify specs exist: `openspec status --change "<name>" --json`
76
+ - If specs empty → **STOP: E2E requires specs.** Use "all" mode instead.
77
+
78
+ **"all" mode** (`/opsx:e2e all` — no OpenSpec needed):
79
+
80
+ - Announce: "Mode: full app exploration"
81
+ - Discover routes via:
82
+ 1. Navigate to `${BASE_URL}/sitemap.xml` (if exists)
83
+ 2. Navigate to `${BASE_URL}/` → extract all links from snapshot
84
+ 3. Fallback common paths: `/`, `/login`, `/dashboard`, `/admin`, `/profile`, `/api/`
85
+ - Group routes: Guest vs Protected (by attempting direct access)
86
+
87
+ ### 2. Detect auth
88
+
89
+ **Change mode**: Read specs and extract functional requirements. Detect auth from keywords.
90
+
91
+ **"all" mode**: Detect auth by attempting to access known protected paths (e.g. `/dashboard`, `/profile`). If redirected to `/login` → auth required.
92
+
93
+ **Auth detection — both modes** (BOTH conditions required):
94
+
95
+ **Condition A — Explicit markers**: "login", "signin", "logout", "authenticate", "protected", "authenticated", "session", "unauthorized", "jwt", "token", "refresh", "middleware"
96
+
97
+ **Condition B — Context indicators**: Protected routes ("/dashboard", "/profile", "/admin"), role mentions ("admin", "user"), redirect flows
98
+
99
+ **Exclude false positives**: HTTP header examples (`Authorization: Bearer ...`) and code snippets do not count.
100
+
101
+ **Confidence**:
102
+
103
+ - High (auto-proceed): Multiple markers AND context indicators
104
+ - Medium (proceed with note): Single marker, context unclear
105
+ - Low (skip auth): No markers found
106
+
107
+ ### 3. Validate environment
108
+
109
+ Run the seed test before generating tests:
110
+
111
+ ```bash
112
+ npx playwright test tests/playwright/seed.spec.ts --project=chromium
113
+ ```
114
+
115
+ Seed test initializes the `page` context — it runs all fixtures, hooks, and globalSetup. Not just a smoke check: it also validates that auth setup, BASE_URL, and Playwright are fully functional.
116
+
117
+ **If seed test fails**: Stop and report. Fix the environment before proceeding.
118
+
119
+ ### 4. Explore application
120
+
121
+ Explore to collect real DOM data before writing test plan. This eliminates blind selector guessing.
122
+
123
+ **Prerequisites**: seed test pass. If auth is required, ensure `auth.setup.ts` has been run (Step 7). BASE_URL must be verified reachable (see 4.1).
124
+
125
+ #### 4.1. Verify BASE_URL + Read app-knowledge.md
126
+
127
+ 1. **Verify BASE_URL**: `browser_navigate(BASE_URL)` → if HTTP 5xx → **STOP: backend error. Fix app first.**
128
+ 2. **Read app-knowledge.md**: known risks, project conventions
129
+ 3. **Routes** (from Step 1): use already-discovered routes — no need to re-extract
130
+
131
+ #### 4.2. Explore each route via Playwright MCP
132
+
133
+ For each route:
134
+
135
+ ```
136
+ browser_navigate → browser_console_messages → browser_snapshot → browser_take_screenshot
137
+ ```
138
+
139
+ **After navigating, check for app-level errors**:
140
+
141
+ | Signal | Meaning | Action |
142
+ | ----------------------------- | --------------------------------- | --------------------------------------------------------------------------------------------- |
143
+ | HTTP 5xx or unreachable | Backend/server error | **STOP** — tell user: "App has a backend error (HTTP <code>). Fix it, then re-run /opsx:e2e." |
144
+ | JS error in console | App runtime error | **STOP** — tell user: "Page has JS errors. Fix them, then re-run /opsx:e2e." |
145
+ | HTTP 404 | Route not in app (metadata issue) | Continue — mark `⚠️ route not found` in app-exploration.md |
146
+ | Auth required, no credentials | Missing auth setup | Continue — skip protected routes, explore login page |
147
+
148
+ **For guest routes** (no auth):
149
+
150
+ ```javascript
151
+ // Navigate directly
152
+ await browser_navigate(`${BASE_URL}/<route>`);
153
+ ```
154
+
155
+ **For protected routes** (auth required):
156
+
157
+ ```javascript
158
+ // Option A: use existing storageState (recommended)
159
+ // Option B: navigate to /login first, fill form, then navigate to target
160
+ // Option C: use browser_run_code to set auth cookies directly
161
+ ```
162
+
163
+ **If credentials are not yet available**:
164
+
165
+ 1. Skip protected routes — mark `⚠️ auth needed — explore after auth.setup.ts`
166
+ 2. Explore the login page itself (guest route) — extract form selectors
167
+ 3. After auth.setup.ts runs, re-run exploration for protected routes
168
+
169
+ Wait for page stability:
170
+
171
+ - Prefer `browser_wait_for` with text or selector
172
+ - Avoid `networkidle` / `load` — too slow or unreliable
173
+ - Ready signal: heading, spinner disappears, or URL change
174
+
175
+ #### 4.3. Parse the snapshot
176
+
177
+ From `browser_snapshot` output, extract **interactive elements** for each route:
178
+
179
+ | Element type | What to capture | Selector priority |
180
+ | -------------------- | ------------------------------------ | ---------------------------------------------------------- |
181
+ | **Buttons** | text, selector | `[data-testid]` > `getByRole` > `getByLabel` > `getByText` |
182
+ | **Form fields** | name, type, label, selector | `[data-testid]` > `name` > `label` |
183
+ | **Navigation links** | text, href, selector | `text` > `href` |
184
+ | **Headings** | text content, selector | for assertions |
185
+ | **Error messages** | text patterns, selector | for error path testing |
186
+ | **Dynamic content** | structure — row counts, card layouts | for data-driven tests |
187
+
188
+ #### 4.4. Write app-exploration.md
189
+
190
+ Output: `openspec/changes/<name>/specs/playwright/app-exploration.md`
191
+
192
+ Use template: `openspec/schemas/playwright-e2e/templates/app-exploration.md`
193
+
194
+ Key fields per route:
195
+
196
+ - **URL**: `${BASE_URL}<path>`
197
+ - **Auth**: none / required (storageState: `<path>`)
198
+ - **Ready signal**: how to know the page is loaded
199
+ - **Elements**: interactive elements with verified selectors (see 4.3 table)
200
+ - **Screenshot**: `__screenshots__/<slug>.png`
201
+
202
+ After exploration, add route-level notes (redirects, dynamic content → see 4.5).
203
+
204
+ #### 4.5. Exploration behavior notes
205
+
206
+ | Situation | Action |
207
+ | ------------------------------------------------- | ---------------------------------------------------------------- |
208
+ | SPA routing (URL changes but page doesn't reload) | Explore via navigation clicks from known routes, not direct URLs |
209
+ | Page loads but no interactive elements | Wait longer for SPA hydration |
210
+ | Dynamic content (user-specific) | Record structure — use `toContainText`, not `toHaveText` |
211
+
212
+ **Idempotency**: If `app-exploration.md` already exists → read it, verify routes still match specs, update only new routes or changed pages.
213
+
214
+ #### 4.6. Update app-knowledge.md
215
+
216
+ After writing `app-exploration.md`, extract **project-level shared knowledge** and append to `tests/playwright/app-knowledge.md`:
217
+
218
+ | Section | What to extract |
219
+ | ------------------------ | ------------------------------------------------------------------------- |
220
+ | Architecture | Monolith or separated? Backend port? Restart command? |
221
+ | Credential Format | Login endpoint, username format (email vs username) |
222
+ | Common Selector Patterns | New patterns discovered that apply across routes |
223
+ | SPA Routing | SPA framework, routing behavior |
224
+ | Project Conventions | BASE_URL, auth method, multi-user roles |
225
+ | Selector Fixes | Healed selectors (see Step 9) — route, old selector, new selector, reason |
226
+
227
+ Append only new/changed items — preserve existing content.
228
+
229
+ #### 4.7. After exploration
230
+
231
+ Pass `app-exploration.md` to:
232
+
233
+ - **Step 5 (Planner)**: reference real routes, auth states, and elements in test-plan.md
234
+ - **Step 6 (Generator)**: use verified selectors instead of inferring
235
+
236
+ Read `tests/playwright/app-knowledge.md` as context for cross-change patterns.
237
+
238
+ ### 5. Generate test plan
239
+
240
+ > **"all" mode: skip this step — go directly to Step 6.**
241
+
242
+ **Change mode — prerequisite**: If `openspec/changes/<name>/specs/playwright/app-exploration.md` does not exist → **STOP**. Run Step 4 (explore application) before generating tests. Without real DOM data from exploration, selectors are guesses and tests will be fragile.
243
+
244
+ **Change mode**: Create `openspec/changes/<name>/specs/playwright/test-plan.md`.
245
+
246
+ **Read inputs**: specs, app-exploration.md, app-knowledge.md
247
+
248
+ **Create test cases**: functional requirement → test case, with `@role` and `@auth` tags. Reference verified selectors from app-exploration.md.
249
+
250
+ Template: `openspec/schemas/playwright-e2e/templates/test-plan.md`
251
+
252
+ **Idempotency**: If test-plan.md exists → read and use, do NOT regenerate.
253
+
254
+ ### 6. Generate test file
255
+
256
+ **"all" mode** → `tests/playwright/app-all.spec.ts` (smoke regression):
257
+
258
+ - For each discovered route: navigate → assert HTTP 200 → assert ready signal visible
259
+ - No detailed assertions — just "this page loads without crashing"
260
+ - This is a regression baseline — catches when existing pages break
261
+
262
+ **Change mode** → `tests/playwright/<name>.spec.ts` (functional):
263
+
264
+ - Read: test-plan.md, app-exploration.md, app-knowledge.md, seed.spec.ts
265
+ - For each test case: verify selectors in real browser, then write Playwright code
266
+
267
+ **Selector verification (change mode)**:
268
+
269
+ 1. Navigate to route with correct auth state
270
+ 2. browser_snapshot to confirm page loaded
271
+ 3. For each selector: verify from current snapshot (see 4.3 table for priority)
272
+ 4. Write test code with verified selectors
273
+ 5. If selector unverifiable → note for Healer (Step 9)
274
+
275
+ **Test coverage — empty states**: For list/detail pages, explore the empty state. If the app shows a "no data" UI when the list is empty, generate a test to verify it. Empty states are often missing from specs but are real user paths.
276
+
277
+ **Output format**:
278
+
279
+ - Follow `seed.spec.ts` structure
280
+ - Use `test.describe(...)` for grouping
281
+ - Each test: `test('描述性名称', async ({ page }) => { ... })`
282
+ - Prefer `data-testid` selectors (see 4.3 table)
283
+
284
+ **Code examples — UI first:**
285
+
286
+ ```typescript
287
+ // ✅ UI 测试 — 用户在界面上的真实操作
288
+ await page.goto(`${BASE_URL}/orders`);
289
+ await page.getByRole("button", { name: "新建订单" }).click();
290
+ await page.getByLabel("订单名称").fill("Test Order");
291
+ await page.getByRole("button", { name: "提交" }).click();
292
+ await expect(page.getByText("订单创建成功")).toBeVisible();
293
+
294
+ // ✅ Error path — 通过 UI 触发错误
295
+ await page.goto(`${BASE_URL}/orders`);
296
+ await page.getByRole("button", { name: "新建订单" }).click();
297
+ await page.getByRole("button", { name: "提交" }).click();
298
+ await expect(page.getByRole("alert")).toContainText("名称不能为空");
299
+
300
+ // ✅ API fallback — 仅在 UI 无法触发时使用
301
+ const res = await page.request.get(`${BASE_URL}/api/orders/99999`);
302
+ expect(res.status()).toBe(404);
303
+ ```
304
+
305
+ ```typescript
306
+ // 🚫 False Pass — 元素不存在时静默跳过
307
+ if (await btn.isVisible().catch(() => false)) { ... }
308
+
309
+ // ✅ CORRECT
310
+ await expect(page.getByRole('button', { name: '取消' })).toBeVisible();
311
+
312
+ // 🚫 用 API 替代 UI — 失去了端到端的意义
313
+ const res = await page.request.post(`${BASE_URL}/api/login`, { data: credentials });
314
+
315
+ // ✅ CORRECT — 通过 UI 登录
316
+ await page.goto(`${BASE_URL}/login`);
317
+ await page.getByLabel('邮箱').fill(process.env.E2E_USERNAME);
318
+ await page.getByLabel('密码').fill(process.env.E2E_PASSWORD);
319
+ await page.getByRole('button', { name: '登录' }).click();
320
+ await expect(page).toHaveURL(/dashboard/);
321
+ ```
322
+
323
+ ```typescript
324
+ // ✅ Fresh browser context for auth guard
325
+ test("unauthenticated user redirected to login", async ({ browser }) => {
326
+ const freshPage = await browser.newContext().newPage();
327
+ await freshPage.goto(`${BASE_URL}/dashboard`);
328
+ await expect(freshPage).toHaveURL(/login|auth/);
329
+ });
330
+ // ✅ Session — logout clears protected state
331
+ await page.getByRole("button", { name: "退出登录" }).click();
332
+ await expect(page).toHaveURL(/login|auth/);
333
+ const freshPage2 = await browser.newContext().newPage();
334
+ await freshPage2.goto(`${BASE_URL}/dashboard`);
335
+ await expect(freshPage2).toHaveURL(/login|auth/); // session revoked
336
+
337
+ // ✅ Browser history — SPA back/forward navigation
338
+ await page.goto(`${BASE_URL}/list`);
339
+ await page.getByRole("link", { name: "详情" }).first().click();
340
+ await expect(page).toHaveURL(/detail/);
341
+ await page.goBack();
342
+ await expect(page).toHaveURL(/list/);
343
+ await page.goForward();
344
+ await expect(page).toHaveURL(/detail/);
345
+
346
+ // ✅ File uploads — UI 操作
347
+ await page.locator('input[type="file"]').setInputFiles("/path/to/file.pdf");
348
+ ```
349
+
350
+ Always include error path tests: UI validation messages, network failure, invalid input. Use `page.request` only for scenarios confirmed unreachable via UI.
351
+
352
+ If the file exists → diff against test-plan, add only missing test cases.
353
+
354
+ ### 7. Configure auth (if required)
355
+
356
+ - **API login**: Generate `auth.setup.ts` using `E2E_USERNAME`/`E2E_PASSWORD` + POST to login endpoint
357
+ - **UI login**: Generate `auth.setup.ts` using browser form fill. Update selectors to match your login page
358
+ - **Multi-user**: Separate `storageState` paths per role
359
+
360
+ **Credential format guidance**:
361
+
362
+ - If the app uses **email** for login → use `CHANGE_ME@example.com`
363
+ - If the app uses **username** (alphanumeric + underscore) → use `test_user_001` (more universal)
364
+ - Check existing test files or login page to determine the format
365
+ - Always set credentials via environment variables — never hardcode
366
+
367
+ **Prompt user**:
368
+
369
+ ```
370
+ Auth required. To set up:
371
+ 1. Customize tests/playwright/credentials.yaml
372
+ 2. Export: export E2E_USERNAME=xxx E2E_PASSWORD=yyy
373
+ 3. Run auth: npx playwright test --project=setup
374
+ 4. Re-run /opsx:e2e to execute tests
375
+ ```
376
+
377
+ **Idempotency**: If `auth.setup.ts` already exists → verify format, update only if stale.
378
+
379
+ ### 8. Configure playwright.config.ts
380
+
381
+ If missing → generate from `openspec/schemas/playwright-e2e/templates/playwright.config.ts`.
382
+
383
+ **Auto-detect BASE_URL** (in priority order):
384
+
385
+ 1. `process.env.BASE_URL` if already set
386
+ 2. `tests/playwright/seed.spec.ts` → extract `BASE_URL` value
387
+ 3. Read `vite.config.ts` (or `vite.config.js`) → extract `server.port` + infer protocol (`https` if `server.https`, else `http`)
388
+ 4. Read `package.json` → `scripts.dev` or `scripts.start` → extract port from `--port` flag
389
+ 5. Fallback: `http://localhost:3000`
390
+
391
+ **Auto-detect dev command**:
392
+
393
+ 1. `package.json` → scripts in order: `dev` → `start` → `serve` → `preview` → `npm run dev`
394
+
395
+ If playwright.config.ts exists → READ first, preserve ALL existing fields, add only missing `webServer` block.
396
+
397
+ ### 9. Execute tests
398
+
399
+ ```bash
400
+ openspec-pw run <name> --project=<role>
401
+ ```
402
+
403
+ The CLI handles: server lifecycle, port mismatch, report generation.
404
+
405
+ If tests fail → use Playwright MCP tools to inspect UI, fix selectors, re-run.
406
+
407
+ **Healer MCP tools** (in order of use):
408
+
409
+ <!-- MCP_VERSION: 0.0.70 -->
410
+
411
+ | Tool | Purpose |
412
+ | -------------------------- | ----------------------------------------------- |
413
+ | `browser_navigate` | Go to the failing test's page |
414
+ | `browser_snapshot` | Get page structure to find equivalent selectors |
415
+ | `browser_console_messages` | Diagnose JS errors that may cause failures |
416
+ | `browser_take_screenshot` | Visually compare before/after fixes |
417
+ | `browser_run_code` | Execute custom fix logic (optional) |
418
+
419
+ **Healer workflow**:
420
+
421
+ 1. Read the failing test → identify failure type
422
+ 2. Classify:
423
+
424
+ | Failure type | Signal | Action |
425
+ | ---------------------------- | --------------------------------------- | ----------------------------------------------------- |
426
+ | **Network/backend** | `fetch failed`, `net::ERR`, 5xx | `browser_console_messages` → `test.skip()` |
427
+ | **Selector changed** | Element not found | `browser_snapshot` → fix selector → re-run |
428
+ | **Assertion mismatch** | Wrong content/value | `browser_snapshot` → compare → fix assertion → re-run |
429
+ | **Timing issue** | `waitFor`/`page.evaluate` timeout | Switch to `request` API or add `waitFor` → re-run |
430
+ | **page.evaluate with fetch** | `fetch` in browser context, CORS errors | Switch to `page.request` API → re-run |
431
+
432
+ 3. **Heal** (≤3 attempts): snapshot → fix → re-run. If healed successfully → append to `app-knowledge.md` → **Selector Fixes** table: route, old selector → new selector, reason.
433
+ 4. **After 3 failures**: collect evidence checklist → `test.skip()` if app bug, report recommendation if test bug
434
+
435
+ ### 10. False Pass Detection
436
+
437
+ Run after test suite completes (even if all pass). Common patterns (see Step 6 Anti-Pattern Warnings for fixes):
438
+
439
+ - **Conditional visibility**: `if (locator.isVisible().catch(() => false))` — if test passes, locator may not exist
440
+ - **Too fast**: < 200ms for a complex flow is suspicious
441
+ - **No fresh auth context**: Protected routes without `browser.newContext()`
442
+
443
+ Report any gaps in a **⚠️ Coverage Gap** section.
444
+
445
+ ### 11. Report results
446
+
447
+ Read report at `openspec/reports/playwright-e2e-<name>-<timestamp>.md`. Present:
448
+
449
+ - Summary table (tests, passed, failed, duration, status)
450
+ - Auto-heal notes
451
+ - Recommendations with `file:line` references
452
+
453
+ Report template: `openspec/schemas/playwright-e2e/templates/report.md`
454
+
455
+ **Update tasks.md** if all tests pass: find E2E-related items, append `✅ Verified via Playwright E2E (<timestamp>)`.
456
+
457
+ ## Report Structure
458
+
459
+ Reference: `openspec/schemas/playwright-e2e/templates/report.md`
460
+
461
+ ## Graceful Degradation
462
+
463
+ | Scenario | Behavior |
464
+ | ------------------------------------------------ | ------------------------------------------------------------------------------------- |
465
+ | No specs (change mode) | Stop — E2E requires specs. Use "all" mode instead. |
466
+ | Sitemap discovery fails ("all" mode) | Continue — use homepage links + common paths fallback |
467
+ | App has JS errors or HTTP 5xx during exploration | **STOP** — see app-knowledge.md → Architecture for restart instructions |
468
+ | app-all.spec.ts exists | Read and use (never regenerate — regression baseline) |
469
+ | app-exploration.md missing (change mode) | **STOP** — Step 4 exploration is mandatory. Explore before generating tests. |
470
+ | app-exploration.md exists | Read and use (verify routes still match specs — re-explore if page structure changed) |
471
+ | app-knowledge.md exists | Read and use (append new patterns only) |
472
+ | test-plan.md exists (change mode) | Read and use (never regenerate) |
473
+ | auth.setup.ts exists | Verify format (update only if stale) |
474
+ | playwright.config.ts exists | Preserve all fields (add only missing) |
475
+ | Test fails (backend) | `test.skip()` + report |
476
+ | Test fails (selector/assertion) | Healer: snapshot → fix → re-run (≤3) |
477
+ | 3 heals failed | Evidence checklist → app bug: `test.skip()`; unclear: report |
478
+ | False pass detected | Add "⚠️ Coverage Gap" to report |
479
+
480
+ ## Guardrails
481
+
482
+ - Read specs from `openspec/changes/<name>/specs/` as source of truth
483
+ - Do NOT generate tests that contradict the specs
484
+ - **DO generate real, runnable Playwright test code** — not placeholders or TODOs
485
+ - Do NOT overwrite files outside: `specs/playwright/`, `tests/playwright/`, `openspec/reports/`, `playwright.config.ts`, `auth.setup.ts`
486
+ - **Always explore before generating** — Step 4 is mandatory for accurate selectors
487
+ - Cap auto-heal at 3 attempts
488
+ - If no change specified → always ask user to select