openspec-playwright 0.1.51 → 0.1.53

package/.claude/skills/openspec-e2e/SKILL.md

@@ -52,7 +52,7 @@ Read all files from `openspec/changes/<name>/specs/*.md`. Extract functional req
 
 Detect if auth is required only when BOTH conditions are met:
 
-**Condition A — Explicit markers**: "login", "signin", "logout", "authenticate", "protected", "authenticated", "session", "unauthorized"
+**Condition A — Explicit markers**: "login", "signin", "logout", "authenticate", "protected", "authenticated", "session", "unauthorized", "jwt", "token", "refresh", "middleware"
 
 **Condition B — Context indicators**: Protected routes ("/dashboard", "/profile", "/admin"), role mentions ("admin", "user"), redirect flows
 
@@ -76,44 +76,33 @@ This validates: app server reachable, auth fixtures initialized, Playwright work
 
 ### 4. Explore application
 
-**Before writing test plan, explore the app to collect real DOM data.** This is the most critical step — it eliminates blind selector guessing.
+Explore to collect real DOM data before writing test plan. This eliminates blind selector guessing.
 
 **Prerequisites**: seed test pass. If auth is required, ensure `auth.setup.ts` has been run (Step 7). BASE_URL must be verified reachable (see 4.1).
 
-#### 4.1. Verify BASE_URL + Read app-knowledge.md + Extract routes from specs
+#### 4.1. Verify BASE_URL + Read app-knowledge.md + Extract routes
 
-**First, verify BASE_URL is reachable**:
-```javascript
-await browser_navigate(`${BASE_URL}/`)
-// Confirm page loaded → proceed
-// If error → check BASE_URL in seed.spec.ts or vite.config.ts
-```
-A correct BASE_URL prevents wasted exploration time on unreachable routes.
-
-**Then**, read `tests/playwright/app-knowledge.md` to understand:
-- **Known risks**: SPA behavior, dynamic content patterns, auth method
-- **Project conventions**: preferred selector strategy, credential format, BASE_URL
-
-This is context, not constraint — explore with open eyes even if patterns differ from history.
-
-**Then**, read all files in `openspec/changes/<name>/specs/*.md`. Extract every URL, route, or path mentioned:
-
-- Full URLs: `http://localhost:3000/dashboard`, `BASE_URL + /admin`
-- Relative paths: `/dashboard`, `/api/auth/login`, `/admin/settings`
-- Infer routes from text: "navigate to the dashboard" → check if `/dashboard` exists
-
-Group routes by role:
-- **Guest routes**: `/`, `/login`, `/about` (no auth needed)
-- **Protected routes**: `/dashboard`, `/profile`, `/admin` (auth required)
+1. **Verify BASE_URL**: `browser_navigate(BASE_URL)` → check HTTP status. If unreachable or HTTP 5xx → **STOP: app has a bug. Tell user to fix the server first.**
+2. **Read app-knowledge.md**: understand known risks (SPA, dynamic content) and project conventions
+3. **Read specs**: extract all URLs/paths, group by Guest vs Protected
 
 #### 4.2. Explore each route via Playwright MCP
 
-For each route, use these tools in order:
+For each route:
 
 ```
-browser_navigate → browser_snapshot → browser_take_screenshot
+browser_navigate → browser_console_messages → browser_snapshot → browser_take_screenshot
 ```
 
+**After navigating, check for app-level errors**:
+
+| Signal | Meaning | Action |
+|--------|---------|--------|
+| HTTP 5xx or unreachable | Server error | **STOP** — tell user: "App has a bug (HTTP <code>). Fix it, then re-run /opsx:e2e." |
+| JS error in console | App runtime error | **STOP** — tell user: "Page has JS errors. Fix them, then re-run /opsx:e2e." |
+| HTTP 404 | Route not in app (metadata issue) | Continue — mark `⚠️ route not found` in app-exploration.md |
+| Auth required, no credentials | Missing auth setup | Continue — skip protected routes, explore login page |
+
 **For guest routes** (no auth):
 ```javascript
 // Navigate directly
@@ -122,103 +111,56 @@ await browser_navigate(`${BASE_URL}/<route>`)
 
 **For protected routes** (auth required):
 ```javascript
-// Option A: use existing storageState (recommended if auth.setup.ts already ran)
+// Option A: use existing storageState (recommended)
 // Option B: navigate to /login first, fill form, then navigate to target
 // Option C: use browser_run_code to set auth cookies directly
 ```
 
 **If credentials are not yet available**:
-1. **Skip protected routes** — mark them as "⚠️ auth needed — explore after auth.setup.ts"
-2. **Explore the login page itself** (it's a guest route) — extract form selectors for later auth.setup.ts
-3. Document login page structure: input names, button text, form action, error patterns
-
-**Auth setup flow**:
-1. Run exploration → discover login page selectors (Step 4)
-2. Customize auth.setup.ts with discovered selectors
-3. Set E2E_USERNAME/E2E_PASSWORD
-4. Run `npx playwright test --project=setup`
-5. Re-run exploration for protected routes
-
-Wait for page stability after navigation:
-- Prefer waiting for a specific element: `browser_wait_for` with text or selector
-- Avoid `networkidle` / `load` — they are too slow or unreliable
-- Use a "page ready" signal: look for a heading, a loading spinner disappearing, or a URL change
+1. Skip protected routes — mark `⚠️ auth needed — explore after auth.setup.ts`
+2. Explore the login page itself (guest route) — extract form selectors
+3. After auth.setup.ts runs, re-run exploration for protected routes
+
+Wait for page stability:
+- Prefer `browser_wait_for` with text or selector
+- Avoid `networkidle` / `load` — too slow or unreliable
+- Ready signal: heading, spinner disappears, or URL change
 
 #### 4.3. Parse the snapshot
 
 From `browser_snapshot` output, extract **interactive elements** for each route:
 
-| Element type | What to capture | Priority |
+| Element type | What to capture | Selector priority |
 |---|---|---|
-| **Buttons** | text, selector (`getByRole`, `getByLabel`, `data-testid`) | data-testid > role > label > text |
-| **Form fields** | name, type, label, selector | data-testid > name > label |
-| **Navigation links** | text, href, selector | text > href |
+| **Buttons** | text, selector | `[data-testid]` > `getByRole` > `getByLabel` > `getByText` |
+| **Form fields** | name, type, label, selector | `[data-testid]` > `name` > `label` |
+| **Navigation links** | text, href, selector | `text` > `href` |
 | **Headings** | text content, selector | for assertions |
 | **Error messages** | text patterns, selector | for error path testing |
-| **Dynamic content** | structure (not content) — row counts, card layouts | for data-driven tests |
-
-**Selector strategy** (in priority order):
-1. `[data-testid="..."]` — most stable, prefer these
-2. `getByRole('button', { name: '...' })` — semantic, stable
-3. `getByLabel('...')` — for form fields
-4. `getByText('...')` — fallback, fragile
-5. CSS selectors — last resort
+| **Dynamic content** | structure — row counts, card layouts | for data-driven tests |
 
 #### 4.4. Write app-exploration.md
 
 Output: `openspec/changes/<name>/specs/playwright/app-exploration.md`
 
-```markdown
-# App Exploration — <name>
-Generated: <timestamp>
-BASE_URL: <from env or seed.spec.ts>
-
-## Route: /
-- **Auth**: none
-- **URL**: ${BASE_URL}/
-- **Ready signal**: page has heading
-- **Elements**:
-  - login link: `a:text("登录")`
-  - signup link: `[data-testid="signup-link"]`
-- **Screenshot**: `__screenshots__/index.png`
-
-## Route: /dashboard (user)
-- **Auth**: required (storageState: playwright/.auth/user.json)
-- **URL**: ${BASE_URL}/dashboard
-- **Ready signal**: [data-testid="dashboard-heading"] visible
-- **Elements**:
-  - heading: `[data-testid="page-title"]`
-  - logout btn: `[data-testid="logout-btn"]`
-  - profile form: `form >> input[name="displayName"]`
-  - settings link: `nav >> text=Settings`
-- **Screenshot**: `__screenshots__/dashboard-user.png`
-
-## Route: /admin (admin)
-- **Auth**: required (storageState: playwright/.auth/admin.json)
-- **URL**: ${BASE_URL}/admin
-- **Ready signal**: [data-testid="admin-panel"] visible
-- **Elements**:
-  - admin panel: `[data-testid="admin-panel"]`
-  - user table: `table#user-table tbody tr`
-  - add user btn: `[data-testid="add-user-btn"]`
-  - delete btn: `button:has-text("Delete")`
-- **Screenshot**: `__screenshots__/admin-panel.png`
-
-## Exploration Notes
-- Route /admin → user gets redirected to /login (no admin role)
-- /dashboard loads user-specific data (test assertions should use toContainText, not toHaveText)
-```
+Use template: `openspec/schemas/playwright-e2e/templates/app-exploration.md`
 
-#### 4.5. Edge cases
+Key fields per route:
+- **URL**: `${BASE_URL}<path>`
+- **Auth**: none / required (storageState: `<path>`)
+- **Ready signal**: how to know the page is loaded
+- **Elements**: interactive elements with verified selectors (see 4.3 table)
+- **Screenshot**: `__screenshots__/<slug>.png`
 
-| Situation | What to do |
-|-----------|-----------|
-| Route 404 | Mark as "⚠️ route not found — verify URL in specs" |
-| Network error | Mark as "⚠️ unreachable — check if server is running" |
-| Auth required, no credentials | Skip routes + explore login page → document selectors → set up auth first | See 4.2 "If credentials are not yet available" |
+After exploration, add route-level notes (redirects, dynamic content → see 4.5).
+
+#### 4.5. Exploration behavior notes
+
+| Situation | Action |
+|-----------|--------|
 | SPA routing (URL changes but page doesn't reload) | Explore via navigation clicks from known routes, not direct URLs |
-| Page loads but no interactive elements | Try waiting longer for SPA hydration |
-| Dynamic content (user-specific) | Record structure, not content — use `toContainText`, not `toHaveText` |
+| Page loads but no interactive elements | Wait longer for SPA hydration |
+| Dynamic content (user-specific) | Record structure — use `toContainText`, not `toHaveText` |
 
 **Idempotency**: If `app-exploration.md` already exists → read it, verify routes still match specs, update only new routes or changed pages.
 
@@ -256,18 +198,9 @@ Create test cases:
 - List each functional requirement as a test case
 - Mark with `@role(user|admin|guest|none)` and `@auth(required|none)`
 - Include happy path AND error paths
-- Reference the **real route URL** from app-exploration.md for each test
-- Reference **verified selectors** from app-exploration.md instead of inferring
-
-```markdown
-### User can view dashboard
-- **Route**: /dashboard (from app-exploration.md)
-- **Auth**: required (user storageState)
-- **Test steps**:
-  1. Go to `/dashboard`
-  2. Assert page heading: `[data-testid="page-title"]` contains "Dashboard"
-  3. Assert logout button visible: `[data-testid="logout-btn"]`
-```
+- Reference the **real route URL** and **verified selectors** from app-exploration.md
+
+Example: see `openspec/schemas/playwright-e2e/templates/test-plan.md`
 
 **Idempotency**: If test-plan.md already exists → read it, use it, do NOT regenerate.
 
@@ -283,7 +216,7 @@ Create `tests/playwright/<name>.spec.ts`:
 
 #### Verify selectors before writing
 
-**For each test case, verify selectors in a real browser BEFORE writing the test code.** This is the most important step — do not skip it.
+**For each test case, verify selectors in a real browser BEFORE writing test code.**
 
 ```
 For each test case in test-plan.md:
@@ -297,7 +230,7 @@ For each test case in test-plan.md:
      a. Check if selector exists in app-exploration.md
      b. If yes → verify it's still valid via browser_snapshot
      c. If no → find equivalent from current snapshot
-     d. Selector priority: data-testid > getByRole > getByLabel > getByText
+     d. Selector priority: see 4.3 table
   7. Write test code with verified selectors
   8. If selector cannot be verified → note it for Healer (Step 9)
 ```
@@ -306,7 +239,7 @@ This ensures every selector in the generated test code has been validated agains
 
 **Generate** Playwright code for each verified test case:
 - Follow `seed.spec.ts` structure
-- Prefer `data-testid`; fallback to `getByRole`, `getByLabel`, `getByText`
+- Prefer `data-testid` selectors (see 4.3 table for priority)
 - Include happy path AND error/edge cases
 - Use `test.describe(...)` for grouping
 - Each test: `test('描述性名称', async ({ page }) => { ... })`
@@ -340,34 +273,29 @@ test('redirects unauthenticated user to login', async ({ browser }) => {
 
 These are the most common mistakes that cause test failures. **Always follow these rules:**
 
-**API calls — use `request` API, NOT `page.evaluate()`:**
+**API calls — use `page.request` directly:**
 ```typescript
-// ❌ WRONG — page.evaluate timeout, wrong context, CORS issues
+// ❌ WRONG — page.evaluate with fetch has timeout, CORS, and context issues
 const result = await page.evaluate(async () => {
   const res = await fetch('/api/data');
   return res.json();
 });
 
-// ✅ CORRECT — direct HTTP, no browser context, fast
+// ✅ CORRECT — page.request is already an APIRequestContext, use directly
 const res = await page.request.get(`${BASE_URL}/api/data`);
 expect(res.status()).toBe(200);
 const data = await res.json();
 ```
 
-**Browser context cleanup — `dispose()` NOT `close()`:**
+**Browser context — use `close()` for BrowserContext, no cleanup for APIRequestContext:**
 ```typescript
-// ❌ WRONG — close() is not a function on APIRequestContext
-const ctx = await page.request.newContext();
-await ctx.dispose(); // actually correct, but this is CommonContext
-const context = await browser.newContext();
-await context.close(); // ← WRONG
-
-// ✅ CORRECT
+// ✅ BrowserContext — close it when done
 const context = await browser.newContext();
-await context.close(); // close() is correct for BrowserContext
+await context.close(); // ← correct
 
-// For APIRequestContext (from page.request):
-// No cleanup needed — it's managed by Playwright automatically
+// ✅ APIRequestContext — page.request is already one, no cleanup needed
+const res = await page.request.get(`${BASE_URL}/api/data`);
+// No dispose() or close() needed
 ```
 
 **File uploads — use `setInputFiles()`, NOT `page.evaluate()` + fetch:**
@@ -465,18 +393,15 @@ If tests fail → use Playwright MCP tools to inspect UI, fix selectors, re-run.
 | **Selector changed** | Element not found | `browser_snapshot` → fix selector → re-run |
 | **Assertion mismatch** | Wrong content/value | `browser_snapshot` → compare → fix assertion → re-run |
 | **Timing issue** | `waitFor`/`page.evaluate` timeout | Switch to `request` API or add `waitFor` → re-run |
-| **Wrong API usage** | `ctx.close is not a function` | Fix: `browser.newContext()` → `close()`; `request.newContext()` → no cleanup needed |
-| **Auth expired** | 401 Unauthorized | Token may have expired — if long suite, recommend splitting or re-auth |
-| **page.evaluate failure** | `fetch` in browser context, CORS errors | Switch to `page.request` API → re-run |
+| **page.evaluate with fetch** | `fetch` in browser context, CORS errors | Switch to `page.request` API → re-run |
 
-3. **Attempt heal** (≤3 times): snapshot → fix → re-run
+3. **Heal** (≤3 attempts): snapshot → fix → re-run
 4. **After 3 failures**: collect evidence checklist → `test.skip()` if app bug, report recommendation if test bug
 
 ### 10. False Pass Detection
 
-Run after test suite completes (even if all pass):
-
-- **Conditional logic**: Look for `if (locator.isVisible().catch(() => false))` — if test passes, locator may not exist
+Run after test suite completes (even if all pass). Common patterns (see Step 6 Anti-Pattern Warnings for fixes):
+- **Conditional visibility**: `if (locator.isVisible().catch(() => false))` — if test passes, locator may not exist
 - **Too fast**: < 200ms for a complex flow is suspicious
 - **No fresh auth context**: Protected routes without `browser.newContext()`
 
@@ -489,45 +414,13 @@ Read report at `openspec/reports/playwright-e2e-<name>-<timestamp>.md`. Present:
 - Auto-heal notes
 - Recommendations with `file:line` references
 
+Report template: `openspec/schemas/playwright-e2e/templates/report.md`
+
 **Update tasks.md** if all tests pass: find E2E-related items, append `✅ Verified via Playwright E2E (<timestamp>)`.
 
 ## Report Structure
 
-```markdown
-# Playwright E2E Report — <name>
-
-## Summary
-| Tests | Passed | Failed | Duration | Status |
-|-------|--------|--------|----------|--------|
-| N     | N      | N      | Xm Xs    | ✅/❌   |
-
-## Results
-### Passed
-| Test | Duration | Notes |
-|------|----------|-------|
-| ...  | ...      | ...   |
-
-### Failed
-| Test | Error | Recommendation |
-|------|-------|----------------|
-| ...  | ...   | file:line — fix |
-
-## Auto-Heal Log
-- Attempt N: selector fix → result
-
-## Coverage
-- [x] Requirement 1
-- [ ] Requirement 2 (unverified)
-
-## ⚠️ Coverage Gaps
-> Tests passed but coverage gaps detected.
-
-| Test | Gap | Recommendation |
-|------|-----|----------------|
-| ... | Conditional visibility check | file:line — use `expect().toBeVisible()` |
-| ... | Auth guard uses inherited session | Add fresh context test |
-| ... | Suspiciously fast execution (<200ms) | Verify test logic executed |
-```
+Reference: `openspec/schemas/playwright-e2e/templates/report.md`
 
 ## Graceful Degradation
 
@@ -535,6 +428,7 @@ Read report at `openspec/reports/playwright-e2e-<name>-<timestamp>.md`. Present:
 |----------|----------|
 | No specs | Stop — E2E requires specs |
 | Seed test fails | Stop — fix environment |
+| App has JS errors or HTTP 5xx during exploration | **STOP** — tell user to fix the app first |
 | No auth required | Skip auth setup |
 | app-exploration.md exists | Read and use (never regenerate) |
 | app-knowledge.md exists | Read and use (append new patterns only) |
@@ -551,7 +445,7 @@ Read report at `openspec/reports/playwright-e2e-<name>-<timestamp>.md`. Present:
 - Read specs from `openspec/changes/<name>/specs/` as source of truth
 - Do NOT generate tests that contradict the specs
 - **DO generate real, runnable Playwright test code** — not placeholders or TODOs
-- Do NOT overwrite files outside: `specs/playwright/`, `tests/playwright/`, `openspec/reports/`, `playwright.config.ts`, `auth.setup.ts`, `app-exploration.md`, `app-knowledge.md`
+- Do NOT overwrite files outside: `specs/playwright/`, `tests/playwright/`, `openspec/reports/`, `playwright.config.ts`, `auth.setup.ts`
 - **Always explore before generating** — Step 4 is mandatory for accurate selectors
 - Cap auto-heal at 3 attempts
 - If no change specified → always ask user to select

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "openspec-playwright",
-  "version": "0.1.51",
+  "version": "0.1.53",
   "description": "OpenSpec + Playwright E2E verification setup tool for Claude Code",
   "type": "module",
   "bin": {

package/release-notes.md

@@ -1,5 +1,5 @@
 ## What's Changed
 
-- fix(SKILL): add BASE_URL smoke check before exploration (Step 4.1)
+- fix(SKILL): stop exploration on app-level errors, continue on metadata issues
 
-**Full Changelog**: https://github.com/wxhou/openspec-playwright/releases/tag/v0.1.51
+**Full Changelog**: https://github.com/wxhou/openspec-playwright/releases/tag/v0.1.53