openspec-playwright 0.1.63 → 0.1.65
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.claude/commands/CLAUDE.md +12 -0
- package/.claude/commands/opsx/CLAUDE.md +12 -0
- package/.claude/commands/opsx/e2e-body.md +44 -0
- package/.claude/commands/opsx/e2e.md +51 -0
- package/.claude/skills/CLAUDE.md +12 -0
- package/.claude/skills/openspec-e2e/CLAUDE.md +14 -0
- package/.claude/skills/openspec-e2e/SKILL.md +488 -0
- package/README.md +8 -17
- package/README.zh-CN.md +8 -17
- package/dist/commands/editors.d.ts +1 -3
- package/dist/commands/editors.js +7 -382
- package/dist/commands/editors.js.map +1 -1
- package/dist/commands/init.js +3 -5
- package/dist/commands/init.js.map +1 -1
- package/dist/commands/uninstall.js +2 -4
- package/dist/commands/uninstall.js.map +1 -1
- package/dist/commands/update.js +2 -4
- package/dist/commands/update.js.map +1 -1
- package/employee-standards.md +44 -0
- package/package.json +3 -1
|
@@ -0,0 +1,12 @@
|
|
|
1
|
+
<claude-mem-context>
|
|
2
|
+
# Recent Activity
|
|
3
|
+
|
|
4
|
+
<!-- This section is auto-generated by claude-mem. Edit content outside the tags. -->
|
|
5
|
+
|
|
6
|
+
### Mar 27, 2026
|
|
7
|
+
|
|
8
|
+
| ID | Time | T | Title | Read |
|
|
9
|
+
| ----- | -------- | --- | ---------------------------------------------------------- | ---- |
|
|
10
|
+
| #5419 | 10:01 AM | 🟣 | Added Claude Code command opsx-e2e.md for E2E verification | ~213 |
|
|
11
|
+
|
|
12
|
+
</claude-mem-context>
|
|
@@ -0,0 +1,12 @@
|
|
|
1
|
+
<claude-mem-context>
|
|
2
|
+
# Recent Activity
|
|
3
|
+
|
|
4
|
+
<!-- This section is auto-generated by claude-mem. Edit content outside the tags. -->
|
|
5
|
+
|
|
6
|
+
### Mar 27, 2026
|
|
7
|
+
|
|
8
|
+
| ID | Time | T | Title | Read |
|
|
9
|
+
| ----- | -------- | --- | ----------------------------------------------------- | ---- |
|
|
10
|
+
| #5434 | 10:15 AM | 🔄 | Organized OpenSpec command files under opsx namespace | ~145 |
|
|
11
|
+
|
|
12
|
+
</claude-mem-context>
|
|
@@ -0,0 +1,44 @@
|
|
|
1
|
+
Run Playwright E2E verification for an OpenSpec change.
|
|
2
|
+
|
|
3
|
+
## Workflow
|
|
4
|
+
|
|
5
|
+
1. **Validate environment**: Run the seed test to confirm your app is reachable.
|
|
6
|
+
|
|
7
|
+
```bash
|
|
8
|
+
npx playwright test tests/playwright/seed.spec.ts --project=chromium
|
|
9
|
+
```
|
|
10
|
+
|
|
11
|
+
If it fails, fix your BASE_URL or start the dev server first.
|
|
12
|
+
|
|
13
|
+
2. **Select the change**: If no change name is provided, run `openspec list --json` and pick one. Then announce: "Using change: `<name>`".
|
|
14
|
+
|
|
15
|
+
3. **Read specs**: Read all files from `openspec/changes/<name>/specs/*.md`.
|
|
16
|
+
|
|
17
|
+
4. **Detect auth**: Check if specs mention login, protected routes, or session handling. See `tests/playwright/auth.setup.ts` for auth setup.
|
|
18
|
+
|
|
19
|
+
5. **Generate test plan**: Create `openspec/changes/<name>/specs/playwright/test-plan.md` listing each test case with `@auth(required|none)` and `@role(...)` tags. Skip if already exists.
|
|
20
|
+
|
|
21
|
+
6. **Generate tests**: Write `tests/playwright/<name>.spec.ts` from the test plan. Follow the patterns in `seed.spec.ts`:
|
|
22
|
+
- Prefer `data-testid`, fallback to `getByRole`, `getByLabel`, `getByText`
|
|
23
|
+
- Include happy path AND error/edge cases
|
|
24
|
+
- Never use conditional `if (isVisible())` — always use `expect().toBeVisible()` (false-pass anti-pattern)
|
|
25
|
+
- Use `browser.newContext()` for auth guard tests (fresh session, no cookies)
|
|
26
|
+
|
|
27
|
+
7. **Run tests**:
|
|
28
|
+
|
|
29
|
+
```bash
|
|
30
|
+
openspec-pw run <change-name>
|
|
31
|
+
```
|
|
32
|
+
|
|
33
|
+
Or with role filtering:
|
|
34
|
+
|
|
35
|
+
```bash
|
|
36
|
+
npx playwright test tests/playwright/<name>.spec.ts --grep "@<role>"
|
|
37
|
+
```
|
|
38
|
+
|
|
39
|
+
8. **Fix failures**: If tests fail, analyze the error:
|
|
40
|
+
- Network/backend error → `test.skip()` + report
|
|
41
|
+
- Selector changed → use Playwright MCP tools to find equivalent selectors, fix, re-run
|
|
42
|
+
- Auto-heal up to 3 attempts
|
|
43
|
+
|
|
44
|
+
9. **Report**: Results are saved to `openspec/reports/playwright-e2e-<name>-<timestamp>.md`. Present the summary table to the user.
|
|
@@ -0,0 +1,51 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: "OPSX: E2E"
|
|
3
|
+
description: Run Playwright E2E verification for an OpenSpec change
|
|
4
|
+
category: OpenSpec
|
|
5
|
+
tags: [openspec, playwright, e2e, testing]
|
|
6
|
+
---
|
|
7
|
+
|
|
8
|
+
Run Playwright E2E verification for an OpenSpec change.
|
|
9
|
+
|
|
10
|
+
## Workflow
|
|
11
|
+
|
|
12
|
+
1. **Validate environment**: Run the seed test to confirm your app is reachable.
|
|
13
|
+
|
|
14
|
+
```bash
|
|
15
|
+
npx playwright test tests/playwright/seed.spec.ts --project=chromium
|
|
16
|
+
```
|
|
17
|
+
|
|
18
|
+
If it fails, fix your BASE_URL or start the dev server first.
|
|
19
|
+
|
|
20
|
+
2. **Select the change**: If no change name is provided, run `openspec list --json` and pick one. Then announce: "Using change: `<name>`".
|
|
21
|
+
|
|
22
|
+
3. **Read specs**: Read all files from `openspec/changes/<name>/specs/*.md`.
|
|
23
|
+
|
|
24
|
+
4. **Detect auth**: Check if specs mention login, protected routes, or session handling. See `tests/playwright/auth.setup.ts` for auth setup.
|
|
25
|
+
|
|
26
|
+
5. **Generate test plan**: Create `openspec/changes/<name>/specs/playwright/test-plan.md` listing each test case with `@auth(required|none)` and `@role(...)` tags. Skip if already exists.
|
|
27
|
+
|
|
28
|
+
6. **Generate tests**: Write `tests/playwright/<name>.spec.ts` from the test plan. Follow the patterns in `seed.spec.ts`:
|
|
29
|
+
- Prefer `data-testid`, fallback to `getByRole`, `getByLabel`, `getByText`
|
|
30
|
+
- Include happy path AND error/edge cases
|
|
31
|
+
- Never use conditional `if (isVisible())` — always use `expect().toBeVisible()` (false-pass anti-pattern)
|
|
32
|
+
- Use `browser.newContext()` for auth guard tests (fresh session, no cookies)
|
|
33
|
+
|
|
34
|
+
7. **Run tests**:
|
|
35
|
+
|
|
36
|
+
```bash
|
|
37
|
+
openspec-pw run <change-name>
|
|
38
|
+
```
|
|
39
|
+
|
|
40
|
+
Or with role filtering:
|
|
41
|
+
|
|
42
|
+
```bash
|
|
43
|
+
npx playwright test tests/playwright/<name>.spec.ts --grep "@<role>"
|
|
44
|
+
```
|
|
45
|
+
|
|
46
|
+
8. **Fix failures**: If tests fail, analyze the error:
|
|
47
|
+
- Network/backend error → `test.skip()` + report
|
|
48
|
+
- Selector changed → use Playwright MCP tools to find equivalent selectors, fix, re-run
|
|
49
|
+
- Auto-heal up to 3 attempts
|
|
50
|
+
|
|
51
|
+
9. **Report**: Results are saved to `openspec/reports/playwright-e2e-<name>-<timestamp>.md`. Present the summary table to the user.
|
|
@@ -0,0 +1,12 @@
|
|
|
1
|
+
<claude-mem-context>
|
|
2
|
+
# Recent Activity
|
|
3
|
+
|
|
4
|
+
<!-- This section is auto-generated by claude-mem. Edit content outside the tags. -->
|
|
5
|
+
|
|
6
|
+
### Mar 27, 2026
|
|
7
|
+
|
|
8
|
+
| ID | Time | T | Title | Read |
|
|
9
|
+
| ----- | ------- | --- | -------------------------------------------------- | ---- |
|
|
10
|
+
| #5372 | 9:44 AM | ✅ | Created skill directory structure for openspec-e2e | ~124 |
|
|
11
|
+
|
|
12
|
+
</claude-mem-context>
|
|
@@ -0,0 +1,14 @@
|
|
|
1
|
+
<claude-mem-context>
|
|
2
|
+
# Recent Activity
|
|
3
|
+
|
|
4
|
+
<!-- This section is auto-generated by claude-mem. Edit content outside the tags. -->
|
|
5
|
+
|
|
6
|
+
### Mar 27, 2026
|
|
7
|
+
|
|
8
|
+
| ID | Time | T | Title | Read |
|
|
9
|
+
| ----- | -------- | --- | ----------------------------------------------------------------------- | ---- |
|
|
10
|
+
| #5511 | 11:22 AM | ✅ | Extended guardrails to protect auth files from overwriting | ~181 |
|
|
11
|
+
| #5510 | 11:21 AM | 🟣 | Enhanced SKILL.md with Playwright auth project configuration | ~228 |
|
|
12
|
+
| #5427 | 10:05 AM | ✅ | Committed style fixes aligning SKILL.md with OpenSpec template standard | ~281 |
|
|
13
|
+
|
|
14
|
+
</claude-mem-context>
|
|
@@ -0,0 +1,488 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: openspec-e2e
|
|
3
|
+
description: Run Playwright E2E verification for an OpenSpec change. Use when the user wants to validate that the implementation works end-to-end by running Playwright tests generated from the specs.
|
|
4
|
+
license: MIT
|
|
5
|
+
compatibility: Requires openspec CLI, Playwright (with browsers installed), and @playwright/mcp (globally installed via `claude mcp add playwright npx @playwright/mcp@latest`).
|
|
6
|
+
metadata:
|
|
7
|
+
author: openspec-playwright
|
|
8
|
+
version: "2.11"
|
|
9
|
+
---
|
|
10
|
+
|
|
11
|
+
## Input
|
|
12
|
+
|
|
13
|
+
- **Change name**: `/opsx:e2e <name>` or `/opsx:e2e all` (full app exploration, no OpenSpec needed)
|
|
14
|
+
- **Specs**: `openspec/changes/<name>/specs/*.md` (if change mode)
|
|
15
|
+
- **Credentials**: `E2E_USERNAME` + `E2E_PASSWORD` env vars
|
|
16
|
+
|
|
17
|
+
## Output
|
|
18
|
+
|
|
19
|
+
- **Test file**: `tests/playwright/<name>.spec.ts` (e.g. `app-all.spec.ts` for "all")
|
|
20
|
+
- **Auth setup**: `tests/playwright/auth.setup.ts` (if auth required)
|
|
21
|
+
- **Report**: `openspec/reports/playwright-e2e-<name>-<timestamp>.md`
|
|
22
|
+
- **Test plan**: `openspec/changes/<name>/specs/playwright/test-plan.md` (change mode only)
|
|
23
|
+
|
|
24
|
+
## Architecture
|
|
25
|
+
|
|
26
|
+
Two modes, same pipeline:
|
|
27
|
+
|
|
28
|
+
| Mode | Command | Route source | Output |
|
|
29
|
+
| ------ | ------------------ | ------------------------ | ----------------- |
|
|
30
|
+
| Change | `/opsx:e2e <name>` | OpenSpec specs | `<name>.spec.ts` |
|
|
31
|
+
| All | `/opsx:e2e all` | sitemap + homepage crawl | `app-all.spec.ts` |
|
|
32
|
+
|
|
33
|
+
Both modes update `app-knowledge.md` and `app-exploration.md`. All `.spec.ts` files run together as regression suite.
|
|
34
|
+
|
|
35
|
+
**Role mapping** (Playwright Test Agents terminology):
|
|
36
|
+
|
|
37
|
+
| Role | This SKILL | What it does |
|
|
38
|
+
| --------- | ---------- | ------------------------------------------------------------ |
|
|
39
|
+
| Planner | Step 4–5 | Explores app via Playwright MCP → produces test-plan.md |
|
|
40
|
+
| Generator | Step 6 | Transforms test-plan.md → `.spec.ts` with verified selectors |
|
|
41
|
+
| Healer | Step 9 | Executes tests, repairs failures via Playwright MCP |
|
|
42
|
+
|
|
43
|
+
## Testing principles
|
|
44
|
+
|
|
45
|
+
**UI first** — Test every user flow through the browser UI. E2E validates that users can accomplish tasks in the real interface, not just that the backend responds correctly.
|
|
46
|
+
|
|
47
|
+
```
|
|
48
|
+
用户操作 → 浏览器 UI → 后端 → 数据库 → UI 反馈
|
|
49
|
+
```
|
|
50
|
+
|
|
51
|
+
**API only as fallback** — Use `page.request` only when UI genuinely cannot cover the scenario:
|
|
52
|
+
|
|
53
|
+
- Triggering HTTP 5xx/4xx error responses (hard to reach via UI)
|
|
54
|
+
- Edge cases requiring pre-condition data that UI cannot set up
|
|
55
|
+
- Cases where Step 4 exploration confirmed no UI element exists
|
|
56
|
+
|
|
57
|
+
**Decision rule**:
|
|
58
|
+
|
|
59
|
+
```
|
|
60
|
+
Can this be tested through the UI?
|
|
61
|
+
→ Yes → page.getByRole/ByLabel/ByText + click/fill/type + assert UI
|
|
62
|
+
→ No → record reason → use page.request
|
|
63
|
+
```
|
|
64
|
+
|
|
65
|
+
**Never use API calls to replace routine UI flows.** If a test completes in < 200ms, it is almost certainly using `page.request` instead of real UI interactions.
|
|
66
|
+
|
|
67
|
+
## Steps
|
|
68
|
+
|
|
69
|
+
### 1. Select the change or mode
|
|
70
|
+
|
|
71
|
+
**Change mode** (`/opsx:e2e <name>`):
|
|
72
|
+
|
|
73
|
+
- Use provided name, or infer from context, or auto-select if only one exists
|
|
74
|
+
- If ambiguous → `openspec list --json` + AskUserQuestion
|
|
75
|
+
- Verify specs exist: `openspec status --change "<name>" --json`
|
|
76
|
+
- If specs empty → **STOP: E2E requires specs.** Use "all" mode instead.
|
|
77
|
+
|
|
78
|
+
**"all" mode** (`/opsx:e2e all` — no OpenSpec needed):
|
|
79
|
+
|
|
80
|
+
- Announce: "Mode: full app exploration"
|
|
81
|
+
- Discover routes via:
|
|
82
|
+
1. Navigate to `${BASE_URL}/sitemap.xml` (if exists)
|
|
83
|
+
2. Navigate to `${BASE_URL}/` → extract all links from snapshot
|
|
84
|
+
3. Fallback common paths: `/`, `/login`, `/dashboard`, `/admin`, `/profile`, `/api/`
|
|
85
|
+
- Group routes: Guest vs Protected (by attempting direct access)
|
|
86
|
+
|
|
87
|
+
### 2. Detect auth
|
|
88
|
+
|
|
89
|
+
**Change mode**: Read specs and extract functional requirements. Detect auth from keywords.
|
|
90
|
+
|
|
91
|
+
**"all" mode**: Detect auth by attempting to access known protected paths (e.g. `/dashboard`, `/profile`). If redirected to `/login` → auth required.
|
|
92
|
+
|
|
93
|
+
**Auth detection — both modes** (BOTH conditions required):
|
|
94
|
+
|
|
95
|
+
**Condition A — Explicit markers**: "login", "signin", "logout", "authenticate", "protected", "authenticated", "session", "unauthorized", "jwt", "token", "refresh", "middleware"
|
|
96
|
+
|
|
97
|
+
**Condition B — Context indicators**: Protected routes ("/dashboard", "/profile", "/admin"), role mentions ("admin", "user"), redirect flows
|
|
98
|
+
|
|
99
|
+
**Exclude false positives**: HTTP header examples (`Authorization: Bearer ...`) and code snippets do not count.
|
|
100
|
+
|
|
101
|
+
**Confidence**:
|
|
102
|
+
|
|
103
|
+
- High (auto-proceed): Multiple markers AND context indicators
|
|
104
|
+
- Medium (proceed with note): Single marker, context unclear
|
|
105
|
+
- Low (skip auth): No markers found
|
|
106
|
+
|
|
107
|
+
### 3. Validate environment
|
|
108
|
+
|
|
109
|
+
Run the seed test before generating tests:
|
|
110
|
+
|
|
111
|
+
```bash
|
|
112
|
+
npx playwright test tests/playwright/seed.spec.ts --project=chromium
|
|
113
|
+
```
|
|
114
|
+
|
|
115
|
+
Seed test initializes the `page` context — it runs all fixtures, hooks, and globalSetup. Not just a smoke check: it also validates that auth setup, BASE_URL, and Playwright are fully functional.
|
|
116
|
+
|
|
117
|
+
**If seed test fails**: Stop and report. Fix the environment before proceeding.
|
|
118
|
+
|
|
119
|
+
### 4. Explore application
|
|
120
|
+
|
|
121
|
+
Explore to collect real DOM data before writing test plan. This eliminates blind selector guessing.
|
|
122
|
+
|
|
123
|
+
**Prerequisites**: seed test pass. If auth is required, ensure `auth.setup.ts` has been run (Step 7). BASE_URL must be verified reachable (see 4.1).
|
|
124
|
+
|
|
125
|
+
#### 4.1. Verify BASE_URL + Read app-knowledge.md
|
|
126
|
+
|
|
127
|
+
1. **Verify BASE_URL**: `browser_navigate(BASE_URL)` → if HTTP 5xx → **STOP: backend error. Fix app first.**
|
|
128
|
+
2. **Read app-knowledge.md**: known risks, project conventions
|
|
129
|
+
3. **Routes** (from Step 1): use already-discovered routes — no need to re-extract
|
|
130
|
+
|
|
131
|
+
#### 4.2. Explore each route via Playwright MCP
|
|
132
|
+
|
|
133
|
+
For each route:
|
|
134
|
+
|
|
135
|
+
```
|
|
136
|
+
browser_navigate → browser_console_messages → browser_snapshot → browser_take_screenshot
|
|
137
|
+
```
|
|
138
|
+
|
|
139
|
+
**After navigating, check for app-level errors**:
|
|
140
|
+
|
|
141
|
+
| Signal | Meaning | Action |
|
|
142
|
+
| ----------------------------- | --------------------------------- | --------------------------------------------------------------------------------------------- |
|
|
143
|
+
| HTTP 5xx or unreachable | Backend/server error | **STOP** — tell user: "App has a backend error (HTTP <code>). Fix it, then re-run /opsx:e2e." |
|
|
144
|
+
| JS error in console | App runtime error | **STOP** — tell user: "Page has JS errors. Fix them, then re-run /opsx:e2e." |
|
|
145
|
+
| HTTP 404 | Route not in app (metadata issue) | Continue — mark `⚠️ route not found` in app-exploration.md |
|
|
146
|
+
| Auth required, no credentials | Missing auth setup | Continue — skip protected routes, explore login page |
|
|
147
|
+
|
|
148
|
+
**For guest routes** (no auth):
|
|
149
|
+
|
|
150
|
+
```javascript
|
|
151
|
+
// Navigate directly
|
|
152
|
+
await browser_navigate(`${BASE_URL}/<route>`);
|
|
153
|
+
```
|
|
154
|
+
|
|
155
|
+
**For protected routes** (auth required):
|
|
156
|
+
|
|
157
|
+
```javascript
|
|
158
|
+
// Option A: use existing storageState (recommended)
|
|
159
|
+
// Option B: navigate to /login first, fill form, then navigate to target
|
|
160
|
+
// Option C: use browser_run_code to set auth cookies directly
|
|
161
|
+
```
|
|
162
|
+
|
|
163
|
+
**If credentials are not yet available**:
|
|
164
|
+
|
|
165
|
+
1. Skip protected routes — mark `⚠️ auth needed — explore after auth.setup.ts`
|
|
166
|
+
2. Explore the login page itself (guest route) — extract form selectors
|
|
167
|
+
3. After auth.setup.ts runs, re-run exploration for protected routes
|
|
168
|
+
|
|
169
|
+
Wait for page stability:
|
|
170
|
+
|
|
171
|
+
- Prefer `browser_wait_for` with text or selector
|
|
172
|
+
- Avoid `networkidle` / `load` — too slow or unreliable
|
|
173
|
+
- Ready signal: heading, spinner disappears, or URL change
|
|
174
|
+
|
|
175
|
+
#### 4.3. Parse the snapshot
|
|
176
|
+
|
|
177
|
+
From `browser_snapshot` output, extract **interactive elements** for each route:
|
|
178
|
+
|
|
179
|
+
| Element type | What to capture | Selector priority |
|
|
180
|
+
| -------------------- | ------------------------------------ | ---------------------------------------------------------- |
|
|
181
|
+
| **Buttons** | text, selector | `[data-testid]` > `getByRole` > `getByLabel` > `getByText` |
|
|
182
|
+
| **Form fields** | name, type, label, selector | `[data-testid]` > `name` > `label` |
|
|
183
|
+
| **Navigation links** | text, href, selector | `text` > `href` |
|
|
184
|
+
| **Headings** | text content, selector | for assertions |
|
|
185
|
+
| **Error messages** | text patterns, selector | for error path testing |
|
|
186
|
+
| **Dynamic content** | structure — row counts, card layouts | for data-driven tests |
|
|
187
|
+
|
|
188
|
+
#### 4.4. Write app-exploration.md
|
|
189
|
+
|
|
190
|
+
Output: `openspec/changes/<name>/specs/playwright/app-exploration.md`
|
|
191
|
+
|
|
192
|
+
Use template: `openspec/schemas/playwright-e2e/templates/app-exploration.md`
|
|
193
|
+
|
|
194
|
+
Key fields per route:
|
|
195
|
+
|
|
196
|
+
- **URL**: `${BASE_URL}<path>`
|
|
197
|
+
- **Auth**: none / required (storageState: `<path>`)
|
|
198
|
+
- **Ready signal**: how to know the page is loaded
|
|
199
|
+
- **Elements**: interactive elements with verified selectors (see 4.3 table)
|
|
200
|
+
- **Screenshot**: `__screenshots__/<slug>.png`
|
|
201
|
+
|
|
202
|
+
After exploration, add route-level notes (redirects, dynamic content → see 4.5).
|
|
203
|
+
|
|
204
|
+
#### 4.5. Exploration behavior notes
|
|
205
|
+
|
|
206
|
+
| Situation | Action |
|
|
207
|
+
| ------------------------------------------------- | ---------------------------------------------------------------- |
|
|
208
|
+
| SPA routing (URL changes but page doesn't reload) | Explore via navigation clicks from known routes, not direct URLs |
|
|
209
|
+
| Page loads but no interactive elements | Wait longer for SPA hydration |
|
|
210
|
+
| Dynamic content (user-specific) | Record structure — use `toContainText`, not `toHaveText` |
|
|
211
|
+
|
|
212
|
+
**Idempotency**: If `app-exploration.md` already exists → read it, verify routes still match specs, update only new routes or changed pages.
|
|
213
|
+
|
|
214
|
+
#### 4.6. Update app-knowledge.md
|
|
215
|
+
|
|
216
|
+
After writing `app-exploration.md`, extract **project-level shared knowledge** and append to `tests/playwright/app-knowledge.md`:
|
|
217
|
+
|
|
218
|
+
| Section | What to extract |
|
|
219
|
+
| ------------------------ | ------------------------------------------------------------------------- |
|
|
220
|
+
| Architecture | Monolith or separated? Backend port? Restart command? |
|
|
221
|
+
| Credential Format | Login endpoint, username format (email vs username) |
|
|
222
|
+
| Common Selector Patterns | New patterns discovered that apply across routes |
|
|
223
|
+
| SPA Routing | SPA framework, routing behavior |
|
|
224
|
+
| Project Conventions | BASE_URL, auth method, multi-user roles |
|
|
225
|
+
| Selector Fixes | Healed selectors (see Step 9) — route, old selector, new selector, reason |
|
|
226
|
+
|
|
227
|
+
Append only new/changed items — preserve existing content.
|
|
228
|
+
|
|
229
|
+
#### 4.7. After exploration
|
|
230
|
+
|
|
231
|
+
Pass `app-exploration.md` to:
|
|
232
|
+
|
|
233
|
+
- **Step 5 (Planner)**: reference real routes, auth states, and elements in test-plan.md
|
|
234
|
+
- **Step 6 (Generator)**: use verified selectors instead of inferring
|
|
235
|
+
|
|
236
|
+
Read `tests/playwright/app-knowledge.md` as context for cross-change patterns.
|
|
237
|
+
|
|
238
|
+
### 5. Generate test plan
|
|
239
|
+
|
|
240
|
+
> **"all" mode: skip this step — go directly to Step 6.**
|
|
241
|
+
|
|
242
|
+
**Change mode — prerequisite**: If `openspec/changes/<name>/specs/playwright/app-exploration.md` does not exist → **STOP**. Run Step 4 (explore application) before generating tests. Without real DOM data from exploration, selectors are guesses and tests will be fragile.
|
|
243
|
+
|
|
244
|
+
**Change mode**: Create `openspec/changes/<name>/specs/playwright/test-plan.md`.
|
|
245
|
+
|
|
246
|
+
**Read inputs**: specs, app-exploration.md, app-knowledge.md
|
|
247
|
+
|
|
248
|
+
**Create test cases**: functional requirement → test case, with `@role` and `@auth` tags. Reference verified selectors from app-exploration.md.
|
|
249
|
+
|
|
250
|
+
Template: `openspec/schemas/playwright-e2e/templates/test-plan.md`
|
|
251
|
+
|
|
252
|
+
**Idempotency**: If test-plan.md exists → read and use, do NOT regenerate.
|
|
253
|
+
|
|
254
|
+
### 6. Generate test file
|
|
255
|
+
|
|
256
|
+
**"all" mode** → `tests/playwright/app-all.spec.ts` (smoke regression):
|
|
257
|
+
|
|
258
|
+
- For each discovered route: navigate → assert HTTP 200 → assert ready signal visible
|
|
259
|
+
- No detailed assertions — just "this page loads without crashing"
|
|
260
|
+
- This is a regression baseline — catches when existing pages break
|
|
261
|
+
|
|
262
|
+
**Change mode** → `tests/playwright/<name>.spec.ts` (functional):
|
|
263
|
+
|
|
264
|
+
- Read: test-plan.md, app-exploration.md, app-knowledge.md, seed.spec.ts
|
|
265
|
+
- For each test case: verify selectors in real browser, then write Playwright code
|
|
266
|
+
|
|
267
|
+
**Selector verification (change mode)**:
|
|
268
|
+
|
|
269
|
+
1. Navigate to route with correct auth state
|
|
270
|
+
2. browser_snapshot to confirm page loaded
|
|
271
|
+
3. For each selector: verify from current snapshot (see 4.3 table for priority)
|
|
272
|
+
4. Write test code with verified selectors
|
|
273
|
+
5. If selector unverifiable → note for Healer (Step 9)
|
|
274
|
+
|
|
275
|
+
**Test coverage — empty states**: For list/detail pages, explore the empty state. If the app shows a "no data" UI when the list is empty, generate a test to verify it. Empty states are often missing from specs but are real user paths.
|
|
276
|
+
|
|
277
|
+
**Output format**:
|
|
278
|
+
|
|
279
|
+
- Follow `seed.spec.ts` structure
|
|
280
|
+
- Use `test.describe(...)` for grouping
|
|
281
|
+
- Each test: `test('描述性名称', async ({ page }) => { ... })`
|
|
282
|
+
- Prefer `data-testid` selectors (see 4.3 table)
|
|
283
|
+
|
|
284
|
+
**Code examples — UI first:**
|
|
285
|
+
|
|
286
|
+
```typescript
|
|
287
|
+
// ✅ UI 测试 — 用户在界面上的真实操作
|
|
288
|
+
await page.goto(`${BASE_URL}/orders`);
|
|
289
|
+
await page.getByRole("button", { name: "新建订单" }).click();
|
|
290
|
+
await page.getByLabel("订单名称").fill("Test Order");
|
|
291
|
+
await page.getByRole("button", { name: "提交" }).click();
|
|
292
|
+
await expect(page.getByText("订单创建成功")).toBeVisible();
|
|
293
|
+
|
|
294
|
+
// ✅ Error path — 通过 UI 触发错误
|
|
295
|
+
await page.goto(`${BASE_URL}/orders`);
|
|
296
|
+
await page.getByRole("button", { name: "新建订单" }).click();
|
|
297
|
+
await page.getByRole("button", { name: "提交" }).click();
|
|
298
|
+
await expect(page.getByRole("alert")).toContainText("名称不能为空");
|
|
299
|
+
|
|
300
|
+
// ✅ API fallback — 仅在 UI 无法触发时使用
|
|
301
|
+
const res = await page.request.get(`${BASE_URL}/api/orders/99999`);
|
|
302
|
+
expect(res.status()).toBe(404);
|
|
303
|
+
```
|
|
304
|
+
|
|
305
|
+
```typescript
|
|
306
|
+
// 🚫 False Pass — 元素不存在时静默跳过
|
|
307
|
+
if (await btn.isVisible().catch(() => false)) { ... }
|
|
308
|
+
|
|
309
|
+
// ✅ CORRECT
|
|
310
|
+
await expect(page.getByRole('button', { name: '取消' })).toBeVisible();
|
|
311
|
+
|
|
312
|
+
// 🚫 用 API 替代 UI — 失去了端到端的意义
|
|
313
|
+
const res = await page.request.post(`${BASE_URL}/api/login`, { data: credentials });
|
|
314
|
+
|
|
315
|
+
// ✅ CORRECT — 通过 UI 登录
|
|
316
|
+
await page.goto(`${BASE_URL}/login`);
|
|
317
|
+
await page.getByLabel('邮箱').fill(process.env.E2E_USERNAME);
|
|
318
|
+
await page.getByLabel('密码').fill(process.env.E2E_PASSWORD);
|
|
319
|
+
await page.getByRole('button', { name: '登录' }).click();
|
|
320
|
+
await expect(page).toHaveURL(/dashboard/);
|
|
321
|
+
```
|
|
322
|
+
|
|
323
|
+
```typescript
|
|
324
|
+
// ✅ Fresh browser context for auth guard
|
|
325
|
+
test("unauthenticated user redirected to login", async ({ browser }) => {
|
|
326
|
+
const freshPage = await browser.newContext().newPage();
|
|
327
|
+
await freshPage.goto(`${BASE_URL}/dashboard`);
|
|
328
|
+
await expect(freshPage).toHaveURL(/login|auth/);
|
|
329
|
+
});
|
|
330
|
+
// ✅ Session — logout clears protected state
|
|
331
|
+
await page.getByRole("button", { name: "退出登录" }).click();
|
|
332
|
+
await expect(page).toHaveURL(/login|auth/);
|
|
333
|
+
const freshPage2 = await browser.newContext().newPage();
|
|
334
|
+
await freshPage2.goto(`${BASE_URL}/dashboard`);
|
|
335
|
+
await expect(freshPage2).toHaveURL(/login|auth/); // session revoked
|
|
336
|
+
|
|
337
|
+
// ✅ Browser history — SPA back/forward navigation
|
|
338
|
+
await page.goto(`${BASE_URL}/list`);
|
|
339
|
+
await page.getByRole("link", { name: "详情" }).first().click();
|
|
340
|
+
await expect(page).toHaveURL(/detail/);
|
|
341
|
+
await page.goBack();
|
|
342
|
+
await expect(page).toHaveURL(/list/);
|
|
343
|
+
await page.goForward();
|
|
344
|
+
await expect(page).toHaveURL(/detail/);
|
|
345
|
+
|
|
346
|
+
// ✅ File uploads — UI 操作
|
|
347
|
+
await page.locator('input[type="file"]').setInputFiles("/path/to/file.pdf");
|
|
348
|
+
```
|
|
349
|
+
|
|
350
|
+
Always include error path tests: UI validation messages, network failure, invalid input. Use `page.request` only for scenarios confirmed unreachable via UI.
|
|
351
|
+
|
|
352
|
+
If the file exists → diff against test-plan, add only missing test cases.
|
|
353
|
+
|
|
354
|
+
### 7. Configure auth (if required)
|
|
355
|
+
|
|
356
|
+
- **API login**: Generate `auth.setup.ts` using `E2E_USERNAME`/`E2E_PASSWORD` + POST to login endpoint
|
|
357
|
+
- **UI login**: Generate `auth.setup.ts` using browser form fill. Update selectors to match your login page
|
|
358
|
+
- **Multi-user**: Separate `storageState` paths per role
|
|
359
|
+
|
|
360
|
+
**Credential format guidance**:
|
|
361
|
+
|
|
362
|
+
- If the app uses **email** for login → use `CHANGE_ME@example.com`
|
|
363
|
+
- If the app uses **username** (alphanumeric + underscore) → use `test_user_001` (more universal)
|
|
364
|
+
- Check existing test files or login page to determine the format
|
|
365
|
+
- Always set credentials via environment variables — never hardcode
|
|
366
|
+
|
|
367
|
+
**Prompt user**:
|
|
368
|
+
|
|
369
|
+
```
|
|
370
|
+
Auth required. To set up:
|
|
371
|
+
1. Customize tests/playwright/credentials.yaml
|
|
372
|
+
2. Export: export E2E_USERNAME=xxx E2E_PASSWORD=yyy
|
|
373
|
+
3. Run auth: npx playwright test --project=setup
|
|
374
|
+
4. Re-run /opsx:e2e to execute tests
|
|
375
|
+
```
|
|
376
|
+
|
|
377
|
+
**Idempotency**: If `auth.setup.ts` already exists → verify format, update only if stale.
|
|
378
|
+
|
|
379
|
+
### 8. Configure playwright.config.ts
|
|
380
|
+
|
|
381
|
+
If missing → generate from `openspec/schemas/playwright-e2e/templates/playwright.config.ts`.
|
|
382
|
+
|
|
383
|
+
**Auto-detect BASE_URL** (in priority order):
|
|
384
|
+
|
|
385
|
+
1. `process.env.BASE_URL` if already set
|
|
386
|
+
2. `tests/playwright/seed.spec.ts` → extract `BASE_URL` value
|
|
387
|
+
3. Read `vite.config.ts` (or `vite.config.js`) → extract `server.port` + infer protocol (`https` if `server.https`, else `http`)
|
|
388
|
+
4. Read `package.json` → `scripts.dev` or `scripts.start` → extract port from `--port` flag
|
|
389
|
+
5. Fallback: `http://localhost:3000`
|
|
390
|
+
|
|
391
|
+
**Auto-detect dev command**:
|
|
392
|
+
|
|
393
|
+
1. `package.json` → scripts in order: `dev` → `start` → `serve` → `preview` → `npm run dev`
|
|
394
|
+
|
|
395
|
+
If playwright.config.ts exists → READ first, preserve ALL existing fields, add only missing `webServer` block.
|
|
396
|
+
|
|
397
|
+
### 9. Execute tests
|
|
398
|
+
|
|
399
|
+
```bash
|
|
400
|
+
openspec-pw run <name> --project=<role>
|
|
401
|
+
```
|
|
402
|
+
|
|
403
|
+
The CLI handles: server lifecycle, port mismatch, report generation.
|
|
404
|
+
|
|
405
|
+
If tests fail → use Playwright MCP tools to inspect UI, fix selectors, re-run.
|
|
406
|
+
|
|
407
|
+
**Healer MCP tools** (in order of use):
|
|
408
|
+
|
|
409
|
+
<!-- MCP_VERSION: 0.0.70 -->
|
|
410
|
+
|
|
411
|
+
| Tool | Purpose |
|
|
412
|
+
| -------------------------- | ----------------------------------------------- |
|
|
413
|
+
| `browser_navigate` | Go to the failing test's page |
|
|
414
|
+
| `browser_snapshot` | Get page structure to find equivalent selectors |
|
|
415
|
+
| `browser_console_messages` | Diagnose JS errors that may cause failures |
|
|
416
|
+
| `browser_take_screenshot` | Visually compare before/after fixes |
|
|
417
|
+
| `browser_run_code` | Execute custom fix logic (optional) |
|
|
418
|
+
|
|
419
|
+
**Healer workflow**:
|
|
420
|
+
|
|
421
|
+
1. Read the failing test → identify failure type
|
|
422
|
+
2. Classify:
|
|
423
|
+
|
|
424
|
+
| Failure type | Signal | Action |
|
|
425
|
+
| ---------------------------- | --------------------------------------- | ----------------------------------------------------- |
|
|
426
|
+
| **Network/backend** | `fetch failed`, `net::ERR`, 5xx | `browser_console_messages` → `test.skip()` |
|
|
427
|
+
| **Selector changed** | Element not found | `browser_snapshot` → fix selector → re-run |
|
|
428
|
+
| **Assertion mismatch** | Wrong content/value | `browser_snapshot` → compare → fix assertion → re-run |
|
|
429
|
+
| **Timing issue** | `waitFor`/`page.evaluate` timeout | Switch to `request` API or add `waitFor` → re-run |
|
|
430
|
+
| **page.evaluate with fetch** | `fetch` in browser context, CORS errors | Switch to `page.request` API → re-run |
|
|
431
|
+
|
|
432
|
+
3. **Heal** (≤3 attempts): snapshot → fix → re-run. If healed successfully → append to `app-knowledge.md` → **Selector Fixes** table: route, old selector → new selector, reason.
|
|
433
|
+
4. **After 3 failures**: collect evidence checklist → `test.skip()` if app bug, report recommendation if test bug
|
|
434
|
+
|
|
435
|
+
### 10. False Pass Detection
|
|
436
|
+
|
|
437
|
+
Run after test suite completes (even if all pass). Common patterns (see Step 6 Anti-Pattern Warnings for fixes):
|
|
438
|
+
|
|
439
|
+
- **Conditional visibility**: `if (locator.isVisible().catch(() => false))` — if test passes, locator may not exist
|
|
440
|
+
- **Too fast**: < 200ms for a complex flow is suspicious
|
|
441
|
+
- **No fresh auth context**: Protected routes without `browser.newContext()`
|
|
442
|
+
|
|
443
|
+
Report any gaps in a **⚠️ Coverage Gap** section.
|
|
444
|
+
|
|
445
|
+
### 11. Report results
|
|
446
|
+
|
|
447
|
+
Read report at `openspec/reports/playwright-e2e-<name>-<timestamp>.md`. Present:
|
|
448
|
+
|
|
449
|
+
- Summary table (tests, passed, failed, duration, status)
|
|
450
|
+
- Auto-heal notes
|
|
451
|
+
- Recommendations with `file:line` references
|
|
452
|
+
|
|
453
|
+
Report template: `openspec/schemas/playwright-e2e/templates/report.md`
|
|
454
|
+
|
|
455
|
+
**Update tasks.md** if all tests pass: find E2E-related items, append `✅ Verified via Playwright E2E (<timestamp>)`.
|
|
456
|
+
|
|
457
|
+
## Report Structure
|
|
458
|
+
|
|
459
|
+
Reference: `openspec/schemas/playwright-e2e/templates/report.md`
|
|
460
|
+
|
|
461
|
+
## Graceful Degradation
|
|
462
|
+
|
|
463
|
+
| Scenario | Behavior |
|
|
464
|
+
| ------------------------------------------------ | ------------------------------------------------------------------------------------- |
|
|
465
|
+
| No specs (change mode) | Stop — E2E requires specs. Use "all" mode instead. |
|
|
466
|
+
| Sitemap discovery fails ("all" mode) | Continue — use homepage links + common paths fallback |
|
|
467
|
+
| App has JS errors or HTTP 5xx during exploration | **STOP** — see app-knowledge.md → Architecture for restart instructions |
|
|
468
|
+
| app-all.spec.ts exists | Read and use (never regenerate — regression baseline) |
|
|
469
|
+
| app-exploration.md missing (change mode) | **STOP** — Step 4 exploration is mandatory. Explore before generating tests. |
|
|
470
|
+
| app-exploration.md exists | Read and use (verify routes still match specs — re-explore if page structure changed) |
|
|
471
|
+
| app-knowledge.md exists | Read and use (append new patterns only) |
|
|
472
|
+
| test-plan.md exists (change mode) | Read and use (never regenerate) |
|
|
473
|
+
| auth.setup.ts exists | Verify format (update only if stale) |
|
|
474
|
+
| playwright.config.ts exists | Preserve all fields (add only missing) |
|
|
475
|
+
| Test fails (backend) | `test.skip()` + report |
|
|
476
|
+
| Test fails (selector/assertion) | Healer: snapshot → fix → re-run (≤3) |
|
|
477
|
+
| 3 heals failed | Evidence checklist → app bug: `test.skip()`; unclear: report |
|
|
478
|
+
| False pass detected | Add "⚠️ Coverage Gap" to report |
|
|
479
|
+
|
|
480
|
+
## Guardrails
|
|
481
|
+
|
|
482
|
+
- Read specs from `openspec/changes/<name>/specs/` as source of truth
|
|
483
|
+
- Do NOT generate tests that contradict the specs
|
|
484
|
+
- **DO generate real, runnable Playwright test code** — not placeholders or TODOs
|
|
485
|
+
- Do NOT overwrite files outside: `specs/playwright/`, `tests/playwright/`, `openspec/reports/`, `playwright.config.ts`, `auth.setup.ts`
|
|
486
|
+
- **Always explore before generating** — Step 4 is mandatory for accurate selectors
|
|
487
|
+
- Cap auto-heal at 3 attempts
|
|
488
|
+
- If no change specified → always ask user to select
|