@athenaflow/plugin-e2e-test-builder 2.0.9
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.claude-plugin/plugin.json +20 -0
- package/.codex-plugin/plugin.json +15 -0
- package/dist/2.0.8/.agents/plugins/marketplace.json +14 -0
- package/dist/2.0.8/claude/plugin/.claude-plugin/plugin.json +20 -0
- package/dist/2.0.8/claude/plugin/package.json +9 -0
- package/dist/2.0.8/claude/plugin/skills/add-e2e-tests/SKILL.md +217 -0
- package/dist/2.0.8/claude/plugin/skills/add-e2e-tests/agents/claude.yaml +1 -0
- package/dist/2.0.8/claude/plugin/skills/add-e2e-tests/agents/openai.yaml +10 -0
- package/dist/2.0.8/claude/plugin/skills/add-e2e-tests/references/authentication.md +8 -0
- package/dist/2.0.8/claude/plugin/skills/add-e2e-tests/references/error-recovery.md +43 -0
- package/dist/2.0.8/claude/plugin/skills/add-e2e-tests/references/scaffolding.md +12 -0
- package/dist/2.0.8/claude/plugin/skills/add-e2e-tests/references/tracker-template.md +53 -0
- package/dist/2.0.8/claude/plugin/skills/analyze-test-codebase/SKILL.md +142 -0
- package/dist/2.0.8/claude/plugin/skills/analyze-test-codebase/agents/claude.yaml +3 -0
- package/dist/2.0.8/claude/plugin/skills/analyze-test-codebase/agents/openai.yaml +4 -0
- package/dist/2.0.8/claude/plugin/skills/fix-flaky-tests/SKILL.md +160 -0
- package/dist/2.0.8/claude/plugin/skills/fix-flaky-tests/agents/claude.yaml +3 -0
- package/dist/2.0.8/claude/plugin/skills/fix-flaky-tests/agents/openai.yaml +10 -0
- package/dist/2.0.8/claude/plugin/skills/fix-flaky-tests/references/fix-patterns.md +91 -0
- package/dist/2.0.8/claude/plugin/skills/generate-test-cases/SKILL.md +184 -0
- package/dist/2.0.8/claude/plugin/skills/generate-test-cases/agents/claude.yaml +3 -0
- package/dist/2.0.8/claude/plugin/skills/generate-test-cases/agents/openai.yaml +10 -0
- package/dist/2.0.8/claude/plugin/skills/generate-test-cases/references/scenario-categories.md +36 -0
- package/dist/2.0.8/claude/plugin/skills/plan-test-coverage/SKILL.md +116 -0
- package/dist/2.0.8/claude/plugin/skills/plan-test-coverage/agents/claude.yaml +3 -0
- package/dist/2.0.8/claude/plugin/skills/plan-test-coverage/agents/openai.yaml +10 -0
- package/dist/2.0.8/claude/plugin/skills/review-test-cases/SKILL.md +147 -0
- package/dist/2.0.8/claude/plugin/skills/review-test-cases/agents/claude.yaml +3 -0
- package/dist/2.0.8/claude/plugin/skills/review-test-cases/agents/openai.yaml +10 -0
- package/dist/2.0.8/claude/plugin/skills/review-test-code/SKILL.md +189 -0
- package/dist/2.0.8/claude/plugin/skills/review-test-code/agents/claude.yaml +3 -0
- package/dist/2.0.8/claude/plugin/skills/review-test-code/agents/openai.yaml +10 -0
- package/dist/2.0.8/claude/plugin/skills/write-test-code/SKILL.md +227 -0
- package/dist/2.0.8/claude/plugin/skills/write-test-code/agents/claude.yaml +3 -0
- package/dist/2.0.8/claude/plugin/skills/write-test-code/agents/openai.yaml +10 -0
- package/dist/2.0.8/claude/plugin/skills/write-test-code/references/anti-patterns.md +88 -0
- package/dist/2.0.8/claude/plugin/skills/write-test-code/references/api-setup-teardown.md +83 -0
- package/dist/2.0.8/claude/plugin/skills/write-test-code/references/auth-patterns.md +63 -0
- package/dist/2.0.8/claude/plugin/skills/write-test-code/references/mapping-tables.md +56 -0
- package/dist/2.0.8/claude/plugin/skills/write-test-code/references/network-interception.md +56 -0
- package/dist/2.0.8/codex/plugin/.codex-plugin/plugin.json +15 -0
- package/dist/2.0.8/codex/plugin/package.json +9 -0
- package/dist/2.0.8/codex/plugin/skills/add-e2e-tests/SKILL.md +217 -0
- package/dist/2.0.8/codex/plugin/skills/add-e2e-tests/agents/claude.yaml +1 -0
- package/dist/2.0.8/codex/plugin/skills/add-e2e-tests/agents/openai.yaml +10 -0
- package/dist/2.0.8/codex/plugin/skills/add-e2e-tests/references/authentication.md +8 -0
- package/dist/2.0.8/codex/plugin/skills/add-e2e-tests/references/error-recovery.md +43 -0
- package/dist/2.0.8/codex/plugin/skills/add-e2e-tests/references/scaffolding.md +12 -0
- package/dist/2.0.8/codex/plugin/skills/add-e2e-tests/references/tracker-template.md +53 -0
- package/dist/2.0.8/codex/plugin/skills/analyze-test-codebase/SKILL.md +142 -0
- package/dist/2.0.8/codex/plugin/skills/analyze-test-codebase/agents/claude.yaml +3 -0
- package/dist/2.0.8/codex/plugin/skills/analyze-test-codebase/agents/openai.yaml +4 -0
- package/dist/2.0.8/codex/plugin/skills/fix-flaky-tests/SKILL.md +160 -0
- package/dist/2.0.8/codex/plugin/skills/fix-flaky-tests/agents/claude.yaml +3 -0
- package/dist/2.0.8/codex/plugin/skills/fix-flaky-tests/agents/openai.yaml +10 -0
- package/dist/2.0.8/codex/plugin/skills/fix-flaky-tests/references/fix-patterns.md +91 -0
- package/dist/2.0.8/codex/plugin/skills/generate-test-cases/SKILL.md +184 -0
- package/dist/2.0.8/codex/plugin/skills/generate-test-cases/agents/claude.yaml +3 -0
- package/dist/2.0.8/codex/plugin/skills/generate-test-cases/agents/openai.yaml +10 -0
- package/dist/2.0.8/codex/plugin/skills/generate-test-cases/references/scenario-categories.md +36 -0
- package/dist/2.0.8/codex/plugin/skills/plan-test-coverage/SKILL.md +116 -0
- package/dist/2.0.8/codex/plugin/skills/plan-test-coverage/agents/claude.yaml +3 -0
- package/dist/2.0.8/codex/plugin/skills/plan-test-coverage/agents/openai.yaml +10 -0
- package/dist/2.0.8/codex/plugin/skills/review-test-cases/SKILL.md +147 -0
- package/dist/2.0.8/codex/plugin/skills/review-test-cases/agents/claude.yaml +3 -0
- package/dist/2.0.8/codex/plugin/skills/review-test-cases/agents/openai.yaml +10 -0
- package/dist/2.0.8/codex/plugin/skills/review-test-code/SKILL.md +189 -0
- package/dist/2.0.8/codex/plugin/skills/review-test-code/agents/claude.yaml +3 -0
- package/dist/2.0.8/codex/plugin/skills/review-test-code/agents/openai.yaml +10 -0
- package/dist/2.0.8/codex/plugin/skills/write-test-code/SKILL.md +227 -0
- package/dist/2.0.8/codex/plugin/skills/write-test-code/agents/claude.yaml +3 -0
- package/dist/2.0.8/codex/plugin/skills/write-test-code/agents/openai.yaml +10 -0
- package/dist/2.0.8/codex/plugin/skills/write-test-code/references/anti-patterns.md +88 -0
- package/dist/2.0.8/codex/plugin/skills/write-test-code/references/api-setup-teardown.md +83 -0
- package/dist/2.0.8/codex/plugin/skills/write-test-code/references/auth-patterns.md +63 -0
- package/dist/2.0.8/codex/plugin/skills/write-test-code/references/mapping-tables.md +56 -0
- package/dist/2.0.8/codex/plugin/skills/write-test-code/references/network-interception.md +56 -0
- package/dist/2.0.8/release.json +18 -0
- package/dist/2.0.9/.agents/plugins/marketplace.json +14 -0
- package/dist/2.0.9/claude/plugin/.claude-plugin/plugin.json +20 -0
- package/dist/2.0.9/claude/plugin/package.json +9 -0
- package/dist/2.0.9/claude/plugin/skills/add-e2e-tests/SKILL.md +215 -0
- package/dist/2.0.9/claude/plugin/skills/add-e2e-tests/agents/claude.yaml +3 -0
- package/dist/2.0.9/claude/plugin/skills/add-e2e-tests/agents/openai.yaml +10 -0
- package/dist/2.0.9/claude/plugin/skills/add-e2e-tests/references/authentication.md +8 -0
- package/dist/2.0.9/claude/plugin/skills/add-e2e-tests/references/error-recovery.md +43 -0
- package/dist/2.0.9/claude/plugin/skills/add-e2e-tests/references/scaffolding.md +12 -0
- package/dist/2.0.9/claude/plugin/skills/add-e2e-tests/references/tracker-template.md +53 -0
- package/dist/2.0.9/claude/plugin/skills/analyze-test-codebase/SKILL.md +142 -0
- package/dist/2.0.9/claude/plugin/skills/analyze-test-codebase/agents/claude.yaml +3 -0
- package/dist/2.0.9/claude/plugin/skills/analyze-test-codebase/agents/openai.yaml +4 -0
- package/dist/2.0.9/claude/plugin/skills/fix-flaky-tests/SKILL.md +160 -0
- package/dist/2.0.9/claude/plugin/skills/fix-flaky-tests/agents/claude.yaml +3 -0
- package/dist/2.0.9/claude/plugin/skills/fix-flaky-tests/agents/openai.yaml +10 -0
- package/dist/2.0.9/claude/plugin/skills/fix-flaky-tests/references/fix-patterns.md +91 -0
- package/dist/2.0.9/claude/plugin/skills/generate-test-cases/SKILL.md +184 -0
- package/dist/2.0.9/claude/plugin/skills/generate-test-cases/agents/claude.yaml +3 -0
- package/dist/2.0.9/claude/plugin/skills/generate-test-cases/agents/openai.yaml +10 -0
- package/dist/2.0.9/claude/plugin/skills/generate-test-cases/references/scenario-categories.md +36 -0
- package/dist/2.0.9/claude/plugin/skills/plan-test-coverage/SKILL.md +117 -0
- package/dist/2.0.9/claude/plugin/skills/plan-test-coverage/agents/claude.yaml +3 -0
- package/dist/2.0.9/claude/plugin/skills/plan-test-coverage/agents/openai.yaml +10 -0
- package/dist/2.0.9/claude/plugin/skills/review-test-cases/SKILL.md +147 -0
- package/dist/2.0.9/claude/plugin/skills/review-test-cases/agents/claude.yaml +3 -0
- package/dist/2.0.9/claude/plugin/skills/review-test-cases/agents/openai.yaml +10 -0
- package/dist/2.0.9/claude/plugin/skills/review-test-code/SKILL.md +189 -0
- package/dist/2.0.9/claude/plugin/skills/review-test-code/agents/claude.yaml +3 -0
- package/dist/2.0.9/claude/plugin/skills/review-test-code/agents/openai.yaml +10 -0
- package/dist/2.0.9/claude/plugin/skills/write-test-code/SKILL.md +227 -0
- package/dist/2.0.9/claude/plugin/skills/write-test-code/agents/claude.yaml +3 -0
- package/dist/2.0.9/claude/plugin/skills/write-test-code/agents/openai.yaml +10 -0
- package/dist/2.0.9/claude/plugin/skills/write-test-code/references/anti-patterns.md +88 -0
- package/dist/2.0.9/claude/plugin/skills/write-test-code/references/api-setup-teardown.md +83 -0
- package/dist/2.0.9/claude/plugin/skills/write-test-code/references/auth-patterns.md +63 -0
- package/dist/2.0.9/claude/plugin/skills/write-test-code/references/mapping-tables.md +56 -0
- package/dist/2.0.9/claude/plugin/skills/write-test-code/references/network-interception.md +56 -0
- package/dist/2.0.9/codex/plugin/.codex-plugin/plugin.json +15 -0
- package/dist/2.0.9/codex/plugin/package.json +9 -0
- package/dist/2.0.9/codex/plugin/skills/add-e2e-tests/SKILL.md +215 -0
- package/dist/2.0.9/codex/plugin/skills/add-e2e-tests/agents/claude.yaml +3 -0
- package/dist/2.0.9/codex/plugin/skills/add-e2e-tests/agents/openai.yaml +10 -0
- package/dist/2.0.9/codex/plugin/skills/add-e2e-tests/references/authentication.md +8 -0
- package/dist/2.0.9/codex/plugin/skills/add-e2e-tests/references/error-recovery.md +43 -0
- package/dist/2.0.9/codex/plugin/skills/add-e2e-tests/references/scaffolding.md +12 -0
- package/dist/2.0.9/codex/plugin/skills/add-e2e-tests/references/tracker-template.md +53 -0
- package/dist/2.0.9/codex/plugin/skills/analyze-test-codebase/SKILL.md +142 -0
- package/dist/2.0.9/codex/plugin/skills/analyze-test-codebase/agents/claude.yaml +3 -0
- package/dist/2.0.9/codex/plugin/skills/analyze-test-codebase/agents/openai.yaml +4 -0
- package/dist/2.0.9/codex/plugin/skills/fix-flaky-tests/SKILL.md +160 -0
- package/dist/2.0.9/codex/plugin/skills/fix-flaky-tests/agents/claude.yaml +3 -0
- package/dist/2.0.9/codex/plugin/skills/fix-flaky-tests/agents/openai.yaml +10 -0
- package/dist/2.0.9/codex/plugin/skills/fix-flaky-tests/references/fix-patterns.md +91 -0
- package/dist/2.0.9/codex/plugin/skills/generate-test-cases/SKILL.md +184 -0
- package/dist/2.0.9/codex/plugin/skills/generate-test-cases/agents/claude.yaml +3 -0
- package/dist/2.0.9/codex/plugin/skills/generate-test-cases/agents/openai.yaml +10 -0
- package/dist/2.0.9/codex/plugin/skills/generate-test-cases/references/scenario-categories.md +36 -0
- package/dist/2.0.9/codex/plugin/skills/plan-test-coverage/SKILL.md +117 -0
- package/dist/2.0.9/codex/plugin/skills/plan-test-coverage/agents/claude.yaml +3 -0
- package/dist/2.0.9/codex/plugin/skills/plan-test-coverage/agents/openai.yaml +10 -0
- package/dist/2.0.9/codex/plugin/skills/review-test-cases/SKILL.md +147 -0
- package/dist/2.0.9/codex/plugin/skills/review-test-cases/agents/claude.yaml +3 -0
- package/dist/2.0.9/codex/plugin/skills/review-test-cases/agents/openai.yaml +10 -0
- package/dist/2.0.9/codex/plugin/skills/review-test-code/SKILL.md +189 -0
- package/dist/2.0.9/codex/plugin/skills/review-test-code/agents/claude.yaml +3 -0
- package/dist/2.0.9/codex/plugin/skills/review-test-code/agents/openai.yaml +10 -0
- package/dist/2.0.9/codex/plugin/skills/write-test-code/SKILL.md +227 -0
- package/dist/2.0.9/codex/plugin/skills/write-test-code/agents/claude.yaml +3 -0
- package/dist/2.0.9/codex/plugin/skills/write-test-code/agents/openai.yaml +10 -0
- package/dist/2.0.9/codex/plugin/skills/write-test-code/references/anti-patterns.md +88 -0
- package/dist/2.0.9/codex/plugin/skills/write-test-code/references/api-setup-teardown.md +83 -0
- package/dist/2.0.9/codex/plugin/skills/write-test-code/references/auth-patterns.md +63 -0
- package/dist/2.0.9/codex/plugin/skills/write-test-code/references/mapping-tables.md +56 -0
- package/dist/2.0.9/codex/plugin/skills/write-test-code/references/network-interception.md +56 -0
- package/dist/2.0.9/release.json +18 -0
- package/package.json +13 -0
- package/skills/add-e2e-tests/SKILL.md +215 -0
- package/skills/add-e2e-tests/agents/claude.yaml +3 -0
- package/skills/add-e2e-tests/agents/openai.yaml +10 -0
- package/skills/add-e2e-tests/references/authentication.md +8 -0
- package/skills/add-e2e-tests/references/error-recovery.md +43 -0
- package/skills/add-e2e-tests/references/scaffolding.md +12 -0
- package/skills/add-e2e-tests/references/tracker-template.md +53 -0
- package/skills/analyze-test-codebase/SKILL.md +142 -0
- package/skills/analyze-test-codebase/agents/claude.yaml +3 -0
- package/skills/analyze-test-codebase/agents/openai.yaml +4 -0
- package/skills/fix-flaky-tests/SKILL.md +160 -0
- package/skills/fix-flaky-tests/agents/claude.yaml +3 -0
- package/skills/fix-flaky-tests/agents/openai.yaml +10 -0
- package/skills/fix-flaky-tests/references/fix-patterns.md +91 -0
- package/skills/generate-test-cases/SKILL.md +184 -0
- package/skills/generate-test-cases/agents/claude.yaml +3 -0
- package/skills/generate-test-cases/agents/openai.yaml +10 -0
- package/skills/generate-test-cases/references/scenario-categories.md +36 -0
- package/skills/plan-test-coverage/SKILL.md +117 -0
- package/skills/plan-test-coverage/agents/claude.yaml +3 -0
- package/skills/plan-test-coverage/agents/openai.yaml +10 -0
- package/skills/review-test-cases/SKILL.md +147 -0
- package/skills/review-test-cases/agents/claude.yaml +3 -0
- package/skills/review-test-cases/agents/openai.yaml +10 -0
- package/skills/review-test-code/SKILL.md +189 -0
- package/skills/review-test-code/agents/claude.yaml +3 -0
- package/skills/review-test-code/agents/openai.yaml +10 -0
- package/skills/write-test-code/SKILL.md +227 -0
- package/skills/write-test-code/agents/claude.yaml +3 -0
- package/skills/write-test-code/agents/openai.yaml +10 -0
- package/skills/write-test-code/references/anti-patterns.md +88 -0
- package/skills/write-test-code/references/api-setup-teardown.md +83 -0
- package/skills/write-test-code/references/auth-patterns.md +63 -0
- package/skills/write-test-code/references/mapping-tables.md +56 -0
- package/skills/write-test-code/references/network-interception.md +56 -0
|
@@ -0,0 +1,88 @@
|
|
|
1
|
+
# Anti-Patterns: Detailed Explanations and Fix Strategies
|
|
2
|
+
|
|
3
|
+
## 1. Raw CSS selectors
|
|
4
|
+
Use semantic locators (`getByRole`, `getByLabel`, `getByTestId`) instead of CSS selectors. CSS selectors are brittle and break when markup changes.
|
|
5
|
+
|
|
6
|
+
**Why:** A class rename, component refactor, or CSS-in-JS migration breaks every CSS-based selector overnight. Semantic locators survive these changes because they target accessible roles and labels, not implementation details.
|
|
7
|
+
|
|
8
|
+
**Fix:** Replace `page.locator('.submit-btn')` with `page.getByRole('button', { name: /submit/i })`. Follow the locator strategy hierarchy in the main skill.
|
|
9
|
+
|
|
10
|
+
## 2. `waitForTimeout()`
|
|
11
|
+
Use proper assertions and event-driven waits. `waitForTimeout()` adds arbitrary delays that slow tests and mask timing issues.
|
|
12
|
+
|
|
13
|
+
**Why:** A 2-second sleep that works locally may be too short in CI (slower machines) or too long everywhere (wasting time). It also hides the real question: "what am I actually waiting for?"
|
|
14
|
+
|
|
15
|
+
**Fix:** Replace with `waitForResponse` for API-dependent UI, `expect(el).toBeVisible()` for element appearance, or `expect(spinner).toBeHidden()` for loading states.
|
|
16
|
+
|
|
17
|
+
## 3. Fragile `.nth()` / `.first()`
|
|
18
|
+
Scope locators to a container instead of relying on position. If unavoidable, add a comment explaining why.
|
|
19
|
+
|
|
20
|
+
**Why:** Element order can change when the page adds a banner, reorders a list, or renders asynchronously. `.first()` silently picks the wrong element, causing false passes or mysterious failures.
|
|
21
|
+
|
|
22
|
+
**Fix:** Use `.filter({ hasText: 'Specific Item' })` or scope to a container: `page.locator('[data-testid="cart"]').getByRole('button')`.
|
|
23
|
+
|
|
24
|
+
## 4. Exact long text matches
|
|
25
|
+
Use regex with key words instead of matching entire strings. Marketing copy and UI text change frequently.
|
|
26
|
+
|
|
27
|
+
**Why:** A copywriter changes "Sign up for free today!" to "Create your free account" and every test matching the full string breaks, even though the feature works fine.
|
|
28
|
+
|
|
29
|
+
**Fix:** Use `page.getByText(/sign up/i)` or `page.getByRole('button', { name: /free/i })` — match the stable semantic keywords.
|
|
30
|
+
|
|
31
|
+
## 5. Unscoped locators
|
|
32
|
+
Scope locators to `main`, `nav`, `dialog`, or other containers when possible. Global locators match unintended elements.
|
|
33
|
+
|
|
34
|
+
**Why:** A page-wide `getByRole('button', { name: /submit/i })` may match a submit button in the header, footer, or a hidden modal — not just the one in your form. This causes the wrong click or ambiguous locator errors.
|
|
35
|
+
|
|
36
|
+
**Fix:** Scope first: `page.locator('main').getByRole('button', { name: /submit/i })` or `page.locator('[role="dialog"]').getByRole('button')`.
|
|
37
|
+
|
|
38
|
+
## 6. Login via UI in every test
|
|
39
|
+
Use `storageState` or API-based auth setup. UI login in every test wastes time and creates coupling to the login flow.
|
|
40
|
+
|
|
41
|
+
**Why:** If every test clicks through the login form, a single login page change breaks the entire suite. It also adds 3-10 seconds per test — multiplied across hundreds of tests, this becomes significant CI time.
|
|
42
|
+
|
|
43
|
+
**Fix:** Log in once in `globalSetup`, save `storageState` to a JSON file, and reuse it. See `references/auth-patterns.md` for the four auth strategies.
|
|
44
|
+
|
|
45
|
+
## 7. UI clicks to set up test data
|
|
46
|
+
Use API requests for data seeding. UI setup is 10-50x slower and more fragile than API calls.
|
|
47
|
+
|
|
48
|
+
**Why:** Creating a product via the admin UI takes 15+ seconds and 10+ actions. An API call takes 200ms and one line. UI setup also couples your test to two features instead of one — if the admin form breaks, your unrelated cart test fails too.
|
|
49
|
+
|
|
50
|
+
**Fix:** Use the `request` fixture: `await request.post('/api/products', { data: { ... } })`. See `references/api-setup-teardown.md`.
|
|
51
|
+
|
|
52
|
+
## 8. No error path tests
|
|
53
|
+
Every feature needs at least one failure scenario test. Cover server errors (500), network timeouts, and empty states.
|
|
54
|
+
|
|
55
|
+
**Why:** Happy-path-only suites give false confidence. The app may crash on a 500, show a blank screen on empty data, or hang on a timeout — none of which are caught without explicit error path tests.
|
|
56
|
+
|
|
57
|
+
**Fix:** Use `page.route()` to mock failures. At minimum: one 500 response, one timeout (`route.abort('timedout')`), one empty state (`route.fulfill({ json: { items: [] } })`). See `references/network-interception.md`.
|
|
58
|
+
|
|
59
|
+
## 9. Hardcoded test data
|
|
60
|
+
NEVER embed real entity IDs (`'ACC-SUB-2026-00025'`), real user names (`'Anas Client 73'`), real monetary amounts, or environment-specific strings in test code. Instead:
|
|
61
|
+
- Create data via API in `beforeEach` and capture the returned ID
|
|
62
|
+
- Use `Date.now()` or `crypto.randomUUID()` suffixes for uniqueness
|
|
63
|
+
- Read values from `process.env` or a test data module
|
|
64
|
+
- For read-only assertions on existing data, use pattern matchers (`expect(text).toMatch(/ACC-SUB-\d{4}-\d{5}/)`) instead of exact values
|
|
65
|
+
|
|
66
|
+
If you find yourself typing a specific ID or name into test code, STOP — that is a hardcoded value.
|
|
67
|
+
|
|
68
|
+
## 10. Tests depending on execution order
|
|
69
|
+
Each test must be independently runnable. Never rely on state left by a previous test.
|
|
70
|
+
|
|
71
|
+
## 11. `expect(await el.isVisible()).toBe(true)`
|
|
72
|
+
Use `await expect(el).toBeVisible()` instead. The Playwright assertion auto-retries, while the manual pattern checks once and fails on timing.
|
|
73
|
+
|
|
74
|
+
## 12. `{ force: true }` on clicks/checks
|
|
75
|
+
Hides real interaction problems (overlapping elements, not scrolled into view, disabled state). Diagnose the root cause instead: use `scrollIntoViewIfNeeded()`, wait for overlay to disappear, or wait for element to be enabled. Only acceptable when interacting with a custom widget that Playwright cannot natively trigger (document why in a comment).
|
|
76
|
+
|
|
77
|
+
## 13. `waitForLoadState('networkidle')` as default strategy
|
|
78
|
+
`networkidle` waits for 500ms of no network activity, which breaks on long-polling, WebSockets, analytics beacons, or chat widgets. Use it ONLY for initial full-page loads where no streaming/polling exists. For post-action waits, use `waitForResponse` targeting the specific API call, or assert directly on the resulting UI state (Playwright auto-retries).
|
|
79
|
+
|
|
80
|
+
## 14. CSS utility class selectors (Tailwind, Bootstrap, etc.)
|
|
81
|
+
`button.rounded-l-lg`, `.flex.items-center`, `.bg-primary` are styling concerns that change during refactors. Treat ALL utility framework classes as volatile — never use them as selectors. If no semantic locator works, request a `data-testid` from the dev team.
|
|
82
|
+
|
|
83
|
+
## 15. Asserting exact server-computed values
|
|
84
|
+
`expect(revenue).toHaveText('12450')` will break when data changes. For dashboard counters, totals, and aggregates:
|
|
85
|
+
- Assert the element exists and contains a number (`toMatch(/\$[\d,]+/)`)
|
|
86
|
+
- Assert non-zero or within a range
|
|
87
|
+
- Assert format correctness (`/^\d{1,3}(,\d{3})*$/`)
|
|
88
|
+
- If exact value matters, seed the data via API first so you control the expected value
|
|
@@ -0,0 +1,83 @@
|
|
|
1
|
+
# API-Driven Test Setup and Teardown
|
|
2
|
+
|
|
3
|
+
## API-Driven Test Setup
|
|
4
|
+
|
|
5
|
+
Use API calls to set up test data instead of clicking through UI. This is 10-50x faster and more reliable.
|
|
6
|
+
|
|
7
|
+
**When to use API setup:** Creating test users, products, orders, seed data. Setting feature flags. Resetting state between tests.
|
|
8
|
+
|
|
9
|
+
**When to use UI setup:** Only when the creation flow IS the test being verified.
|
|
10
|
+
|
|
11
|
+
```typescript
|
|
12
|
+
test('TC-CART-001: User sees items in cart', async ({ page, request }) => {
|
|
13
|
+
// Arrange: seed data via API (fast, deterministic)
|
|
14
|
+
await request.post('/api/cart/items', {
|
|
15
|
+
data: { productId: 'SKU-123', quantity: 2 }
|
|
16
|
+
});
|
|
17
|
+
|
|
18
|
+
// Act: navigate to verify UI
|
|
19
|
+
await page.goto('/cart');
|
|
20
|
+
|
|
21
|
+
// Assert
|
|
22
|
+
await expect(page.getByRole('listitem')).toHaveCount(2);
|
|
23
|
+
});
|
|
24
|
+
```
|
|
25
|
+
|
|
26
|
+
**Reusable API fixture pattern:**
|
|
27
|
+
```typescript
|
|
28
|
+
export const test = base.extend<{ apiClient: APIRequestContext }>({
|
|
29
|
+
apiClient: async ({ playwright }, use) => {
|
|
30
|
+
const ctx = await playwright.request.newContext({
|
|
31
|
+
baseURL: process.env.API_BASE_URL,
|
|
32
|
+
extraHTTPHeaders: { Authorization: `Bearer ${process.env.API_TOKEN}` },
|
|
33
|
+
});
|
|
34
|
+
await use(ctx);
|
|
35
|
+
await ctx.dispose();
|
|
36
|
+
},
|
|
37
|
+
});
|
|
38
|
+
```
|
|
39
|
+
|
|
40
|
+
## Test Data Teardown
|
|
41
|
+
|
|
42
|
+
Tests that create persistent data (database records, uploaded files, user accounts) MUST clean up after themselves. Leaked test data accumulates across runs and causes false positives/negatives in other tests (pagination counts drift, filter results change, list assertions break).
|
|
43
|
+
|
|
44
|
+
### Strategy 1: API teardown in afterEach (Recommended)
|
|
45
|
+
|
|
46
|
+
```typescript
|
|
47
|
+
let createdTicketId: string;
|
|
48
|
+
|
|
49
|
+
test.beforeEach(async ({ request }) => {
|
|
50
|
+
const resp = await request.post('/api/tickets', {
|
|
51
|
+
data: { title: `Test ${Date.now()}` }
|
|
52
|
+
});
|
|
53
|
+
createdTicketId = (await resp.json()).id;
|
|
54
|
+
});
|
|
55
|
+
|
|
56
|
+
test.afterEach(async ({ request }) => {
|
|
57
|
+
if (createdTicketId) {
|
|
58
|
+
await request.delete(`/api/tickets/${createdTicketId}`);
|
|
59
|
+
}
|
|
60
|
+
});
|
|
61
|
+
```
|
|
62
|
+
|
|
63
|
+
### Strategy 2: Fixture with automatic cleanup
|
|
64
|
+
|
|
65
|
+
```typescript
|
|
66
|
+
export const test = base.extend<{ testTicket: { id: string; title: string } }>({
|
|
67
|
+
testTicket: async ({ request }, use) => {
|
|
68
|
+
const resp = await request.post('/api/tickets', {
|
|
69
|
+
data: { title: `Test ${Date.now()}` }
|
|
70
|
+
});
|
|
71
|
+
const ticket = await resp.json();
|
|
72
|
+
await use(ticket);
|
|
73
|
+
// cleanup runs automatically when test finishes
|
|
74
|
+
await request.delete(`/api/tickets/${ticket.id}`);
|
|
75
|
+
},
|
|
76
|
+
});
|
|
77
|
+
```
|
|
78
|
+
|
|
79
|
+
### Strategy 3: Bulk cleanup in globalTeardown
|
|
80
|
+
|
|
81
|
+
For environments where individual deletion is impractical, tag test data (e.g., `title LIKE 'Test %'`) and delete in batch during `globalTeardown.ts`.
|
|
82
|
+
|
|
83
|
+
If the cleanup API endpoint is unknown, do not invent one. Leave a clear `TODO` with the missing endpoint details, document the cleanup gap in the test file or tracker, and prefer fixture-scoped or environment reset strategies that you can verify. If cleanup is genuinely impossible (no API, no database access), document this as a known limitation in the test file header AND add an `afterEach` that logs a warning.
|
|
@@ -0,0 +1,63 @@
|
|
|
1
|
+
# Authentication Setup Patterns
|
|
2
|
+
|
|
3
|
+
Choose the right auth strategy based on the project's needs.
|
|
4
|
+
|
|
5
|
+
## Strategy 1: storageState (Recommended for most projects)
|
|
6
|
+
|
|
7
|
+
Log in once in global setup, save cookies/localStorage to a JSON file, and reuse across all tests:
|
|
8
|
+
|
|
9
|
+
```typescript
|
|
10
|
+
// global-setup.ts
|
|
11
|
+
import { chromium, FullConfig } from '@playwright/test';
|
|
12
|
+
|
|
13
|
+
async function globalSetup(config: FullConfig) {
|
|
14
|
+
const browser = await chromium.launch();
|
|
15
|
+
const page = await browser.newPage();
|
|
16
|
+
await page.goto('/login');
|
|
17
|
+
await page.getByLabel(/email/i).fill(process.env.TEST_USER_EMAIL!);
|
|
18
|
+
await page.getByLabel(/password/i).fill(process.env.TEST_USER_PASSWORD!);
|
|
19
|
+
await page.getByRole('button', { name: /sign in/i }).click();
|
|
20
|
+
await page.waitForURL('/dashboard');
|
|
21
|
+
await page.context().storageState({ path: 'tests/.auth/user.json' });
|
|
22
|
+
await browser.close();
|
|
23
|
+
}
|
|
24
|
+
export default globalSetup;
|
|
25
|
+
```
|
|
26
|
+
|
|
27
|
+
Reference in config: `use: { storageState: 'tests/.auth/user.json' }`
|
|
28
|
+
|
|
29
|
+
## Strategy 2: Worker-scoped fixture (for parallel workers needing separate accounts)
|
|
30
|
+
|
|
31
|
+
```typescript
|
|
32
|
+
export const test = base.extend<{}, { workerStorageState: string }>({
|
|
33
|
+
storageState: ({ workerStorageState }, use) => use(workerStorageState),
|
|
34
|
+
workerStorageState: [async ({ browser }, use, testInfo) => {
|
|
35
|
+
const page = await browser.newPage({ storageState: undefined });
|
|
36
|
+
// Login with worker-specific account...
|
|
37
|
+
const path = `tests/.auth/worker-${testInfo.parallelIndex}.json`;
|
|
38
|
+
await page.context().storageState({ path });
|
|
39
|
+
await use(path);
|
|
40
|
+
await page.close();
|
|
41
|
+
}, { scope: 'worker' }],
|
|
42
|
+
});
|
|
43
|
+
```
|
|
44
|
+
|
|
45
|
+
## Strategy 3: Multi-role testing (admin + user in same test)
|
|
46
|
+
|
|
47
|
+
```typescript
|
|
48
|
+
test('TC-ADMIN-001: Admin sees user profile', async ({ browser }) => {
|
|
49
|
+
const adminContext = await browser.newContext({ storageState: 'tests/.auth/admin.json' });
|
|
50
|
+
const userContext = await browser.newContext({ storageState: 'tests/.auth/user.json' });
|
|
51
|
+
const adminPage = await adminContext.newPage();
|
|
52
|
+
const userPage = await userContext.newPage();
|
|
53
|
+
// Interact with both pages...
|
|
54
|
+
await adminContext.close();
|
|
55
|
+
await userContext.close();
|
|
56
|
+
});
|
|
57
|
+
```
|
|
58
|
+
|
|
59
|
+
## Strategy 4: Per-test login
|
|
60
|
+
|
|
61
|
+
Only when testing login itself or permission-specific scenarios.
|
|
62
|
+
|
|
63
|
+
Never hardcode tokens. Use environment variables or `.env.test`.
|
|
@@ -0,0 +1,56 @@
|
|
|
1
|
+
# Mapping Tables
|
|
2
|
+
|
|
3
|
+
Standard translations for converting journey specs and exploration results to Playwright API calls.
|
|
4
|
+
|
|
5
|
+
## Scope-to-Locator
|
|
6
|
+
|
|
7
|
+
| Journey Scope | Playwright Scoping |
|
|
8
|
+
|---------------|-------------------|
|
|
9
|
+
| `page` | No scoping needed |
|
|
10
|
+
| `header` | `page.locator('header')` |
|
|
11
|
+
| `main` | `page.locator('main')` |
|
|
12
|
+
| `nav` | `page.locator('nav')` |
|
|
13
|
+
| `dialog` | `page.locator('[role="dialog"]')` |
|
|
14
|
+
|
|
15
|
+
## Action-to-Playwright
|
|
16
|
+
|
|
17
|
+
| Journey Action | Playwright Code |
|
|
18
|
+
|----------------|-----------------|
|
|
19
|
+
| `goto` | `await page.goto(url)` |
|
|
20
|
+
| `click` | `await locator.click()` |
|
|
21
|
+
| `fill` | `await locator.fill(value)` |
|
|
22
|
+
| `select` | `await locator.selectOption(value)` |
|
|
23
|
+
| `assert` | `await expect(locator).toBeVisible()` |
|
|
24
|
+
|
|
25
|
+
## Assertion Mapping
|
|
26
|
+
|
|
27
|
+
| Observed Effect | Playwright Assertion |
|
|
28
|
+
|----------------|---------------------|
|
|
29
|
+
| `url changed to /cart` | `await expect(page).toHaveURL(/cart/)` |
|
|
30
|
+
| `text 'Added' visible` | `await expect(page.getByText(/added/i)).toBeVisible()` |
|
|
31
|
+
| `radio 256GB checked` | `await expect(locator).toBeChecked()` |
|
|
32
|
+
| `button now enabled` | `await expect(locator).toBeEnabled()` |
|
|
33
|
+
|
|
34
|
+
## Target Kind to Locator
|
|
35
|
+
|
|
36
|
+
| Target Kind | Value Pattern | Playwright Locator |
|
|
37
|
+
|-------------|--------------|-------------------|
|
|
38
|
+
| `role` | `button name~Add to Bag` | `getByRole('button', { name: /add to bag/i })` |
|
|
39
|
+
| `role` | `radio name~256GB` | `getByRole('radio', { name: /256gb/i })` |
|
|
40
|
+
| `label` | `Email address` | `getByLabel(/email address/i)` |
|
|
41
|
+
| `testid` | `checkout-button` | `getByTestId('checkout-button')` |
|
|
42
|
+
|
|
43
|
+
## Low Confidence Handling (<0.7)
|
|
44
|
+
|
|
45
|
+
When journey step confidence is low:
|
|
46
|
+
1. Add extra assertions to verify state
|
|
47
|
+
2. Include fallback locators as comments
|
|
48
|
+
3. Consider retry logic for flaky interactions
|
|
49
|
+
|
|
50
|
+
```typescript
|
|
51
|
+
// Primary locator
|
|
52
|
+
const storageRadio = page.getByRole('radio', { name: /256gb/i });
|
|
53
|
+
// Fallback: page.getByLabel(/256gb/i)
|
|
54
|
+
await storageRadio.click();
|
|
55
|
+
await expect(storageRadio).toBeChecked({ timeout: 5000 });
|
|
56
|
+
```
|
|
@@ -0,0 +1,56 @@
|
|
|
1
|
+
# Network Interception and Error Path Testing
|
|
2
|
+
|
|
3
|
+
## Network Interception
|
|
4
|
+
|
|
5
|
+
Use `page.route()` to intercept and mock network requests for deterministic error testing.
|
|
6
|
+
|
|
7
|
+
**Mock server errors:**
|
|
8
|
+
```typescript
|
|
9
|
+
await page.route('**/api/checkout', route =>
|
|
10
|
+
route.fulfill({ status: 500, body: JSON.stringify({ error: 'Payment declined' }) })
|
|
11
|
+
);
|
|
12
|
+
```
|
|
13
|
+
|
|
14
|
+
**Patch real responses (modify, don't replace):**
|
|
15
|
+
```typescript
|
|
16
|
+
await page.route('**/api/products', async route => {
|
|
17
|
+
const response = await route.fetch();
|
|
18
|
+
const json = await response.json();
|
|
19
|
+
json.results = json.results.slice(0, 1); // reduce to 1 item
|
|
20
|
+
await route.fulfill({ response, json });
|
|
21
|
+
});
|
|
22
|
+
```
|
|
23
|
+
|
|
24
|
+
**Assert backend was called:**
|
|
25
|
+
```typescript
|
|
26
|
+
const [response] = await Promise.all([
|
|
27
|
+
page.waitForResponse(resp =>
|
|
28
|
+
resp.url().includes('/api/order') && resp.status() === 201
|
|
29
|
+
),
|
|
30
|
+
page.getByRole('button', { name: /place order/i }).click(),
|
|
31
|
+
]);
|
|
32
|
+
expect(response.status()).toBe(201);
|
|
33
|
+
```
|
|
34
|
+
|
|
35
|
+
**Block heavy resources to speed up tests:**
|
|
36
|
+
```typescript
|
|
37
|
+
await page.route('**/*.{png,jpg,jpeg,gif,svg}', route => route.abort());
|
|
38
|
+
```
|
|
39
|
+
|
|
40
|
+
## Error Path Testing
|
|
41
|
+
|
|
42
|
+
Every feature needs error path tests. Use network interception (see above) to simulate failures. At minimum, every feature test suite should cover:
|
|
43
|
+
|
|
44
|
+
- **Server error** — `route.fulfill({ status: 500, ... })` — verify error UI appears
|
|
45
|
+
- **Network timeout** — `route.abort('timedout')` — verify retry option or error message
|
|
46
|
+
- **Empty state** — `route.fulfill({ status: 200, json: { items: [] } })` — verify empty state UI
|
|
47
|
+
|
|
48
|
+
```typescript
|
|
49
|
+
test('TC-DASHBOARD-005: Shows empty state when no data', async ({ page }) => {
|
|
50
|
+
await page.route('**/api/items', route =>
|
|
51
|
+
route.fulfill({ status: 200, json: { items: [] } })
|
|
52
|
+
);
|
|
53
|
+
await page.goto('/dashboard');
|
|
54
|
+
await expect(page.getByText(/no items/i)).toBeVisible();
|
|
55
|
+
});
|
|
56
|
+
```
|
|
@@ -0,0 +1,18 @@
|
|
|
1
|
+
{
|
|
2
|
+
"schemaVersion": 1,
|
|
3
|
+
"pluginRef": "e2e-test-builder@athena-workflow-marketplace",
|
|
4
|
+
"pluginName": "e2e-test-builder",
|
|
5
|
+
"marketplaceName": "athena-workflow-marketplace",
|
|
6
|
+
"version": "2.0.9",
|
|
7
|
+
"artifacts": {
|
|
8
|
+
"claude": {
|
|
9
|
+
"type": "directory",
|
|
10
|
+
"path": "./claude/plugin"
|
|
11
|
+
},
|
|
12
|
+
"codex": {
|
|
13
|
+
"type": "marketplace",
|
|
14
|
+
"marketplacePath": "./.agents/plugins/marketplace.json",
|
|
15
|
+
"pluginPath": "./codex/plugin"
|
|
16
|
+
}
|
|
17
|
+
}
|
|
18
|
+
}
|
package/package.json
ADDED
|
@@ -0,0 +1,13 @@
|
|
|
1
|
+
{
|
|
2
|
+
"name": "@athenaflow/plugin-e2e-test-builder",
|
|
3
|
+
"version": "2.0.9",
|
|
4
|
+
"description": "Full-pipeline Playwright E2E test generation \u2014 explores your live site via browser, detects existing test conventions, plans coverage gaps, produces reviewed test specs, writes production-grade test code with quality gates, and stabilizes flaky tests",
|
|
5
|
+
"license": "MIT",
|
|
6
|
+
"publishConfig": {
|
|
7
|
+
"access": "public"
|
|
8
|
+
},
|
|
9
|
+
"scripts": {
|
|
10
|
+
"build:artifacts": "node ../../scripts/build-plugin-artifacts.mjs .",
|
|
11
|
+
"prepack": "npm run build:artifacts"
|
|
12
|
+
}
|
|
13
|
+
}
|
|
@@ -0,0 +1,215 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: add-e2e-tests
|
|
3
|
+
description: >
|
|
4
|
+
THE DEFAULT ENTRY POINT for all Playwright / E2E test work. This skill should be used FIRST
|
|
5
|
+
whenever the user wants to add, create, or set up end-to-end tests for any feature, page, or
|
|
6
|
+
application. Runs the full pipeline: analyze codebase, explore the live site, plan coverage,
|
|
7
|
+
generate TC-ID specs, run quality-gate reviews, write production-grade test code, and execute.
|
|
8
|
+
Delegates to sub-skills (analyze-test-codebase, plan-test-coverage, generate-test-cases,
|
|
9
|
+
review-test-cases, write-test-code, review-test-code, fix-flaky-tests) internally — do NOT
|
|
10
|
+
skip to sub-skills directly unless the user explicitly requests a narrow activity.
|
|
11
|
+
Iterative and resumable via tracker file. Uses subagent delegation to save context.
|
|
12
|
+
allowed-tools: Read Write Edit Glob Grep Bash Task
|
|
13
|
+
---
|
|
14
|
+
|
|
15
|
+
# Add E2E Tests
|
|
16
|
+
|
|
17
|
+
Go from zero to passing Playwright tests for the target feature in one interactive session.
|
|
18
|
+
|
|
19
|
+
## Input
|
|
20
|
+
|
|
21
|
+
Parse the target URL and feature description from: $ARGUMENTS
|
|
22
|
+
|
|
23
|
+
Derive a **feature slug** from the feature description (e.g., "Login flow" → `login`, "Checkout with payment" → `checkout`). Use this slug for file naming throughout.
|
|
24
|
+
|
|
25
|
+
## Session Protocol
|
|
26
|
+
|
|
27
|
+
### 1. Orient: Understand the Project, the Product, and Your Capabilities
|
|
28
|
+
|
|
29
|
+
Before planning any work, build deep situational awareness. This step determines the quality of everything that follows — rushed orientation leads to missed test cases and wasted effort.
|
|
30
|
+
|
|
31
|
+
**Check for existing progress:**
|
|
32
|
+
- If `e2e-tracker.md` exists in the project root, read it and resume from where you left off — skip to **step 2 (Plan)** with the remaining work.
|
|
33
|
+
- If no tracker exists, this is a fresh start. Proceed with orientation below.
|
|
34
|
+
|
|
35
|
+
#### First: create initial tasks and tracker
|
|
36
|
+
|
|
37
|
+
As soon as you parse the user's request:
|
|
38
|
+
|
|
39
|
+
1. **Create the tracker** — write `e2e-tracker.md` with the goal (URL, feature, slug) and a skeleton plan.
|
|
40
|
+
2. **Create high-level tasks** for the work ahead — analyze codebase, explore the product, plan coverage, generate test specs, write tests, verify tests.
|
|
41
|
+
|
|
42
|
+
These are your starting skeleton. As you work through orientation and discover the actual shape of the work, refine both the tasks and the tracker — break tasks into granular sub-tasks, add new ones, remove ones that don't apply.
|
|
43
|
+
|
|
44
|
+
Treat the task list as a visible milestone log. Keep it concise, but update it continuously. Do not leave broad tasks open until the end and then mark everything complete in one batch.
|
|
45
|
+
|
|
46
|
+
#### 1a. Understand the codebase
|
|
47
|
+
|
|
48
|
+
- Does a Playwright config exist (`playwright.config.{ts,js,mjs}`)? If not, you will need to scaffold one (see Scaffolding section).
|
|
49
|
+
- Are there existing tests? What conventions do they follow — naming, locators, fixtures, page objects, auth?
|
|
50
|
+
- Load the `analyze-test-codebase` skill and follow its methodology.
|
|
51
|
+
|
|
52
|
+
#### 1b. Understand the product
|
|
53
|
+
|
|
54
|
+
This is the most important part of orientation. You cannot write good tests for a product you don't understand.
|
|
55
|
+
|
|
56
|
+
- **Read existing test cases** — if `test-cases/*.md` files exist, read them to understand what journeys have been mapped. Look at what's covered AND what's missing.
|
|
57
|
+
- **Browse the actual product** — load the `agent-web-interface-guide` skill and use the browser MCP tools to walk through the feature you're testing. Don't just skim the page — interact with it as a user would: fill forms, click buttons, trigger validation, navigate between pages, check error states.
|
|
58
|
+
- **Map the user journey in detail** — understand the complete flow: entry points, happy paths, error paths, edge cases, what happens with invalid input, what happens when the user goes back, what conditional UI exists.
|
|
59
|
+
|
|
60
|
+
Why this matters: absent explicit exploration, agents tend to write tests based on assumptions about how a product works rather than how it actually works. The result is tests that target imaginary behavior or miss critical real behavior. Spending time here prevents both.
|
|
61
|
+
|
|
62
|
+
#### 1c. Know your skills
|
|
63
|
+
|
|
64
|
+
You have access to specialized skills that contain deep domain knowledge. Load the relevant skill before performing each activity — skills prevent improvisation and encode best practices.
|
|
65
|
+
|
|
66
|
+
| Activity | Skill |
|
|
67
|
+
|----------|-------|
|
|
68
|
+
| Analyzing test setup, config, conventions | `analyze-test-codebase` |
|
|
69
|
+
| Deciding what to test, coverage gaps, priorities | `plan-test-coverage` |
|
|
70
|
+
| Opening a URL, browsing, using browser MCP tools | `agent-web-interface-guide` |
|
|
71
|
+
| Creating TC-ID specs from site exploration | `generate-test-cases` |
|
|
72
|
+
| Reviewing TC-ID specs before implementation | `review-test-cases` |
|
|
73
|
+
| Writing, editing, or refactoring test code | `write-test-code` |
|
|
74
|
+
| Reviewing test code before execution signoff | `review-test-code` |
|
|
75
|
+
| Debugging test failures, checking stability | `fix-flaky-tests` |
|
|
76
|
+
|
|
77
|
+
Before doing a substantial activity, load the skill that covers that activity so you can follow its workflow rather than improvising.
|
|
78
|
+
|
|
79
|
+
#### 1d. Update the tracker with orientation findings
|
|
80
|
+
|
|
81
|
+
After orienting, update the tracker with what you learned about the codebase and product, conventions discovered, and your refined plan. The tracker must always answer these four questions for anyone reading it cold:
|
|
82
|
+
|
|
83
|
+
1. What is the goal?
|
|
84
|
+
2. What has been done?
|
|
85
|
+
3. What is remaining?
|
|
86
|
+
4. What should I do next?
|
|
87
|
+
|
|
88
|
+
See [references/tracker-template.md](references/tracker-template.md) for a concrete template.
|
|
89
|
+
|
|
90
|
+
### 2. Plan: Refine Tasks Into Granular Checkpoints
|
|
91
|
+
|
|
92
|
+
By now you have initial tasks and a tracker from step 1. Refine tasks into granular checkpoints. The plan should flow from what you learned during orientation, not from a fixed template.
|
|
93
|
+
|
|
94
|
+
#### Task granularity
|
|
95
|
+
|
|
96
|
+
Think in small checkpoints, not big phases. Each task should represent a concrete, verifiable unit of progress.
|
|
97
|
+
|
|
98
|
+
Too coarse: "Analyze codebase", "Write tests", "Verify tests"
|
|
99
|
+
|
|
100
|
+
Right granularity:
|
|
101
|
+
- "Read playwright.config.ts — extract baseURL, testDir, projects"
|
|
102
|
+
- "Read 2 existing test files — identify locator strategy and naming pattern"
|
|
103
|
+
- "Write conventions report to e2e-plan/conventions.md"
|
|
104
|
+
- "Navigate to /login — catalog all form fields, buttons, and validation messages"
|
|
105
|
+
- "Submit login form empty — record all validation error messages and their positions"
|
|
106
|
+
- "Submit login with invalid email format — record inline validation behavior"
|
|
107
|
+
- "Write TC-LOGIN-001: happy path login with valid credentials"
|
|
108
|
+
- "Write TC-LOGIN-002: login with empty email shows required field error"
|
|
109
|
+
- "Run login.spec.ts and record full output"
|
|
110
|
+
- "Fix TC-LOGIN-003: selector not found — browse page and re-extract selector"
|
|
111
|
+
- "Re-run login.spec.ts — verify fix didn't break other tests"
|
|
112
|
+
- "Check all TC-IDs from spec are present in test files"
|
|
113
|
+
|
|
114
|
+
**Never be conservative.** More tasks is better than fewer. If you discover new work mid-session (a test fails, a selector changed, a form has unexpected validation), add tasks dynamically. The task list is a living document that reflects the real state of the work.
|
|
115
|
+
|
|
116
|
+
Create tasks for verification steps too (running tests, checking coverage, browsing to confirm selectors), not just implementation.
|
|
117
|
+
|
|
118
|
+
Update task status as each checkpoint completes. A good pattern is: finish exploration and mark it complete, finish coverage/spec work and mark it complete, finish implementation and mark it complete, then finish review/execution and mark it complete. Do not keep all milestones open until session end.
|
|
119
|
+
|
|
120
|
+
### 3. Execute
|
|
121
|
+
|
|
122
|
+
Work through your tasks. Load the relevant skill before each activity.
|
|
123
|
+
|
|
124
|
+
#### Planning uses the browser heavily
|
|
125
|
+
|
|
126
|
+
When planning what to test (coverage planning, test case generation), use the browser extensively. Don't just catalog elements — interact with the product to discover:
|
|
127
|
+
- What validation messages appear for each field?
|
|
128
|
+
- What happens when you submit with missing data?
|
|
129
|
+
- What error states exist (network errors, empty states, permission errors)?
|
|
130
|
+
- What does the flow look like end-to-end, not just page-by-page?
|
|
131
|
+
- What edge cases exist (special characters, long inputs, rapid clicks)?
|
|
132
|
+
- What UI changes conditionally (loading states, disabled buttons, progressive disclosure)?
|
|
133
|
+
|
|
134
|
+
Every test case you generate should trace back to something you actually observed or deliberately triggered in the browser. This is how you avoid introducing useless test cases (testing imaginary behavior) and avoid missing important ones (behavior you didn't think to check).
|
|
135
|
+
|
|
136
|
+
#### Subagent delegation
|
|
137
|
+
|
|
138
|
+
Delegate heavy browser exploration and test writing to subagents when that saves context for orchestration, verification, and debugging. When delegating:
|
|
139
|
+
- Pass the relevant file paths (conventions, coverage plan, test specs)
|
|
140
|
+
- Instruct the subagent to invoke the appropriate skill (subagents inherit access to plugin skills)
|
|
141
|
+
- Specify concrete output expectations (file path, format, TC-ID conventions)
|
|
142
|
+
|
|
143
|
+
#### Quality gates
|
|
144
|
+
|
|
145
|
+
Two review gates and a test execution checkpoint are mandatory during execution. The review gates are review-only — they produce findings but do not modify files.
|
|
146
|
+
|
|
147
|
+
**Gate 1: Review test case specs** (after `generate-test-cases`, before `write-test-code`)
|
|
148
|
+
1. Load the `review-test-cases` skill and run it against `test-cases/<feature>.md`
|
|
149
|
+
2. If verdict is **NEEDS REVISION** — address all blockers in the spec before proceeding to implementation
|
|
150
|
+
3. If verdict is **PASS WITH WARNINGS** — address warnings if quick, otherwise note them and proceed
|
|
151
|
+
4. Record the review verdict in the tracker
|
|
152
|
+
|
|
153
|
+
**Gate 2: Review test code** (after `write-test-code`, before final test execution)
|
|
154
|
+
1. Load the `review-test-code` skill and run it against the implemented test files
|
|
155
|
+
2. If verdict is **NEEDS REVISION** — fix all blockers before running tests for signoff
|
|
156
|
+
3. If verdict is **PASS WITH WARNINGS** — fix warnings that affect stability, proceed with execution
|
|
157
|
+
4. Record the review verdict in the tracker
|
|
158
|
+
|
|
159
|
+
**Checkpoint: Test execution**
|
|
160
|
+
1. Run the tests: `npx playwright test <file> --reporter=list 2>&1`
|
|
161
|
+
2. Record full output — green test output is the only proof of correctness
|
|
162
|
+
3. If tests fail, load the `fix-flaky-tests` skill and follow its structured diagnostic approach. Do not guess-and-retry.
|
|
163
|
+
4. Maximum 3 fix-and-rerun cycles per test. If stuck after 3 cycles, record the diagnostic output in the tracker and move on.
|
|
164
|
+
|
|
165
|
+
**Test execution and coverage checks must never be delegated to subagents.** Run `npx playwright test` directly and record the output.
|
|
166
|
+
|
|
167
|
+
#### Update the tracker as you work
|
|
168
|
+
|
|
169
|
+
Do not wait until session end. After each meaningful chunk of progress (completing a step, discovering a blocker, producing an artifact), update the tracker. If your context window resets, only what's in the tracker survives.
|
|
170
|
+
|
|
171
|
+
Keep the tracker and task list synchronized. If you record progress in the tracker, update the corresponding task status in the same phase of work.
|
|
172
|
+
|
|
173
|
+
#### Error recovery
|
|
174
|
+
|
|
175
|
+
If infrastructure failures occur (browser MCP unavailable, clone failures, npm install errors), see [references/error-recovery.md](references/error-recovery.md) for diagnostic steps. General pattern: diagnose, attempt one known fix, if still stuck record in tracker and ask the user.
|
|
176
|
+
|
|
177
|
+
### 4. End of Session
|
|
178
|
+
|
|
179
|
+
Before exiting:
|
|
180
|
+
1. Ensure the tracker reflects all progress, discoveries, and blockers from this session
|
|
181
|
+
2. Write clear instructions for what the next session should do
|
|
182
|
+
3. If all work is complete and all tests pass with full TC-ID coverage: write `<!-- E2E_COMPLETE -->` as the last line of the tracker
|
|
183
|
+
4. If an unrecoverable blocker prevents progress: write `<!-- E2E_BLOCKED: reason -->` as the last line
|
|
184
|
+
|
|
185
|
+
Do not write terminal markers prematurely. Only after you are confident the work is truly done or truly stuck.
|
|
186
|
+
|
|
187
|
+
## Scaffolding
|
|
188
|
+
|
|
189
|
+
If Playwright is not set up in the target project, follow the procedure in [references/scaffolding.md](references/scaffolding.md) to clone the boilerplate, merge configuration, and install dependencies. Log all scaffolding steps in the tracker.
|
|
190
|
+
|
|
191
|
+
## Authentication
|
|
192
|
+
|
|
193
|
+
If the target feature requires login, follow [references/authentication.md](references/authentication.md). Key rule: never hardcode credentials — use environment variables or `storageState`.
|
|
194
|
+
|
|
195
|
+
## Principles
|
|
196
|
+
|
|
197
|
+
- **Skills carry the knowledge** — load the relevant skill before each activity; do not improvise
|
|
198
|
+
- **Subagent-driven** — delegate heavy browser and writing work to subagents to save context
|
|
199
|
+
- **Follow existing conventions** — match the project's test style, not a generic template
|
|
200
|
+
- **Traceable** — every test links back to a TC-ID from the spec
|
|
201
|
+
- **Use what the project provides** — if the scaffolded boilerplate includes Page Object Models (BasePage, pages/), path aliases (tsconfig paths), or utility modules, USE them in generated tests. Do not ship infrastructure that tests ignore. If a boilerplate file is unused after test generation, either integrate it or remove it — dead code in test infrastructure causes confusion.
|
|
202
|
+
- **No arbitrary waits** — use Playwright's built-in auto-wait and event-driven waits
|
|
203
|
+
- **API before UI for setup** — use API calls (`request` fixture) for test data; reserve UI for what you are verifying
|
|
204
|
+
- **Test failures, not just success** — every feature needs error path coverage
|
|
205
|
+
- **Artifacts live in standard locations** — `e2e-plan/` for analysis, `test-cases/` for specs, project test dir for test files
|
|
206
|
+
|
|
207
|
+
## Example Usage
|
|
208
|
+
|
|
209
|
+
```
|
|
210
|
+
Claude Code: /add-e2e-tests https://myapp.com/checkout Checkout flow with cart, shipping, and payment
|
|
211
|
+
Codex: $add-e2e-tests https://myapp.com/checkout Checkout flow with cart, shipping, and payment
|
|
212
|
+
|
|
213
|
+
Claude Code: /add-e2e-tests https://myapp.com/login User authentication including social login
|
|
214
|
+
Codex: $add-e2e-tests https://myapp.com/login User authentication including social login
|
|
215
|
+
```
|
|
@@ -0,0 +1,10 @@
|
|
|
1
|
+
interface:
|
|
2
|
+
display_name: "Run Full E2E Workflow"
|
|
3
|
+
short_description: "Orchestrate browser-led E2E work from exploration to verified tests"
|
|
4
|
+
default_prompt: "Run the full E2E workflow for this feature: maintain a concise progress tracker, explore the live product first, then plan, spec, review, implement, and verify the Playwright tests."
|
|
5
|
+
|
|
6
|
+
dependencies:
|
|
7
|
+
tools:
|
|
8
|
+
- type: "mcp"
|
|
9
|
+
value: "agent-web-interface"
|
|
10
|
+
description: "Browser automation MCP used to explore the site and verify flows"
|
|
@@ -0,0 +1,8 @@
|
|
|
1
|
+
# Authentication Handling for E2E Tests
|
|
2
|
+
|
|
3
|
+
If the target feature requires login or any form of authentication:
|
|
4
|
+
|
|
5
|
+
1. **Check existing infrastructure** — look for existing test fixtures, environment variables, or auth setup files that already handle authentication. Load the `analyze-test-codebase` skill to find auth patterns.
|
|
6
|
+
2. **Ask the user if no auth setup exists** — request credentials or an auth strategy (stored auth state, API tokens, test accounts). Do not proceed with tests that require login until auth is resolved.
|
|
7
|
+
3. **Never hardcode credentials** — use environment variables, Playwright's `storageState`, or the project's existing auth fixture pattern.
|
|
8
|
+
4. **Handle mid-session auth discovery** — if you discover auth is needed mid-session (e.g., a page redirects to login), ask the user immediately and add auth setup as a prerequisite task.
|