@athenaflow/plugin-e2e-test-builder 2.0.9

package/skills/add-e2e-tests/references/error-recovery.md

@@ -0,0 +1,43 @@
+# Error Recovery for Infrastructure Failures
+
+When infrastructure failures occur during E2E test building, follow the general pattern: diagnose, attempt one known fix, if still stuck record in tracker and ask the user.
+
+## Browser MCP unavailable
+
+The browser MCP server (`agent-web-interface`) must be running for site exploration.
+
+1. Verify the MCP server is configured in the project (check `.mcp.json` or plugin config).
+2. Ask the user to confirm the MCP server is running or to restart it.
+3. If unreachable after user intervention, mark the session as blocked: `<!-- E2E_BLOCKED: browser MCP server unreachable -->`.
+
+## Boilerplate clone fails
+
+When `git clone git@github.com:lespaceman/playwright-typescript-e2e-boilerplate.git` fails:
+
+1. Check if SSH keys are configured — the error will usually indicate `Permission denied (publickey)`.
+2. Fall back to HTTPS: `git clone https://github.com/lespaceman/playwright-typescript-e2e-boilerplate.git`.
+3. If both fail, ask the user to verify network access and GitHub authentication.
+
+## npm install fails
+
+1. Check the Node.js version — Playwright requires Node 18+. Run `node --version`.
+2. Clear the npm cache: `npm cache clean --force`.
+3. Check for lockfile conflicts — if both `package-lock.json` and `yarn.lock` exist, ask the user which package manager to use.
+4. If dependency resolution fails, try `npm install --legacy-peer-deps` as a fallback.
+
+## Playwright browser install fails
+
+When `npx playwright install --with-deps chromium` fails:
+
+1. Try installing just chromium without system deps: `npx playwright install chromium`.
+2. Check permissions — on Linux, system dependency installation may require sudo.
+3. If behind a corporate proxy, ask the user for proxy configuration.
+4. As a last resort, ask the user to run the install command manually and confirm when done.
+
+## General pattern
+
+For any infrastructure failure not listed above:
+
+1. **Diagnose** — read the error message carefully, check logs, identify the root cause.
+2. **Attempt one known fix** — apply the most likely solution based on the error.
+3. **If still stuck** — record the full error output and diagnostic steps taken in the tracker, then ask the user for help. Do not loop through multiple speculative fixes.

package/skills/add-e2e-tests/references/scaffolding.md

@@ -0,0 +1,12 @@
+# Scaffolding Playwright from Boilerplate
+
+If Playwright is not set up in the target project, follow this procedure:
+
+1. Clone `git@github.com:lespaceman/playwright-typescript-e2e-boilerplate.git` into a temporary directory.
+2. Copy config, fixtures, pages, and utils into the project. Do not overwrite existing files.
+3. Update `baseURL` to the target URL and remove example tests.
+   - **Test execution strategy:** If the project needs role-based or category-based test filtering, configure it via Playwright `--grep` tags or file naming conventions (`*.admin.spec.ts`), NOT via `testIgnore` regex patterns. A `testIgnore` regex becomes a maintenance trap — every new test file requires updating the regex. If the boilerplate includes a `testIgnore` regex, replace it with tag-based filtering.
+4. Merge devDependencies into `package.json`.
+5. Run `npm install && npx playwright install --with-deps chromium`.
+6. Clean up the temp clone.
+7. Log all scaffolding steps in the tracker.

package/skills/add-e2e-tests/references/tracker-template.md

@@ -0,0 +1,53 @@
+# Tracker Template: e2e-tracker.md
+
+Use this as a starting template when creating the tracker file. Adapt sections as needed.
+
+---
+
+```markdown
+# E2E Test Tracker
+
+## Goal
+
+- **URL:** https://myapp.com/checkout
+- **Feature:** Checkout flow with cart, shipping, and payment
+- **Slug:** checkout
+
+## Progress
+
+### Session 1 — 2026-04-01
+
+- Analyzed codebase: Playwright 1.42, TypeScript, existing auth fixture in `tests/fixtures/auth.ts`
+- Browsed checkout flow: 4-step wizard (cart review, shipping, payment, confirmation)
+- Discovered 3 form validation states per step
+- Generated test spec: `test-cases/checkout.md` with TC-CHECKOUT-001 through TC-CHECKOUT-012
+- Review gate passed with warnings (TC-CHECKOUT-008 needs clarification on coupon edge case)
+
+### Session 2 — 2026-04-02
+
+- Wrote `tests/checkout.spec.ts` covering TC-CHECKOUT-001 through TC-CHECKOUT-012
+- Code review gate passed
+- Ran tests: 10/12 passing, 2 failing (TC-CHECKOUT-006: selector stale, TC-CHECKOUT-011: timeout)
+- Fixed TC-CHECKOUT-006: updated selector from `.price` to `[data-testid="cart-total"]`
+- Fixed TC-CHECKOUT-011: added `waitForResponse` before assertion
+
+## Remaining
+
+- Re-run full suite to confirm fixes
+- Verify TC-ID coverage: all 12 specs should map to test code
+
+## Next Steps
+
+- Run `npx playwright test tests/checkout.spec.ts --reporter=list` and record output
+- If all green, mark complete
+- If failures remain, diagnose with `fix-flaky-tests` skill
+
+<!-- E2E_COMPLETE -->
+```
+
+---
+
+**Terminal markers** (write as the last line of the tracker when appropriate):
+
+- `<!-- E2E_COMPLETE -->` — all tests pass, all TC-IDs covered, work is done
+- `<!-- E2E_BLOCKED: reason -->` — unrecoverable blocker prevents further progress (e.g., `<!-- E2E_BLOCKED: login requires 2FA token we cannot automate -->`)

package/skills/analyze-test-codebase/SKILL.md

@@ -0,0 +1,142 @@
+---
+name: analyze-test-codebase
+description: >
+  Scans and reports on an existing Playwright test codebase. This skill should be used to inspect
+  the current Playwright setup before writing, reviewing, or fixing tests, and should be loaded
+  early when working in a new project to understand existing patterns. Covers: Playwright config,
+  page objects, fixtures, helpers, auth/globalSetup patterns, test conventions, and directory
+  structure. Triggers: "understand", "check", "show me", "inspect", or "analyze"
+  the current test setup, config, infrastructure, patterns, or conventions. In the full add-tests
+  workflow, this skill serves as an early read-only analysis sub-step before planning detailed
+  coverage or writing code. This skill examines existing files and outputs a structured report.
+  It does NOT write or fix tests, install Playwright, or explore live websites — for live site
+  exploration, use agent-web-interface-guide instead.
+allowed-tools: Read Glob Grep Bash
+---
+
+# Analyze Test Codebase
+
+Scan and analyze an existing Playwright test codebase to understand its conventions, configuration, and patterns before writing new tests.
+
+## Workflow
+
+1. **Locate Playwright configuration** — search for `playwright.config.ts`, `playwright.config.js`, or `playwright.config.mjs` in the project root and common subdirectories.
+
+2. **Extract configuration details**:
+   - `baseURL` — the target application URL
+   - `testDir` — where tests live
+   - `projects` — browser/device configurations
+   - `use` — default options (viewport, headless, trace, screenshot)
+   - `webServer` — if Playwright starts the app
+   - `globalSetup` / `globalTeardown` — auth or data seeding scripts
+   - `reporter` — configured reporters
+
+3. **Scan test directory structure**:
+   ```
+   Glob for **/*.spec.ts, **/*.test.ts, **/*.spec.js, **/*.test.js
+   ```
+   - Count total test files
+   - Identify naming conventions (`.spec.ts` vs `.test.ts`)
+   - Map directory organization (by feature? by page? flat?)
+
+4. **Detect patterns in existing tests** — read 2-3 representative test files and identify:
+   - **Locator strategy**: `getByRole`, `getByTestId`, CSS selectors, XPath
+   - **Wait strategy**: auto-waits, explicit `waitFor*`, any `waitForTimeout` (flag these)
+   - **Test structure**: AAA pattern, describe/test nesting, use of `test.step`
+   - **Data management**: fixtures, test data files, factory functions
+   - **Page Objects**: POM pattern usage, fixture-based injection
+   - **Auth handling**: `storageState`, global setup, per-test login
+   - **Custom fixtures**: extended `test` with custom fixtures
+   - **Helper utilities**: shared functions, custom assertions
+   - **Network mocking**: `page.route()` usage, HAR recording (`routeFromHAR`), API interceptors
+   - **Visual regression**: `toHaveScreenshot()`, `toMatchSnapshot()`, snapshot directories
+   - **Accessibility testing**: `@axe-core/playwright` usage, custom a11y assertions
+   - **Cross-browser config**: multiple projects in playwright.config (chromium, firefox, webkit)
+   - **Retry configuration**: `retries` count, trace settings (`on-first-retry`)
+   - **Parallelism**: `fullyParallel`, `workers` count, test isolation strategy
+
+5. **Check for supporting infrastructure**:
+   - `fixtures/` or `helpers/` directories
+   - `pages/` or `pom/` (Page Object Model)
+   - `.env` or `.env.test` files (test-specific environment configuration)
+   - CI configuration (`.github/workflows/`, `Jenkinsfile`, etc.)
+   - `package.json` scripts for running tests
+   - `*.har` files or HAR directories (recorded API responses for mocking)
+   - Snapshot directories (`__snapshots__`, `screenshots/`) for visual regression baselines
+   - Docker or `docker-compose` for test environment setup
+   - Global setup/teardown scripts — what they do (auth, seeding, cleanup)
+
+6. **Generate report** — output a structured summary:
+
+```markdown
+## Test Codebase Analysis
+
+### Framework
+- **Playwright** version X.Y.Z
+- Config: `playwright.config.ts`
+- Test directory: `tests/`
+
+### Conventions
+- File naming: `*.spec.ts`
+- Locator preference: getByRole > getByTestId
+- Structure: describe blocks by feature
+- Page Objects: Yes, in `tests/pages/`
+- Custom fixtures: Yes, in `tests/fixtures/`
+
+### Auth
+- Method: storageState via global setup
+- Auth file: `tests/.auth/user.json`
+
+### Existing Coverage
+- Total test files: N
+- Feature areas covered: [list]
+- Test count: ~N tests
+
+### Network Mocking
+- Method: page.route() / HAR / none
+- Patterns: [list any existing mocking patterns]
+
+### Visual Testing
+- Enabled: Yes/No
+- Tool: toHaveScreenshot() / third-party / none
+- Baseline directory: [path if exists]
+
+### Accessibility Testing
+- Enabled: Yes/No
+- Tool: @axe-core/playwright / custom assertions / none
+
+### Cross-Browser & CI
+- Browser projects: [list from config]
+- Retries: N (CI) / N (local)
+- Workers: N / fullyParallel: Yes/No
+- CI platform: GitHub Actions / Jenkins / none detected
+- Trace: [setting from config]
+
+### Recommendations for New Tests
+- Follow existing `*.spec.ts` naming
+- Use page objects from `tests/pages/`
+- Import test data from `tests/fixtures/testData.ts`
+- Use `baseURL` for navigation (relative paths)
+- Auth: reuse existing storageState setup
+```
+
+## Out of Scope
+
+This skill only reads and reports on the codebase. For related tasks, use the appropriate skill:
+
+| Task | Skill |
+|------|-------|
+| Browsing a live site, interacting with UI elements | `agent-web-interface-guide` |
+| Deciding what to test, coverage gaps, priorities | `plan-test-coverage` |
+| Writing or modifying executable test code | `write-test-code` |
+| Diagnosing flaky or failing tests | `fix-flaky-tests` |
+
+## Example Usage
+
+```
+Claude Code: /analyze-test-codebase
+Codex: $analyze-test-codebase
+
+Claude Code: /analyze-test-codebase ./my-app
+Codex: $analyze-test-codebase ./my-app
+```

package/skills/analyze-test-codebase/agents/claude.yaml

@@ -0,0 +1,3 @@
+frontmatter:
+  argument-hint: "[optional: path to project root]"
+  user-invocable: true

package/skills/analyze-test-codebase/agents/openai.yaml

@@ -0,0 +1,4 @@
+interface:
+  display_name: "Analyze Existing Test Setup"
+  short_description: "Inspect Playwright conventions before planning or writing tests"
+  default_prompt: "Analyze this Playwright codebase and summarize the current setup as an early workflow step before planning detailed coverage or writing test code."

package/skills/fix-flaky-tests/SKILL.md

@@ -0,0 +1,160 @@
+---
+name: fix-flaky-tests
+description: >
+  This skill should be used when a Playwright test is failing, flaky, timing out, or behaving
+  inconsistently. It provides structured root cause analysis for: stabilizing intermittent tests,
+  debugging timeouts ("Test timeout of 30000ms exceeded"), fixing race conditions, investigating
+  local-vs-CI divergence, running repeated stability checks (--repeat-each).
+  IMPORTANT: If running tests with --repeat-each, --retries, or multiple times to check stability,
+  STOP and load this skill first — it has structured root cause analysis that prevents brute-force
+  approaches. Triggers: "stabilize", "intermittent", "flaky", "keeps failing", "fails in CI",
+  "timeout on", "race condition", "run N times to check stability", "verify tests are stable".
+  NOT for writing new tests (use write-test-code) or analyzing setup (use analyze-test-codebase).
+allowed-tools: Read Write Edit Bash Glob Grep Task
+---
+
+# Fix Flaky Tests
+
+Systematically diagnose and fix intermittent Playwright test failures using root cause analysis. A flaky test is worse than no test — it trains teams to ignore failures.
+
+## Input
+
+Parse the test file path or test name from: $ARGUMENTS
+
+If no argument provided, ask: "Which test file or test name is flaky?"
+
+## Workflow
+
+### Step 1: Reproduce and Classify
+
+1. **Read the test file** to understand what it tests and how
+2. **Run the test multiple times** to observe the failure pattern:
+   ```bash
+   npx playwright test <file> --repeat-each=5 --reporter=list 2>&1
+   ```
+3. **Run in isolation** if it passed above — it may only fail with other tests:
+   ```bash
+   npx playwright test --reporter=list 2>&1
+   ```
+4. **Classify the failure** into one of these root cause categories:
+
+| Category | Symptoms |
+|----------|----------|
+| **Timing** | Timeout errors, "element not found", "not visible yet" |
+| **State leakage** | Passes alone, fails when run with other tests |
+| **Data dependency** | Fails when expected data doesn't exist or has changed |
+| **Race condition** | Action fires before page is ready (hydration, animation) |
+| **Selector fragility** | Element found but wrong one, or `.first()` picks different element |
+| **Environment** | Passes locally, fails in CI (viewport, speed, resources) |
+
+### Step 2: Root Cause Analysis
+
+Investigate based on the classification:
+
+**Timing issues:**
+- Look for assertions immediately after actions with no wait for the resulting state change
+- Check if the test asserts before an API response arrives — search for missing `waitForResponse`
+- Look for animations/transitions that affect element state (CSS transitions, skeleton screens)
+- Check for `waitForTimeout` being used as a "fix" — this is a symptom, not a cure
+- Check if `networkidle` or `load` waitUntil would help for navigation
+
+**State leakage:**
+- Run the failing test alone: `npx playwright test --grep "<test name>"`
+- Check if tests share mutable state: global variables, database rows, cookies, localStorage
+- Look for missing cleanup in `afterEach`/`afterAll`
+- Check if `storageState` bleeds between tests or test files
+- Check for test data created by one test that another test depends on
+
+**Race conditions:**
+- Identify the race: what two things are happening concurrently?
+- Check for click handlers that fire before JavaScript hydration completes
+- Look for optimistic UI updates that revert on API response
+- Check for actions during navigation transitions (click during page load)
+- Look for double-clicks or rapid interactions that trigger duplicate actions
+
+**Selector fragility:**
+- Navigate to the page in the browser and verify the selector currently matches the intended element
+- Check if the selector matches multiple elements — `.first()` or `.nth()` is a smell
+- Look for dynamically generated IDs, classes, or attributes
+- Check for conditional rendering that changes element order or presence
+- Verify locators against current DOM structure using `find` and `get_element`
+
+**Environment issues:**
+- Compare CI viewport size vs local — element may be off-screen in CI
+- Check for timezone-dependent assertions (dates, timestamps)
+- Check for locale-dependent formatting (numbers, currency)
+- Check if CI has slower network/CPU affecting timing
+- Look for third-party scripts (analytics, chat widgets) that load differently in CI
+
+### Step 3: Apply the Correct Fix
+
+Use the right fix pattern for the diagnosed root cause. **Never apply a fix without understanding the cause.** See [references/fix-patterns.md](references/fix-patterns.md) for full code examples.
+
+| Category | Principle |
+|----------|-----------|
+| **Timing** | Replace sleeps with event-driven waits (`waitForResponse`, auto-retrying assertions) |
+| **State isolation** | Unique data per test, API-based reset in `beforeEach`, no shared mutable state |
+| **Race condition** | Use `Promise.all` for action + expected response; wait for hydration before interaction |
+| **Selector** | Scope locators to containers with unique content; avoid `.first()` and position-dependent selectors |
+| **Environment** | Explicit viewport, timezone-agnostic assertions, block interfering third-party scripts |
+
+### Step 4: Verify the Fix
+
+1. **Run the test 5+ times** to confirm stability:
+   ```bash
+   npx playwright test <file> --repeat-each=5 --reporter=list 2>&1
+   ```
+2. **Run with the full test suite** to verify no state leakage:
+   ```bash
+   npx playwright test --reporter=list 2>&1
+   ```
+3. If still flaky → return to Step 2 with the new failure output. The initial classification may have been wrong.
+4. **Maximum 3 fix-and-rerun cycles.** If the test is still flaky after 3 attempts, stop and report the diagnostic findings (root cause hypothesis, fixes attempted, remaining failure output) so the user can decide next steps. Do not continue looping.
+
+### Step 5: Summarize
+
+Report:
+1. **Root cause** — what made the test flaky and why
+2. **Fix applied** — what changed and why this fix addresses the root cause
+3. **Verification** — how many consecutive runs passed
+4. **Prevention** — what pattern to follow in future tests to avoid this class of flakiness
+
+## Flakiness Checklist (Less Obvious Causes)
+
+When the standard categories don't fit, check these:
+
+- [ ] **Viewport size** — element off-screen in CI (smaller viewport)
+- [ ] **Font rendering** — text matching fails due to font differences across OS
+- [ ] **Timezone** — date/time assertions fail in different timezones
+- [ ] **Locale** — number/currency formatting differs (1,000 vs 1.000)
+- [ ] **Third-party scripts** — analytics/chat widgets change DOM or block clicks
+- [ ] **Cookie consent banners** — overlay blocks click targets
+- [ ] **Feature flags** — different features enabled in different environments
+- [ ] **Database state** — shared test database with stale or conflicting data
+- [ ] **Parallel execution** — tests interfere when run in parallel workers
+- [ ] **Browser caching** — cached responses differ from fresh ones
+- [ ] **Service workers** — intercepting requests differently than expected
+- [ ] **Lazy loading** — elements not yet in DOM when test tries to interact
+
+## Anti-Patterns: What is NOT a Fix
+
+These mask the problem. Never apply them without a real fix:
+
+| "Fix" | Why It's Wrong | Real Fix |
+|-------|---------------|----------|
+| `waitForTimeout(3000)` | Hides timing race, will break under load | Wait for the specific event |
+| `.first()` added | Hides selector ambiguity | Narrow the selector |
+| Increased timeout to 30s | Hides missing wait or slow setup | Find what you're actually waiting for |
+| `test.skip()` | Ignoring the problem | Diagnose and fix |
+| `retries: 3` without fix | Masks real failures, wastes CI time | Fix the root cause, then keep retries as safety net |
+| `{ force: true }` | Bypasses actionability checks, hides overlapping elements or disabled state | Find and fix the actionability issue: wait for overlay to disappear, scroll element into view, or wait for enabled state |
+| `try/catch` swallowing errors | Test passes but doesn't verify anything | Fix the assertion |
+
+## Multiple Flaky Tests
+
+When a suite has several flaky tests:
+
+1. **Triage first.** Run the full suite once and group failures by root cause category (timing, state leakage, etc.). Shared root causes (broken fixture, leaking state) should be fixed once, not per-test.
+2. **Fix shared infrastructure issues first.** A bad `beforeEach`, a leaking `storageState`, or a missing cleanup can cause many tests to fail. One fix resolves many failures.
+3. **Split independent fixes across subagents** when the fix scopes do not overlap (different test files, no shared fixtures). Pass each subagent the test file path, this diagnostic workflow, and the root cause classification table.
+4. The 3 fix-and-rerun cycle limit applies **per test**, not globally.

package/skills/fix-flaky-tests/agents/claude.yaml

@@ -0,0 +1,3 @@
+frontmatter:
+  argument-hint: "<path to flaky test file or test name>"
+  user-invocable: true

package/skills/fix-flaky-tests/agents/openai.yaml

@@ -0,0 +1,10 @@
+interface:
+  display_name: "Fix Flaky Playwright Tests"
+  short_description: "Diagnose unstable Playwright tests and fix the root cause"
+  default_prompt: "Diagnose this flaky Playwright test, reproduce the failure, and fix the root cause."
+
+dependencies:
+  tools:
+    - type: "mcp"
+      value: "agent-web-interface"
+      description: "Browser automation MCP used to reproduce and inspect unstable flows"

package/skills/fix-flaky-tests/references/fix-patterns.md

@@ -0,0 +1,91 @@
+# Fix Patterns by Root Cause
+
+Code examples for each root cause category. Apply only after diagnosing the cause in Step 2.
+
+## Timing Fixes — Replace Sleeps with Event-Driven Waits
+
+```typescript
+// BAD: arbitrary sleep
+await page.waitForTimeout(2000);
+await expect(element).toBeVisible();
+
+// GOOD: wait for the network event that loads the content
+await page.waitForResponse(resp => resp.url().includes('/api/data'));
+await expect(element).toBeVisible();
+
+// GOOD: wait for loading indicator to disappear
+await expect(page.getByRole('progressbar')).toBeHidden();
+await expect(element).toBeVisible();
+
+// GOOD: wait for navigation to complete
+await page.goto('/page', { waitUntil: 'networkidle' });
+
+// GOOD: use auto-retrying assertion (retries until timeout)
+await expect(page.getByText(/loaded/i)).toBeVisible({ timeout: 10000 });
+```
+
+## State Isolation Fixes
+
+```typescript
+// Unique data per test
+const uniqueEmail = `test-${Date.now()}@example.com`;
+
+// Reset state via API before each test
+test.beforeEach(async ({ request }) => {
+  await request.post('/api/test/reset');
+});
+
+// Use fresh browser context (default in Playwright, but verify)
+// Do NOT share page or context between tests
+```
+
+## Race Condition Fixes
+
+```typescript
+// Wait for hydration/framework readiness
+await page.waitForFunction(() =>
+  document.querySelector('[data-hydrated="true"]')
+);
+
+// Use Promise.all for action + expected response
+const [response] = await Promise.all([
+  page.waitForResponse('**/api/submit'),
+  submitButton.click(),
+]);
+
+// Wait for animation/transition to complete
+await expect(modal).toBeVisible();
+await page.waitForFunction(() =>
+  !document.querySelector('.modal-animating')
+);
+```
+
+## Selector Fixes
+
+```typescript
+// BAD: position-dependent, matches wrong element if order changes
+page.locator('.item').first();
+
+// GOOD: scoped to container with unique content
+page.getByRole('listitem').filter({ hasText: 'Specific Item' });
+
+// GOOD: use test IDs for ambiguous elements
+page.getByTestId('cart-item-sku-123');
+
+// GOOD: scope to a region first, then find within
+page.locator('main').getByRole('button', { name: /submit/i });
+```
+
+## Environment Fixes
+
+```typescript
+// Set explicit viewport in test or config
+test.use({ viewport: { width: 1280, height: 720 } });
+
+// Use timezone-agnostic assertions
+await expect(dateElement).toContainText(/\d{4}/); // year, not full date string
+
+// Block third-party scripts that interfere
+await page.route('**/analytics/**', route => route.abort());
+await page.route('**/chat-widget/**', route => route.abort());
+```