@athenaflow/plugin-e2e-test-builder 2.0.9 → 2.0.10

package/dist/2.0.9/codex/plugin/skills/fix-flaky-tests/SKILL.md

@@ -1,160 +0,0 @@
----
-name: fix-flaky-tests
-description: >
-  This skill should be used when a Playwright test is failing, flaky, timing out, or behaving
-  inconsistently. It provides structured root cause analysis for: stabilizing intermittent tests,
-  debugging timeouts ("Test timeout of 30000ms exceeded"), fixing race conditions, investigating
-  local-vs-CI divergence, running repeated stability checks (--repeat-each).
-  IMPORTANT: If running tests with --repeat-each, --retries, or multiple times to check stability,
-  STOP and load this skill first — it has structured root cause analysis that prevents brute-force
-  approaches. Triggers: "stabilize", "intermittent", "flaky", "keeps failing", "fails in CI",
-  "timeout on", "race condition", "run N times to check stability", "verify tests are stable".
-  NOT for writing new tests (use write-test-code) or analyzing setup (use analyze-test-codebase).
-allowed-tools: Read Write Edit Bash Glob Grep Task
----
-
-# Fix Flaky Tests
-
-Systematically diagnose and fix intermittent Playwright test failures using root cause analysis. A flaky test is worse than no test — it trains teams to ignore failures.
-
-## Input
-
-Parse the test file path or test name from: $ARGUMENTS
-
-If no argument provided, ask: "Which test file or test name is flaky?"
-
-## Workflow
-
-### Step 1: Reproduce and Classify
-
-1. **Read the test file** to understand what it tests and how
-2. **Run the test multiple times** to observe the failure pattern:
-   ```bash
-   npx playwright test <file> --repeat-each=5 --reporter=list 2>&1
-   ```
-3. **Run in isolation** if it passed above — it may only fail with other tests:
-   ```bash
-   npx playwright test --reporter=list 2>&1
-   ```
-4. **Classify the failure** into one of these root cause categories:
-
-| Category | Symptoms |
-|----------|----------|
-| **Timing** | Timeout errors, "element not found", "not visible yet" |
-| **State leakage** | Passes alone, fails when run with other tests |
-| **Data dependency** | Fails when expected data doesn't exist or has changed |
-| **Race condition** | Action fires before page is ready (hydration, animation) |
-| **Selector fragility** | Element found but wrong one, or `.first()` picks different element |
-| **Environment** | Passes locally, fails in CI (viewport, speed, resources) |
-
-### Step 2: Root Cause Analysis
-
-Investigate based on the classification:
-
-**Timing issues:**
-- Look for assertions immediately after actions with no wait for the resulting state change
-- Check if the test asserts before an API response arrives — search for missing `waitForResponse`
-- Look for animations/transitions that affect element state (CSS transitions, skeleton screens)
-- Check for `waitForTimeout` being used as a "fix" — this is a symptom, not a cure
-- Check if `networkidle` or `load` waitUntil would help for navigation
-
-**State leakage:**
-- Run the failing test alone: `npx playwright test --grep "<test name>"`
-- Check if tests share mutable state: global variables, database rows, cookies, localStorage
-- Look for missing cleanup in `afterEach`/`afterAll`
-- Check if `storageState` bleeds between tests or test files
-- Check for test data created by one test that another test depends on
-
-**Race conditions:**
-- Identify the race: what two things are happening concurrently?
-- Check for click handlers that fire before JavaScript hydration completes
-- Look for optimistic UI updates that revert on API response
-- Check for actions during navigation transitions (click during page load)
-- Look for double-clicks or rapid interactions that trigger duplicate actions
-
-**Selector fragility:**
-- Navigate to the page in the browser and verify the selector currently matches the intended element
-- Check if the selector matches multiple elements — `.first()` or `.nth()` is a smell
-- Look for dynamically generated IDs, classes, or attributes
-- Check for conditional rendering that changes element order or presence
-- Verify locators against current DOM structure using `find` and `get_element`
-
-**Environment issues:**
-- Compare CI viewport size vs local — element may be off-screen in CI
-- Check for timezone-dependent assertions (dates, timestamps)
-- Check for locale-dependent formatting (numbers, currency)
-- Check if CI has slower network/CPU affecting timing
-- Look for third-party scripts (analytics, chat widgets) that load differently in CI
-
-### Step 3: Apply the Correct Fix
-
-Use the right fix pattern for the diagnosed root cause. **Never apply a fix without understanding the cause.** See [references/fix-patterns.md](references/fix-patterns.md) for full code examples.
-
-| Category | Principle |
-|----------|-----------|
-| **Timing** | Replace sleeps with event-driven waits (`waitForResponse`, auto-retrying assertions) |
-| **State isolation** | Unique data per test, API-based reset in `beforeEach`, no shared mutable state |
-| **Race condition** | Use `Promise.all` for action + expected response; wait for hydration before interaction |
-| **Selector** | Scope locators to containers with unique content; avoid `.first()` and position-dependent selectors |
-| **Environment** | Explicit viewport, timezone-agnostic assertions, block interfering third-party scripts |
-
-### Step 4: Verify the Fix
-
-1. **Run the test 5+ times** to confirm stability:
-   ```bash
-   npx playwright test <file> --repeat-each=5 --reporter=list 2>&1
-   ```
-2. **Run with the full test suite** to verify no state leakage:
-   ```bash
-   npx playwright test --reporter=list 2>&1
-   ```
-3. If still flaky → return to Step 2 with the new failure output. The initial classification may have been wrong.
-4. **Maximum 3 fix-and-rerun cycles.** If the test is still flaky after 3 attempts, stop and report the diagnostic findings (root cause hypothesis, fixes attempted, remaining failure output) so the user can decide next steps. Do not continue looping.
-
-### Step 5: Summarize
-
-Report:
-1. **Root cause** — what made the test flaky and why
-2. **Fix applied** — what changed and why this fix addresses the root cause
-3. **Verification** — how many consecutive runs passed
-4. **Prevention** — what pattern to follow in future tests to avoid this class of flakiness
-
-## Flakiness Checklist (Less Obvious Causes)
-
-When the standard categories don't fit, check these:
-
-- [ ] **Viewport size** — element off-screen in CI (smaller viewport)
-- [ ] **Font rendering** — text matching fails due to font differences across OS
-- [ ] **Timezone** — date/time assertions fail in different timezones
-- [ ] **Locale** — number/currency formatting differs (1,000 vs 1.000)
-- [ ] **Third-party scripts** — analytics/chat widgets change DOM or block clicks
-- [ ] **Cookie consent banners** — overlay blocks click targets
-- [ ] **Feature flags** — different features enabled in different environments
-- [ ] **Database state** — shared test database with stale or conflicting data
-- [ ] **Parallel execution** — tests interfere when run in parallel workers
-- [ ] **Browser caching** — cached responses differ from fresh ones
-- [ ] **Service workers** — intercepting requests differently than expected
-- [ ] **Lazy loading** — elements not yet in DOM when test tries to interact
-
-## Anti-Patterns: What is NOT a Fix
-
-These mask the problem. Never apply them without a real fix:
-
-| "Fix" | Why It's Wrong | Real Fix |
-|-------|---------------|----------|
-| `waitForTimeout(3000)` | Hides timing race, will break under load | Wait for the specific event |
-| `.first()` added | Hides selector ambiguity | Narrow the selector |
-| Increased timeout to 30s | Hides missing wait or slow setup | Find what you're actually waiting for |
-| `test.skip()` | Ignoring the problem | Diagnose and fix |
-| `retries: 3` without fix | Masks real failures, wastes CI time | Fix the root cause, then keep retries as safety net |
-| `{ force: true }` | Bypasses actionability checks, hides overlapping elements or disabled state | Find and fix the actionability issue: wait for overlay to disappear, scroll element into view, or wait for enabled state |
-| `try/catch` swallowing errors | Test passes but doesn't verify anything | Fix the assertion |
-
-## Multiple Flaky Tests
-
-When a suite has several flaky tests:
-
-1. **Triage first.** Run the full suite once and group failures by root cause category (timing, state leakage, etc.). Shared root causes (broken fixture, leaking state) should be fixed once, not per-test.
-2. **Fix shared infrastructure issues first.** A bad `beforeEach`, a leaking `storageState`, or a missing cleanup can cause many tests to fail. One fix resolves many failures.
-3. **Split independent fixes across subagents** when the fix scopes do not overlap (different test files, no shared fixtures). Pass each subagent the test file path, this diagnostic workflow, and the root cause classification table.
-4. The 3 fix-and-rerun cycle limit applies **per test**, not globally.

package/dist/2.0.9/codex/plugin/skills/fix-flaky-tests/agents/claude.yaml

@@ -1,3 +0,0 @@
-frontmatter:
-  argument-hint: "<path to flaky test file or test name>"
-  user-invocable: true

package/dist/2.0.9/codex/plugin/skills/fix-flaky-tests/agents/openai.yaml

@@ -1,10 +0,0 @@
-interface:
-  display_name: "Fix Flaky Playwright Tests"
-  short_description: "Diagnose unstable Playwright tests and fix the root cause"
-  default_prompt: "Diagnose this flaky Playwright test, reproduce the failure, and fix the root cause."
-
-dependencies:
-  tools:
-    - type: "mcp"
-      value: "agent-web-interface"
-      description: "Browser automation MCP used to reproduce and inspect unstable flows"

package/dist/2.0.9/codex/plugin/skills/fix-flaky-tests/references/fix-patterns.md

@@ -1,91 +0,0 @@
-# Fix Patterns by Root Cause
-
-Code examples for each root cause category. Apply only after diagnosing the cause in Step 2.
-
-## Timing Fixes — Replace Sleeps with Event-Driven Waits
-
-```typescript
-// BAD: arbitrary sleep
-await page.waitForTimeout(2000);
-await expect(element).toBeVisible();
-
-// GOOD: wait for the network event that loads the content
-await page.waitForResponse(resp => resp.url().includes('/api/data'));
-await expect(element).toBeVisible();
-
-// GOOD: wait for loading indicator to disappear
-await expect(page.getByRole('progressbar')).toBeHidden();
-await expect(element).toBeVisible();
-
-// GOOD: wait for navigation to complete
-await page.goto('/page', { waitUntil: 'networkidle' });
-
-// GOOD: use auto-retrying assertion (retries until timeout)
-await expect(page.getByText(/loaded/i)).toBeVisible({ timeout: 10000 });
-```
-
-## State Isolation Fixes
-
-```typescript
-// Unique data per test
-const uniqueEmail = `test-${Date.now()}@example.com`;
-
-// Reset state via API before each test
-test.beforeEach(async ({ request }) => {
-  await request.post('/api/test/reset');
-});
-
-// Use fresh browser context (default in Playwright, but verify)
-// Do NOT share page or context between tests
-```
-
-## Race Condition Fixes
-
-```typescript
-// Wait for hydration/framework readiness
-await page.waitForFunction(() =>
-  document.querySelector('[data-hydrated="true"]')
-);
-
-// Use Promise.all for action + expected response
-const [response] = await Promise.all([
-  page.waitForResponse('**/api/submit'),
-  submitButton.click(),
-]);
-
-// Wait for animation/transition to complete
-await expect(modal).toBeVisible();
-await page.waitForFunction(() =>
-  !document.querySelector('.modal-animating')
-);
-```
-
-## Selector Fixes
-
-```typescript
-// BAD: position-dependent, matches wrong element if order changes
-page.locator('.item').first();
-
-// GOOD: scoped to container with unique content
-page.getByRole('listitem').filter({ hasText: 'Specific Item' });
-
-// GOOD: use test IDs for ambiguous elements
-page.getByTestId('cart-item-sku-123');
-
-// GOOD: scope to a region first, then find within
-page.locator('main').getByRole('button', { name: /submit/i });
-```
-
-## Environment Fixes
-
-```typescript
-// Set explicit viewport in test or config
-test.use({ viewport: { width: 1280, height: 720 } });
-
-// Use timezone-agnostic assertions
-await expect(dateElement).toContainText(/\d{4}/); // year, not full date string
-
-// Block third-party scripts that interfere
-await page.route('**/analytics/**', route => route.abort());
-await page.route('**/chat-widget/**', route => route.abort());
-```

package/dist/2.0.9/codex/plugin/skills/generate-test-cases/agents/claude.yaml

@@ -1,3 +0,0 @@
-frontmatter:
-  argument-hint: "<url> <user journey description>"
-  user-invocable: true

package/dist/2.0.9/codex/plugin/skills/generate-test-cases/agents/openai.yaml

@@ -1,10 +0,0 @@
-interface:
-  display_name: "Write TC-ID Specs"
-  short_description: "Explore the feature and write detailed TC-ID test case specs"
-  default_prompt: "Explore this feature and generate detailed TC-ID test case specifications."
-
-dependencies:
-  tools:
-    - type: "mcp"
-      value: "agent-web-interface"
-      description: "Browser automation MCP used to inspect the live flow before writing specs"

package/dist/2.0.9/codex/plugin/skills/generate-test-cases/references/scenario-categories.md

@@ -1,36 +0,0 @@
-# Scenario Categories — Detailed Checklists
-
-These checklists support Step 4 of the generate-test-cases skill. Each category covers scenarios that may not be directly triggerable during browser exploration but must be included in comprehensive test specifications.
-
-## Network & Performance
-
-- Network failure during form submission (mock 500, timeout)
-- Slow API response (loading states, skeleton screens, spinners)
-- Large data sets (pagination, infinite scroll, 100+ items)
-- Offline behavior (if PWA or service worker is present)
-
-## Accessibility (WCAG 2.1 AA)
-
-- Keyboard-only navigation through the entire flow (Tab, Enter, Escape)
-- Screen reader announcements for dynamic content (ARIA live regions)
-- Focus management after modal open/close, page transitions
-- Color contrast for error states and disabled elements
-- Form error association (`aria-describedby` linking errors to fields)
-
-## Visual Consistency
-
-- Layout stability (no unexpected content shifts after load)
-- Responsive behavior at standard breakpoints (mobile 375px, tablet 768px, desktop 1280px)
-- Dark mode rendering if supported
-
-## Cross-browser Considerations
-
-- Safari-specific behavior (date inputs, smooth scrolling, storage quirks)
-- Firefox form validation differences
-- Mobile browser touch targets and gestures
-
-## Concurrent & Session
-
-- Session expiry mid-flow (cookie cleared during multi-step)
-- Concurrent access (two tabs, same user)
-- Race conditions (double-click submit, rapid navigation)

package/dist/2.0.9/codex/plugin/skills/plan-test-coverage/agents/claude.yaml

@@ -1,3 +0,0 @@
-frontmatter:
-  argument-hint: "<url> <feature or area to test>"
-  user-invocable: true

package/dist/2.0.9/codex/plugin/skills/plan-test-coverage/agents/openai.yaml

@@ -1,10 +0,0 @@
-interface:
-  display_name: "Plan Coverage Priorities"
-  short_description: "Identify coverage gaps and prioritize what to test first"
-  default_prompt: "Review this feature and produce a prioritized E2E coverage plan without writing specs or code."
-
-dependencies:
-  tools:
-    - type: "mcp"
-      value: "agent-web-interface"
-      description: "Browser automation MCP used for lightweight site inspection"

package/dist/2.0.9/codex/plugin/skills/review-test-cases/SKILL.md

@@ -1,147 +0,0 @@
----
-name: review-test-cases
-description: >
-  This skill should be used when a quality review of TC-ID test case specifications is needed before writing executable
-  test code. It reviews the spec artifact only; it does not implement or rewrite tests.
-  Triggers: "review test cases", "check test specs", "review TC-IDs", "audit test coverage",
-  "are my test cases good", "validate test specs", "review test-cases/*.md",
-  "check for gaps in test cases", "review before writing tests", "quality check test specs".
-  Inserted as a quality gate between generate-test-cases and write-test-code — catches
-  gaps, duplication, weak assertions, missing error paths, and invented scenarios before they get
-  encoded into test code. Review-only — does NOT modify the spec file, does NOT write test code.
-  The write-test-code skill should be used for implementation.
-allowed-tools: Read Glob Grep Task
----
-
-# Review Test Cases
-
-Review TC-ID test case specifications for completeness, accuracy, and quality before they are implemented as executable Playwright tests. This is a quality gate — catch problems in the spec, not in the code.
-
-## Input
-
-Parse the spec file path from: $ARGUMENTS
-
-If no argument provided, search for `test-cases/*.md` files and review the most recently modified one.
-
-## Workflow
-
-### Step 1: Load the Spec and Context
-
-1. Read the test case spec file
-2. Read any related files for context:
-   - `e2e-plan/conventions.md` or `e2e-plan/coverage-plan.md` if they exist
-   - `e2e-tracker.md` if it exists (to understand what was explored)
-3. Extract the target URL from the spec header
-
-### Step 2: Run the Review Checklist
-
-Evaluate every test case against each criterion. Track findings by severity:
-
-- **BLOCKER** — must fix before writing tests (missing critical paths, invented behavior, wrong URL)
-- **WARNING** — should fix, will cause problems in implementation (vague steps, weak assertions, duplication)
-- **SUGGESTION** — optional improvement (priority adjustment, better categorization, additional edge case)
-
-#### 2a. Coverage Completeness
-
-| Check | What to Look For |
-|-------|-----------------|
-| Happy path present | At least one Critical-priority test covers the primary success flow end-to-end |
-| Error paths covered (MINIMUM) | Every spec MUST have at least: (1) one server error test (500), (2) one network failure test (timeout/offline), (3) one empty state test. If auth is involved: (4) one session expiry test. Missing any of these is a BLOCKER, not a suggestion |
-| Boundary conditions | Min/max values, empty inputs, special characters, long strings |
-| Authentication edge cases | Session expiry, unauthorized access, role-based differences (if applicable) |
-| Navigation edge cases | Back/forward, direct URL access, refresh mid-flow |
-| Missing user actions | Every interactive element on the page should appear in at least one test case |
-
-#### 2b. Specification Quality
-
-| Check | What to Look For |
-|-------|-----------------|
-| Steps are concrete | "Click the Submit button" not "submit the form"; "Enter 'test@example.com' in Email field" not "enter email" |
-| Expected results are observable | Specific text, URL change, element state — not "page updates" or "works correctly" |
-| Preconditions are explicit | Auth state, test data, feature flags, starting URL — nothing assumed |
-| TC-IDs are sequential | No gaps, no duplicates, correct feature prefix |
-| Priority is justified | Critical = blocks core journey; not everything is Critical |
-| Categories are accurate | Happy Path vs Validation vs Edge Case — correctly classified |
-
-#### 2c. Invented vs Observed
-
-This is the most important check. Test cases should trace back to behavior that was actually observed or deliberately triggered during exploration, not assumed.
-
-Red flags for invented scenarios:
-- Specific error message text that wasn't observed (e.g., "Please enter a valid email" when the actual message might differ)
-- Assumptions about validation rules without exploration evidence (e.g., "minimum 8 characters" without trying it)
-- Test cases for UI elements that may not exist (e.g., "retry button" on error page without visiting the error page)
-- Server-side behavior assumptions (e.g., "rate limit after 5 attempts" without evidence)
-
-When suspicious: delegate a spot-check to a subagent with browser access (Task tool). Pass it the target URL, the specific TC-IDs under suspicion, and the claims to verify (element existence, error message text, validation behavior). The subagent should return structured evidence: what it found, what matched, what differed.
-
-#### 2d. Duplication and Overlap
-
-- Flag test cases that test the same behavior with trivially different inputs
-- Flag test cases where the steps are identical but expected results differ only cosmetically
-- Merging candidates: cases that could be combined into a single parameterized test without losing coverage
-
-#### 2e. Implementability
-
-- Flag steps that cannot be automated with Playwright (e.g., "verify email arrives", "check database directly")
-- Flag preconditions that require manual setup with no automation path
-- Flag assertions that require visual comparison without specifying tolerance
-- Flag test cases that depend on third-party services (payment processors, OAuth providers) without a mock strategy
-
-### Step 3: Produce the Review Report
-
-Output a structured review with this format:
-
-```markdown
-# Test Case Review: <feature>
-
-**Spec file:** <path>
-**Total test cases:** <count>
-**Review date:** <date>
-
-## Verdict: PASS | PASS WITH WARNINGS | NEEDS REVISION
-
-## Blockers (<count>)
-- **TC-<ID>**: <issue description>
-
-## Warnings (<count>)
-- **TC-<ID>**: <issue description>
-
-## Suggestions (<count>)
-- **TC-<ID>**: <issue description>
-
-## Coverage Gaps
-- <Missing scenario that should be added>
-
-## Duplication
-- **TC-<ID>** and **TC-<ID>**: <overlap description>
-
-## Summary
-<2-3 sentences on overall spec quality and what to address before implementation>
-```
-
-### Step 4: Verdict Rules
-
-- **PASS** — no blockers, 2 or fewer warnings. Proceed to write-test-code.
-- **PASS WITH WARNINGS** — no blockers, 3+ warnings. Can proceed but should address warnings.
-- **NEEDS REVISION** — 1+ blockers. Do not proceed to write-test-code until blockers are resolved.
-
-Example: 0 blockers + 2 warnings = PASS. 0 blockers + 3 warnings = PASS WITH WARNINGS. 1+ blockers = NEEDS REVISION regardless of warning count.
-
-## Principles
-
-- **Review-only** — never modify the spec file; report findings for the author to act on
-- **Evidence over opinion** — cite specific TC-IDs and quote specific steps/assertions when flagging issues
-- **Spot-check against live site** — delegate to a subagent with browser access to verify 2-3 suspicious claims rather than trusting all text at face value
-- **Bounded output** — the review report should be actionable and finite, not an exhaustive rewrite
-- **Severity matters** — distinguish blockers from suggestions; not every imperfection is worth fixing before implementation
-
-## Example Usage
-
-```
-Claude Code: /review-test-cases test-cases/login.md
-Codex: $review-test-cases test-cases/login.md
-
-Claude Code: /review-test-cases test-cases/checkout.md
-Codex: $review-test-cases test-cases/checkout.md
-```

package/dist/2.0.9/codex/plugin/skills/review-test-cases/agents/claude.yaml

@@ -1,3 +0,0 @@
-frontmatter:
-  argument-hint: "<path to test-cases/*.md spec file>"
-  user-invocable: true

package/dist/2.0.9/codex/plugin/skills/review-test-cases/agents/openai.yaml

@@ -1,10 +0,0 @@
-interface:
-  display_name: "Review TC-ID Specs"
-  short_description: "Review TC-ID specs for gaps, duplication, and weak assertions"
-  default_prompt: "Review these TC-ID test case specifications before implementation and flag quality issues."
-
-dependencies:
-  tools:
-    - type: "mcp"
-      value: "agent-web-interface"
-      description: "Browser automation MCP used to spot-check observed behavior claims against live site"