@athenaflow/plugin-e2e-test-builder 2.0.9
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.claude-plugin/plugin.json +20 -0
- package/.codex-plugin/plugin.json +15 -0
- package/dist/2.0.8/.agents/plugins/marketplace.json +14 -0
- package/dist/2.0.8/claude/plugin/.claude-plugin/plugin.json +20 -0
- package/dist/2.0.8/claude/plugin/package.json +9 -0
- package/dist/2.0.8/claude/plugin/skills/add-e2e-tests/SKILL.md +217 -0
- package/dist/2.0.8/claude/plugin/skills/add-e2e-tests/agents/claude.yaml +1 -0
- package/dist/2.0.8/claude/plugin/skills/add-e2e-tests/agents/openai.yaml +10 -0
- package/dist/2.0.8/claude/plugin/skills/add-e2e-tests/references/authentication.md +8 -0
- package/dist/2.0.8/claude/plugin/skills/add-e2e-tests/references/error-recovery.md +43 -0
- package/dist/2.0.8/claude/plugin/skills/add-e2e-tests/references/scaffolding.md +12 -0
- package/dist/2.0.8/claude/plugin/skills/add-e2e-tests/references/tracker-template.md +53 -0
- package/dist/2.0.8/claude/plugin/skills/analyze-test-codebase/SKILL.md +142 -0
- package/dist/2.0.8/claude/plugin/skills/analyze-test-codebase/agents/claude.yaml +3 -0
- package/dist/2.0.8/claude/plugin/skills/analyze-test-codebase/agents/openai.yaml +4 -0
- package/dist/2.0.8/claude/plugin/skills/fix-flaky-tests/SKILL.md +160 -0
- package/dist/2.0.8/claude/plugin/skills/fix-flaky-tests/agents/claude.yaml +3 -0
- package/dist/2.0.8/claude/plugin/skills/fix-flaky-tests/agents/openai.yaml +10 -0
- package/dist/2.0.8/claude/plugin/skills/fix-flaky-tests/references/fix-patterns.md +91 -0
- package/dist/2.0.8/claude/plugin/skills/generate-test-cases/SKILL.md +184 -0
- package/dist/2.0.8/claude/plugin/skills/generate-test-cases/agents/claude.yaml +3 -0
- package/dist/2.0.8/claude/plugin/skills/generate-test-cases/agents/openai.yaml +10 -0
- package/dist/2.0.8/claude/plugin/skills/generate-test-cases/references/scenario-categories.md +36 -0
- package/dist/2.0.8/claude/plugin/skills/plan-test-coverage/SKILL.md +116 -0
- package/dist/2.0.8/claude/plugin/skills/plan-test-coverage/agents/claude.yaml +3 -0
- package/dist/2.0.8/claude/plugin/skills/plan-test-coverage/agents/openai.yaml +10 -0
- package/dist/2.0.8/claude/plugin/skills/review-test-cases/SKILL.md +147 -0
- package/dist/2.0.8/claude/plugin/skills/review-test-cases/agents/claude.yaml +3 -0
- package/dist/2.0.8/claude/plugin/skills/review-test-cases/agents/openai.yaml +10 -0
- package/dist/2.0.8/claude/plugin/skills/review-test-code/SKILL.md +189 -0
- package/dist/2.0.8/claude/plugin/skills/review-test-code/agents/claude.yaml +3 -0
- package/dist/2.0.8/claude/plugin/skills/review-test-code/agents/openai.yaml +10 -0
- package/dist/2.0.8/claude/plugin/skills/write-test-code/SKILL.md +227 -0
- package/dist/2.0.8/claude/plugin/skills/write-test-code/agents/claude.yaml +3 -0
- package/dist/2.0.8/claude/plugin/skills/write-test-code/agents/openai.yaml +10 -0
- package/dist/2.0.8/claude/plugin/skills/write-test-code/references/anti-patterns.md +88 -0
- package/dist/2.0.8/claude/plugin/skills/write-test-code/references/api-setup-teardown.md +83 -0
- package/dist/2.0.8/claude/plugin/skills/write-test-code/references/auth-patterns.md +63 -0
- package/dist/2.0.8/claude/plugin/skills/write-test-code/references/mapping-tables.md +56 -0
- package/dist/2.0.8/claude/plugin/skills/write-test-code/references/network-interception.md +56 -0
- package/dist/2.0.8/codex/plugin/.codex-plugin/plugin.json +15 -0
- package/dist/2.0.8/codex/plugin/package.json +9 -0
- package/dist/2.0.8/codex/plugin/skills/add-e2e-tests/SKILL.md +217 -0
- package/dist/2.0.8/codex/plugin/skills/add-e2e-tests/agents/claude.yaml +1 -0
- package/dist/2.0.8/codex/plugin/skills/add-e2e-tests/agents/openai.yaml +10 -0
- package/dist/2.0.8/codex/plugin/skills/add-e2e-tests/references/authentication.md +8 -0
- package/dist/2.0.8/codex/plugin/skills/add-e2e-tests/references/error-recovery.md +43 -0
- package/dist/2.0.8/codex/plugin/skills/add-e2e-tests/references/scaffolding.md +12 -0
- package/dist/2.0.8/codex/plugin/skills/add-e2e-tests/references/tracker-template.md +53 -0
- package/dist/2.0.8/codex/plugin/skills/analyze-test-codebase/SKILL.md +142 -0
- package/dist/2.0.8/codex/plugin/skills/analyze-test-codebase/agents/claude.yaml +3 -0
- package/dist/2.0.8/codex/plugin/skills/analyze-test-codebase/agents/openai.yaml +4 -0
- package/dist/2.0.8/codex/plugin/skills/fix-flaky-tests/SKILL.md +160 -0
- package/dist/2.0.8/codex/plugin/skills/fix-flaky-tests/agents/claude.yaml +3 -0
- package/dist/2.0.8/codex/plugin/skills/fix-flaky-tests/agents/openai.yaml +10 -0
- package/dist/2.0.8/codex/plugin/skills/fix-flaky-tests/references/fix-patterns.md +91 -0
- package/dist/2.0.8/codex/plugin/skills/generate-test-cases/SKILL.md +184 -0
- package/dist/2.0.8/codex/plugin/skills/generate-test-cases/agents/claude.yaml +3 -0
- package/dist/2.0.8/codex/plugin/skills/generate-test-cases/agents/openai.yaml +10 -0
- package/dist/2.0.8/codex/plugin/skills/generate-test-cases/references/scenario-categories.md +36 -0
- package/dist/2.0.8/codex/plugin/skills/plan-test-coverage/SKILL.md +116 -0
- package/dist/2.0.8/codex/plugin/skills/plan-test-coverage/agents/claude.yaml +3 -0
- package/dist/2.0.8/codex/plugin/skills/plan-test-coverage/agents/openai.yaml +10 -0
- package/dist/2.0.8/codex/plugin/skills/review-test-cases/SKILL.md +147 -0
- package/dist/2.0.8/codex/plugin/skills/review-test-cases/agents/claude.yaml +3 -0
- package/dist/2.0.8/codex/plugin/skills/review-test-cases/agents/openai.yaml +10 -0
- package/dist/2.0.8/codex/plugin/skills/review-test-code/SKILL.md +189 -0
- package/dist/2.0.8/codex/plugin/skills/review-test-code/agents/claude.yaml +3 -0
- package/dist/2.0.8/codex/plugin/skills/review-test-code/agents/openai.yaml +10 -0
- package/dist/2.0.8/codex/plugin/skills/write-test-code/SKILL.md +227 -0
- package/dist/2.0.8/codex/plugin/skills/write-test-code/agents/claude.yaml +3 -0
- package/dist/2.0.8/codex/plugin/skills/write-test-code/agents/openai.yaml +10 -0
- package/dist/2.0.8/codex/plugin/skills/write-test-code/references/anti-patterns.md +88 -0
- package/dist/2.0.8/codex/plugin/skills/write-test-code/references/api-setup-teardown.md +83 -0
- package/dist/2.0.8/codex/plugin/skills/write-test-code/references/auth-patterns.md +63 -0
- package/dist/2.0.8/codex/plugin/skills/write-test-code/references/mapping-tables.md +56 -0
- package/dist/2.0.8/codex/plugin/skills/write-test-code/references/network-interception.md +56 -0
- package/dist/2.0.8/release.json +18 -0
- package/dist/2.0.9/.agents/plugins/marketplace.json +14 -0
- package/dist/2.0.9/claude/plugin/.claude-plugin/plugin.json +20 -0
- package/dist/2.0.9/claude/plugin/package.json +9 -0
- package/dist/2.0.9/claude/plugin/skills/add-e2e-tests/SKILL.md +215 -0
- package/dist/2.0.9/claude/plugin/skills/add-e2e-tests/agents/claude.yaml +3 -0
- package/dist/2.0.9/claude/plugin/skills/add-e2e-tests/agents/openai.yaml +10 -0
- package/dist/2.0.9/claude/plugin/skills/add-e2e-tests/references/authentication.md +8 -0
- package/dist/2.0.9/claude/plugin/skills/add-e2e-tests/references/error-recovery.md +43 -0
- package/dist/2.0.9/claude/plugin/skills/add-e2e-tests/references/scaffolding.md +12 -0
- package/dist/2.0.9/claude/plugin/skills/add-e2e-tests/references/tracker-template.md +53 -0
- package/dist/2.0.9/claude/plugin/skills/analyze-test-codebase/SKILL.md +142 -0
- package/dist/2.0.9/claude/plugin/skills/analyze-test-codebase/agents/claude.yaml +3 -0
- package/dist/2.0.9/claude/plugin/skills/analyze-test-codebase/agents/openai.yaml +4 -0
- package/dist/2.0.9/claude/plugin/skills/fix-flaky-tests/SKILL.md +160 -0
- package/dist/2.0.9/claude/plugin/skills/fix-flaky-tests/agents/claude.yaml +3 -0
- package/dist/2.0.9/claude/plugin/skills/fix-flaky-tests/agents/openai.yaml +10 -0
- package/dist/2.0.9/claude/plugin/skills/fix-flaky-tests/references/fix-patterns.md +91 -0
- package/dist/2.0.9/claude/plugin/skills/generate-test-cases/SKILL.md +184 -0
- package/dist/2.0.9/claude/plugin/skills/generate-test-cases/agents/claude.yaml +3 -0
- package/dist/2.0.9/claude/plugin/skills/generate-test-cases/agents/openai.yaml +10 -0
- package/dist/2.0.9/claude/plugin/skills/generate-test-cases/references/scenario-categories.md +36 -0
- package/dist/2.0.9/claude/plugin/skills/plan-test-coverage/SKILL.md +117 -0
- package/dist/2.0.9/claude/plugin/skills/plan-test-coverage/agents/claude.yaml +3 -0
- package/dist/2.0.9/claude/plugin/skills/plan-test-coverage/agents/openai.yaml +10 -0
- package/dist/2.0.9/claude/plugin/skills/review-test-cases/SKILL.md +147 -0
- package/dist/2.0.9/claude/plugin/skills/review-test-cases/agents/claude.yaml +3 -0
- package/dist/2.0.9/claude/plugin/skills/review-test-cases/agents/openai.yaml +10 -0
- package/dist/2.0.9/claude/plugin/skills/review-test-code/SKILL.md +189 -0
- package/dist/2.0.9/claude/plugin/skills/review-test-code/agents/claude.yaml +3 -0
- package/dist/2.0.9/claude/plugin/skills/review-test-code/agents/openai.yaml +10 -0
- package/dist/2.0.9/claude/plugin/skills/write-test-code/SKILL.md +227 -0
- package/dist/2.0.9/claude/plugin/skills/write-test-code/agents/claude.yaml +3 -0
- package/dist/2.0.9/claude/plugin/skills/write-test-code/agents/openai.yaml +10 -0
- package/dist/2.0.9/claude/plugin/skills/write-test-code/references/anti-patterns.md +88 -0
- package/dist/2.0.9/claude/plugin/skills/write-test-code/references/api-setup-teardown.md +83 -0
- package/dist/2.0.9/claude/plugin/skills/write-test-code/references/auth-patterns.md +63 -0
- package/dist/2.0.9/claude/plugin/skills/write-test-code/references/mapping-tables.md +56 -0
- package/dist/2.0.9/claude/plugin/skills/write-test-code/references/network-interception.md +56 -0
- package/dist/2.0.9/codex/plugin/.codex-plugin/plugin.json +15 -0
- package/dist/2.0.9/codex/plugin/package.json +9 -0
- package/dist/2.0.9/codex/plugin/skills/add-e2e-tests/SKILL.md +215 -0
- package/dist/2.0.9/codex/plugin/skills/add-e2e-tests/agents/claude.yaml +3 -0
- package/dist/2.0.9/codex/plugin/skills/add-e2e-tests/agents/openai.yaml +10 -0
- package/dist/2.0.9/codex/plugin/skills/add-e2e-tests/references/authentication.md +8 -0
- package/dist/2.0.9/codex/plugin/skills/add-e2e-tests/references/error-recovery.md +43 -0
- package/dist/2.0.9/codex/plugin/skills/add-e2e-tests/references/scaffolding.md +12 -0
- package/dist/2.0.9/codex/plugin/skills/add-e2e-tests/references/tracker-template.md +53 -0
- package/dist/2.0.9/codex/plugin/skills/analyze-test-codebase/SKILL.md +142 -0
- package/dist/2.0.9/codex/plugin/skills/analyze-test-codebase/agents/claude.yaml +3 -0
- package/dist/2.0.9/codex/plugin/skills/analyze-test-codebase/agents/openai.yaml +4 -0
- package/dist/2.0.9/codex/plugin/skills/fix-flaky-tests/SKILL.md +160 -0
- package/dist/2.0.9/codex/plugin/skills/fix-flaky-tests/agents/claude.yaml +3 -0
- package/dist/2.0.9/codex/plugin/skills/fix-flaky-tests/agents/openai.yaml +10 -0
- package/dist/2.0.9/codex/plugin/skills/fix-flaky-tests/references/fix-patterns.md +91 -0
- package/dist/2.0.9/codex/plugin/skills/generate-test-cases/SKILL.md +184 -0
- package/dist/2.0.9/codex/plugin/skills/generate-test-cases/agents/claude.yaml +3 -0
- package/dist/2.0.9/codex/plugin/skills/generate-test-cases/agents/openai.yaml +10 -0
- package/dist/2.0.9/codex/plugin/skills/generate-test-cases/references/scenario-categories.md +36 -0
- package/dist/2.0.9/codex/plugin/skills/plan-test-coverage/SKILL.md +117 -0
- package/dist/2.0.9/codex/plugin/skills/plan-test-coverage/agents/claude.yaml +3 -0
- package/dist/2.0.9/codex/plugin/skills/plan-test-coverage/agents/openai.yaml +10 -0
- package/dist/2.0.9/codex/plugin/skills/review-test-cases/SKILL.md +147 -0
- package/dist/2.0.9/codex/plugin/skills/review-test-cases/agents/claude.yaml +3 -0
- package/dist/2.0.9/codex/plugin/skills/review-test-cases/agents/openai.yaml +10 -0
- package/dist/2.0.9/codex/plugin/skills/review-test-code/SKILL.md +189 -0
- package/dist/2.0.9/codex/plugin/skills/review-test-code/agents/claude.yaml +3 -0
- package/dist/2.0.9/codex/plugin/skills/review-test-code/agents/openai.yaml +10 -0
- package/dist/2.0.9/codex/plugin/skills/write-test-code/SKILL.md +227 -0
- package/dist/2.0.9/codex/plugin/skills/write-test-code/agents/claude.yaml +3 -0
- package/dist/2.0.9/codex/plugin/skills/write-test-code/agents/openai.yaml +10 -0
- package/dist/2.0.9/codex/plugin/skills/write-test-code/references/anti-patterns.md +88 -0
- package/dist/2.0.9/codex/plugin/skills/write-test-code/references/api-setup-teardown.md +83 -0
- package/dist/2.0.9/codex/plugin/skills/write-test-code/references/auth-patterns.md +63 -0
- package/dist/2.0.9/codex/plugin/skills/write-test-code/references/mapping-tables.md +56 -0
- package/dist/2.0.9/codex/plugin/skills/write-test-code/references/network-interception.md +56 -0
- package/dist/2.0.9/release.json +18 -0
- package/package.json +13 -0
- package/skills/add-e2e-tests/SKILL.md +215 -0
- package/skills/add-e2e-tests/agents/claude.yaml +3 -0
- package/skills/add-e2e-tests/agents/openai.yaml +10 -0
- package/skills/add-e2e-tests/references/authentication.md +8 -0
- package/skills/add-e2e-tests/references/error-recovery.md +43 -0
- package/skills/add-e2e-tests/references/scaffolding.md +12 -0
- package/skills/add-e2e-tests/references/tracker-template.md +53 -0
- package/skills/analyze-test-codebase/SKILL.md +142 -0
- package/skills/analyze-test-codebase/agents/claude.yaml +3 -0
- package/skills/analyze-test-codebase/agents/openai.yaml +4 -0
- package/skills/fix-flaky-tests/SKILL.md +160 -0
- package/skills/fix-flaky-tests/agents/claude.yaml +3 -0
- package/skills/fix-flaky-tests/agents/openai.yaml +10 -0
- package/skills/fix-flaky-tests/references/fix-patterns.md +91 -0
- package/skills/generate-test-cases/SKILL.md +184 -0
- package/skills/generate-test-cases/agents/claude.yaml +3 -0
- package/skills/generate-test-cases/agents/openai.yaml +10 -0
- package/skills/generate-test-cases/references/scenario-categories.md +36 -0
- package/skills/plan-test-coverage/SKILL.md +117 -0
- package/skills/plan-test-coverage/agents/claude.yaml +3 -0
- package/skills/plan-test-coverage/agents/openai.yaml +10 -0
- package/skills/review-test-cases/SKILL.md +147 -0
- package/skills/review-test-cases/agents/claude.yaml +3 -0
- package/skills/review-test-cases/agents/openai.yaml +10 -0
- package/skills/review-test-code/SKILL.md +189 -0
- package/skills/review-test-code/agents/claude.yaml +3 -0
- package/skills/review-test-code/agents/openai.yaml +10 -0
- package/skills/write-test-code/SKILL.md +227 -0
- package/skills/write-test-code/agents/claude.yaml +3 -0
- package/skills/write-test-code/agents/openai.yaml +10 -0
- package/skills/write-test-code/references/anti-patterns.md +88 -0
- package/skills/write-test-code/references/api-setup-teardown.md +83 -0
- package/skills/write-test-code/references/auth-patterns.md +63 -0
- package/skills/write-test-code/references/mapping-tables.md +56 -0
- package/skills/write-test-code/references/network-interception.md +56 -0
|
@@ -0,0 +1,142 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: analyze-test-codebase
|
|
3
|
+
description: >
|
|
4
|
+
Scans and reports on an existing Playwright test codebase. This skill should be used to inspect
|
|
5
|
+
the current Playwright setup before writing, reviewing, or fixing tests, and should be loaded
|
|
6
|
+
early when working in a new project to understand existing patterns. Covers: Playwright config,
|
|
7
|
+
page objects, fixtures, helpers, auth/globalSetup patterns, test conventions, and directory
|
|
8
|
+
structure. Triggers: "understand", "check", "show me", "inspect", or "analyze"
|
|
9
|
+
the current test setup, config, infrastructure, patterns, or conventions. In the full add-tests
|
|
10
|
+
workflow, this skill serves as an early read-only analysis sub-step before planning detailed
|
|
11
|
+
coverage or writing code. This skill examines existing files and outputs a structured report.
|
|
12
|
+
It does NOT write or fix tests, install Playwright, or explore live websites — for live site
|
|
13
|
+
exploration, use agent-web-interface-guide instead.
|
|
14
|
+
allowed-tools: Read Glob Grep Bash
|
|
15
|
+
---
|
|
16
|
+
|
|
17
|
+
# Analyze Test Codebase
|
|
18
|
+
|
|
19
|
+
Scan and analyze an existing Playwright test codebase to understand its conventions, configuration, and patterns before writing new tests.
|
|
20
|
+
|
|
21
|
+
## Workflow
|
|
22
|
+
|
|
23
|
+
1. **Locate Playwright configuration** — search for `playwright.config.ts`, `playwright.config.js`, or `playwright.config.mjs` in the project root and common subdirectories.
|
|
24
|
+
|
|
25
|
+
2. **Extract configuration details**:
|
|
26
|
+
- `baseURL` — the target application URL
|
|
27
|
+
- `testDir` — where tests live
|
|
28
|
+
- `projects` — browser/device configurations
|
|
29
|
+
- `use` — default options (viewport, headless, trace, screenshot)
|
|
30
|
+
- `webServer` — if Playwright starts the app
|
|
31
|
+
- `globalSetup` / `globalTeardown` — auth or data seeding scripts
|
|
32
|
+
- `reporter` — configured reporters
|
|
33
|
+
|
|
34
|
+
3. **Scan test directory structure**:
|
|
35
|
+
```
|
|
36
|
+
Glob for **/*.spec.ts, **/*.test.ts, **/*.spec.js, **/*.test.js
|
|
37
|
+
```
|
|
38
|
+
- Count total test files
|
|
39
|
+
- Identify naming conventions (`.spec.ts` vs `.test.ts`)
|
|
40
|
+
- Map directory organization (by feature? by page? flat?)
|
|
41
|
+
|
|
42
|
+
4. **Detect patterns in existing tests** — read 2-3 representative test files and identify:
|
|
43
|
+
- **Locator strategy**: `getByRole`, `getByTestId`, CSS selectors, XPath
|
|
44
|
+
- **Wait strategy**: auto-waits, explicit `waitFor*`, any `waitForTimeout` (flag these)
|
|
45
|
+
- **Test structure**: AAA pattern, describe/test nesting, use of `test.step`
|
|
46
|
+
- **Data management**: fixtures, test data files, factory functions
|
|
47
|
+
- **Page Objects**: POM pattern usage, fixture-based injection
|
|
48
|
+
- **Auth handling**: `storageState`, global setup, per-test login
|
|
49
|
+
- **Custom fixtures**: extended `test` with custom fixtures
|
|
50
|
+
- **Helper utilities**: shared functions, custom assertions
|
|
51
|
+
- **Network mocking**: `page.route()` usage, HAR recording (`routeFromHAR`), API interceptors
|
|
52
|
+
- **Visual regression**: `toHaveScreenshot()`, `toMatchSnapshot()`, snapshot directories
|
|
53
|
+
- **Accessibility testing**: `@axe-core/playwright` usage, custom a11y assertions
|
|
54
|
+
- **Cross-browser config**: multiple projects in playwright.config (chromium, firefox, webkit)
|
|
55
|
+
- **Retry configuration**: `retries` count, trace settings (`on-first-retry`)
|
|
56
|
+
- **Parallelism**: `fullyParallel`, `workers` count, test isolation strategy
|
|
57
|
+
|
|
58
|
+
5. **Check for supporting infrastructure**:
|
|
59
|
+
- `fixtures/` or `helpers/` directories
|
|
60
|
+
- `pages/` or `pom/` (Page Object Model)
|
|
61
|
+
- `.env` or `.env.test` files (test-specific environment configuration)
|
|
62
|
+
- CI configuration (`.github/workflows/`, `Jenkinsfile`, etc.)
|
|
63
|
+
- `package.json` scripts for running tests
|
|
64
|
+
- `*.har` files or HAR directories (recorded API responses for mocking)
|
|
65
|
+
- Snapshot directories (`__snapshots__`, `screenshots/`) for visual regression baselines
|
|
66
|
+
- Docker or `docker-compose` for test environment setup
|
|
67
|
+
- Global setup/teardown scripts — what they do (auth, seeding, cleanup)
|
|
68
|
+
|
|
69
|
+
6. **Generate report** — output a structured summary:
|
|
70
|
+
|
|
71
|
+
```markdown
|
|
72
|
+
## Test Codebase Analysis
|
|
73
|
+
|
|
74
|
+
### Framework
|
|
75
|
+
- **Playwright** version X.Y.Z
|
|
76
|
+
- Config: `playwright.config.ts`
|
|
77
|
+
- Test directory: `tests/`
|
|
78
|
+
|
|
79
|
+
### Conventions
|
|
80
|
+
- File naming: `*.spec.ts`
|
|
81
|
+
- Locator preference: getByRole > getByTestId
|
|
82
|
+
- Structure: describe blocks by feature
|
|
83
|
+
- Page Objects: Yes, in `tests/pages/`
|
|
84
|
+
- Custom fixtures: Yes, in `tests/fixtures/`
|
|
85
|
+
|
|
86
|
+
### Auth
|
|
87
|
+
- Method: storageState via global setup
|
|
88
|
+
- Auth file: `tests/.auth/user.json`
|
|
89
|
+
|
|
90
|
+
### Existing Coverage
|
|
91
|
+
- Total test files: N
|
|
92
|
+
- Feature areas covered: [list]
|
|
93
|
+
- Test count: ~N tests
|
|
94
|
+
|
|
95
|
+
### Network Mocking
|
|
96
|
+
- Method: page.route() / HAR / none
|
|
97
|
+
- Patterns: [list any existing mocking patterns]
|
|
98
|
+
|
|
99
|
+
### Visual Testing
|
|
100
|
+
- Enabled: Yes/No
|
|
101
|
+
- Tool: toHaveScreenshot() / third-party / none
|
|
102
|
+
- Baseline directory: [path if exists]
|
|
103
|
+
|
|
104
|
+
### Accessibility Testing
|
|
105
|
+
- Enabled: Yes/No
|
|
106
|
+
- Tool: @axe-core/playwright / custom assertions / none
|
|
107
|
+
|
|
108
|
+
### Cross-Browser & CI
|
|
109
|
+
- Browser projects: [list from config]
|
|
110
|
+
- Retries: N (CI) / N (local)
|
|
111
|
+
- Workers: N / fullyParallel: Yes/No
|
|
112
|
+
- CI platform: GitHub Actions / Jenkins / none detected
|
|
113
|
+
- Trace: [setting from config]
|
|
114
|
+
|
|
115
|
+
### Recommendations for New Tests
|
|
116
|
+
- Follow existing `*.spec.ts` naming
|
|
117
|
+
- Use page objects from `tests/pages/`
|
|
118
|
+
- Import test data from `tests/fixtures/testData.ts`
|
|
119
|
+
- Use `baseURL` for navigation (relative paths)
|
|
120
|
+
- Auth: reuse existing storageState setup
|
|
121
|
+
```
|
|
122
|
+
|
|
123
|
+
## Out of Scope
|
|
124
|
+
|
|
125
|
+
This skill only reads and reports on the codebase. For related tasks, use the appropriate skill:
|
|
126
|
+
|
|
127
|
+
| Task | Skill |
|
|
128
|
+
|------|-------|
|
|
129
|
+
| Browsing a live site, interacting with UI elements | `agent-web-interface-guide` |
|
|
130
|
+
| Deciding what to test, coverage gaps, priorities | `plan-test-coverage` |
|
|
131
|
+
| Writing or modifying executable test code | `write-test-code` |
|
|
132
|
+
| Diagnosing flaky or failing tests | `fix-flaky-tests` |
|
|
133
|
+
|
|
134
|
+
## Example Usage
|
|
135
|
+
|
|
136
|
+
```
|
|
137
|
+
Claude Code: /analyze-test-codebase
|
|
138
|
+
Codex: $analyze-test-codebase
|
|
139
|
+
|
|
140
|
+
Claude Code: /analyze-test-codebase ./my-app
|
|
141
|
+
Codex: $analyze-test-codebase ./my-app
|
|
142
|
+
```
|
|
@@ -0,0 +1,4 @@
|
|
|
1
|
+
interface:
|
|
2
|
+
display_name: "Analyze Existing Test Setup"
|
|
3
|
+
short_description: "Inspect Playwright conventions before planning or writing tests"
|
|
4
|
+
default_prompt: "Analyze this Playwright codebase and summarize the current setup as an early workflow step before planning detailed coverage or writing test code."
|
|
@@ -0,0 +1,160 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: fix-flaky-tests
|
|
3
|
+
description: >
|
|
4
|
+
This skill should be used when a Playwright test is failing, flaky, timing out, or behaving
|
|
5
|
+
inconsistently. It provides structured root cause analysis for: stabilizing intermittent tests,
|
|
6
|
+
debugging timeouts ("Test timeout of 30000ms exceeded"), fixing race conditions, investigating
|
|
7
|
+
local-vs-CI divergence, running repeated stability checks (--repeat-each).
|
|
8
|
+
IMPORTANT: If running tests with --repeat-each, --retries, or multiple times to check stability,
|
|
9
|
+
STOP and load this skill first — it has structured root cause analysis that prevents brute-force
|
|
10
|
+
approaches. Triggers: "stabilize", "intermittent", "flaky", "keeps failing", "fails in CI",
|
|
11
|
+
"timeout on", "race condition", "run N times to check stability", "verify tests are stable".
|
|
12
|
+
NOT for writing new tests (use write-test-code) or analyzing setup (use analyze-test-codebase).
|
|
13
|
+
allowed-tools: Read Write Edit Bash Glob Grep Task
|
|
14
|
+
---
|
|
15
|
+
|
|
16
|
+
# Fix Flaky Tests
|
|
17
|
+
|
|
18
|
+
Systematically diagnose and fix intermittent Playwright test failures using root cause analysis. A flaky test is worse than no test — it trains teams to ignore failures.
|
|
19
|
+
|
|
20
|
+
## Input
|
|
21
|
+
|
|
22
|
+
Parse the test file path or test name from: $ARGUMENTS
|
|
23
|
+
|
|
24
|
+
If no argument provided, ask: "Which test file or test name is flaky?"
|
|
25
|
+
|
|
26
|
+
## Workflow
|
|
27
|
+
|
|
28
|
+
### Step 1: Reproduce and Classify
|
|
29
|
+
|
|
30
|
+
1. **Read the test file** to understand what it tests and how
|
|
31
|
+
2. **Run the test multiple times** to observe the failure pattern:
|
|
32
|
+
```bash
|
|
33
|
+
npx playwright test <file> --repeat-each=5 --reporter=list 2>&1
|
|
34
|
+
```
|
|
35
|
+
3. **Run in isolation** if it passed above — it may only fail with other tests:
|
|
36
|
+
```bash
|
|
37
|
+
npx playwright test --reporter=list 2>&1
|
|
38
|
+
```
|
|
39
|
+
4. **Classify the failure** into one of these root cause categories:
|
|
40
|
+
|
|
41
|
+
| Category | Symptoms |
|
|
42
|
+
|----------|----------|
|
|
43
|
+
| **Timing** | Timeout errors, "element not found", "not visible yet" |
|
|
44
|
+
| **State leakage** | Passes alone, fails when run with other tests |
|
|
45
|
+
| **Data dependency** | Fails when expected data doesn't exist or has changed |
|
|
46
|
+
| **Race condition** | Action fires before page is ready (hydration, animation) |
|
|
47
|
+
| **Selector fragility** | Element found but wrong one, or `.first()` picks different element |
|
|
48
|
+
| **Environment** | Passes locally, fails in CI (viewport, speed, resources) |
|
|
49
|
+
|
|
50
|
+
### Step 2: Root Cause Analysis
|
|
51
|
+
|
|
52
|
+
Investigate based on the classification:
|
|
53
|
+
|
|
54
|
+
**Timing issues:**
|
|
55
|
+
- Look for assertions immediately after actions with no wait for the resulting state change
|
|
56
|
+
- Check if the test asserts before an API response arrives — search for missing `waitForResponse`
|
|
57
|
+
- Look for animations/transitions that affect element state (CSS transitions, skeleton screens)
|
|
58
|
+
- Check for `waitForTimeout` being used as a "fix" — this is a symptom, not a cure
|
|
59
|
+
- Check if `networkidle` or `load` waitUntil would help for navigation
|
|
60
|
+
|
|
61
|
+
**State leakage:**
|
|
62
|
+
- Run the failing test alone: `npx playwright test --grep "<test name>"`
|
|
63
|
+
- Check if tests share mutable state: global variables, database rows, cookies, localStorage
|
|
64
|
+
- Look for missing cleanup in `afterEach`/`afterAll`
|
|
65
|
+
- Check if `storageState` bleeds between tests or test files
|
|
66
|
+
- Check for test data created by one test that another test depends on
|
|
67
|
+
|
|
68
|
+
**Race conditions:**
|
|
69
|
+
- Identify the race: what two things are happening concurrently?
|
|
70
|
+
- Check for click handlers that fire before JavaScript hydration completes
|
|
71
|
+
- Look for optimistic UI updates that revert on API response
|
|
72
|
+
- Check for actions during navigation transitions (click during page load)
|
|
73
|
+
- Look for double-clicks or rapid interactions that trigger duplicate actions
|
|
74
|
+
|
|
75
|
+
**Selector fragility:**
|
|
76
|
+
- Navigate to the page in the browser and verify the selector currently matches the intended element
|
|
77
|
+
- Check if the selector matches multiple elements — `.first()` or `.nth()` is a smell
|
|
78
|
+
- Look for dynamically generated IDs, classes, or attributes
|
|
79
|
+
- Check for conditional rendering that changes element order or presence
|
|
80
|
+
- Verify locators against current DOM structure using `find` and `get_element`
|
|
81
|
+
|
|
82
|
+
**Environment issues:**
|
|
83
|
+
- Compare CI viewport size vs local — element may be off-screen in CI
|
|
84
|
+
- Check for timezone-dependent assertions (dates, timestamps)
|
|
85
|
+
- Check for locale-dependent formatting (numbers, currency)
|
|
86
|
+
- Check if CI has slower network/CPU affecting timing
|
|
87
|
+
- Look for third-party scripts (analytics, chat widgets) that load differently in CI
|
|
88
|
+
|
|
89
|
+
### Step 3: Apply the Correct Fix
|
|
90
|
+
|
|
91
|
+
Use the right fix pattern for the diagnosed root cause. **Never apply a fix without understanding the cause.** See [references/fix-patterns.md](references/fix-patterns.md) for full code examples.
|
|
92
|
+
|
|
93
|
+
| Category | Principle |
|
|
94
|
+
|----------|-----------|
|
|
95
|
+
| **Timing** | Replace sleeps with event-driven waits (`waitForResponse`, auto-retrying assertions) |
|
|
96
|
+
| **State isolation** | Unique data per test, API-based reset in `beforeEach`, no shared mutable state |
|
|
97
|
+
| **Race condition** | Use `Promise.all` for action + expected response; wait for hydration before interaction |
|
|
98
|
+
| **Selector** | Scope locators to containers with unique content; avoid `.first()` and position-dependent selectors |
|
|
99
|
+
| **Environment** | Explicit viewport, timezone-agnostic assertions, block interfering third-party scripts |
|
|
100
|
+
|
|
101
|
+
### Step 4: Verify the Fix
|
|
102
|
+
|
|
103
|
+
1. **Run the test 5+ times** to confirm stability:
|
|
104
|
+
```bash
|
|
105
|
+
npx playwright test <file> --repeat-each=5 --reporter=list 2>&1
|
|
106
|
+
```
|
|
107
|
+
2. **Run with the full test suite** to verify no state leakage:
|
|
108
|
+
```bash
|
|
109
|
+
npx playwright test --reporter=list 2>&1
|
|
110
|
+
```
|
|
111
|
+
3. If still flaky → return to Step 2 with the new failure output. The initial classification may have been wrong.
|
|
112
|
+
4. **Maximum 3 fix-and-rerun cycles.** If the test is still flaky after 3 attempts, stop and report the diagnostic findings (root cause hypothesis, fixes attempted, remaining failure output) so the user can decide next steps. Do not continue looping.
|
|
113
|
+
|
|
114
|
+
### Step 5: Summarize
|
|
115
|
+
|
|
116
|
+
Report:
|
|
117
|
+
1. **Root cause** — what made the test flaky and why
|
|
118
|
+
2. **Fix applied** — what changed and why this fix addresses the root cause
|
|
119
|
+
3. **Verification** — how many consecutive runs passed
|
|
120
|
+
4. **Prevention** — what pattern to follow in future tests to avoid this class of flakiness
|
|
121
|
+
|
|
122
|
+
## Flakiness Checklist (Less Obvious Causes)
|
|
123
|
+
|
|
124
|
+
When the standard categories don't fit, check these:
|
|
125
|
+
|
|
126
|
+
- [ ] **Viewport size** — element off-screen in CI (smaller viewport)
|
|
127
|
+
- [ ] **Font rendering** — text matching fails due to font differences across OS
|
|
128
|
+
- [ ] **Timezone** — date/time assertions fail in different timezones
|
|
129
|
+
- [ ] **Locale** — number/currency formatting differs (1,000 vs 1.000)
|
|
130
|
+
- [ ] **Third-party scripts** — analytics/chat widgets change DOM or block clicks
|
|
131
|
+
- [ ] **Cookie consent banners** — overlay blocks click targets
|
|
132
|
+
- [ ] **Feature flags** — different features enabled in different environments
|
|
133
|
+
- [ ] **Database state** — shared test database with stale or conflicting data
|
|
134
|
+
- [ ] **Parallel execution** — tests interfere when run in parallel workers
|
|
135
|
+
- [ ] **Browser caching** — cached responses differ from fresh ones
|
|
136
|
+
- [ ] **Service workers** — intercepting requests differently than expected
|
|
137
|
+
- [ ] **Lazy loading** — elements not yet in DOM when test tries to interact
|
|
138
|
+
|
|
139
|
+
## Anti-Patterns: What is NOT a Fix
|
|
140
|
+
|
|
141
|
+
These mask the problem. Never apply them without a real fix:
|
|
142
|
+
|
|
143
|
+
| "Fix" | Why It's Wrong | Real Fix |
|
|
144
|
+
|-------|---------------|----------|
|
|
145
|
+
| `waitForTimeout(3000)` | Hides timing race, will break under load | Wait for the specific event |
|
|
146
|
+
| `.first()` added | Hides selector ambiguity | Narrow the selector |
|
|
147
|
+
| Increased timeout to 30s | Hides missing wait or slow setup | Find what you're actually waiting for |
|
|
148
|
+
| `test.skip()` | Ignoring the problem | Diagnose and fix |
|
|
149
|
+
| `retries: 3` without fix | Masks real failures, wastes CI time | Fix the root cause, then keep retries as safety net |
|
|
150
|
+
| `{ force: true }` | Bypasses actionability checks, hides overlapping elements or disabled state | Find and fix the actionability issue: wait for overlay to disappear, scroll element into view, or wait for enabled state |
|
|
151
|
+
| `try/catch` swallowing errors | Test passes but doesn't verify anything | Fix the assertion |
|
|
152
|
+
|
|
153
|
+
## Multiple Flaky Tests
|
|
154
|
+
|
|
155
|
+
When a suite has several flaky tests:
|
|
156
|
+
|
|
157
|
+
1. **Triage first.** Run the full suite once and group failures by root cause category (timing, state leakage, etc.). Shared root causes (broken fixture, leaking state) should be fixed once, not per-test.
|
|
158
|
+
2. **Fix shared infrastructure issues first.** A bad `beforeEach`, a leaking `storageState`, or a missing cleanup can cause many tests to fail. One fix resolves many failures.
|
|
159
|
+
3. **Split independent fixes across subagents** when the fix scopes do not overlap (different test files, no shared fixtures). Pass each subagent the test file path, this diagnostic workflow, and the root cause classification table.
|
|
160
|
+
4. The 3 fix-and-rerun cycle limit applies **per test**, not globally.
|
|
@@ -0,0 +1,10 @@
|
|
|
1
|
+
interface:
|
|
2
|
+
display_name: "Fix Flaky Playwright Tests"
|
|
3
|
+
short_description: "Diagnose unstable Playwright tests and fix the root cause"
|
|
4
|
+
default_prompt: "Diagnose this flaky Playwright test, reproduce the failure, and fix the root cause."
|
|
5
|
+
|
|
6
|
+
dependencies:
|
|
7
|
+
tools:
|
|
8
|
+
- type: "mcp"
|
|
9
|
+
value: "agent-web-interface"
|
|
10
|
+
description: "Browser automation MCP used to reproduce and inspect unstable flows"
|
|
@@ -0,0 +1,91 @@
|
|
|
1
|
+
# Fix Patterns by Root Cause
|
|
2
|
+
|
|
3
|
+
Code examples for each root cause category. Apply only after diagnosing the cause in Step 2.
|
|
4
|
+
|
|
5
|
+
## Timing Fixes — Replace Sleeps with Event-Driven Waits
|
|
6
|
+
|
|
7
|
+
```typescript
|
|
8
|
+
// BAD: arbitrary sleep
|
|
9
|
+
await page.waitForTimeout(2000);
|
|
10
|
+
await expect(element).toBeVisible();
|
|
11
|
+
|
|
12
|
+
// GOOD: wait for the network event that loads the content
|
|
13
|
+
await page.waitForResponse(resp => resp.url().includes('/api/data'));
|
|
14
|
+
await expect(element).toBeVisible();
|
|
15
|
+
|
|
16
|
+
// GOOD: wait for loading indicator to disappear
|
|
17
|
+
await expect(page.getByRole('progressbar')).toBeHidden();
|
|
18
|
+
await expect(element).toBeVisible();
|
|
19
|
+
|
|
20
|
+
// GOOD: wait for navigation to complete
|
|
21
|
+
await page.goto('/page', { waitUntil: 'networkidle' });
|
|
22
|
+
|
|
23
|
+
// GOOD: use auto-retrying assertion (retries until timeout)
|
|
24
|
+
await expect(page.getByText(/loaded/i)).toBeVisible({ timeout: 10000 });
|
|
25
|
+
```
|
|
26
|
+
|
|
27
|
+
## State Isolation Fixes
|
|
28
|
+
|
|
29
|
+
```typescript
|
|
30
|
+
// Unique data per test
|
|
31
|
+
const uniqueEmail = `test-${Date.now()}@example.com`;
|
|
32
|
+
|
|
33
|
+
// Reset state via API before each test
|
|
34
|
+
test.beforeEach(async ({ request }) => {
|
|
35
|
+
await request.post('/api/test/reset');
|
|
36
|
+
});
|
|
37
|
+
|
|
38
|
+
// Use fresh browser context (default in Playwright, but verify)
|
|
39
|
+
// Do NOT share page or context between tests
|
|
40
|
+
```
|
|
41
|
+
|
|
42
|
+
## Race Condition Fixes
|
|
43
|
+
|
|
44
|
+
```typescript
|
|
45
|
+
// Wait for hydration/framework readiness
|
|
46
|
+
await page.waitForFunction(() =>
|
|
47
|
+
document.querySelector('[data-hydrated="true"]')
|
|
48
|
+
);
|
|
49
|
+
|
|
50
|
+
// Use Promise.all for action + expected response
|
|
51
|
+
const [response] = await Promise.all([
|
|
52
|
+
page.waitForResponse('**/api/submit'),
|
|
53
|
+
submitButton.click(),
|
|
54
|
+
]);
|
|
55
|
+
|
|
56
|
+
// Wait for animation/transition to complete
|
|
57
|
+
await expect(modal).toBeVisible();
|
|
58
|
+
await page.waitForFunction(() =>
|
|
59
|
+
!document.querySelector('.modal-animating')
|
|
60
|
+
);
|
|
61
|
+
```
|
|
62
|
+
|
|
63
|
+
## Selector Fixes
|
|
64
|
+
|
|
65
|
+
```typescript
|
|
66
|
+
// BAD: position-dependent, matches wrong element if order changes
|
|
67
|
+
page.locator('.item').first();
|
|
68
|
+
|
|
69
|
+
// GOOD: scoped to container with unique content
|
|
70
|
+
page.getByRole('listitem').filter({ hasText: 'Specific Item' });
|
|
71
|
+
|
|
72
|
+
// GOOD: use test IDs for ambiguous elements
|
|
73
|
+
page.getByTestId('cart-item-sku-123');
|
|
74
|
+
|
|
75
|
+
// GOOD: scope to a region first, then find within
|
|
76
|
+
page.locator('main').getByRole('button', { name: /submit/i });
|
|
77
|
+
```
|
|
78
|
+
|
|
79
|
+
## Environment Fixes
|
|
80
|
+
|
|
81
|
+
```typescript
|
|
82
|
+
// Set explicit viewport in test or config
|
|
83
|
+
test.use({ viewport: { width: 1280, height: 720 } });
|
|
84
|
+
|
|
85
|
+
// Use timezone-agnostic assertions
|
|
86
|
+
await expect(dateElement).toContainText(/\d{4}/); // year, not full date string
|
|
87
|
+
|
|
88
|
+
// Block third-party scripts that interfere
|
|
89
|
+
await page.route('**/analytics/**', route => route.abort());
|
|
90
|
+
await page.route('**/chat-widget/**', route => route.abort());
|
|
91
|
+
```
|
|
@@ -0,0 +1,184 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: generate-test-cases
|
|
3
|
+
description: >
|
|
4
|
+
Use when the user wants detailed TC-ID test case specifications for a web app feature, not executable code. It explores the target flow, covers happy paths, validation failures, edge cases, and required error states, then writes structured specs under `test-cases/`. Use it after coverage planning or when the user explicitly asks for test cases or TC-IDs. It does not write Playwright code.
|
|
5
|
+
allowed-tools: Read Write Bash Glob Grep Task
|
|
6
|
+
---
|
|
7
|
+
|
|
8
|
+
# Generate Test Cases
|
|
9
|
+
|
|
10
|
+
Generate comprehensive, structured test case specifications for a web application by exploring it live in a browser.
|
|
11
|
+
|
|
12
|
+
## Input
|
|
13
|
+
|
|
14
|
+
Parse the target URL and user journey description from: $ARGUMENTS
|
|
15
|
+
|
|
16
|
+
## Workflow
|
|
17
|
+
|
|
18
|
+
### Step 1: Understand the Journey and Existing Coverage
|
|
19
|
+
|
|
20
|
+
Parse the journey description to identify:
|
|
21
|
+
- **Base URL** and target feature area
|
|
22
|
+
- **Primary user goal** (what the happy path achieves)
|
|
23
|
+
- **Key interaction points** (forms, buttons, navigation, selections)
|
|
24
|
+
- **Implicit requirements** (validation, authentication, authorization)
|
|
25
|
+
|
|
26
|
+
Check for existing test coverage before exploring:
|
|
27
|
+
- Search for existing test files related to this feature (`Grep` for feature keywords in `**/*.spec.ts`, `**/*.test.ts`)
|
|
28
|
+
- Read any existing `test-cases/*.md` spec files for this feature
|
|
29
|
+
- Note existing TC-IDs to avoid conflicts — continue numbering from the highest existing ID
|
|
30
|
+
- Focus on gaps in existing coverage
|
|
31
|
+
|
|
32
|
+
### Step 2: Explore the Happy Path
|
|
33
|
+
|
|
34
|
+
Use a subagent for browser exploration when that saves context. Pass it:
|
|
35
|
+
- The URL and journey description
|
|
36
|
+
- Instructions to walk through each step using `find`, `get_form`, `get_field`
|
|
37
|
+
- Instructions to catalog all interactive elements, form fields, navigation options
|
|
38
|
+
- Instructions to use `get_element` on key elements to capture the best Playwright selector
|
|
39
|
+
- Instructions to return results in this structured format for each step:
|
|
40
|
+
|
|
41
|
+
```
|
|
42
|
+
Step: <what was done>
|
|
43
|
+
URL: <current URL after action>
|
|
44
|
+
Elements found:
|
|
45
|
+
- Submit button: getByRole('button', { name: /submit/i })
|
|
46
|
+
- Email field: getByLabel(/email/i)
|
|
47
|
+
- Error message: getByText(/required/i)
|
|
48
|
+
Observations: <what appeared, validation messages, state changes>
|
|
49
|
+
```
|
|
50
|
+
|
|
51
|
+
This structured output ensures selectors survive the handoff to the spec file and ultimately to `write-test-code`.
|
|
52
|
+
|
|
53
|
+
### Step 3: Explore Alternative and Failure Paths
|
|
54
|
+
|
|
55
|
+
Launch another subagent, or continue in the main thread if the flow is small, to systematically probe beyond the happy path:
|
|
56
|
+
|
|
57
|
+
**Validation & Error Handling:**
|
|
58
|
+
- Submit forms with empty required fields
|
|
59
|
+
- Enter invalid formats (wrong email, short passwords, letters in number fields)
|
|
60
|
+
- Exceed field length limits, use special characters and Unicode
|
|
61
|
+
|
|
62
|
+
**Boundary Conditions:**
|
|
63
|
+
- Min/max values for numeric fields
|
|
64
|
+
- Single character and max length strings
|
|
65
|
+
- Zero quantities, negative numbers, date boundaries
|
|
66
|
+
|
|
67
|
+
**State & Navigation:**
|
|
68
|
+
- Browser back/forward during multi-step flows
|
|
69
|
+
- Page refresh mid-flow
|
|
70
|
+
- Accessing later steps directly via URL
|
|
71
|
+
|
|
72
|
+
**UI & Interaction:**
|
|
73
|
+
- Rapid repeated clicks on submit buttons
|
|
74
|
+
- Dropdown default values, empty options
|
|
75
|
+
- Loading states, disabled states, conditional visibility
|
|
76
|
+
|
|
77
|
+
**Access & Authorization (observe only):**
|
|
78
|
+
- Redirect behavior for unauthenticated users
|
|
79
|
+
- Permission-related error messages
|
|
80
|
+
|
|
81
|
+
### Step 4: Reason About Additional Scenarios
|
|
82
|
+
|
|
83
|
+
After exploration, reason about scenarios that could not be directly triggered but must be covered:
|
|
84
|
+
|
|
85
|
+
- **Network & Performance** — failure modes, slow responses, large data sets, offline behavior
|
|
86
|
+
- **Accessibility (WCAG 2.1 AA)** — keyboard navigation, screen reader support, focus management, contrast
|
|
87
|
+
- **Visual Consistency** — layout stability, responsive breakpoints, dark mode
|
|
88
|
+
- **Cross-browser** — Safari/Firefox/mobile-specific behavioral differences
|
|
89
|
+
- **Concurrent & Session** — session expiry, multi-tab conflicts, race conditions
|
|
90
|
+
|
|
91
|
+
See [references/scenario-categories.md](references/scenario-categories.md) for detailed checklists within each category.
|
|
92
|
+
|
|
93
|
+
### Step 5: Generate Test Case Specifications
|
|
94
|
+
|
|
95
|
+
Write structured test cases to `test-cases/<feature-name>.md`.
|
|
96
|
+
|
|
97
|
+
## Output Specification
|
|
98
|
+
|
|
99
|
+
### Test Case Format
|
|
100
|
+
|
|
101
|
+
```markdown
|
|
102
|
+
### TC-<FEATURE>-<NUMBER>: <Descriptive title>
|
|
103
|
+
|
|
104
|
+
**Priority:** Critical | High | Medium | Low
|
|
105
|
+
**Category:** Happy Path | Validation | Error Handling | Edge Case | Boundary | Security | Accessibility | Visual | Performance | Network Error | UX
|
|
106
|
+
**Preconditions:**
|
|
107
|
+
- <What must be true before this test>
|
|
108
|
+
|
|
109
|
+
**Steps:**
|
|
110
|
+
1. <Action the tester performs>
|
|
111
|
+
2. <Next action>
|
|
112
|
+
|
|
113
|
+
**Expected Result:**
|
|
114
|
+
- <What should happen>
|
|
115
|
+
|
|
116
|
+
**Selectors observed:**
|
|
117
|
+
- <element>: `getByRole('button', { name: /submit/i })` or `getByLabel(/email/i)` — from `get_element`/`find` during exploration
|
|
118
|
+
- (Include selectors for key interactive elements so `write-test-code` doesn't have to rediscover them)
|
|
119
|
+
|
|
120
|
+
**Notes:**
|
|
121
|
+
- <Additional context discovered during exploration>
|
|
122
|
+
```
|
|
123
|
+
|
|
124
|
+
### Output Organization
|
|
125
|
+
|
|
126
|
+
```markdown
|
|
127
|
+
# Test Cases: <Feature Name>
|
|
128
|
+
|
|
129
|
+
**URL:** <base URL>
|
|
130
|
+
**Generated:** <date>
|
|
131
|
+
**Journey:** <brief description>
|
|
132
|
+
|
|
133
|
+
## Summary
|
|
134
|
+
- Total test cases: <count>
|
|
135
|
+
- Critical: <count> | High: <count> | Medium: <count> | Low: <count>
|
|
136
|
+
|
|
137
|
+
## Happy Path
|
|
138
|
+
## Validation & Error Handling
|
|
139
|
+
## Edge Cases
|
|
140
|
+
## Boundary Conditions
|
|
141
|
+
## Security & Access
|
|
142
|
+
## Network Error Scenarios
|
|
143
|
+
## Visual & Responsive
|
|
144
|
+
## Performance & Loading
|
|
145
|
+
## Accessibility & UX
|
|
146
|
+
```
|
|
147
|
+
|
|
148
|
+
### File Naming Convention
|
|
149
|
+
|
|
150
|
+
When generating specs that span multiple roles or test categories, recommend role-based file naming (`*.admin.spec.ts`, `*.user.spec.ts`) or Playwright tag annotations (`@admin`, `@smoke`) in the spec. This enables selective execution via `--grep @admin` or glob patterns instead of fragile `testIgnore` regex in playwright.config.ts. NEVER recommend a `testIgnore` regex that must be updated for every new test file.
|
|
151
|
+
|
|
152
|
+
### TC-ID Convention
|
|
153
|
+
|
|
154
|
+
- Format: `TC-<FEATURE>-<NNN>` where NNN is zero-padded to 3 digits
|
|
155
|
+
- Feature abbreviation: short and clear (LOGIN, CHECKOUT, SEARCH, SIGNUP)
|
|
156
|
+
- Start at 001, sequential, unique within the document
|
|
157
|
+
- Happy path first, then validation, then edge cases
|
|
158
|
+
|
|
159
|
+
## Quality Standards
|
|
160
|
+
|
|
161
|
+
- Every test case must be **independently executable** — no hidden dependencies
|
|
162
|
+
- Steps must be **concrete and unambiguous** — "click the Submit button" not "submit the form"
|
|
163
|
+
- Expected results must be **observable and verifiable** — include actual error messages observed
|
|
164
|
+
- Priority must be **justified** — Critical = blocks core journey, High = significant, Medium = secondary, Low = cosmetic
|
|
165
|
+
- Every feature spec MUST include at minimum: one network error scenario (500/timeout), one empty state scenario, and one session/auth edge case (if the feature requires auth). These are non-negotiable — omitting them is a BLOCKER in the review-test-cases quality gate.
|
|
166
|
+
- **Test case count guidance:** Aim for 15-30 test cases per feature area as a baseline. Fewer than 10 suggests missing error paths or edge cases. More than 40 suggests the feature should be split into sub-features with separate spec files. Prioritize breadth of category coverage over depth within a single category.
|
|
167
|
+
|
|
168
|
+
## Blocking Conditions
|
|
169
|
+
|
|
170
|
+
Report and work around:
|
|
171
|
+
- **Login/auth walls**: Document as precondition, test observable behavior
|
|
172
|
+
- **CAPTCHA**: Report, skip, note in preconditions
|
|
173
|
+
- **Payment gateways**: Don't enter real data, document flow up to that point
|
|
174
|
+
- **Rate limiting**: Slow down, note rate limit behavior as a test case
|
|
175
|
+
|
|
176
|
+
## Example Usage
|
|
177
|
+
|
|
178
|
+
```
|
|
179
|
+
Claude Code: /generate-test-cases https://example.com/login User logs in with email and password, sees dashboard
|
|
180
|
+
Codex: $generate-test-cases https://example.com/login User logs in with email and password, sees dashboard
|
|
181
|
+
|
|
182
|
+
Claude Code: /generate-test-cases https://shop.example.com User searches for product, adds to cart, proceeds to checkout
|
|
183
|
+
Codex: $generate-test-cases https://shop.example.com User searches for product, adds to cart, proceeds to checkout
|
|
184
|
+
```
|
|
@@ -0,0 +1,10 @@
|
|
|
1
|
+
interface:
|
|
2
|
+
display_name: "Write TC-ID Specs"
|
|
3
|
+
short_description: "Explore the feature and write detailed TC-ID test case specs"
|
|
4
|
+
default_prompt: "Explore this feature and generate detailed TC-ID test case specifications."
|
|
5
|
+
|
|
6
|
+
dependencies:
|
|
7
|
+
tools:
|
|
8
|
+
- type: "mcp"
|
|
9
|
+
value: "agent-web-interface"
|
|
10
|
+
description: "Browser automation MCP used to inspect the live flow before writing specs"
|
|
@@ -0,0 +1,36 @@
|
|
|
1
|
+
# Scenario Categories — Detailed Checklists
|
|
2
|
+
|
|
3
|
+
These checklists support Step 4 of the generate-test-cases skill. Each category covers scenarios that may not be directly triggerable during browser exploration but must be included in comprehensive test specifications.
|
|
4
|
+
|
|
5
|
+
## Network & Performance
|
|
6
|
+
|
|
7
|
+
- Network failure during form submission (mock 500, timeout)
|
|
8
|
+
- Slow API response (loading states, skeleton screens, spinners)
|
|
9
|
+
- Large data sets (pagination, infinite scroll, 100+ items)
|
|
10
|
+
- Offline behavior (if PWA or service worker is present)
|
|
11
|
+
|
|
12
|
+
## Accessibility (WCAG 2.1 AA)
|
|
13
|
+
|
|
14
|
+
- Keyboard-only navigation through the entire flow (Tab, Enter, Escape)
|
|
15
|
+
- Screen reader announcements for dynamic content (ARIA live regions)
|
|
16
|
+
- Focus management after modal open/close, page transitions
|
|
17
|
+
- Color contrast for error states and disabled elements
|
|
18
|
+
- Form error association (`aria-describedby` linking errors to fields)
|
|
19
|
+
|
|
20
|
+
## Visual Consistency
|
|
21
|
+
|
|
22
|
+
- Layout stability (no unexpected content shifts after load)
|
|
23
|
+
- Responsive behavior at standard breakpoints (mobile 375px, tablet 768px, desktop 1280px)
|
|
24
|
+
- Dark mode rendering if supported
|
|
25
|
+
|
|
26
|
+
## Cross-browser Considerations
|
|
27
|
+
|
|
28
|
+
- Safari-specific behavior (date inputs, smooth scrolling, storage quirks)
|
|
29
|
+
- Firefox form validation differences
|
|
30
|
+
- Mobile browser touch targets and gestures
|
|
31
|
+
|
|
32
|
+
## Concurrent & Session
|
|
33
|
+
|
|
34
|
+
- Session expiry mid-flow (cookie cleared during multi-step)
|
|
35
|
+
- Concurrent access (two tabs, same user)
|
|
36
|
+
- Race conditions (double-click submit, rapid navigation)
|