@specwright/plugin 0.1.1 → 0.1.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.claude_agent-memory/execution-manager/MEMORY.md +12 -0
- package/.claude_agent-memory/playwright-test-healer/MEMORY.md +15 -0
- package/.claude_agent-memory/playwright-test-planner/MEMORY.md +21 -0
- package/.claude_agents/bdd-generator.md +534 -0
- package/.claude_agents/code-generator.md +464 -0
- package/.claude_agents/execution-manager.md +441 -0
- package/.claude_agents/input-processor.md +367 -0
- package/.claude_agents/jira-processor.md +403 -0
- package/.claude_agents/playwright/playwright-test-generator.md +69 -0
- package/.claude_agents/playwright/playwright-test-healer.md +66 -0
- package/.claude_agents/playwright/playwright-test-planner.md +236 -0
- package/.claude_rules/architecture.md +102 -0
- package/.claude_rules/conventions.md +82 -0
- package/.claude_rules/debugging.md +90 -0
- package/.claude_rules/dependencies.md +54 -0
- package/.claude_rules/onboarding.md +122 -0
- package/.claude_skills/e2e-automate/SKILL.md +179 -0
- package/.claude_skills/e2e-generate/SKILL.md +69 -0
- package/.claude_skills/e2e-heal/SKILL.md +99 -0
- package/.claude_skills/e2e-plan/SKILL.md +38 -0
- package/.claude_skills/e2e-process/SKILL.md +63 -0
- package/.claude_skills/e2e-run/SKILL.md +42 -0
- package/.claude_skills/e2e-validate/SKILL.md +50 -0
- package/package.json +8 -2
|
@@ -0,0 +1,122 @@
|
|
|
1
|
+
# Onboarding
|
|
2
|
+
|
|
3
|
+
Context for developers new to this project.
|
|
4
|
+
|
|
5
|
+
---
|
|
6
|
+
|
|
7
|
+
### Running E2E Tests for the First Time
|
|
8
|
+
|
|
9
|
+
**Context:** New developers often run `playwright test` directly and get failures.
|
|
10
|
+
**Decision/Finding:** The correct sequence is always:
|
|
11
|
+
|
|
12
|
+
```bash
|
|
13
|
+
# 1. Ensure you have valid test credentials in .env
|
|
14
|
+
# TEST_USER_EMAIL=your-email@example.com
|
|
15
|
+
# TEST_USER_PASSWORD=your-password
|
|
16
|
+
|
|
17
|
+
# 2. Run the full BDD suite (bddgen runs automatically)
|
|
18
|
+
pnpm test:bdd
|
|
19
|
+
|
|
20
|
+
# 3. View results
|
|
21
|
+
pnpm report:playwright
|
|
22
|
+
```
|
|
23
|
+
|
|
24
|
+
Never run `npx playwright test` directly — it skips `bddgen` and uses stale `.features-gen/` specs.
|
|
25
|
+
|
|
26
|
+
---
|
|
27
|
+
|
|
28
|
+
### Auth Credentials
|
|
29
|
+
|
|
30
|
+
**Context:** E2E tests require valid login credentials.
|
|
31
|
+
**Decision/Finding:** Set `TEST_USER_EMAIL` and `TEST_USER_PASSWORD` in your `.env` file at the project root. The `auth.setup.js` file reads from `authenticationData.js` which pulls from env vars. See `e2e-tests/.env.testing` for a template of all available env vars.
|
|
32
|
+
|
|
33
|
+
Never hardcode credentials in `authenticationData.js` or any committed file.
|
|
34
|
+
|
|
35
|
+
---
|
|
36
|
+
|
|
37
|
+
### How `bddgen` Works
|
|
38
|
+
|
|
39
|
+
**Context:** Developers are confused why test files in `.features-gen/` exist and whether to edit them.
|
|
40
|
+
**Decision/Finding:** `bddgen` (part of `playwright-bdd`) compiles Gherkin `.feature` files into Playwright `*.spec.js` files inside `.features-gen/`. This directory is gitignored and auto-regenerated. Never edit files in `.features-gen/` — changes are overwritten. Edit the source `.feature` and `steps.js` files instead.
|
|
41
|
+
|
|
42
|
+
---
|
|
43
|
+
|
|
44
|
+
### Adding a New E2E Module
|
|
45
|
+
|
|
46
|
+
**Context:** Step-by-step for creating tests for a new app feature.
|
|
47
|
+
**Decision/Finding:**
|
|
48
|
+
|
|
49
|
+
1. Create directory: `e2e-tests/features/playwright-bdd/@Modules/@YourModule/`
|
|
50
|
+
2. Create feature file: `@YourModule/your-feature.feature`
|
|
51
|
+
3. Create co-located steps: `@YourModule/steps.js`
|
|
52
|
+
4. Import fixtures: `import { Given, When, Then } from '../../../../playwright/fixtures.js'`
|
|
53
|
+
5. Tag the feature: `@your-module @your-feature` (lowercase module tag)
|
|
54
|
+
6. If steps are reused by other modules, move them to `shared/` instead
|
|
55
|
+
7. Run `pnpm test:bdd` — bddgen picks up new files automatically
|
|
56
|
+
|
|
57
|
+
---
|
|
58
|
+
|
|
59
|
+
### Adding a Cross-Module Workflow Test
|
|
60
|
+
|
|
61
|
+
**Context:** When features span multiple modules with shared data.
|
|
62
|
+
**Decision/Finding:** Workflow tests go in `e2e-tests/features/playwright-bdd/@Workflows/@YourWorkflow/`. Use numbered directories:
|
|
63
|
+
|
|
64
|
+
- `@0-Precondition/` — serial, creates shared data (`@precondition @cross-feature-data`)
|
|
65
|
+
- `@1-Consumer/` — loads predata via `Given I load predata from "workflow-name"` (`@workflow-consumer`)
|
|
66
|
+
|
|
67
|
+
Steps that touch multiple modules must be in `shared/` — never co-locate them inside an `@`-prefixed directory.
|
|
68
|
+
|
|
69
|
+
---
|
|
70
|
+
|
|
71
|
+
### Module-Specific Test Scripts
|
|
72
|
+
|
|
73
|
+
**Context:** Running tests for a single module without the full suite.
|
|
74
|
+
**Decision/Finding:**
|
|
75
|
+
|
|
76
|
+
```bash
|
|
77
|
+
pnpm test:bdd:counter # Counter module only (serial)
|
|
78
|
+
pnpm test:bdd:users # Users module only (parallel)
|
|
79
|
+
pnpm test:bdd:bookings # Bookings workflow only (serial)
|
|
80
|
+
pnpm test:bdd:homepage # HomePage module only (parallel)
|
|
81
|
+
pnpm test:bdd:auth # Authentication tests only
|
|
82
|
+
pnpm test:bdd:workflows # All @Workflows features
|
|
83
|
+
```
|
|
84
|
+
|
|
85
|
+
---
|
|
86
|
+
|
|
87
|
+
### Agent Pipeline — Quick Start
|
|
88
|
+
|
|
89
|
+
**Context:** Using the multi-agent test generation pipeline.
|
|
90
|
+
**Decision/Finding:**
|
|
91
|
+
|
|
92
|
+
1. Configure `instructions.js` in `e2e-tests/` (see `instructions.example.js`)
|
|
93
|
+
2. Ensure dev server is running: `pnpm dev`
|
|
94
|
+
3. Run `/e2e-automate` skill via Claude Code CLI
|
|
95
|
+
4. The skill handles 10 phases: config → process → explore → approve → generate → execute → review
|
|
96
|
+
5. Generated files appear in `e2e-tests/features/playwright-bdd/@Modules/` or `@Workflows/`
|
|
97
|
+
|
|
98
|
+
Available skills: `/e2e-automate`, `/e2e-plan`, `/e2e-generate`, `/e2e-heal`, `/e2e-validate`, `/e2e-process`, `/e2e-run`
|
|
99
|
+
|
|
100
|
+
---
|
|
101
|
+
|
|
102
|
+
### E2E Directory Map (Quick Reference)
|
|
103
|
+
|
|
104
|
+
```
|
|
105
|
+
e2e-tests/
|
|
106
|
+
├── features/playwright-bdd/ ← Gherkin feature files + co-located steps
|
|
107
|
+
│ ├── @Modules/ ← single-module tests
|
|
108
|
+
│ ├── @Workflows/ ← cross-module workflow tests
|
|
109
|
+
│ └── shared/ ← globally scoped step definitions
|
|
110
|
+
├── playwright/
|
|
111
|
+
│ ├── fixtures.js ← custom fixtures (import from here, not playwright-bdd)
|
|
112
|
+
│ ├── auth.setup.js ← two-step localStorage login
|
|
113
|
+
│ ├── auth-storage/ ← .auth/user.json (gitignored)
|
|
114
|
+
│ ├── generated/ ← seed.spec.js (exploration output)
|
|
115
|
+
│ └── test-data/ ← scoped JSON files (auto-created at runtime)
|
|
116
|
+
├── data/
|
|
117
|
+
│ ├── authenticationData.js ← credentials from env vars
|
|
118
|
+
│ ├── testConfig.js ← routes, timeouts, baseUrl
|
|
119
|
+
│ └── migrations/ ← instructions.js for agent pipeline
|
|
120
|
+
├── plans/ ← generated test plans (intermediate, cleaned up)
|
|
121
|
+
└── reports/ ← execution reports, review plans
|
|
122
|
+
```
|
|
@@ -0,0 +1,179 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: e2e-automate
|
|
3
|
+
description: Run the full 10-phase test automation pipeline — chains /e2e-process, /e2e-plan, /e2e-validate, /e2e-generate, /e2e-heal skills sequentially.
|
|
4
|
+
context: fork
|
|
5
|
+
---
|
|
6
|
+
|
|
7
|
+
# Test Automation Pipeline
|
|
8
|
+
|
|
9
|
+
Execute the complete test automation workflow by chaining skills sequentially.
|
|
10
|
+
|
|
11
|
+
## Pipeline Overview
|
|
12
|
+
|
|
13
|
+
```
|
|
14
|
+
Phase 1-2: Read config → Detect route (jira/file/text)
|
|
15
|
+
Phase 3: /e2e-process (convert input to parsed test plan MD)
|
|
16
|
+
Phase 4: /e2e-plan (explore app + generate test plan)
|
|
17
|
+
Phase 5: /e2e-validate (validate seed file — optional)
|
|
18
|
+
Phase 6: ⛔ USER APPROVAL CHECKPOINT
|
|
19
|
+
Phase 7: /e2e-generate (create .feature + steps.js)
|
|
20
|
+
Phase 8: /e2e-heal (run tests + auto-heal — optional)
|
|
21
|
+
Phase 9: Cleanup
|
|
22
|
+
Phase 10: Final review (quality score + formatted summary)
|
|
23
|
+
```
|
|
24
|
+
|
|
25
|
+
## Execution Steps
|
|
26
|
+
|
|
27
|
+
### Phase 1: Initialization
|
|
28
|
+
|
|
29
|
+
Read `/e2e-tests/instructions.js` and validate configuration.
|
|
30
|
+
|
|
31
|
+
### Phase 2: Detection & Routing
|
|
32
|
+
|
|
33
|
+
For each config entry, detect the input type:
|
|
34
|
+
|
|
35
|
+
- `inputs.jira.url` exists → Jira mode
|
|
36
|
+
- `filePath` exists → File mode
|
|
37
|
+
- `instructions[]` exists → Text mode
|
|
38
|
+
- Otherwise → log error and skip
|
|
39
|
+
|
|
40
|
+
### Phase 3: Input Processing
|
|
41
|
+
|
|
42
|
+
Invoke `/e2e-process` with the appropriate input for each config entry:
|
|
43
|
+
|
|
44
|
+
- Jira mode: pass the Jira URL
|
|
45
|
+
- File mode: pass the file path
|
|
46
|
+
- Text mode: pass as text instructions
|
|
47
|
+
|
|
48
|
+
**Output:** Parsed MD file at `/e2e-tests/plans/{moduleName}-parsed.md`
|
|
49
|
+
|
|
50
|
+
### Phase 4: Exploration & Planning
|
|
51
|
+
|
|
52
|
+
Invoke `/e2e-plan` with the `pageURL` from each config entry.
|
|
53
|
+
|
|
54
|
+
- If `explore: true`: the skill explores the app via MCP, discovers selectors, writes seed file
|
|
55
|
+
- Saves test plan to `/e2e-tests/plans/{moduleName}-{fileName}-plan.md`
|
|
56
|
+
- **After exploration**: update `.claude/agent-memory/playwright-test-planner/MEMORY.md` with ALL discovered selectors, navigation paths, and patterns. Read the file first, then Edit to add new entries.
|
|
57
|
+
|
|
58
|
+
### Phase 5: Exploration Validation (Optional)
|
|
59
|
+
|
|
60
|
+
**Skip if `runExploredCases` is false.**
|
|
61
|
+
|
|
62
|
+
Invoke `/e2e-validate` to run the seed file tests and auto-heal failures.
|
|
63
|
+
|
|
64
|
+
### Phase 6: User Approval (MANDATORY)
|
|
65
|
+
|
|
66
|
+
⛔ **BLOCKING** — Do not proceed without explicit user approval.
|
|
67
|
+
|
|
68
|
+
Present the test plan summary including:
|
|
69
|
+
|
|
70
|
+
- Scenario count and names for each module
|
|
71
|
+
- **File paths to the full plan files** so the user can review detailed plans with selectors, steps, and data tables
|
|
72
|
+
|
|
73
|
+
Then ask:
|
|
74
|
+
|
|
75
|
+
1. Approve & Generate
|
|
76
|
+
2. View Full Plan
|
|
77
|
+
3. Modify & Retry
|
|
78
|
+
|
|
79
|
+
### Phase 7: BDD Generation
|
|
80
|
+
|
|
81
|
+
Invoke `/e2e-generate` with the approved plan file path for each config entry.
|
|
82
|
+
Creates complete `.feature` + `steps.js` files.
|
|
83
|
+
|
|
84
|
+
### Phase 8: Test Execution & Healing (Optional)
|
|
85
|
+
|
|
86
|
+
**Skip if `runGeneratedCases` is false.**
|
|
87
|
+
|
|
88
|
+
Invoke `/e2e-heal` to run the generated BDD tests and auto-heal failures (max 3 iterations).
|
|
89
|
+
|
|
90
|
+
### Phase 9: Cleanup & Aggregation
|
|
91
|
+
|
|
92
|
+
Aggregate all results: configs processed, files generated, tests passed/failed, healing stats.
|
|
93
|
+
|
|
94
|
+
**Clean up intermediate files:**
|
|
95
|
+
|
|
96
|
+
```bash
|
|
97
|
+
rm -f e2e-tests/plans/*.md
|
|
98
|
+
```
|
|
99
|
+
|
|
100
|
+
### Phase 10: Final Review
|
|
101
|
+
|
|
102
|
+
Calculate a quality score from the aggregated pipeline data and display a formatted summary.
|
|
103
|
+
|
|
104
|
+
**Quality Score Formula:**
|
|
105
|
+
|
|
106
|
+
```
|
|
107
|
+
qualityScore = (
|
|
108
|
+
inputProcessingScore * 0.25 + // (successful configs / total configs) × 100
|
|
109
|
+
selectorDiscoveryScore * 0.20 + // (validated selectors / total discovered) × 100, or 0 if no exploration
|
|
110
|
+
testExecutionScore * 0.30 + // (passed tests / total tests) × 100, or 70 if execution skipped
|
|
111
|
+
healingSuccessScore * 0.25 // (auto-fixed / total failures) × 100, or 50 if healing skipped
|
|
112
|
+
)
|
|
113
|
+
```
|
|
114
|
+
|
|
115
|
+
**Rating:** 95-100 Excellent ⭐⭐⭐⭐⭐ | 85-94 Very Good ⭐⭐⭐⭐ | 75-84 Good ⭐⭐⭐ | 60-74 Fair ⭐⭐ | 0-59 Poor ⭐
|
|
116
|
+
|
|
117
|
+
**Status:** score>=95 → "READY FOR PRODUCTION" | >=85 → "READY WITH MINOR FIXES" | >=75 → "REQUIRES ATTENTION" | >=60 → "NEEDS IMPROVEMENT" | else → "SIGNIFICANT ISSUES"
|
|
118
|
+
|
|
119
|
+
**Display format:**
|
|
120
|
+
|
|
121
|
+
```
|
|
122
|
+
╔══════════════════════════════════════════════════════════════╗
|
|
123
|
+
║ E2E AUTOMATION FINAL REVIEW ║
|
|
124
|
+
║ [{current date/time}] ║
|
|
125
|
+
╚══════════════════════════════════════════════════════════════╝
|
|
126
|
+
|
|
127
|
+
📊 QUALITY SCORE: {score}/100 {stars}
|
|
128
|
+
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
|
|
129
|
+
{rating}
|
|
130
|
+
|
|
131
|
+
🎯 GENERATION SUMMARY
|
|
132
|
+
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
|
|
133
|
+
Configs Processed: {successful}/{total}
|
|
134
|
+
Feature Files: {count} created
|
|
135
|
+
Step Definitions: {count} files ({totalSteps} steps)
|
|
136
|
+
Scenarios: {count} total
|
|
137
|
+
|
|
138
|
+
📁 GENERATED FILES
|
|
139
|
+
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
|
|
140
|
+
{list each .feature and steps.js with full path}
|
|
141
|
+
|
|
142
|
+
🧪 TEST EXECUTION
|
|
143
|
+
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
|
|
144
|
+
{if tests ran: Passed: X/Y (Z%), Duration: Ns}
|
|
145
|
+
{if skipped: "Skipped (runGeneratedCases: false)"}
|
|
146
|
+
|
|
147
|
+
🔧 HEALING
|
|
148
|
+
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
|
|
149
|
+
{if healing ran: Attempts: N, Auto-fixed: N, Success Rate: N%}
|
|
150
|
+
{if skipped: "Skipped"}
|
|
151
|
+
|
|
152
|
+
⏭️ SKIPPED PHASES
|
|
153
|
+
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
|
|
154
|
+
{list each skipped phase and reason}
|
|
155
|
+
|
|
156
|
+
📋 NEXT STEPS
|
|
157
|
+
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
|
|
158
|
+
1. Run tests: pnpm test:bdd
|
|
159
|
+
2. Fix failures: /e2e-heal
|
|
160
|
+
3. View report: pnpm report:playwright
|
|
161
|
+
|
|
162
|
+
╔══════════════════════════════════════════════════════════════╗
|
|
163
|
+
║ STATUS: {overallStatus} ║
|
|
164
|
+
╚══════════════════════════════════════════════════════════════╝
|
|
165
|
+
```
|
|
166
|
+
|
|
167
|
+
## Progress Display
|
|
168
|
+
|
|
169
|
+
Show a todo list at start, update as each phase completes:
|
|
170
|
+
|
|
171
|
+
- □ Not started → 🔄 In progress → ✅ Complete / ⏭️ Skipped / ❌ Failed
|
|
172
|
+
|
|
173
|
+
## Error Handling
|
|
174
|
+
|
|
175
|
+
If any config fails: log error, continue with next config, mark as failed in summary.
|
|
176
|
+
|
|
177
|
+
## CRITICAL: Sequential Execution
|
|
178
|
+
|
|
179
|
+
Execute phases ONE BY ONE. Never run phases in parallel. Always wait for the current phase before starting the next.
|
|
@@ -0,0 +1,69 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: e2e-generate
|
|
3
|
+
description: Generate Playwright BDD tests from a test plan — chains bdd-generator (feature files + step skeletons) → code-generator (Playwright implementations).
|
|
4
|
+
argument-hint: <plan-file-or-feature>
|
|
5
|
+
context: fork
|
|
6
|
+
---
|
|
7
|
+
|
|
8
|
+
# Test Generation
|
|
9
|
+
|
|
10
|
+
Generate complete BDD test files from a test plan.
|
|
11
|
+
|
|
12
|
+
## Agent Chain
|
|
13
|
+
|
|
14
|
+
```
|
|
15
|
+
@agent-bdd-generator
|
|
16
|
+
→ Creates .feature file (Gherkin scenarios, data tables, tags)
|
|
17
|
+
→ Creates steps.js skeleton (imports, step patterns, processDataTable wiring)
|
|
18
|
+
→ Output: READY_FOR_CODE_GENERATION
|
|
19
|
+
|
|
20
|
+
@agent-code-generator
|
|
21
|
+
→ Reads seed file for validated selectors
|
|
22
|
+
→ Fills in Playwright implementation code
|
|
23
|
+
→ Applies code quality rules (no RegExp constructors, semantic locators, auto-retrying assertions)
|
|
24
|
+
→ Output: Complete runnable steps.js
|
|
25
|
+
```
|
|
26
|
+
|
|
27
|
+
## Steps
|
|
28
|
+
|
|
29
|
+
### Step 1: Generate BDD Files
|
|
30
|
+
|
|
31
|
+
Invoke `@agent-bdd-generator` with:
|
|
32
|
+
|
|
33
|
+
- Plan file path from $ARGUMENTS (or most recent plan in `/e2e-tests/plans/`)
|
|
34
|
+
- Module config (moduleName, category, subModuleName, fileName)
|
|
35
|
+
|
|
36
|
+
The agent creates both `.feature` and `steps.js` skeleton with correct imports, step patterns, and data table wiring.
|
|
37
|
+
|
|
38
|
+
### Step 2: Generate Playwright Code
|
|
39
|
+
|
|
40
|
+
Invoke `@agent-code-generator` with:
|
|
41
|
+
|
|
42
|
+
- The steps.js skeleton from Step 1
|
|
43
|
+
- Seed file at `e2e-tests/playwright/generated/seed.spec.js` (if exists)
|
|
44
|
+
- Plan file for context
|
|
45
|
+
|
|
46
|
+
The agent reads validated selectors from the seed file and fills in complete Playwright implementations.
|
|
47
|
+
|
|
48
|
+
### Step 3: Verify (Optional)
|
|
49
|
+
|
|
50
|
+
If the user requests verification, invoke `@agent-playwright-test-generator` to:
|
|
51
|
+
|
|
52
|
+
- Execute the generated steps in a real browser via MCP
|
|
53
|
+
- Confirm selectors work against the live app
|
|
54
|
+
|
|
55
|
+
## Input: $ARGUMENTS
|
|
56
|
+
|
|
57
|
+
Pass the plan file path or module name:
|
|
58
|
+
|
|
59
|
+
```
|
|
60
|
+
/e2e-generate /e2e-tests/plans/feature-plan.md
|
|
61
|
+
/e2e-generate @ModuleName
|
|
62
|
+
```
|
|
63
|
+
|
|
64
|
+
## Output
|
|
65
|
+
|
|
66
|
+
Generated files in `/e2e-tests/features/playwright-bdd/{category}/@{Module}/`:
|
|
67
|
+
|
|
68
|
+
- `{feature}.feature` — Gherkin scenarios with tags and 3-column data tables
|
|
69
|
+
- `steps.js` — Complete step definitions with Playwright code
|
|
@@ -0,0 +1,99 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: e2e-heal
|
|
3
|
+
description: Run tests, diagnose failures, investigate source code, apply direct fixes, and auto-heal remaining issues with up to 3 healing iterations. Generates review plan for unresolved failures.
|
|
4
|
+
argument-hint: <test-file-or-tag>
|
|
5
|
+
context: fork
|
|
6
|
+
---
|
|
7
|
+
|
|
8
|
+
# Test Healing
|
|
9
|
+
|
|
10
|
+
Run tests, triage failures, investigate source code, apply direct fixes, and auto-heal remaining issues.
|
|
11
|
+
|
|
12
|
+
## Agent Chain
|
|
13
|
+
|
|
14
|
+
```
|
|
15
|
+
@agent-execution-manager (run + triage + source-fix)
|
|
16
|
+
↕ loop (max 3 iterations)
|
|
17
|
+
@agent-playwright-test-healer (fix timing/visibility failures via MCP)
|
|
18
|
+
```
|
|
19
|
+
|
|
20
|
+
## Steps
|
|
21
|
+
|
|
22
|
+
### Step 1: Run Tests & Triage
|
|
23
|
+
|
|
24
|
+
Invoke `@agent-execution-manager` in **bdd mode** with any filters from $ARGUMENTS.
|
|
25
|
+
|
|
26
|
+
The agent:
|
|
27
|
+
|
|
28
|
+
- Runs `npx bddgen && npx playwright test`
|
|
29
|
+
- Categorizes failures (selector, timeout, assertion, network, data, server)
|
|
30
|
+
- Investigates source code for fixable failures
|
|
31
|
+
- **Directly edits step files** to fix clear selector mismatches
|
|
32
|
+
- Returns execution report with `recommendation`
|
|
33
|
+
|
|
34
|
+
### Step 2: Check Results
|
|
35
|
+
|
|
36
|
+
- If `recommendation` is `all_passed` → done, go to Step 6 (memory)
|
|
37
|
+
- If `recommendation` is `fixes_applied` → re-run to verify (go to Step 1)
|
|
38
|
+
- If `recommendation` is `needs_healing` → proceed to Step 3
|
|
39
|
+
- If `recommendation` is `needs_review` → generate review plan, go to Step 6
|
|
40
|
+
|
|
41
|
+
### Step 3: Heal Remaining Failures
|
|
42
|
+
|
|
43
|
+
Invoke `@agent-playwright-test-healer` with the `healableFailures` from Step 1.
|
|
44
|
+
|
|
45
|
+
The healer:
|
|
46
|
+
|
|
47
|
+
- Debugs each failure interactively using MCP browser tools
|
|
48
|
+
- Captures page snapshots to understand context
|
|
49
|
+
- Updates selectors, assertions, and timing
|
|
50
|
+
- Verifies fixes by re-running the test
|
|
51
|
+
|
|
52
|
+
### Step 4: Re-run (Loop)
|
|
53
|
+
|
|
54
|
+
Invoke `@agent-execution-manager` again to verify healer's fixes.
|
|
55
|
+
|
|
56
|
+
- If all pass → go to Step 6 (memory)
|
|
57
|
+
- If still failing and iterations < 3 → go to Step 3
|
|
58
|
+
- If iterations exhausted → generate review plan, go to Step 6
|
|
59
|
+
|
|
60
|
+
### Step 5: Generate Review Plan (if failures remain)
|
|
61
|
+
|
|
62
|
+
After exhausting iterations, generate:
|
|
63
|
+
`/e2e-tests/reports/review-plan-{module}-{timestamp}.md`
|
|
64
|
+
|
|
65
|
+
### Step 6: Update Agent Memory (MANDATORY — always execute)
|
|
66
|
+
|
|
67
|
+
After ALL healing work is complete (whether tests pass or review plan generated), update memory files:
|
|
68
|
+
|
|
69
|
+
**Update `.claude/agent-memory/playwright-test-healer/MEMORY.md`** with:
|
|
70
|
+
|
|
71
|
+
- **Project Conventions**: Stable patterns discovered (e.g., "Elemental Card doesn't forward data-testid")
|
|
72
|
+
- **Selector Fixes**: Each fix applied (date, module, old selector, new selector, reason) — max 20 entries
|
|
73
|
+
- **Anti-Patterns**: Approaches that consistently fail and their alternatives
|
|
74
|
+
|
|
75
|
+
**Update `.claude/agent-memory/execution-manager/MEMORY.md`** with:
|
|
76
|
+
|
|
77
|
+
- **Module → Source Path Mappings**: Any new E2E module → src/ path discovered
|
|
78
|
+
- **Selector Patterns**: ARIA quirks, force-click requirements found during source investigation
|
|
79
|
+
- **Known Risks**: Timing issues, environment gotchas discovered
|
|
80
|
+
|
|
81
|
+
Read each memory file first, then use the Edit tool to add new entries in-place. Do not overwrite existing entries — append new rows to tables.
|
|
82
|
+
|
|
83
|
+
**This step is NOT optional.** Execute it even if all tests passed on the first run.
|
|
84
|
+
|
|
85
|
+
## Input: $ARGUMENTS
|
|
86
|
+
|
|
87
|
+
```
|
|
88
|
+
/e2e-heal # Heal all failing tests
|
|
89
|
+
/e2e-heal @ModuleName # Heal tests matching tag
|
|
90
|
+
/e2e-heal path/to/feature # Heal specific feature
|
|
91
|
+
```
|
|
92
|
+
|
|
93
|
+
## Output
|
|
94
|
+
|
|
95
|
+
- Fixed test files with updated selectors/assertions
|
|
96
|
+
- Execution report with pass/fail metrics
|
|
97
|
+
- Review plan file (if failures remain after 3 iterations)
|
|
98
|
+
- Updated agent memory files with discovered patterns
|
|
99
|
+
- Tests marked as `test.fixme()` if unfixable
|
|
@@ -0,0 +1,38 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: e2e-plan
|
|
3
|
+
description: Create a test plan for a web page or feature using browser exploration, selector discovery, and scenario design.
|
|
4
|
+
argument-hint: <page-url-or-feature>
|
|
5
|
+
context: fork
|
|
6
|
+
agent: playwright-test-planner
|
|
7
|
+
---
|
|
8
|
+
|
|
9
|
+
# Test Planning Skill
|
|
10
|
+
|
|
11
|
+
Create a comprehensive test plan by exploring the application in a real browser.
|
|
12
|
+
|
|
13
|
+
## What This Does
|
|
14
|
+
|
|
15
|
+
1. Opens browser and navigates to the target page
|
|
16
|
+
2. Explores UI elements via MCP recording workflow
|
|
17
|
+
3. Discovers and validates selectors (priority: testId > role > text > label)
|
|
18
|
+
4. Identifies testable scenarios (happy path, edge cases, negative)
|
|
19
|
+
5. Generates a structured test plan + seed file with validated selectors
|
|
20
|
+
6. **Updates agent memory** with ALL discovered selectors and patterns
|
|
21
|
+
|
|
22
|
+
## Usage
|
|
23
|
+
|
|
24
|
+
```
|
|
25
|
+
/e2e-plan /home # Plan tests for a page
|
|
26
|
+
/e2e-plan http://localhost:5173/users # Plan tests for a URL
|
|
27
|
+
/e2e-plan "Login flow" # Plan tests for a feature
|
|
28
|
+
```
|
|
29
|
+
|
|
30
|
+
## Input: $ARGUMENTS
|
|
31
|
+
|
|
32
|
+
Pass the page URL or feature description to plan tests for.
|
|
33
|
+
|
|
34
|
+
## Output (3 required)
|
|
35
|
+
|
|
36
|
+
1. **Seed file**: `e2e-tests/playwright/generated/seed.spec.js` (validated selectors)
|
|
37
|
+
2. **Test plan**: `e2e-tests/plans/{feature}-plan.md` (scenarios, steps, data dependencies)
|
|
38
|
+
3. **Memory update**: `.claude/agent-memory/playwright-test-planner/MEMORY.md` — ALL discovered selectors, navigation paths, reusable patterns, and known limitations written to the memory file
|
|
@@ -0,0 +1,63 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: e2e-process
|
|
3
|
+
description: Process Jira tickets or files (Excel, CSV, PDF) into parsed test plan markdown.
|
|
4
|
+
argument-hint: <jira-url-or-file-path>
|
|
5
|
+
context: fork
|
|
6
|
+
---
|
|
7
|
+
|
|
8
|
+
# Input Processing
|
|
9
|
+
|
|
10
|
+
Process Jira tickets, files, or text instructions into a parsed test plan markdown file.
|
|
11
|
+
|
|
12
|
+
## Agent Chain
|
|
13
|
+
|
|
14
|
+
```
|
|
15
|
+
(detect input type)
|
|
16
|
+
├─ Jira URL → @agent-jira-processor → @agent-input-processor
|
|
17
|
+
├─ File path → @agent-input-processor
|
|
18
|
+
└─ Text instructions → @agent-input-processor
|
|
19
|
+
```
|
|
20
|
+
|
|
21
|
+
## Steps
|
|
22
|
+
|
|
23
|
+
### Step 1: Detect Input Type
|
|
24
|
+
|
|
25
|
+
From $ARGUMENTS, determine the input type:
|
|
26
|
+
|
|
27
|
+
- Starts with `http` and contains `atlassian.net` or `jira` → **Jira mode**
|
|
28
|
+
- Has a file extension (.xlsx, .csv, .json, .pdf, .docx) → **File mode**
|
|
29
|
+
- Otherwise → **Text mode** (treat as instructions)
|
|
30
|
+
|
|
31
|
+
### Step 2a: Jira Mode
|
|
32
|
+
|
|
33
|
+
Invoke `@agent-jira-processor` with the Jira URL.
|
|
34
|
+
It fetches requirements, analyzes gaps, asks clarifying questions.
|
|
35
|
+
Take its refined output and pass to Step 3.
|
|
36
|
+
|
|
37
|
+
### Step 2b: File/Text Mode
|
|
38
|
+
|
|
39
|
+
Skip to Step 3 directly.
|
|
40
|
+
|
|
41
|
+
### Step 3: Generate Parsed MD
|
|
42
|
+
|
|
43
|
+
Invoke `@agent-input-processor` with:
|
|
44
|
+
|
|
45
|
+
- **Jira mode:** refined jiraData from Step 2a
|
|
46
|
+
- **File mode:** filePath from $ARGUMENTS
|
|
47
|
+
- **Text mode:** instructions from $ARGUMENTS
|
|
48
|
+
|
|
49
|
+
### Step 4: Report
|
|
50
|
+
|
|
51
|
+
Output: path to generated test plan MD at `/e2e-tests/plans/{module}-parsed.md`
|
|
52
|
+
|
|
53
|
+
## Input: $ARGUMENTS
|
|
54
|
+
|
|
55
|
+
```
|
|
56
|
+
/e2e-process https://org.atlassian.net/browse/PROJ-123 # Jira ticket
|
|
57
|
+
/e2e-process /path/to/test-cases.xlsx # Excel file
|
|
58
|
+
/e2e-process /path/to/requirements.pdf # PDF file
|
|
59
|
+
```
|
|
60
|
+
|
|
61
|
+
## Output
|
|
62
|
+
|
|
63
|
+
Parsed test plan at `/e2e-tests/plans/{module}-parsed.md` with status `READY_FOR_VALIDATION`.
|
|
@@ -0,0 +1,42 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: e2e-run
|
|
3
|
+
description: Run BDD tests and report results. Use to quickly execute E2E tests without the full automation pipeline.
|
|
4
|
+
argument-hint: <project-or-tag>
|
|
5
|
+
context: fork
|
|
6
|
+
---
|
|
7
|
+
|
|
8
|
+
# Quick Test Runner
|
|
9
|
+
|
|
10
|
+
Run BDD tests quickly with optional filtering.
|
|
11
|
+
|
|
12
|
+
## What This Does
|
|
13
|
+
|
|
14
|
+
1. Runs `npx bddgen` to compile feature files
|
|
15
|
+
2. Runs `npx playwright test` with optional filters from $ARGUMENTS
|
|
16
|
+
3. Reports pass/fail summary
|
|
17
|
+
4. Opens HTML report if failures detected
|
|
18
|
+
|
|
19
|
+
## Usage
|
|
20
|
+
|
|
21
|
+
```
|
|
22
|
+
/e2e-run # Run all BDD tests
|
|
23
|
+
/e2e-run --project auth-tests # Run specific project
|
|
24
|
+
/e2e-run --grep @smoke # Run tests matching tag
|
|
25
|
+
/e2e-run --project setup --project main-e2e # Run multiple projects
|
|
26
|
+
```
|
|
27
|
+
|
|
28
|
+
## Input: $ARGUMENTS
|
|
29
|
+
|
|
30
|
+
Optional Playwright CLI arguments (--project, --grep, --headed, --debug, etc.).
|
|
31
|
+
|
|
32
|
+
## Execution
|
|
33
|
+
|
|
34
|
+
```bash
|
|
35
|
+
npx bddgen && npx playwright test $ARGUMENTS
|
|
36
|
+
```
|
|
37
|
+
|
|
38
|
+
If tests fail, automatically run:
|
|
39
|
+
|
|
40
|
+
```bash
|
|
41
|
+
npx playwright show-report
|
|
42
|
+
```
|
|
@@ -0,0 +1,50 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: e2e-validate
|
|
3
|
+
description: Validate explored seed file tests before BDD generation — runs seed tests with random data and auto-heals failures.
|
|
4
|
+
context: fork
|
|
5
|
+
---
|
|
6
|
+
|
|
7
|
+
# Seed File Validation
|
|
8
|
+
|
|
9
|
+
Validate explored test cases from the seed file before proceeding with BDD generation.
|
|
10
|
+
|
|
11
|
+
## Agent Chain
|
|
12
|
+
|
|
13
|
+
```
|
|
14
|
+
@agent-execution-manager (seed mode — run seed.spec.js)
|
|
15
|
+
↕ loop (max 3 iterations)
|
|
16
|
+
@agent-playwright-test-healer (fix selector/timing failures)
|
|
17
|
+
```
|
|
18
|
+
|
|
19
|
+
## Steps
|
|
20
|
+
|
|
21
|
+
### Step 1: Execute Seed File
|
|
22
|
+
|
|
23
|
+
Invoke `@agent-execution-manager` in **seed mode**.
|
|
24
|
+
It runs `e2e-tests/playwright/generated/seed.spec.js` with random test data.
|
|
25
|
+
|
|
26
|
+
### Step 2: Check Results
|
|
27
|
+
|
|
28
|
+
- `all_passed` → report "Ready for BDD generation"
|
|
29
|
+
- `needs_healing` → proceed to Step 3
|
|
30
|
+
- `needs_review` → report issues to user
|
|
31
|
+
|
|
32
|
+
### Step 3: Heal Failures
|
|
33
|
+
|
|
34
|
+
Invoke `@agent-playwright-test-healer` with the healable failures.
|
|
35
|
+
The healer fixes selectors in the seed file.
|
|
36
|
+
|
|
37
|
+
### Step 4: Re-run (Loop)
|
|
38
|
+
|
|
39
|
+
Invoke `@agent-execution-manager` seed mode again.
|
|
40
|
+
|
|
41
|
+
- All pass → done
|
|
42
|
+
- Still failing and iterations < 3 → go to Step 3
|
|
43
|
+
- Exhausted → report remaining failures
|
|
44
|
+
|
|
45
|
+
## Output
|
|
46
|
+
|
|
47
|
+
```
|
|
48
|
+
✅ READY FOR BDD GENERATION — all seed tests passing
|
|
49
|
+
⚠️ NEEDS REVIEW — {n} failures remain after 3 healing iterations
|
|
50
|
+
```
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@specwright/plugin",
|
|
3
|
-
"version": "0.1.
|
|
3
|
+
"version": "0.1.2",
|
|
4
4
|
"description": "Drop-in E2E test automation framework plugin — Playwright BDD + Claude Code agents",
|
|
5
5
|
"license": "MIT",
|
|
6
6
|
"bin": {
|
|
@@ -10,11 +10,17 @@
|
|
|
10
10
|
"cli.js",
|
|
11
11
|
"install.sh",
|
|
12
12
|
"PLUGIN.md",
|
|
13
|
+
"README.md",
|
|
13
14
|
"README-TESTING.md",
|
|
14
15
|
"playwright.config.ts",
|
|
15
16
|
"package.json.snippet",
|
|
16
17
|
".gitignore.snippet",
|
|
17
|
-
".
|
|
18
|
+
".claude_agents/",
|
|
19
|
+
".claude_skills/",
|
|
20
|
+
".claude_rules/",
|
|
21
|
+
".claude_agent-memory/",
|
|
22
|
+
".claude_memory_MEMORY.md",
|
|
23
|
+
".claude_README.md",
|
|
18
24
|
"e2e-tests/"
|
|
19
25
|
],
|
|
20
26
|
"publishConfig": {
|