@sun-asterisk/sungen 2.4.0 → 2.4.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/dist/cli/index.js +1 -1
- package/dist/generators/gherkin-parser/index.d.ts +8 -0
- package/dist/generators/gherkin-parser/index.d.ts.map +1 -1
- package/dist/generators/gherkin-parser/index.js +12 -0
- package/dist/generators/gherkin-parser/index.js.map +1 -1
- package/dist/generators/test-generator/adapters/playwright/templates/steps/assertions/table-match-data.hbs +15 -0
- package/dist/generators/test-generator/patterns/index.d.ts.map +1 -1
- package/dist/generators/test-generator/patterns/index.js +2 -1
- package/dist/generators/test-generator/patterns/index.js.map +1 -1
- package/dist/generators/test-generator/patterns/table-patterns.d.ts +4 -1
- package/dist/generators/test-generator/patterns/table-patterns.d.ts.map +1 -1
- package/dist/generators/test-generator/patterns/table-patterns.js +49 -1
- package/dist/generators/test-generator/patterns/table-patterns.js.map +1 -1
- package/dist/orchestrator/templates/ai-instructions/claude-cmd-run-test.md +19 -16
- package/dist/orchestrator/templates/ai-instructions/claude-skill-error-mapping.md +5 -5
- package/dist/orchestrator/templates/ai-instructions/claude-skill-gherkin-syntax.md +8 -1
- package/dist/orchestrator/templates/ai-instructions/claude-skill-selector-fix.md +46 -61
- package/dist/orchestrator/templates/ai-instructions/claude-skill-tc-generation.md +20 -0
- package/dist/orchestrator/templates/ai-instructions/claude-skill-viewpoint.md +3 -1
- package/dist/orchestrator/templates/ai-instructions/copilot-cmd-run-test.md +19 -16
- package/dist/orchestrator/templates/ai-instructions/github-skill-sungen-error-mapping.md +5 -5
- package/dist/orchestrator/templates/ai-instructions/github-skill-sungen-gherkin-syntax.md +8 -1
- package/dist/orchestrator/templates/ai-instructions/github-skill-sungen-selector-fix.md +46 -61
- package/dist/orchestrator/templates/ai-instructions/github-skill-sungen-tc-generation.md +20 -0
- package/dist/orchestrator/templates/ai-instructions/github-skill-sungen-viewpoint.md +3 -1
- package/dist/orchestrator/templates/readme.md +82 -19
- package/package.json +1 -1
- package/src/cli/index.ts +1 -1
- package/src/generators/gherkin-parser/index.ts +23 -0
- package/src/generators/test-generator/adapters/playwright/templates/steps/assertions/table-match-data.hbs +15 -0
- package/src/generators/test-generator/patterns/index.ts +2 -1
- package/src/generators/test-generator/patterns/table-patterns.ts +58 -1
- package/src/orchestrator/templates/ai-instructions/claude-cmd-run-test.md +19 -16
- package/src/orchestrator/templates/ai-instructions/claude-skill-error-mapping.md +5 -5
- package/src/orchestrator/templates/ai-instructions/claude-skill-gherkin-syntax.md +8 -1
- package/src/orchestrator/templates/ai-instructions/claude-skill-selector-fix.md +46 -61
- package/src/orchestrator/templates/ai-instructions/claude-skill-tc-generation.md +20 -0
- package/src/orchestrator/templates/ai-instructions/claude-skill-viewpoint.md +3 -1
- package/src/orchestrator/templates/ai-instructions/copilot-cmd-run-test.md +19 -16
- package/src/orchestrator/templates/ai-instructions/github-skill-sungen-error-mapping.md +5 -5
- package/src/orchestrator/templates/ai-instructions/github-skill-sungen-gherkin-syntax.md +8 -1
- package/src/orchestrator/templates/ai-instructions/github-skill-sungen-selector-fix.md +46 -61
- package/src/orchestrator/templates/ai-instructions/github-skill-sungen-tc-generation.md +20 -0
- package/src/orchestrator/templates/ai-instructions/github-skill-sungen-viewpoint.md +3 -1
- package/src/orchestrator/templates/readme.md +82 -19
|
@@ -1,29 +1,32 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: run-test
|
|
3
|
-
description: '
|
|
3
|
+
description: 'Compile and run Playwright tests — auto-fixes selectors on failure'
|
|
4
4
|
argument-hint: [screen-name]
|
|
5
5
|
allowed-tools: Read, Grep, Bash, Glob, Edit, AskUserQuestion
|
|
6
6
|
---
|
|
7
7
|
|
|
8
8
|
## Role
|
|
9
9
|
|
|
10
|
-
You are a **Senior Developer
|
|
10
|
+
You are a **Senior Developer**. Use `sungen-selector-fix`, `sungen-selector-keys`, and `sungen-error-mapping` skills.
|
|
11
11
|
|
|
12
12
|
## Parameters
|
|
13
13
|
|
|
14
14
|
Parse **screen** from `$ARGUMENTS`. If missing, ask the user.
|
|
15
15
|
|
|
16
|
-
##
|
|
17
|
-
|
|
18
|
-
1. Verify `qa/screens/<screen>/` has `.feature` + `test-data.yaml
|
|
19
|
-
2.
|
|
20
|
-
3.
|
|
21
|
-
4.
|
|
22
|
-
5.
|
|
23
|
-
|
|
24
|
-
|
|
25
|
-
|
|
26
|
-
|
|
27
|
-
|
|
28
|
-
|
|
29
|
-
|
|
16
|
+
## Phase 1: Compile & Run (no MCP)
|
|
17
|
+
|
|
18
|
+
1. Verify `qa/screens/<screen>/` has `.feature` + `test-data.yaml`
|
|
19
|
+
2. Ensure `selectors.yaml` has page selector. If missing, ask user for URL path
|
|
20
|
+
3. `sungen generate --screen <screen>`
|
|
21
|
+
4. `npx playwright test specs/generated/<screen>/*.spec.ts --reporter=line`
|
|
22
|
+
5. If all pass → done
|
|
23
|
+
|
|
24
|
+
## Phase 2: Targeted Fix (only if failures)
|
|
25
|
+
|
|
26
|
+
6. Parse failures → group by root cause
|
|
27
|
+
7. Navigate to page ONCE via MCP → ONE `browser_snapshot`
|
|
28
|
+
8. Fix broken selectors per `sungen-selector-fix` skill
|
|
29
|
+
9. Recompile → re-run only failing tests
|
|
30
|
+
10. Repeat up to 3 attempts
|
|
31
|
+
11. Final full run for regression check
|
|
32
|
+
12. Still failing → ask user
|
|
@@ -81,12 +81,12 @@ If `toHaveText` fails on an input → the Gherkin step has wrong target type. Fi
|
|
|
81
81
|
|
|
82
82
|
| Symptom | Fix |
|
|
83
83
|
|---|---|
|
|
84
|
-
| Redirect to login page | Auth expired.
|
|
85
|
-
| `storageState` file not found |
|
|
86
|
-
| Most tests timeout on first step | Auth expired — re-
|
|
87
|
-
| Page shows home instead of target | SPA + expired auth. Re-
|
|
84
|
+
| Redirect to login page | Auth expired. Ask user to log in manually via MCP browser |
|
|
85
|
+
| `storageState` file not found | Ask user to log in manually via MCP browser, then save storage state |
|
|
86
|
+
| Most tests timeout on first step | Auth expired — ask user to re-authenticate via MCP browser |
|
|
87
|
+
| Page shows home instead of target | SPA + expired auth. Re-authenticate + add `wait for` step |
|
|
88
88
|
|
|
89
|
-
**Never
|
|
89
|
+
**Never use `sungen makeauth`.** Always let the user log in manually via the MCP browser.
|
|
90
90
|
|
|
91
91
|
---
|
|
92
92
|
|
|
@@ -50,7 +50,7 @@ User check [T] radio # select radio
|
|
|
50
50
|
User uncheck [T] checkbox # uncheck
|
|
51
51
|
User uncheck [T] toggle # toggle off
|
|
52
52
|
User select [T] dropdown with {{v}} # select option
|
|
53
|
-
User
|
|
53
|
+
User fill [T] uploader with {{f}} # file upload
|
|
54
54
|
```
|
|
55
55
|
|
|
56
56
|
### Interaction
|
|
@@ -161,10 +161,17 @@ User see [Table] table with {{count}} # row count
|
|
|
161
161
|
User see [Table] table is empty # empty table
|
|
162
162
|
User see [Col] column with {{v}} # cell value (row scoped)
|
|
163
163
|
User click [Act] button in [Table] table with {{v}} # action in row
|
|
164
|
+
User see [Table] table match data: # exact table match (inline DataTable)
|
|
165
|
+
| Header1 | Header2 |
|
|
166
|
+
| {{value1}} | {{value2}} |
|
|
164
167
|
```
|
|
165
168
|
|
|
169
|
+
**Priority**: Use row scope patterns for single-row verification + actions. Use `table match data` only when verifying multiple rows at once.
|
|
170
|
+
|
|
166
171
|
Row scope: `see [Ref] row in [Table] table with {{v}}` defines `tableRow`. Subsequent `see [Col] column with {{v}}` checks the cell in that row. With `columns` config → exact `nth(index)`, without → `filter({ hasText })`.
|
|
167
172
|
|
|
173
|
+
Table match data: `see [Table] table match data:` followed by a Cucumber DataTable. First row = headers, remaining rows = expected data. Uses **filter-based matching** — verifies rows containing expected values exist, resilient to data changes, extra rows, and row reordering. Use `{{variable}}` for cell values.
|
|
174
|
+
|
|
168
175
|
### States
|
|
169
176
|
|
|
170
177
|
`hidden` `visible` `disabled` `enabled` `checked` `unchecked` `focused` `empty` `loading` `selected` `sorted ascending` `sorted descending`
|
|
@@ -1,41 +1,51 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: sungen-selector-fix
|
|
3
|
-
description: 'Selector
|
|
3
|
+
description: 'Selector fixing strategy — diagnose failures, explore live page targeted, fix broken selectors. Auto-loaded by run-test command.'
|
|
4
4
|
user-invocable: false
|
|
5
5
|
---
|
|
6
6
|
|
|
7
|
-
##
|
|
7
|
+
## Strategy: Fix Only What Breaks
|
|
8
8
|
|
|
9
|
-
|
|
9
|
+
Most selectors auto-infer from Gherkin element types (see `sungen-selector-keys` skill). Only add YAML entries for selectors that fail during test execution.
|
|
10
|
+
|
|
11
|
+
**Minimal `selectors.yaml`**: page selectors (required), overrides for elements where auto-infer doesn't work. Never generate a full page catalog upfront.
|
|
10
12
|
|
|
11
13
|
---
|
|
12
14
|
|
|
13
|
-
## Step 1:
|
|
15
|
+
## Step 1: Diagnose Failures
|
|
14
16
|
|
|
15
|
-
|
|
16
|
-
2. `browser_navigate` to `baseURL`
|
|
17
|
-
3. If redirected to login → ask user to log in manually via MCP browser
|
|
18
|
-
4. Navigate to target page → `browser_snapshot`
|
|
17
|
+
Parse Playwright error output to categorize failures:
|
|
19
18
|
|
|
20
|
-
|
|
19
|
+
| Error pattern | Root cause | Fix target |
|
|
20
|
+
|---|---|---|
|
|
21
|
+
| `No element found` / `strict mode violation` | Selector mismatch | `selectors.yaml` |
|
|
22
|
+
| `toBeVisible` timeout | Wrong name or missing element | `selectors.yaml` |
|
|
23
|
+
| `toHaveText` / `toHaveValue` mismatch | Wrong expected data | `test-data.yaml` |
|
|
24
|
+
| `page.goto` error | Wrong URL | page selector in `selectors.yaml` |
|
|
25
|
+
| `frame` error | Element inside iframe | add `frame` field |
|
|
26
|
+
|
|
27
|
+
**Group by root cause** — if 5 tests fail because `[Submit]` button has a different name, that's 1 fix, not 5.
|
|
28
|
+
|
|
29
|
+
**Check `test-results/` first** — Playwright captures failure screenshots automatically. Use these to diagnose before any MCP exploration.
|
|
21
30
|
|
|
22
31
|
---
|
|
23
32
|
|
|
24
|
-
## Step 2:
|
|
33
|
+
## Step 2: Targeted MCP Exploration
|
|
25
34
|
|
|
26
|
-
|
|
35
|
+
Only when `test-results/` screenshots are insufficient:
|
|
27
36
|
|
|
28
|
-
|
|
37
|
+
1. Read `baseURL` from `playwright.config.ts`
|
|
38
|
+
2. `browser_navigate` to target page
|
|
39
|
+
3. If redirected to login → ask user to log in manually via MCP browser
|
|
40
|
+
4. Take **ONE** `browser_snapshot` — fix all broken selectors from this single snapshot
|
|
29
41
|
|
|
30
|
-
|
|
31
|
-
```js
|
|
32
|
-
Array.from(document.querySelectorAll('[data-testid]'))
|
|
33
|
-
.map(e => ({ testid: e.dataset.testid, tag: e.tagName, text: e.textContent.trim().slice(0, 60) }))
|
|
34
|
-
```
|
|
42
|
+
**Never use `sungen makeauth`.** Never use `browser_evaluate` to inject cookies.
|
|
35
43
|
|
|
36
44
|
---
|
|
37
45
|
|
|
38
|
-
## Step 3:
|
|
46
|
+
## Step 3: Fix Broken Selectors
|
|
47
|
+
|
|
48
|
+
For each failed selector, find the correct locator from the snapshot:
|
|
39
49
|
|
|
40
50
|
Selector priority (use first applicable):
|
|
41
51
|
|
|
@@ -50,24 +60,24 @@ Selector priority (use first applicable):
|
|
|
50
60
|
|
|
51
61
|
**Exact name rule**: copy name character-for-character from snapshot. Never infer from Gherkin label.
|
|
52
62
|
|
|
53
|
-
|
|
54
|
-
```
|
|
55
|
-
|
|
56
|
-
|
|
57
|
-
value: '[role="searchbox"]'
|
|
63
|
+
Check for `data-testid` attributes if role-based matching fails:
|
|
64
|
+
```js
|
|
65
|
+
Array.from(document.querySelectorAll('[data-testid]'))
|
|
66
|
+
.map(e => ({ testid: e.dataset.testid, tag: e.tagName, text: e.textContent.trim().slice(0, 60) }))
|
|
58
67
|
```
|
|
59
68
|
|
|
60
|
-
|
|
61
|
-
|
|
62
|
-
|
|
63
|
-
|
|
64
|
-
|
|
69
|
+
Common fixes:
|
|
70
|
+
- Name mismatch → copy exact name from snapshot
|
|
71
|
+
- Multiple matches → add `nth` or `exact: true`
|
|
72
|
+
- No accessible name → use `testid` or `locator` (CSS)
|
|
73
|
+
- Element in iframe → add `frame` field
|
|
74
|
+
- Dynamic content → use `testid` or structural `role` + `nth`
|
|
65
75
|
|
|
66
76
|
---
|
|
67
77
|
|
|
68
78
|
## Step 4: Table Selectors
|
|
69
79
|
|
|
70
|
-
For table patterns
|
|
80
|
+
For table patterns, add table selectors with `columns` config:
|
|
71
81
|
|
|
72
82
|
```yaml
|
|
73
83
|
users:
|
|
@@ -88,15 +98,6 @@ users:
|
|
|
88
98
|
|
|
89
99
|
**How to build `columns`**: count column headers in snapshot (left to right, 0-indexed). Map each `[Col] column` reference from feature file to its index.
|
|
90
100
|
|
|
91
|
-
**Why**: enables exact cell assertion via `cell.nth(index).toHaveText(value)` instead of approximate `cell.filter({ hasText })`.
|
|
92
|
-
|
|
93
|
-
**Row scope Gherkin** — these patterns use the table selector:
|
|
94
|
-
```gherkin
|
|
95
|
-
Then User see [Username] row in [Users] table with {{user_name}}
|
|
96
|
-
Then User see [Status] column with {{expected_status}}
|
|
97
|
-
When User click [Edit] button in [Users] table with {{user_name}}
|
|
98
|
-
```
|
|
99
|
-
|
|
100
101
|
---
|
|
101
102
|
|
|
102
103
|
## Step 5: Detail Screens with Dynamic IDs
|
|
@@ -113,28 +114,12 @@ user detail:
|
|
|
113
114
|
|
|
114
115
|
---
|
|
115
116
|
|
|
116
|
-
## Step 6:
|
|
117
|
-
|
|
118
|
-
**Fix 80%+ of selector issues before first test run.**
|
|
119
|
-
|
|
120
|
-
After generating `selectors.yaml`, verify each entry against the live snapshot:
|
|
121
|
-
- `role` + `name` → search snapshot for `role "name"`
|
|
122
|
-
- `testid` → `browser_evaluate`: `document.querySelector('[data-testid="xxx"]')`
|
|
123
|
-
- `placeholder` → search snapshot for textbox with placeholder
|
|
124
|
-
- `page` → verify URL path exists
|
|
125
|
-
|
|
126
|
-
Common fixes: name mismatch → copy from snapshot, element in iframe → add `frame`, multiple matches → add `nth` or `exact: true`.
|
|
127
|
-
|
|
128
|
-
---
|
|
129
|
-
|
|
130
|
-
## Step 7: Run & Fix Loop
|
|
131
|
-
|
|
132
|
-
Run tests in batches of 20. Group failures by root cause (same selector, same error type). Fix once, verify batch, move to next.
|
|
133
|
-
|
|
134
|
-
```
|
|
135
|
-
compile → batch run 20 → fix root cause → recompile → re-run failures → next batch
|
|
136
|
-
```
|
|
117
|
+
## Step 6: Fix Loop
|
|
137
118
|
|
|
138
|
-
After
|
|
119
|
+
After fixing selectors:
|
|
120
|
+
1. Recompile: `sungen generate --screen <screen>`
|
|
121
|
+
2. Re-run only failing tests: `npx playwright test --grep "VP-XXX-001|VP-XXX-002" --reporter=line`
|
|
122
|
+
3. If new failures → repeat (max 3 attempts)
|
|
123
|
+
4. After all fixes → run ALL tests once for regression check
|
|
139
124
|
|
|
140
|
-
If still failing after
|
|
125
|
+
If still failing after 3 attempts → ask user for guidance.
|
|
@@ -100,8 +100,28 @@ Feature: <Screen> Screen
|
|
|
100
100
|
Then User see [Name] column in [Users] table
|
|
101
101
|
And User see [Email] column in [Users] table
|
|
102
102
|
And User see [Status] column in [Users] table
|
|
103
|
+
|
|
104
|
+
# --- Data & Validate ---
|
|
105
|
+
|
|
106
|
+
Scenario: VP-VAL-010 Table displays correct data
|
|
107
|
+
Then User see [Users] table match data:
|
|
108
|
+
| Name | Email | Status |
|
|
109
|
+
| {{name_1}} | {{email_1}} | {{status_1}} |
|
|
110
|
+
| {{name_2}} | {{email_2}} | {{status_2}} |
|
|
111
|
+
|
|
112
|
+
Scenario: VP-VAL-011 Edit button targets correct row
|
|
113
|
+
Given User see [Target] row in [Users] table with {{name_1}}
|
|
114
|
+
When User click [Edit] button in [Users] table with {{name_1}}
|
|
115
|
+
Then User see [Name] field with {{name_1}}
|
|
103
116
|
```
|
|
104
117
|
|
|
118
|
+
### When to use DataTable vs Row Scope
|
|
119
|
+
|
|
120
|
+
| Pattern | Use when |
|
|
121
|
+
|---|---|
|
|
122
|
+
| `table match data:` + DataTable | Verifying **multiple rows** exist with expected values |
|
|
123
|
+
| `row in [Table] table with {{v}}` + `column with {{v}}` | Checking **single row** details or **acting** on a row (click, edit) |
|
|
124
|
+
|
|
105
125
|
**Naming**: `VP-<CATEGORY>-<NNN>` prefix.
|
|
106
126
|
|
|
107
127
|
**Test data** — `qa/screens/<screen>/test-data/<screen>.yaml`, grouped by section.
|
|
@@ -52,6 +52,8 @@ user-invocable: false
|
|
|
52
52
|
- Consistency: total record count on UI matches server data
|
|
53
53
|
- Row Limit: displayed rows never exceed page size/limit
|
|
54
54
|
- Cell Integrity: cell data matches database, correct format (date, currency, status)
|
|
55
|
+
- **Use `table match data:` with inline DataTable** for multi-row content verification (filter-based, resilient to data changes)
|
|
56
|
+
- Use row scope (`row in [Table] table with {{v}}` + `column with {{v}}`) for single-row detail checks or when you need actions on a row
|
|
55
57
|
|
|
56
58
|
**Logic**
|
|
57
59
|
- Sorting: column sort refreshes data with correct order, updates header icon
|
|
@@ -82,7 +84,7 @@ user-invocable: false
|
|
|
82
84
|
|
|
83
85
|
**Security**
|
|
84
86
|
- Injection: XSS/SQL encoded as plain text, never executed
|
|
85
|
-
- Wildcards:
|
|
87
|
+
- Wildcards: `%, _, *` treated as normal text (escaped)
|
|
86
88
|
|
|
87
89
|
---
|
|
88
90
|
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: sungen-run-test
|
|
3
|
-
description: '
|
|
3
|
+
description: 'Compile and run Playwright tests — auto-fixes selectors on failure. Uses sungen-selector-fix, sungen-selector-keys, and sungen-error-mapping skills.'
|
|
4
4
|
argument-hint: '[screen-name]'
|
|
5
5
|
agent: 'agent'
|
|
6
6
|
tools: [vscode, execute, read, agent, edit, search, web, browser, todo, 'playwright/*']
|
|
@@ -10,23 +10,26 @@ tools: [vscode, execute, read, agent, edit, search, web, browser, todo, 'playwri
|
|
|
10
10
|
|
|
11
11
|
## Role
|
|
12
12
|
|
|
13
|
-
You are a **Senior Developer
|
|
13
|
+
You are a **Senior Developer**. Use `sungen-selector-fix`, `sungen-selector-keys`, and `sungen-error-mapping` skills.
|
|
14
14
|
|
|
15
15
|
## Parameters
|
|
16
16
|
|
|
17
17
|
- **screen** — ${input:screen:screen name (e.g., login, dashboard)}
|
|
18
18
|
|
|
19
|
-
##
|
|
20
|
-
|
|
21
|
-
1. Verify `qa/screens/${input:screen}/` has `.feature` + `test-data.yaml
|
|
22
|
-
2.
|
|
23
|
-
3.
|
|
24
|
-
4.
|
|
25
|
-
5.
|
|
26
|
-
|
|
27
|
-
|
|
28
|
-
|
|
29
|
-
|
|
30
|
-
|
|
31
|
-
|
|
32
|
-
|
|
19
|
+
## Phase 1: Compile & Run (no MCP)
|
|
20
|
+
|
|
21
|
+
1. Verify `qa/screens/${input:screen}/` has `.feature` + `test-data.yaml`
|
|
22
|
+
2. Ensure `selectors.yaml` has page selector. If missing, ask user for URL path
|
|
23
|
+
3. `sungen generate --screen ${input:screen}`
|
|
24
|
+
4. `npx playwright test specs/generated/${input:screen}/*.spec.ts --reporter=line`
|
|
25
|
+
5. If all pass → done
|
|
26
|
+
|
|
27
|
+
## Phase 2: Targeted Fix (only if failures)
|
|
28
|
+
|
|
29
|
+
6. Parse failures → group by root cause
|
|
30
|
+
7. Navigate to page ONCE via #tool:playwright → ONE `browser_snapshot`
|
|
31
|
+
8. Fix broken selectors per `sungen-selector-fix` skill
|
|
32
|
+
9. Recompile → re-run only failing tests
|
|
33
|
+
10. Repeat up to 3 attempts
|
|
34
|
+
11. Final full run for regression check
|
|
35
|
+
12. Still failing → ask user
|
|
@@ -81,12 +81,12 @@ If `toHaveText` fails on an input → the Gherkin step has wrong target type. Fi
|
|
|
81
81
|
|
|
82
82
|
| Symptom | Fix |
|
|
83
83
|
|---|---|
|
|
84
|
-
| Redirect to login page | Auth expired.
|
|
85
|
-
| `storageState` file not found |
|
|
86
|
-
| Most tests timeout on first step | Auth expired — re-
|
|
87
|
-
| Page shows home instead of target | SPA + expired auth. Re-
|
|
84
|
+
| Redirect to login page | Auth expired. Ask user to log in manually via MCP browser |
|
|
85
|
+
| `storageState` file not found | Ask user to log in manually via MCP browser, then save storage state |
|
|
86
|
+
| Most tests timeout on first step | Auth expired — ask user to re-authenticate via MCP browser |
|
|
87
|
+
| Page shows home instead of target | SPA + expired auth. Re-authenticate + add `wait for` step |
|
|
88
88
|
|
|
89
|
-
**Never
|
|
89
|
+
**Never use `sungen makeauth`.** Always let the user log in manually via the MCP browser.
|
|
90
90
|
|
|
91
91
|
---
|
|
92
92
|
|
|
@@ -50,7 +50,7 @@ User check [T] radio # select radio
|
|
|
50
50
|
User uncheck [T] checkbox # uncheck
|
|
51
51
|
User uncheck [T] toggle # toggle off
|
|
52
52
|
User select [T] dropdown with {{v}} # select option
|
|
53
|
-
User
|
|
53
|
+
User fill [T] uploader with {{f}} # file upload
|
|
54
54
|
```
|
|
55
55
|
|
|
56
56
|
### Interaction
|
|
@@ -161,10 +161,17 @@ User see [Table] table with {{count}} # row count
|
|
|
161
161
|
User see [Table] table is empty # empty table
|
|
162
162
|
User see [Col] column with {{v}} # cell value (row scoped)
|
|
163
163
|
User click [Act] button in [Table] table with {{v}} # action in row
|
|
164
|
+
User see [Table] table match data: # exact table match (inline DataTable)
|
|
165
|
+
| Header1 | Header2 |
|
|
166
|
+
| {{value1}} | {{value2}} |
|
|
164
167
|
```
|
|
165
168
|
|
|
169
|
+
**Priority**: Use row scope patterns for single-row verification + actions. Use `table match data` only when verifying multiple rows at once.
|
|
170
|
+
|
|
166
171
|
Row scope: `see [Ref] row in [Table] table with {{v}}` defines `tableRow`. Subsequent `see [Col] column with {{v}}` checks the cell in that row. With `columns` config → exact `nth(index)`, without → `filter({ hasText })`.
|
|
167
172
|
|
|
173
|
+
Table match data: `see [Table] table match data:` followed by a Cucumber DataTable. First row = headers, remaining rows = expected data. Uses **filter-based matching** — verifies rows containing expected values exist, resilient to data changes, extra rows, and row reordering. Use `{{variable}}` for cell values.
|
|
174
|
+
|
|
168
175
|
### States
|
|
169
176
|
|
|
170
177
|
`hidden` `visible` `disabled` `enabled` `checked` `unchecked` `focused` `empty` `loading` `selected` `sorted ascending` `sorted descending`
|
|
@@ -1,41 +1,51 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: sungen-selector-fix
|
|
3
|
-
description: 'Selector
|
|
3
|
+
description: 'Selector fixing strategy — diagnose failures, explore live page targeted, fix broken selectors. Auto-loaded by run-test command.'
|
|
4
4
|
user-invocable: false
|
|
5
5
|
---
|
|
6
6
|
|
|
7
|
-
##
|
|
7
|
+
## Strategy: Fix Only What Breaks
|
|
8
8
|
|
|
9
|
-
|
|
9
|
+
Most selectors auto-infer from Gherkin element types (see `sungen-selector-keys` skill). Only add YAML entries for selectors that fail during test execution.
|
|
10
|
+
|
|
11
|
+
**Minimal `selectors.yaml`**: page selectors (required), overrides for elements where auto-infer doesn't work. Never generate a full page catalog upfront.
|
|
10
12
|
|
|
11
13
|
---
|
|
12
14
|
|
|
13
|
-
## Step 1:
|
|
15
|
+
## Step 1: Diagnose Failures
|
|
14
16
|
|
|
15
|
-
|
|
16
|
-
2. `browser_navigate` to `baseURL`
|
|
17
|
-
3. If redirected to login → ask user to log in manually via MCP browser
|
|
18
|
-
4. Navigate to target page → `browser_snapshot`
|
|
17
|
+
Parse Playwright error output to categorize failures:
|
|
19
18
|
|
|
20
|
-
|
|
19
|
+
| Error pattern | Root cause | Fix target |
|
|
20
|
+
|---|---|---|
|
|
21
|
+
| `No element found` / `strict mode violation` | Selector mismatch | `selectors.yaml` |
|
|
22
|
+
| `toBeVisible` timeout | Wrong name or missing element | `selectors.yaml` |
|
|
23
|
+
| `toHaveText` / `toHaveValue` mismatch | Wrong expected data | `test-data.yaml` |
|
|
24
|
+
| `page.goto` error | Wrong URL | page selector in `selectors.yaml` |
|
|
25
|
+
| `frame` error | Element inside iframe | add `frame` field |
|
|
26
|
+
|
|
27
|
+
**Group by root cause** — if 5 tests fail because `[Submit]` button has a different name, that's 1 fix, not 5.
|
|
28
|
+
|
|
29
|
+
**Check `test-results/` first** — Playwright captures failure screenshots automatically. Use these to diagnose before any MCP exploration.
|
|
21
30
|
|
|
22
31
|
---
|
|
23
32
|
|
|
24
|
-
## Step 2:
|
|
33
|
+
## Step 2: Targeted MCP Exploration
|
|
25
34
|
|
|
26
|
-
|
|
35
|
+
Only when `test-results/` screenshots are insufficient:
|
|
27
36
|
|
|
28
|
-
|
|
37
|
+
1. Read `baseURL` from `playwright.config.ts`
|
|
38
|
+
2. `browser_navigate` to target page
|
|
39
|
+
3. If redirected to login → ask user to log in manually via MCP browser
|
|
40
|
+
4. Take **ONE** `browser_snapshot` — fix all broken selectors from this single snapshot
|
|
29
41
|
|
|
30
|
-
|
|
31
|
-
```js
|
|
32
|
-
Array.from(document.querySelectorAll('[data-testid]'))
|
|
33
|
-
.map(e => ({ testid: e.dataset.testid, tag: e.tagName, text: e.textContent.trim().slice(0, 60) }))
|
|
34
|
-
```
|
|
42
|
+
**Never use `sungen makeauth`.** Never use `browser_evaluate` to inject cookies.
|
|
35
43
|
|
|
36
44
|
---
|
|
37
45
|
|
|
38
|
-
## Step 3:
|
|
46
|
+
## Step 3: Fix Broken Selectors
|
|
47
|
+
|
|
48
|
+
For each failed selector, find the correct locator from the snapshot:
|
|
39
49
|
|
|
40
50
|
Selector priority (use first applicable):
|
|
41
51
|
|
|
@@ -50,24 +60,24 @@ Selector priority (use first applicable):
|
|
|
50
60
|
|
|
51
61
|
**Exact name rule**: copy name character-for-character from snapshot. Never infer from Gherkin label.
|
|
52
62
|
|
|
53
|
-
|
|
54
|
-
```
|
|
55
|
-
|
|
56
|
-
|
|
57
|
-
value: '[role="searchbox"]'
|
|
63
|
+
Check for `data-testid` attributes if role-based matching fails:
|
|
64
|
+
```js
|
|
65
|
+
Array.from(document.querySelectorAll('[data-testid]'))
|
|
66
|
+
.map(e => ({ testid: e.dataset.testid, tag: e.tagName, text: e.textContent.trim().slice(0, 60) }))
|
|
58
67
|
```
|
|
59
68
|
|
|
60
|
-
|
|
61
|
-
|
|
62
|
-
|
|
63
|
-
|
|
64
|
-
|
|
69
|
+
Common fixes:
|
|
70
|
+
- Name mismatch → copy exact name from snapshot
|
|
71
|
+
- Multiple matches → add `nth` or `exact: true`
|
|
72
|
+
- No accessible name → use `testid` or `locator` (CSS)
|
|
73
|
+
- Element in iframe → add `frame` field
|
|
74
|
+
- Dynamic content → use `testid` or structural `role` + `nth`
|
|
65
75
|
|
|
66
76
|
---
|
|
67
77
|
|
|
68
78
|
## Step 4: Table Selectors
|
|
69
79
|
|
|
70
|
-
For table patterns
|
|
80
|
+
For table patterns, add table selectors with `columns` config:
|
|
71
81
|
|
|
72
82
|
```yaml
|
|
73
83
|
users:
|
|
@@ -88,15 +98,6 @@ users:
|
|
|
88
98
|
|
|
89
99
|
**How to build `columns`**: count column headers in snapshot (left to right, 0-indexed). Map each `[Col] column` reference from feature file to its index.
|
|
90
100
|
|
|
91
|
-
**Why**: enables exact cell assertion via `cell.nth(index).toHaveText(value)` instead of approximate `cell.filter({ hasText })`.
|
|
92
|
-
|
|
93
|
-
**Row scope Gherkin** — these patterns use the table selector:
|
|
94
|
-
```gherkin
|
|
95
|
-
Then User see [Username] row in [Users] table with {{user_name}}
|
|
96
|
-
Then User see [Status] column with {{expected_status}}
|
|
97
|
-
When User click [Edit] button in [Users] table with {{user_name}}
|
|
98
|
-
```
|
|
99
|
-
|
|
100
101
|
---
|
|
101
102
|
|
|
102
103
|
## Step 5: Detail Screens with Dynamic IDs
|
|
@@ -113,28 +114,12 @@ user detail:
|
|
|
113
114
|
|
|
114
115
|
---
|
|
115
116
|
|
|
116
|
-
## Step 6:
|
|
117
|
-
|
|
118
|
-
**Fix 80%+ of selector issues before first test run.**
|
|
119
|
-
|
|
120
|
-
After generating `selectors.yaml`, verify each entry against the live snapshot:
|
|
121
|
-
- `role` + `name` → search snapshot for `role "name"`
|
|
122
|
-
- `testid` → `browser_evaluate`: `document.querySelector('[data-testid="xxx"]')`
|
|
123
|
-
- `placeholder` → search snapshot for textbox with placeholder
|
|
124
|
-
- `page` → verify URL path exists
|
|
125
|
-
|
|
126
|
-
Common fixes: name mismatch → copy from snapshot, element in iframe → add `frame`, multiple matches → add `nth` or `exact: true`.
|
|
127
|
-
|
|
128
|
-
---
|
|
129
|
-
|
|
130
|
-
## Step 7: Run & Fix Loop
|
|
131
|
-
|
|
132
|
-
Run tests in batches of 20. Group failures by root cause (same selector, same error type). Fix once, verify batch, move to next.
|
|
133
|
-
|
|
134
|
-
```
|
|
135
|
-
compile → batch run 20 → fix root cause → recompile → re-run failures → next batch
|
|
136
|
-
```
|
|
117
|
+
## Step 6: Fix Loop
|
|
137
118
|
|
|
138
|
-
After
|
|
119
|
+
After fixing selectors:
|
|
120
|
+
1. Recompile: `sungen generate --screen <screen>`
|
|
121
|
+
2. Re-run only failing tests: `npx playwright test --grep "VP-XXX-001|VP-XXX-002" --reporter=line`
|
|
122
|
+
3. If new failures → repeat (max 3 attempts)
|
|
123
|
+
4. After all fixes → run ALL tests once for regression check
|
|
139
124
|
|
|
140
|
-
If still failing after
|
|
125
|
+
If still failing after 3 attempts → ask user for guidance.
|
|
@@ -100,8 +100,28 @@ Feature: <Screen> Screen
|
|
|
100
100
|
Then User see [Name] column in [Users] table
|
|
101
101
|
And User see [Email] column in [Users] table
|
|
102
102
|
And User see [Status] column in [Users] table
|
|
103
|
+
|
|
104
|
+
# --- Data & Validate ---
|
|
105
|
+
|
|
106
|
+
Scenario: VP-VAL-010 Table displays correct data
|
|
107
|
+
Then User see [Users] table match data:
|
|
108
|
+
| Name | Email | Status |
|
|
109
|
+
| {{name_1}} | {{email_1}} | {{status_1}} |
|
|
110
|
+
| {{name_2}} | {{email_2}} | {{status_2}} |
|
|
111
|
+
|
|
112
|
+
Scenario: VP-VAL-011 Edit button targets correct row
|
|
113
|
+
Given User see [Target] row in [Users] table with {{name_1}}
|
|
114
|
+
When User click [Edit] button in [Users] table with {{name_1}}
|
|
115
|
+
Then User see [Name] field with {{name_1}}
|
|
103
116
|
```
|
|
104
117
|
|
|
118
|
+
### When to use DataTable vs Row Scope
|
|
119
|
+
|
|
120
|
+
| Pattern | Use when |
|
|
121
|
+
|---|---|
|
|
122
|
+
| `table match data:` + DataTable | Verifying **multiple rows** exist with expected values |
|
|
123
|
+
| `row in [Table] table with {{v}}` + `column with {{v}}` | Checking **single row** details or **acting** on a row (click, edit) |
|
|
124
|
+
|
|
105
125
|
**Naming**: `VP-<CATEGORY>-<NNN>` prefix.
|
|
106
126
|
|
|
107
127
|
**Test data** — `qa/screens/<screen>/test-data/<screen>.yaml`, grouped by section.
|
|
@@ -52,6 +52,8 @@ user-invocable: false
|
|
|
52
52
|
- Consistency: total record count on UI matches server data
|
|
53
53
|
- Row Limit: displayed rows never exceed page size/limit
|
|
54
54
|
- Cell Integrity: cell data matches database, correct format (date, currency, status)
|
|
55
|
+
- **Use `table match data:` with inline DataTable** for multi-row content verification (filter-based, resilient to data changes)
|
|
56
|
+
- Use row scope (`row in [Table] table with {{v}}` + `column with {{v}}`) for single-row detail checks or when you need actions on a row
|
|
55
57
|
|
|
56
58
|
**Logic**
|
|
57
59
|
- Sorting: column sort refreshes data with correct order, updates header icon
|
|
@@ -82,7 +84,7 @@ user-invocable: false
|
|
|
82
84
|
|
|
83
85
|
**Security**
|
|
84
86
|
- Injection: XSS/SQL encoded as plain text, never executed
|
|
85
|
-
- Wildcards:
|
|
87
|
+
- Wildcards: `%, _, *` treated as normal text (escaped)
|
|
86
88
|
|
|
87
89
|
---
|
|
88
90
|
|