@sun-asterisk/sungen 2.3.1 → 2.4.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +10 -11
- package/dist/cli/index.js +1 -1
- package/dist/generators/test-generator/adapters/playwright/templates/steps/actions/table-action-in-row.hbs +1 -1
- package/dist/generators/test-generator/adapters/playwright/templates/steps/assertions/table-column-exists.hbs +5 -1
- package/dist/generators/test-generator/adapters/playwright/templates/steps/assertions/table-row-exists.hbs +7 -1
- package/dist/generators/test-generator/adapters/playwright/templates/steps/assertions/table-row-not-exists.hbs +5 -1
- package/dist/generators/test-generator/patterns/table-patterns.d.ts +9 -0
- package/dist/generators/test-generator/patterns/table-patterns.d.ts.map +1 -1
- package/dist/generators/test-generator/patterns/table-patterns.js +94 -98
- package/dist/generators/test-generator/patterns/table-patterns.js.map +1 -1
- package/dist/generators/test-generator/patterns/types.d.ts +2 -0
- package/dist/generators/test-generator/patterns/types.d.ts.map +1 -1
- package/dist/generators/test-generator/step-mapper.d.ts +13 -0
- package/dist/generators/test-generator/step-mapper.d.ts.map +1 -1
- package/dist/generators/test-generator/step-mapper.js +80 -0
- package/dist/generators/test-generator/step-mapper.js.map +1 -1
- package/dist/orchestrator/ai-rules-updater.d.ts.map +1 -1
- package/dist/orchestrator/ai-rules-updater.js +8 -6
- package/dist/orchestrator/ai-rules-updater.js.map +1 -1
- package/dist/orchestrator/project-initializer.d.ts.map +1 -1
- package/dist/orchestrator/project-initializer.js +33 -2
- package/dist/orchestrator/project-initializer.js.map +1 -1
- package/dist/orchestrator/screen-manager.js +1 -1
- package/dist/orchestrator/screen-manager.js.map +1 -1
- package/dist/orchestrator/templates/ai-instructions/claude-cmd-add-screen.md +5 -5
- package/{src/orchestrator/templates/ai-instructions/claude-cmd-make-tc.md → dist/orchestrator/templates/ai-instructions/claude-cmd-create-test.md} +7 -7
- package/dist/orchestrator/templates/ai-instructions/claude-cmd-review.md +21 -0
- package/{src/orchestrator/templates/ai-instructions/claude-cmd-make-test.md → dist/orchestrator/templates/ai-instructions/claude-cmd-run-test.md} +2 -2
- package/dist/orchestrator/templates/ai-instructions/claude-config.md +65 -12
- package/dist/orchestrator/templates/ai-instructions/claude-skill-error-mapping.md +128 -52
- package/dist/orchestrator/templates/ai-instructions/claude-skill-gherkin-syntax.md +9 -40
- package/dist/orchestrator/templates/ai-instructions/claude-skill-selector-fix.md +67 -239
- package/dist/orchestrator/templates/ai-instructions/claude-skill-tc-generation.md +40 -208
- package/dist/orchestrator/templates/ai-instructions/claude-skill-tc-review.md +104 -0
- package/dist/orchestrator/templates/ai-instructions/claude-skill-viewpoint.md +186 -0
- package/dist/orchestrator/templates/ai-instructions/copilot-cmd-add-screen.md +4 -4
- package/dist/orchestrator/templates/ai-instructions/{copilot-cmd-make-tc.md → copilot-cmd-create-test.md} +8 -8
- package/dist/orchestrator/templates/ai-instructions/copilot-cmd-review.md +24 -0
- package/{src/orchestrator/templates/ai-instructions/copilot-cmd-make-test.md → dist/orchestrator/templates/ai-instructions/copilot-cmd-run-test.md} +3 -3
- package/dist/orchestrator/templates/ai-instructions/copilot-config.md +66 -13
- package/dist/orchestrator/templates/ai-instructions/github-skill-sungen-error-mapping.md +128 -52
- package/dist/orchestrator/templates/ai-instructions/github-skill-sungen-gherkin-syntax.md +9 -40
- package/dist/orchestrator/templates/ai-instructions/github-skill-sungen-selector-fix.md +67 -214
- package/dist/orchestrator/templates/ai-instructions/github-skill-sungen-tc-generation.md +40 -208
- package/dist/orchestrator/templates/ai-instructions/github-skill-sungen-tc-review.md +104 -0
- package/dist/orchestrator/templates/ai-instructions/github-skill-sungen-viewpoint.md +186 -0
- package/dist/orchestrator/templates/readme.md +4 -4
- package/package.json +1 -1
- package/src/cli/index.ts +1 -1
- package/src/generators/test-generator/adapters/playwright/templates/steps/actions/table-action-in-row.hbs +1 -1
- package/src/generators/test-generator/adapters/playwright/templates/steps/assertions/table-column-exists.hbs +5 -1
- package/src/generators/test-generator/adapters/playwright/templates/steps/assertions/table-row-exists.hbs +7 -1
- package/src/generators/test-generator/adapters/playwright/templates/steps/assertions/table-row-not-exists.hbs +5 -1
- package/src/generators/test-generator/patterns/table-patterns.ts +98 -111
- package/src/generators/test-generator/patterns/types.ts +2 -0
- package/src/generators/test-generator/step-mapper.ts +87 -1
- package/src/orchestrator/ai-rules-updater.ts +8 -6
- package/src/orchestrator/project-initializer.ts +38 -2
- package/src/orchestrator/screen-manager.ts +1 -1
- package/src/orchestrator/templates/ai-instructions/claude-cmd-add-screen.md +5 -5
- package/{dist/orchestrator/templates/ai-instructions/claude-cmd-make-tc.md → src/orchestrator/templates/ai-instructions/claude-cmd-create-test.md} +7 -7
- package/src/orchestrator/templates/ai-instructions/claude-cmd-review.md +21 -0
- package/{dist/orchestrator/templates/ai-instructions/claude-cmd-make-test.md → src/orchestrator/templates/ai-instructions/claude-cmd-run-test.md} +2 -2
- package/src/orchestrator/templates/ai-instructions/claude-config.md +65 -12
- package/src/orchestrator/templates/ai-instructions/claude-skill-error-mapping.md +128 -52
- package/src/orchestrator/templates/ai-instructions/claude-skill-gherkin-syntax.md +9 -40
- package/src/orchestrator/templates/ai-instructions/claude-skill-selector-fix.md +67 -239
- package/src/orchestrator/templates/ai-instructions/claude-skill-tc-generation.md +40 -208
- package/src/orchestrator/templates/ai-instructions/claude-skill-tc-review.md +104 -0
- package/src/orchestrator/templates/ai-instructions/claude-skill-viewpoint.md +186 -0
- package/src/orchestrator/templates/ai-instructions/copilot-cmd-add-screen.md +4 -4
- package/src/orchestrator/templates/ai-instructions/{copilot-cmd-make-tc.md → copilot-cmd-create-test.md} +8 -8
- package/src/orchestrator/templates/ai-instructions/copilot-cmd-review.md +24 -0
- package/{dist/orchestrator/templates/ai-instructions/copilot-cmd-make-test.md → src/orchestrator/templates/ai-instructions/copilot-cmd-run-test.md} +3 -3
- package/src/orchestrator/templates/ai-instructions/copilot-config.md +66 -13
- package/src/orchestrator/templates/ai-instructions/github-skill-sungen-error-mapping.md +128 -52
- package/src/orchestrator/templates/ai-instructions/github-skill-sungen-gherkin-syntax.md +9 -40
- package/src/orchestrator/templates/ai-instructions/github-skill-sungen-selector-fix.md +67 -214
- package/src/orchestrator/templates/ai-instructions/github-skill-sungen-tc-generation.md +40 -208
- package/src/orchestrator/templates/ai-instructions/github-skill-sungen-tc-review.md +104 -0
- package/src/orchestrator/templates/ai-instructions/github-skill-sungen-viewpoint.md +186 -0
- package/src/orchestrator/templates/readme.md +4 -4
- package/dist/generators/test-generator/adapters/playwright/templates/steps/assertions/table-cell-by-filter.hbs +0 -2
- package/dist/generators/test-generator/adapters/playwright/templates/steps/assertions/table-cell-by-index.hbs +0 -2
- package/dist/orchestrator/templates/ai-instructions/claude-skill-gherkin-review.md +0 -228
- package/dist/orchestrator/templates/ai-instructions/github-skill-sungen-gherkin-review.md +0 -228
- package/src/generators/test-generator/adapters/playwright/templates/steps/assertions/table-cell-by-filter.hbs +0 -2
- package/src/generators/test-generator/adapters/playwright/templates/steps/assertions/table-cell-by-index.hbs +0 -2
- package/src/orchestrator/templates/ai-instructions/claude-skill-gherkin-review.md +0 -228
- package/src/orchestrator/templates/ai-instructions/github-skill-sungen-gherkin-review.md +0 -228
|
@@ -1,69 +1,145 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: sungen-error-mapping
|
|
3
|
-
description: '
|
|
3
|
+
description: 'Error diagnosis and fix strategy — systematic debugging flow, error patterns, fix priorities. Auto-loaded during run-test.'
|
|
4
4
|
user-invocable: false
|
|
5
5
|
---
|
|
6
6
|
|
|
7
|
-
##
|
|
7
|
+
## Diagnosis Flow (before fixing anything)
|
|
8
8
|
|
|
9
|
-
|
|
10
|
-
|---|---|
|
|
11
|
-
| strict mode violation | add `nth`, `exact: true`, or specific `name` in selectors.yaml |
|
|
12
|
-
| Element is not an input | change `type` or `value` in selectors.yaml |
|
|
13
|
-
| Timeout waiting for selector | fix `type`/`value`/`name` to match element in selectors.yaml |
|
|
14
|
-
| toBeVisible Timeout | verify accessible name or use `testid` in selectors.yaml |
|
|
15
|
-
| toHaveText mismatch | fix expected text in `test-data.yaml` — OR if element is an input (field/textarea/search/dropdown), change Gherkin type so compiler uses `toHaveValue` instead |
|
|
16
|
-
| toHaveValue mismatch | fix expected value in `test-data.yaml` — this is for input elements (field, textarea, search, dropdown, slider, date-picker) |
|
|
17
|
-
| toContainText mismatch | fix expected partial text in `test-data.yaml` |
|
|
18
|
-
| Navigation failed | fix page `value` path in selectors.yaml |
|
|
19
|
-
| not a select | set `variant: 'custom'` in selectors.yaml |
|
|
20
|
-
| Frame not found | fix `frame` value in selectors.yaml |
|
|
21
|
-
| dialog was dismissed | alert step is AFTER the trigger — move `click [OK] alert` BEFORE the button click that opens the dialog |
|
|
22
|
-
| page.once('dialog') never fired | no browser dialog appeared — check if the action actually triggers a `window.alert/confirm/prompt`, not a CSS modal |
|
|
23
|
-
|
|
24
|
-
### Assertion type mismatch (toHaveText vs toHaveValue)
|
|
25
|
-
|
|
26
|
-
Sungen picks the assertion based on element type:
|
|
27
|
-
- **Input types** (`field`, `textarea`, `search`, `dropdown`, `slider`, `date-picker`) → `toHaveValue()`
|
|
28
|
-
- **Text types** (everything else: `message`, `header`, `label`, `row`, etc.) → `toHaveText()`
|
|
29
|
-
- **Partial match** (uses `contains` keyword) → `toContainText()`
|
|
30
|
-
|
|
31
|
-
If you see `toHaveText` failing on an input element, the Gherkin step has the wrong target type. Fix: change the target type in the `.feature` file (e.g., `see [Email] message with {{v}}` → `see [Email] field with {{v}}`).
|
|
32
|
-
|
|
33
|
-
## Auth Errors → Fix
|
|
9
|
+
**Step 1: Read the error** — extract: error type, element name, expected vs actual, file:line.
|
|
34
10
|
|
|
35
|
-
|
|
36
|
-
|
|
37
|
-
|
|
38
|
-
|
|
39
|
-
|
|
40
|
-
|
|
11
|
+
**Step 2: Read error context** — check `test-results/` for the snapshot at failure time. This shows the exact page state when the test failed — more reliable than re-navigating with MCP.
|
|
12
|
+
|
|
13
|
+
**Step 3: Compare** — match the failing selector against the snapshot. Ask:
|
|
14
|
+
- Does the element exist on the page?
|
|
15
|
+
- Does the accessible name match exactly?
|
|
16
|
+
- Are there multiple matches (strict mode)?
|
|
17
|
+
- Is the element inside an iframe or dialog?
|
|
18
|
+
- Is the page in the expected state (correct URL, loaded)?
|
|
19
|
+
|
|
20
|
+
Then choose the fix from the patterns below.
|
|
21
|
+
|
|
22
|
+
---
|
|
23
|
+
|
|
24
|
+
## Fix Priority (try in order)
|
|
25
|
+
|
|
26
|
+
1. **Auth issue** — page redirected to login? Fix auth first, everything else is noise
|
|
27
|
+
2. **Element not found** — wrong name/type/value in selectors.yaml. Re-snapshot, copy exact name
|
|
28
|
+
3. **Multiple matches** — add `nth`, `exact: true`, or `scope` to narrow down
|
|
29
|
+
4. **Wrong assertion** — `toHaveText` vs `toHaveValue` mismatch, wrong expected data
|
|
30
|
+
5. **Timing** — SPA not loaded, async content. Add `wait for` step in .feature
|
|
31
|
+
|
|
32
|
+
---
|
|
33
|
+
|
|
34
|
+
## Playwright Errors
|
|
35
|
+
|
|
36
|
+
### Selector errors → fix in `selectors.yaml`
|
|
37
|
+
|
|
38
|
+
| Error | Diagnosis | Fix |
|
|
39
|
+
|---|---|---|
|
|
40
|
+
| strict mode violation | Multiple elements match | Add `nth: 0`, `exact: true`, or more specific `name` |
|
|
41
|
+
| Timeout / not found | Element doesn't exist or name wrong | Re-snapshot → copy exact accessible name. Check iframe/dialog scope |
|
|
42
|
+
| Element is not an input | Wrong element type targeted | Change `type` or `value` to match actual element |
|
|
43
|
+
| not a select | Custom dropdown, not native `<select>` | Set `variant: 'custom'` |
|
|
44
|
+
| Frame not found | iframe selector wrong or doesn't exist | Fix `frame` value, verify iframe in snapshot |
|
|
45
|
+
|
|
46
|
+
### Assertion errors → fix in `test-data.yaml` or `.feature`
|
|
47
|
+
|
|
48
|
+
| Error | Diagnosis | Fix |
|
|
49
|
+
|---|---|---|
|
|
50
|
+
| toHaveText mismatch | Expected text differs from actual | Fix value in test-data. If element is input type → change Gherkin type to `field`/`textarea` (triggers `toHaveValue` instead) |
|
|
51
|
+
| toHaveValue mismatch | Expected value differs from actual | Fix value in test-data |
|
|
52
|
+
| toContainText mismatch | Partial text not found | Fix expected partial text in test-data |
|
|
53
|
+
| toBeVisible timeout | Element exists but hidden, or name wrong | Check: is element conditionally visible? Wrong name? Inside dialog? |
|
|
54
|
+
| toHaveCount mismatch | Row count differs | Fix expected count in test-data. Verify: is table loaded? Filtered? |
|
|
41
55
|
|
|
42
|
-
|
|
56
|
+
### Assertion type rule
|
|
43
57
|
|
|
44
|
-
|
|
58
|
+
Sungen picks assertion based on element type:
|
|
59
|
+
- **Input** (`field`, `textarea`, `search`, `dropdown`, `slider`) → `toHaveValue()`
|
|
60
|
+
- **Text** (everything else: `message`, `heading`, `label`, `row`) → `toHaveText()`
|
|
61
|
+
- **Partial** (`contains` keyword) → `toContainText()`
|
|
45
62
|
|
|
46
|
-
|
|
63
|
+
If `toHaveText` fails on an input → the Gherkin step has wrong target type. Fix: change type in `.feature`.
|
|
47
64
|
|
|
48
|
-
|
|
65
|
+
---
|
|
66
|
+
|
|
67
|
+
## Table-Specific Errors
|
|
68
|
+
|
|
69
|
+
| Error | Diagnosis | Fix |
|
|
70
|
+
|---|---|---|
|
|
71
|
+
| `tableRow is not defined` | Column assertion without preceding row scope step | Add `User see [Ref] row in [Table] table with {{v}}` before `User see [Col] column with {{v}}` |
|
|
72
|
+
| `toHaveText` on cell fails (with columns) | Wrong column index in `columns` config | Re-count columns in snapshot (0-indexed). Fix `index` in selectors.yaml |
|
|
73
|
+
| `toBeVisible` on cell fails (no columns) | `filter({ hasText })` didn't match | Check exact cell text in snapshot. Fix value in test-data |
|
|
74
|
+
| Row filter matches 0 rows | Filter text doesn't match any row content | Re-snapshot → find actual row text. Fix filter value in test-data |
|
|
75
|
+
| Row filter matches multiple rows | Filter text is too generic (matches multiple rows) | Use more specific filter text (unique identifier like email, ID) |
|
|
76
|
+
| Table not found | Wrong table name or table not rendered | Re-snapshot → copy exact table accessible name |
|
|
77
|
+
|
|
78
|
+
---
|
|
79
|
+
|
|
80
|
+
## Auth Errors
|
|
81
|
+
|
|
82
|
+
| Symptom | Fix |
|
|
49
83
|
|---|---|
|
|
50
|
-
|
|
|
51
|
-
|
|
|
52
|
-
|
|
|
53
|
-
|
|
|
54
|
-
| Multiple session invalidation errors | Too many contexts created — ensure `contextCache.get(cacheKey)` returns existing context |
|
|
84
|
+
| Redirect to login page | Auth expired. Tell user: `sungen makeauth <role> --url <baseURL>` |
|
|
85
|
+
| `storageState` file not found | Run `sungen makeauth <role> --url <baseURL>` |
|
|
86
|
+
| Most tests timeout on first step | Auth expired — re-run makeauth |
|
|
87
|
+
| Page shows home instead of target | SPA + expired auth. Re-run makeauth + add `wait for` step |
|
|
55
88
|
|
|
56
|
-
**
|
|
57
|
-
- `base.ts` changes affect ALL tests — edit carefully
|
|
58
|
-
- Small project-specific tweaks are OK (e.g., custom overlay dismiss, timeout tuning, extra wait logic)
|
|
59
|
-
- Do NOT move individual selector/data fixes into `base.ts` — those belong in `selectors.yaml` / `test-data.yaml`
|
|
60
|
-
- If unsure whether a fix belongs in `base.ts` or selectors, ask the user
|
|
89
|
+
**Never run `sungen makeauth` yourself.** Tell the user.
|
|
61
90
|
|
|
62
|
-
|
|
91
|
+
---
|
|
92
|
+
|
|
93
|
+
## Sungen Compile Errors
|
|
63
94
|
|
|
64
95
|
| Error | Fix |
|
|
65
96
|
|---|---|
|
|
66
|
-
| Unknown step pattern |
|
|
67
|
-
| Missing selector |
|
|
68
|
-
| Missing variable |
|
|
69
|
-
| Invalid selector type |
|
|
97
|
+
| Unknown step pattern | Rewrite step to match `sungen-gherkin-syntax` patterns |
|
|
98
|
+
| Missing selector | Add key to `selectors.yaml` |
|
|
99
|
+
| Missing variable | Add key to `test-data.yaml` |
|
|
100
|
+
| Invalid selector type | Use: role/testid/placeholder/label/text/locator/page/upload/frame |
|
|
101
|
+
|
|
102
|
+
---
|
|
103
|
+
|
|
104
|
+
## Performance & Infrastructure Errors → Fix in `specs/base.ts`
|
|
105
|
+
|
|
106
|
+
All generated `.spec.ts` import from `specs/base.ts` — shared context caching, navigation, overlay cleanup. AI **can and should** tune `base.ts` to match the project.
|
|
107
|
+
|
|
108
|
+
| Symptom | Root cause | Fix |
|
|
109
|
+
|---|---|---|
|
|
110
|
+
| Server 429 (rate limited) | Too many browser contexts | Fix `contextCache` to reuse sessions per `storageState` |
|
|
111
|
+
| Tests slow with `--workers=1` | Redundant navigation | Fix goto patch: skip if `currentPath === url` |
|
|
112
|
+
| Previous test's modal blocks next | Overlay not cleaned up | Add/improve Escape + backdrop click in cleanup hook |
|
|
113
|
+
| All tests fail on first navigation | `page.url()` is `about:blank` | Add try/catch in goto patch |
|
|
114
|
+
| Flaky timeouts on SPA pages | Default timeout too short for app | Increase `actionTimeout` / `navigationTimeout` |
|
|
115
|
+
| Tests pass individually but fail in batch | Shared state leaking between tests | Isolate context per test or reset state in `beforeEach` |
|
|
116
|
+
|
|
117
|
+
### What AI CAN fix in `base.ts`
|
|
118
|
+
|
|
119
|
+
- Timeout tuning (increase for slow APIs, decrease for fast apps)
|
|
120
|
+
- Custom overlay/modal dismiss logic (project has unique close patterns)
|
|
121
|
+
- Navigation wait strategy (`networkidle` vs `domcontentloaded` vs custom)
|
|
122
|
+
- Context caching strategy (per-role, per-session)
|
|
123
|
+
- Cookie/localStorage cleanup between tests
|
|
124
|
+
|
|
125
|
+
### What AI should NOT move into `base.ts`
|
|
126
|
+
|
|
127
|
+
- Individual selector fixes → belong in `selectors.yaml`
|
|
128
|
+
- Test data values → belong in `test-data.yaml`
|
|
129
|
+
- Single-test workarounds → belong in the `.spec.ts` directly
|
|
130
|
+
|
|
131
|
+
---
|
|
132
|
+
|
|
133
|
+
## Smart Fix Strategy
|
|
134
|
+
|
|
135
|
+
### When multiple tests fail on the same element
|
|
136
|
+
Fix the **root cause once** — don't fix each test individually. Group by selector key, fix selectors.yaml, recompile, re-run batch.
|
|
137
|
+
|
|
138
|
+
### When you're unsure about the fix
|
|
139
|
+
**Re-snapshot** — `browser_navigate` to the failing page, take `browser_snapshot`, compare the live state against what the test expects. This is faster than guessing.
|
|
140
|
+
|
|
141
|
+
### When the fix might break other tests
|
|
142
|
+
After fixing selectors.yaml, run ALL tests (not just the failing one) to catch regressions from renamed/changed selectors.
|
|
143
|
+
|
|
144
|
+
### When nothing works after 3 attempts
|
|
145
|
+
Ask the user. The issue might be: dynamic content, race condition, environment-specific state, or a real application bug.
|
|
@@ -33,7 +33,6 @@ AND → inherits from preceding keyword
|
|
|
33
33
|
User is on [T] page # navigate to page
|
|
34
34
|
User is on [T] page with {{v}} # navigate with data (e.g., detail page with ID)
|
|
35
35
|
User is on [T] dialog # enter dialog scope
|
|
36
|
-
User is logged in | is not logged in # auth state
|
|
37
36
|
```
|
|
38
37
|
|
|
39
38
|
### Form
|
|
@@ -155,29 +154,16 @@ User see [T] page # URL assertion
|
|
|
155
154
|
### Table
|
|
156
155
|
|
|
157
156
|
```
|
|
158
|
-
User see [Col] column in [Table] table
|
|
159
|
-
User see [Table] table
|
|
160
|
-
User see [
|
|
161
|
-
User see [Table] table with {{count}}
|
|
162
|
-
User see [Table] table is empty
|
|
163
|
-
User see [
|
|
164
|
-
User
|
|
165
|
-
User click [Act] in [Table] table row with {{f}} # action in row
|
|
157
|
+
User see [Col] column in [Table] table # column exists
|
|
158
|
+
User see [Ref] row in [Table] table with {{v}} # row exists (enters row scope)
|
|
159
|
+
User see [Ref] row in [Table] table with {{v}} is hidden # row hidden
|
|
160
|
+
User see [Table] table with {{count}} # row count
|
|
161
|
+
User see [Table] table is empty # empty table
|
|
162
|
+
User see [Col] column with {{v}} # cell value (row scoped)
|
|
163
|
+
User click [Act] button in [Table] table with {{v}} # action in row
|
|
166
164
|
```
|
|
167
165
|
|
|
168
|
-
|
|
169
|
-
|
|
170
|
-
```
|
|
171
|
-
User click [Submit] button in [User Info] form # button inside specific form
|
|
172
|
-
User fill [Email] field in [Registration] form with {{v}} # field inside specific form
|
|
173
|
-
User see [Total] text in [Summary] section with {{v}} # text inside specific section
|
|
174
|
-
User click [Delete] button in [Active Users] table # button inside specific table
|
|
175
|
-
```
|
|
176
|
-
|
|
177
|
-
- **Optional** — only use when page has 2+ similar UI blocks
|
|
178
|
-
- **Valid parent types**: `table`, `list`, `section`, `dialog`, `form`
|
|
179
|
-
- **Max 2 levels**: `[Target] in [Parent]`. **NEVER** nest 3 levels: `[A] in [B] in [C]`
|
|
180
|
-
- Parent resolves from selectors YAML first, falls back to auto-infer `getByRole(parentType, { name })`
|
|
166
|
+
Row scope: `see [Ref] row in [Table] table with {{v}}` defines `tableRow`. Subsequent `see [Col] column with {{v}}` checks the cell in that row. With `columns` config → exact `nth(index)`, without → `filter({ hasText })`.
|
|
181
167
|
|
|
182
168
|
### States
|
|
183
169
|
|
|
@@ -196,24 +182,7 @@ User click [Delete] button in [Active Users] table # button inside specifi
|
|
|
196
182
|
|
|
197
183
|
### Auto-infer (no YAML entry needed)
|
|
198
184
|
|
|
199
|
-
|
|
200
|
-
|---|---|
|
|
201
|
-
| `[Submit] button` | `getByRole('button', { name: 'Submit' })` |
|
|
202
|
-
| `[Home] link` | `getByRole('link', { name: 'Home' })` |
|
|
203
|
-
| `[Welcome] heading` / `header` | `getByRole('heading', { name: 'Welcome' })` |
|
|
204
|
-
| `[Email] field` | `getByPlaceholder('Email')` |
|
|
205
|
-
| `[Success] text` / `message` / `label` | `getByText('Success')` |
|
|
206
|
-
| `[Terms] checkbox` | `getByRole('checkbox', { name: 'Terms' })` |
|
|
207
|
-
| `[Male] radio` | `getByRole('radio', { name: 'Male' })` |
|
|
208
|
-
| `[Global] search` | `getByRole('searchbox', { name: 'Global' })` |
|
|
209
|
-
| `[Vietnam] option` | `getByRole('option', { name: 'Vietnam' })` |
|
|
210
|
-
| `[Price] slider` | `getByRole('slider', { name: 'Price' })` |
|
|
211
|
-
| `[Notification] toggle` | `getByRole('switch', { name: 'Notification' })` |
|
|
212
|
-
| `[Profile] tab` | `getByRole('tab', { name: 'Profile' })` |
|
|
213
|
-
| `[Orders] table` | `getByRole('table', { name: 'Orders' })` |
|
|
214
|
-
| `[Products] list` | `getByRole('list', { name: 'Products' })` |
|
|
215
|
-
| `[Name] column` | `getByRole('columnheader', { name: 'Name' })` |
|
|
216
|
-
| `[Confirm] dialog` / `modal` / `drawer` | `getByRole('dialog', { name: 'Confirm' })` |
|
|
185
|
+
Most elements auto-infer from `[Label] type` → `getByRole(type, { name: 'Label' })`. Only add YAML when the accessible name differs, needs `nth`, or needs `testid`. Full auto-infer table → see `sungen-selector-keys` skill.
|
|
217
186
|
|
|
218
187
|
## YAML Keys
|
|
219
188
|
|
|
@@ -1,312 +1,140 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: sungen-selector-fix
|
|
3
|
-
description: 'Selector generation and fixing strategy
|
|
3
|
+
description: 'Selector generation and fixing strategy — explore live page, generate selectors.yaml, validate, auto-fix. Auto-loaded by run-test command.'
|
|
4
4
|
user-invocable: false
|
|
5
5
|
---
|
|
6
6
|
|
|
7
|
-
##
|
|
7
|
+
## When to Generate
|
|
8
8
|
|
|
9
|
-
|
|
10
|
-
|
|
11
|
-
Selectors are generated during `/sungen:make-test`, NOT during `/sungen:make-tc`. This ensures selectors are always validated against the live page.
|
|
9
|
+
Selectors are generated during `/sungen:run-test`, NOT during `/sungen:create-test`.
|
|
12
10
|
|
|
13
11
|
---
|
|
14
12
|
|
|
15
|
-
|
|
13
|
+
## Step 1: Authenticate & Snapshot
|
|
16
14
|
|
|
17
15
|
1. Read `baseURL` from `playwright.config.ts`
|
|
18
16
|
2. `browser_navigate` to `baseURL`
|
|
19
|
-
3. If redirected to login
|
|
20
|
-
|
|
21
|
-
4. Once confirmed, `browser_navigate` to the target page and take `browser_snapshot`
|
|
17
|
+
3. If redirected to login → ask user to log in manually via MCP browser
|
|
18
|
+
4. Navigate to target page → `browser_snapshot`
|
|
22
19
|
|
|
23
|
-
**Never use `sungen makeauth`.**
|
|
24
|
-
**Never use `browser_evaluate` to inject cookies or localStorage.** Use `browser_evaluate` only to read DOM info (e.g. detect `data-testid` attributes, check element properties).
|
|
20
|
+
**Never use `sungen makeauth`.** Never use `browser_evaluate` to inject cookies.
|
|
25
21
|
|
|
26
22
|
---
|
|
27
23
|
|
|
28
|
-
|
|
29
|
-
|
|
30
|
-
**Never guess or infer selector names from Gherkin labels. Always copy exact values from the snapshot.**
|
|
24
|
+
## Step 2: Build Element Catalog
|
|
31
25
|
|
|
32
|
-
|
|
26
|
+
**Never guess names from Gherkin labels. Copy exact values from snapshot.**
|
|
33
27
|
|
|
34
|
-
|
|
35
|
-
- **Links**: list each `link "..."` with its exact accessible name
|
|
36
|
-
- **Headings**: list each `heading "..."` with level and exact name
|
|
37
|
-
- **Inputs**: list each `textbox`, `searchbox`, `combobox` — note if it has an accessible name or not
|
|
38
|
-
- **Regions/landmarks**: `region "..."`, `navigation "..."`, `dialog "..."`
|
|
39
|
-
- **Images**: `img "..."` with accessible name (alt text)
|
|
28
|
+
Catalog all accessible elements from snapshot: buttons, links, headings, inputs, regions, images.
|
|
40
29
|
|
|
41
|
-
Then check for `data-testid` attributes
|
|
30
|
+
Then check for `data-testid` attributes:
|
|
42
31
|
```js
|
|
43
|
-
// browser_evaluate — read-only, safe:
|
|
44
32
|
Array.from(document.querySelectorAll('[data-testid]'))
|
|
45
33
|
.map(e => ({ testid: e.dataset.testid, tag: e.tagName, text: e.textContent.trim().slice(0, 60) }))
|
|
46
34
|
```
|
|
47
35
|
|
|
48
36
|
---
|
|
49
37
|
|
|
50
|
-
|
|
51
|
-
|
|
52
|
-
For each `[Reference]` in the `.feature` file, find the best match in the element catalog.
|
|
38
|
+
## Step 3: Map Gherkin Labels → Selectors
|
|
53
39
|
|
|
54
|
-
|
|
40
|
+
Selector priority (use first applicable):
|
|
55
41
|
|
|
56
|
-
| Priority | type | When
|
|
42
|
+
| Priority | type | When |
|
|
57
43
|
|---|---|---|
|
|
58
|
-
| 1 | `testid` | `data-testid`
|
|
59
|
-
| 2 | `role` + exact name | Interactive elements
|
|
60
|
-
| 3 | `placeholder` | Input with
|
|
61
|
-
| 4 | `label` | Form field with `<label>`
|
|
62
|
-
| 5 | `locator` (CSS
|
|
63
|
-
| 6 | `text` | Static text only
|
|
44
|
+
| 1 | `testid` | `data-testid` exists |
|
|
45
|
+
| 2 | `role` + exact name | Interactive elements |
|
|
46
|
+
| 3 | `placeholder` | Input with placeholder |
|
|
47
|
+
| 4 | `label` | Form field with `<label>` |
|
|
48
|
+
| 5 | `locator` (CSS) | No accessible name |
|
|
49
|
+
| 6 | `text` | Static text only |
|
|
64
50
|
|
|
65
|
-
|
|
66
|
-
|
|
67
|
-
Copy the accessible name **character-for-character** from the snapshot. Do not transform, translate, or infer from the Gherkin label.
|
|
68
|
-
|
|
69
|
-
```yaml
|
|
70
|
-
# BAD — name inferred from Gherkin label, not from snapshot
|
|
71
|
-
notification bell:
|
|
72
|
-
type: 'role'
|
|
73
|
-
value: 'button'
|
|
74
|
-
name: 'Notification Bell'
|
|
75
|
-
|
|
76
|
-
# GOOD — exact name from snapshot: button "Notifications"
|
|
77
|
-
notification bell:
|
|
78
|
-
type: 'role'
|
|
79
|
-
value: 'button'
|
|
80
|
-
name: 'Notifications'
|
|
81
|
-
```
|
|
82
|
-
|
|
83
|
-
#### When there is no accessible name
|
|
84
|
-
|
|
85
|
-
Use a `locator` with a CSS attribute selector or ARIA role:
|
|
51
|
+
**Exact name rule**: copy name character-for-character from snapshot. Never infer from Gherkin label.
|
|
86
52
|
|
|
53
|
+
**No accessible name**: use `locator` with CSS/ARIA attribute:
|
|
87
54
|
```yaml
|
|
88
55
|
search field:
|
|
89
56
|
type: 'locator'
|
|
90
57
|
value: '[role="searchbox"]'
|
|
91
|
-
|
|
92
|
-
icon button:
|
|
93
|
-
type: 'locator'
|
|
94
|
-
value: '[aria-label="Close"]'
|
|
95
58
|
```
|
|
96
59
|
|
|
97
|
-
|
|
98
|
-
|
|
99
|
-
### Step 4: Handle Dynamic Content
|
|
60
|
+
**Dynamic content**: never use `type: text` with specific value. Use `testid` or structural `role` + `nth`.
|
|
100
61
|
|
|
101
|
-
|
|
62
|
+
**Multiple matches**: add `nth: 0` for first occurrence.
|
|
102
63
|
|
|
103
|
-
|
|
104
|
-
|
|
105
|
-
```yaml
|
|
106
|
-
# BAD — specific text breaks when data changes
|
|
107
|
-
card title:
|
|
108
|
-
type: 'text'
|
|
109
|
-
value: 'Some specific title'
|
|
110
|
-
|
|
111
|
-
# GOOD — structural: first heading inside a card container
|
|
112
|
-
card title:
|
|
113
|
-
type: 'role'
|
|
114
|
-
value: 'heading'
|
|
115
|
-
nth: 1 # skip page-level headings, get first card heading
|
|
116
|
-
|
|
117
|
-
# BEST — if the app has data-testid
|
|
118
|
-
card title:
|
|
119
|
-
type: 'testid'
|
|
120
|
-
value: 'card-title'
|
|
121
|
-
```
|
|
122
|
-
|
|
123
|
-
To determine the correct `nth` offset, count how many matching elements appear before the target in the snapshot (e.g. how many headings come before the first card heading).
|
|
64
|
+
Auto-infer rules and lookup priority → see `sungen-selector-keys` skill.
|
|
124
65
|
|
|
125
66
|
---
|
|
126
67
|
|
|
127
|
-
|
|
68
|
+
## Step 4: Table Selectors
|
|
128
69
|
|
|
129
|
-
For
|
|
130
|
-
1. Navigate to the **list page** first via MCP browser to find a real record ID
|
|
131
|
-
2. Use that ID in the page selector value
|
|
132
|
-
3. Use `User is on [X] page` — sungen resolves the path from the selector
|
|
70
|
+
For table patterns using the v2.3 syntax, generate table selectors with `columns` config:
|
|
133
71
|
|
|
134
72
|
```yaml
|
|
135
|
-
|
|
136
|
-
|
|
137
|
-
|
|
138
|
-
|
|
73
|
+
users:
|
|
74
|
+
type: 'role'
|
|
75
|
+
value: 'table'
|
|
76
|
+
name: 'Users'
|
|
77
|
+
columns:
|
|
78
|
+
username:
|
|
79
|
+
index: 0
|
|
80
|
+
header: 'Username'
|
|
81
|
+
email:
|
|
82
|
+
index: 1
|
|
83
|
+
header: 'Email'
|
|
84
|
+
status:
|
|
85
|
+
index: 2
|
|
86
|
+
header: 'Status'
|
|
139
87
|
```
|
|
140
88
|
|
|
141
|
-
|
|
142
|
-
|
|
143
|
-
---
|
|
144
|
-
|
|
145
|
-
### Step 5: Handle SPA / Client-side Routing
|
|
146
|
-
|
|
147
|
-
Many modern apps (Next.js, Nuxt, SvelteKit, etc.) return the same HTML shell for all URLs and route client-side. `domcontentloaded` fires on the shell before the target page renders.
|
|
89
|
+
**How to build `columns`**: count column headers in snapshot (left to right, 0-indexed). Map each `[Col] column` reference from feature file to its index.
|
|
148
90
|
|
|
149
|
-
**
|
|
150
|
-
|
|
151
|
-
**Fix in the feature file**: Add an explicit assertion after navigation to confirm the correct page has loaded before proceeding:
|
|
91
|
+
**Why**: enables exact cell assertion via `cell.nth(index).toHaveText(value)` instead of approximate `cell.filter({ hasText })`.
|
|
152
92
|
|
|
93
|
+
**Row scope Gherkin** — these patterns use the table selector:
|
|
153
94
|
```gherkin
|
|
154
|
-
|
|
155
|
-
|
|
156
|
-
|
|
157
|
-
When User click [Some Button] button
|
|
158
|
-
...
|
|
95
|
+
Then User see [Username] row in [Users] table with {{user_name}}
|
|
96
|
+
Then User see [Status] column with {{expected_status}}
|
|
97
|
+
When User click [Edit] button in [Users] table with {{user_name}}
|
|
159
98
|
```
|
|
160
99
|
|
|
161
|
-
The "page loaded" check should use a **stable, always-visible element** on that specific page — typically the main page heading.
|
|
162
|
-
|
|
163
100
|
---
|
|
164
101
|
|
|
165
|
-
|
|
102
|
+
## Step 5: Detail Screens with Dynamic IDs
|
|
166
103
|
|
|
167
|
-
|
|
104
|
+
For screens like `/admin/users/:id`:
|
|
105
|
+
1. Navigate to list page via MCP to find a real record ID
|
|
106
|
+
2. Hardcode the ID in page selector
|
|
168
107
|
|
|
169
108
|
```yaml
|
|
170
|
-
|
|
171
|
-
type: '
|
|
172
|
-
value: '
|
|
173
|
-
name: 'Dashboard'
|
|
174
|
-
nth: 0 # first occurrence (desktop nav)
|
|
109
|
+
user detail:
|
|
110
|
+
type: 'page'
|
|
111
|
+
value: '/admin/users/de42d800-0f5a-490e-9dcf-344fedbd34a5'
|
|
175
112
|
```
|
|
176
113
|
|
|
177
|
-
Use `nth: 0` for the first match of dynamic/repeated elements.
|
|
178
|
-
|
|
179
|
-
---
|
|
180
|
-
|
|
181
|
-
### Auto-Infer (skip YAML entry)
|
|
182
|
-
|
|
183
|
-
Many elements don't need a YAML entry — sungen auto-infers from the Gherkin label:
|
|
184
|
-
|
|
185
|
-
| Gherkin | Auto-inferred locator |
|
|
186
|
-
|---|---|
|
|
187
|
-
| `[Submit] button` | `getByRole('button', { name: 'Submit' })` |
|
|
188
|
-
| `[Home] link` | `getByRole('link', { name: 'Home' })` |
|
|
189
|
-
| `[Welcome] heading` / `header` | `getByRole('heading', { name: 'Welcome' })` |
|
|
190
|
-
| `[Email] field` | `getByPlaceholder('Email')` |
|
|
191
|
-
| `[Success] text` / `message` / `label` | `getByText('Success')` |
|
|
192
|
-
| `[Terms] checkbox` | `getByRole('checkbox', { name: 'Terms' })` |
|
|
193
|
-
| `[Global] search` | `getByRole('searchbox', { name: 'Global' })` |
|
|
194
|
-
| `[Vietnam] option` | `getByRole('option', { name: 'Vietnam' })` |
|
|
195
|
-
| `[Price] slider` | `getByRole('slider', { name: 'Price' })` |
|
|
196
|
-
| `[Notification] toggle` | `getByRole('switch', { name: 'Notification' })` |
|
|
197
|
-
| `[Profile] tab` | `getByRole('tab', { name: 'Profile' })` |
|
|
198
|
-
| `[Confirm] dialog` / `modal` / `drawer` | `getByRole('dialog', { name: 'Confirm' })` |
|
|
199
|
-
|
|
200
|
-
**Only add a YAML entry when auto-infer won't work:**
|
|
201
|
-
- The accessible name differs from the Gherkin label
|
|
202
|
-
- Needs `nth` to disambiguate
|
|
203
|
-
- Needs `testid` or CSS locator
|
|
204
|
-
- Is dynamic content (needs structural selector)
|
|
205
|
-
- Is a bare input with no label or placeholder
|
|
206
|
-
|
|
207
114
|
---
|
|
208
115
|
|
|
209
|
-
|
|
210
|
-
|
|
211
|
-
**Most failures are selector mismatches.** Validate selectors against the live page BEFORE running any test — this eliminates slow compile→run→read→fix cycles.
|
|
212
|
-
|
|
213
|
-
After generating `selectors.yaml` (Step 3), verify each entry:
|
|
214
|
-
|
|
215
|
-
#### How to validate
|
|
216
|
-
|
|
217
|
-
Use `browser_evaluate` to check if each selector actually finds an element on the page:
|
|
218
|
-
|
|
219
|
-
```js
|
|
220
|
-
// Validate a role selector
|
|
221
|
-
document.querySelectorAll('[role="button"]').length;
|
|
222
|
-
// or check accessible name exists
|
|
223
|
-
Array.from(document.querySelectorAll('button'))
|
|
224
|
-
.filter(el => el.textContent.includes('Submit') || el.getAttribute('aria-label')?.includes('Submit'))
|
|
225
|
-
.length;
|
|
226
|
-
```
|
|
227
|
-
|
|
228
|
-
Or use `browser_snapshot` and cross-check:
|
|
229
|
-
1. Read all `[Reference]` entries from `selectors.yaml`
|
|
230
|
-
2. Take a `browser_snapshot`
|
|
231
|
-
3. For each entry, verify:
|
|
232
|
-
- **role + name**: does the snapshot contain `button "Submit"` or `link "Home"`?
|
|
233
|
-
- **testid**: does `browser_evaluate` find `[data-testid="xxx"]`?
|
|
234
|
-
- **placeholder**: does the snapshot contain a textbox with that placeholder?
|
|
235
|
-
- **locator**: does `browser_evaluate` find `document.querySelector('xxx')`?
|
|
236
|
-
4. Fix mismatches immediately — no need to run tests
|
|
116
|
+
## Step 6: Validate Before Running
|
|
237
117
|
|
|
238
|
-
|
|
118
|
+
**Fix 80%+ of selector issues before first test run.**
|
|
239
119
|
|
|
240
|
-
|
|
241
|
-
|
|
242
|
-
|
|
243
|
-
|
|
244
|
-
|
|
245
|
-
| `locator` (CSS) | `browser_evaluate`: `document.querySelector('xxx')` |
|
|
246
|
-
| `page` | Verify URL path exists (navigate and check) |
|
|
120
|
+
After generating `selectors.yaml`, verify each entry against the live snapshot:
|
|
121
|
+
- `role` + `name` → search snapshot for `role "name"`
|
|
122
|
+
- `testid` → `browser_evaluate`: `document.querySelector('[data-testid="xxx"]')`
|
|
123
|
+
- `placeholder` → search snapshot for textbox with placeholder
|
|
124
|
+
- `page` → verify URL path exists
|
|
247
125
|
|
|
248
|
-
|
|
249
|
-
|
|
250
|
-
- Name mismatch → copy exact name from snapshot
|
|
251
|
-
- Element not found → check if element is inside iframe/dialog (needs `frame` or scope)
|
|
252
|
-
- Multiple matches → add `nth: 0` or `exact: true`
|
|
253
|
-
- No accessible name → switch to `testid` or `locator`
|
|
254
|
-
|
|
255
|
-
**Target: fix 80%+ of selector issues before the first test run.**
|
|
126
|
+
Common fixes: name mismatch → copy from snapshot, element in iframe → add `frame`, multiple matches → add `nth` or `exact: true`.
|
|
256
127
|
|
|
257
128
|
---
|
|
258
129
|
|
|
259
|
-
|
|
260
|
-
|
|
261
|
-
After proactive validation, run tests in **batches of 20** for faster feedback:
|
|
262
|
-
|
|
263
|
-
```
|
|
264
|
-
1. COMPILE — sungen generate --screen <screen>
|
|
265
|
-
|
|
266
|
-
2. BATCH RUN — run 20 tests at a time:
|
|
267
|
-
npx playwright test <spec> --grep "VP-UI-001|VP-UI-002|...|VP-UI-020" --reporter=line
|
|
268
|
-
|
|
269
|
-
3. IF FAILURES in batch:
|
|
270
|
-
a. Group failures by root cause (same selector, same error type)
|
|
271
|
-
b. Fix selectors.yaml or test-data.yaml
|
|
272
|
-
c. Recompile: sungen generate --screen <screen>
|
|
273
|
-
d. Re-run ONLY the failing tests from this batch
|
|
274
|
-
e. If fixed → move to next batch
|
|
275
|
-
|
|
276
|
-
4. NEXT BATCH — repeat with next 20 tests (VP-UI-021...VP-UI-040)
|
|
130
|
+
## Step 7: Run & Fix Loop
|
|
277
131
|
|
|
278
|
-
|
|
279
|
-
npx playwright test <spec> --reporter=line
|
|
280
|
-
This catches regressions from selector changes.
|
|
132
|
+
Run tests in batches of 20. Group failures by root cause (same selector, same error type). Fix once, verify batch, move to next.
|
|
281
133
|
|
|
282
|
-
6. If still failing after 5 fix attempts per batch → ask user about direct .spec.ts fix
|
|
283
134
|
```
|
|
284
|
-
|
|
285
|
-
#### Building batch `--grep` patterns
|
|
286
|
-
|
|
287
|
-
Extract scenario names and batch them:
|
|
288
|
-
```bash
|
|
289
|
-
# Batch 1: first 20
|
|
290
|
-
npx playwright test <spec> --grep "VP-UI-001|VP-UI-002|...|VP-UI-020" --reporter=line
|
|
291
|
-
|
|
292
|
-
# Batch 2: next 20
|
|
293
|
-
npx playwright test <spec> --grep "VP-VAL-001|VP-VAL-002|...|VP-VAL-020" --reporter=line
|
|
135
|
+
compile → batch run 20 → fix root cause → recompile → re-run failures → next batch
|
|
294
136
|
```
|
|
295
137
|
|
|
296
|
-
|
|
297
|
-
|
|
298
|
-
Common patterns where 1 fix resolves many failures:
|
|
299
|
-
- **Same selector** — e.g., all `[Email Error]` tests fail → fix `email error` in selectors.yaml once
|
|
300
|
-
- **Same error type** — e.g., all `strict mode violation` → add `exact: true` or `nth`
|
|
301
|
-
- **Same assertion** — e.g., all `toHaveText` on inputs fail → change Gherkin pattern
|
|
302
|
-
|
|
303
|
-
Fix the root cause first, verify with the batch, then move on.
|
|
304
|
-
|
|
305
|
-
---
|
|
306
|
-
|
|
307
|
-
### Key Rules (from sungen-selector-keys)
|
|
138
|
+
After all batches pass → run ALL tests once for regression check.
|
|
308
139
|
|
|
309
|
-
|
|
310
|
-
- Keys use **spaces** (not dots) as word separators
|
|
311
|
-
- Same label, different element types → add `--type` suffix
|
|
312
|
-
- Same label, nth occurrence → add `--N` suffix
|
|
140
|
+
If still failing after 5 attempts per batch → ask user about direct `.spec.ts` fix.
|