@sun-asterisk/sungen 2.3.1 → 2.4.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (90) hide show
  1. package/README.md +10 -11
  2. package/dist/cli/index.js +1 -1
  3. package/dist/generators/test-generator/adapters/playwright/templates/steps/actions/table-action-in-row.hbs +1 -1
  4. package/dist/generators/test-generator/adapters/playwright/templates/steps/assertions/table-column-exists.hbs +5 -1
  5. package/dist/generators/test-generator/adapters/playwright/templates/steps/assertions/table-row-exists.hbs +7 -1
  6. package/dist/generators/test-generator/adapters/playwright/templates/steps/assertions/table-row-not-exists.hbs +5 -1
  7. package/dist/generators/test-generator/patterns/table-patterns.d.ts +9 -0
  8. package/dist/generators/test-generator/patterns/table-patterns.d.ts.map +1 -1
  9. package/dist/generators/test-generator/patterns/table-patterns.js +94 -98
  10. package/dist/generators/test-generator/patterns/table-patterns.js.map +1 -1
  11. package/dist/generators/test-generator/patterns/types.d.ts +2 -0
  12. package/dist/generators/test-generator/patterns/types.d.ts.map +1 -1
  13. package/dist/generators/test-generator/step-mapper.d.ts +13 -0
  14. package/dist/generators/test-generator/step-mapper.d.ts.map +1 -1
  15. package/dist/generators/test-generator/step-mapper.js +80 -0
  16. package/dist/generators/test-generator/step-mapper.js.map +1 -1
  17. package/dist/orchestrator/ai-rules-updater.d.ts.map +1 -1
  18. package/dist/orchestrator/ai-rules-updater.js +8 -6
  19. package/dist/orchestrator/ai-rules-updater.js.map +1 -1
  20. package/dist/orchestrator/project-initializer.d.ts.map +1 -1
  21. package/dist/orchestrator/project-initializer.js +33 -2
  22. package/dist/orchestrator/project-initializer.js.map +1 -1
  23. package/dist/orchestrator/screen-manager.js +1 -1
  24. package/dist/orchestrator/screen-manager.js.map +1 -1
  25. package/dist/orchestrator/templates/ai-instructions/claude-cmd-add-screen.md +5 -5
  26. package/{src/orchestrator/templates/ai-instructions/claude-cmd-make-tc.md → dist/orchestrator/templates/ai-instructions/claude-cmd-create-test.md} +7 -7
  27. package/dist/orchestrator/templates/ai-instructions/claude-cmd-review.md +21 -0
  28. package/{src/orchestrator/templates/ai-instructions/claude-cmd-make-test.md → dist/orchestrator/templates/ai-instructions/claude-cmd-run-test.md} +2 -2
  29. package/dist/orchestrator/templates/ai-instructions/claude-config.md +65 -12
  30. package/dist/orchestrator/templates/ai-instructions/claude-skill-error-mapping.md +128 -52
  31. package/dist/orchestrator/templates/ai-instructions/claude-skill-gherkin-syntax.md +9 -40
  32. package/dist/orchestrator/templates/ai-instructions/claude-skill-selector-fix.md +67 -239
  33. package/dist/orchestrator/templates/ai-instructions/claude-skill-tc-generation.md +40 -208
  34. package/dist/orchestrator/templates/ai-instructions/claude-skill-tc-review.md +104 -0
  35. package/dist/orchestrator/templates/ai-instructions/claude-skill-viewpoint.md +186 -0
  36. package/dist/orchestrator/templates/ai-instructions/copilot-cmd-add-screen.md +4 -4
  37. package/dist/orchestrator/templates/ai-instructions/{copilot-cmd-make-tc.md → copilot-cmd-create-test.md} +8 -8
  38. package/dist/orchestrator/templates/ai-instructions/copilot-cmd-review.md +24 -0
  39. package/{src/orchestrator/templates/ai-instructions/copilot-cmd-make-test.md → dist/orchestrator/templates/ai-instructions/copilot-cmd-run-test.md} +3 -3
  40. package/dist/orchestrator/templates/ai-instructions/copilot-config.md +66 -13
  41. package/dist/orchestrator/templates/ai-instructions/github-skill-sungen-error-mapping.md +128 -52
  42. package/dist/orchestrator/templates/ai-instructions/github-skill-sungen-gherkin-syntax.md +9 -40
  43. package/dist/orchestrator/templates/ai-instructions/github-skill-sungen-selector-fix.md +67 -214
  44. package/dist/orchestrator/templates/ai-instructions/github-skill-sungen-tc-generation.md +40 -208
  45. package/dist/orchestrator/templates/ai-instructions/github-skill-sungen-tc-review.md +104 -0
  46. package/dist/orchestrator/templates/ai-instructions/github-skill-sungen-viewpoint.md +186 -0
  47. package/dist/orchestrator/templates/readme.md +4 -4
  48. package/package.json +1 -1
  49. package/src/cli/index.ts +1 -1
  50. package/src/generators/test-generator/adapters/playwright/templates/steps/actions/table-action-in-row.hbs +1 -1
  51. package/src/generators/test-generator/adapters/playwright/templates/steps/assertions/table-column-exists.hbs +5 -1
  52. package/src/generators/test-generator/adapters/playwright/templates/steps/assertions/table-row-exists.hbs +7 -1
  53. package/src/generators/test-generator/adapters/playwright/templates/steps/assertions/table-row-not-exists.hbs +5 -1
  54. package/src/generators/test-generator/patterns/table-patterns.ts +98 -111
  55. package/src/generators/test-generator/patterns/types.ts +2 -0
  56. package/src/generators/test-generator/step-mapper.ts +87 -1
  57. package/src/orchestrator/ai-rules-updater.ts +8 -6
  58. package/src/orchestrator/project-initializer.ts +38 -2
  59. package/src/orchestrator/screen-manager.ts +1 -1
  60. package/src/orchestrator/templates/ai-instructions/claude-cmd-add-screen.md +5 -5
  61. package/{dist/orchestrator/templates/ai-instructions/claude-cmd-make-tc.md → src/orchestrator/templates/ai-instructions/claude-cmd-create-test.md} +7 -7
  62. package/src/orchestrator/templates/ai-instructions/claude-cmd-review.md +21 -0
  63. package/{dist/orchestrator/templates/ai-instructions/claude-cmd-make-test.md → src/orchestrator/templates/ai-instructions/claude-cmd-run-test.md} +2 -2
  64. package/src/orchestrator/templates/ai-instructions/claude-config.md +65 -12
  65. package/src/orchestrator/templates/ai-instructions/claude-skill-error-mapping.md +128 -52
  66. package/src/orchestrator/templates/ai-instructions/claude-skill-gherkin-syntax.md +9 -40
  67. package/src/orchestrator/templates/ai-instructions/claude-skill-selector-fix.md +67 -239
  68. package/src/orchestrator/templates/ai-instructions/claude-skill-tc-generation.md +40 -208
  69. package/src/orchestrator/templates/ai-instructions/claude-skill-tc-review.md +104 -0
  70. package/src/orchestrator/templates/ai-instructions/claude-skill-viewpoint.md +186 -0
  71. package/src/orchestrator/templates/ai-instructions/copilot-cmd-add-screen.md +4 -4
  72. package/src/orchestrator/templates/ai-instructions/{copilot-cmd-make-tc.md → copilot-cmd-create-test.md} +8 -8
  73. package/src/orchestrator/templates/ai-instructions/copilot-cmd-review.md +24 -0
  74. package/{dist/orchestrator/templates/ai-instructions/copilot-cmd-make-test.md → src/orchestrator/templates/ai-instructions/copilot-cmd-run-test.md} +3 -3
  75. package/src/orchestrator/templates/ai-instructions/copilot-config.md +66 -13
  76. package/src/orchestrator/templates/ai-instructions/github-skill-sungen-error-mapping.md +128 -52
  77. package/src/orchestrator/templates/ai-instructions/github-skill-sungen-gherkin-syntax.md +9 -40
  78. package/src/orchestrator/templates/ai-instructions/github-skill-sungen-selector-fix.md +67 -214
  79. package/src/orchestrator/templates/ai-instructions/github-skill-sungen-tc-generation.md +40 -208
  80. package/src/orchestrator/templates/ai-instructions/github-skill-sungen-tc-review.md +104 -0
  81. package/src/orchestrator/templates/ai-instructions/github-skill-sungen-viewpoint.md +186 -0
  82. package/src/orchestrator/templates/readme.md +4 -4
  83. package/dist/generators/test-generator/adapters/playwright/templates/steps/assertions/table-cell-by-filter.hbs +0 -2
  84. package/dist/generators/test-generator/adapters/playwright/templates/steps/assertions/table-cell-by-index.hbs +0 -2
  85. package/dist/orchestrator/templates/ai-instructions/claude-skill-gherkin-review.md +0 -228
  86. package/dist/orchestrator/templates/ai-instructions/github-skill-sungen-gherkin-review.md +0 -228
  87. package/src/generators/test-generator/adapters/playwright/templates/steps/assertions/table-cell-by-filter.hbs +0 -2
  88. package/src/generators/test-generator/adapters/playwright/templates/steps/assertions/table-cell-by-index.hbs +0 -2
  89. package/src/orchestrator/templates/ai-instructions/claude-skill-gherkin-review.md +0 -228
  90. package/src/orchestrator/templates/ai-instructions/github-skill-sungen-gherkin-review.md +0 -228
@@ -1,69 +1,145 @@
1
1
  ---
2
2
  name: sungen-error-mapping
3
- description: 'Playwright and Sungen error to fix mapping. Auto-loaded when running or debugging tests.'
3
+ description: 'Error diagnosis and fix strategy — systematic debugging flow, error patterns, fix priorities. Auto-loaded during run-test.'
4
4
  user-invocable: false
5
5
  ---
6
6
 
7
- ## Playwright Errors Fix
7
+ ## Diagnosis Flow (before fixing anything)
8
8
 
9
- | Error | Fix |
10
- |---|---|
11
- | strict mode violation | add `nth`, `exact: true`, or specific `name` in selectors.yaml |
12
- | Element is not an input | change `type` or `value` in selectors.yaml |
13
- | Timeout waiting for selector | fix `type`/`value`/`name` to match element in selectors.yaml |
14
- | toBeVisible Timeout | verify accessible name or use `testid` in selectors.yaml |
15
- | toHaveText mismatch | fix expected text in `test-data.yaml` — OR if element is an input (field/textarea/search/dropdown), change Gherkin type so compiler uses `toHaveValue` instead |
16
- | toHaveValue mismatch | fix expected value in `test-data.yaml` — this is for input elements (field, textarea, search, dropdown, slider, date-picker) |
17
- | toContainText mismatch | fix expected partial text in `test-data.yaml` |
18
- | Navigation failed | fix page `value` path in selectors.yaml |
19
- | not a select | set `variant: 'custom'` in selectors.yaml |
20
- | Frame not found | fix `frame` value in selectors.yaml |
21
- | dialog was dismissed | alert step is AFTER the trigger — move `click [OK] alert` BEFORE the button click that opens the dialog |
22
- | page.once('dialog') never fired | no browser dialog appeared — check if the action actually triggers a `window.alert/confirm/prompt`, not a CSS modal |
23
-
24
- ### Assertion type mismatch (toHaveText vs toHaveValue)
25
-
26
- Sungen picks the assertion based on element type:
27
- - **Input types** (`field`, `textarea`, `search`, `dropdown`, `slider`, `date-picker`) → `toHaveValue()`
28
- - **Text types** (everything else: `message`, `header`, `label`, `row`, etc.) → `toHaveText()`
29
- - **Partial match** (uses `contains` keyword) → `toContainText()`
30
-
31
- If you see `toHaveText` failing on an input element, the Gherkin step has the wrong target type. Fix: change the target type in the `.feature` file (e.g., `see [Email] message with {{v}}` → `see [Email] field with {{v}}`).
32
-
33
- ## Auth Errors → Fix
9
+ **Step 1: Read the error** — extract: error type, element name, expected vs actual, file:line.
34
10
 
35
- | Symptom | Fix |
36
- |---|---|
37
- | Tests redirect to login page | `specs/.auth/<role>.json` missing or expired. Tell user: `sungen makeauth <role> --url <baseURL>` |
38
- | `storageState` file not found | Run `sungen makeauth <role> --url <baseURL>` to create auth state |
39
- | Most tests timeout on first step | Auth expired — re-run `sungen makeauth <role> --url <baseURL>` |
40
- | Page shows home/login instead of target | SPA routing + expired auth. Re-run `sungen makeauth`, add `wait for` step after navigation |
11
+ **Step 2: Read error context** — check `test-results/` for the snapshot at failure time. This shows the exact page state when the test failed — more reliable than re-navigating with MCP.
12
+
13
+ **Step 3: Compare** match the failing selector against the snapshot. Ask:
14
+ - Does the element exist on the page?
15
+ - Does the accessible name match exactly?
16
+ - Are there multiple matches (strict mode)?
17
+ - Is the element inside an iframe or dialog?
18
+ - Is the page in the expected state (correct URL, loaded)?
19
+
20
+ Then choose the fix from the patterns below.
21
+
22
+ ---
23
+
24
+ ## Fix Priority (try in order)
25
+
26
+ 1. **Auth issue** — page redirected to login? Fix auth first, everything else is noise
27
+ 2. **Element not found** — wrong name/type/value in selectors.yaml. Re-snapshot, copy exact name
28
+ 3. **Multiple matches** — add `nth`, `exact: true`, or `scope` to narrow down
29
+ 4. **Wrong assertion** — `toHaveText` vs `toHaveValue` mismatch, wrong expected data
30
+ 5. **Timing** — SPA not loaded, async content. Add `wait for` step in .feature
31
+
32
+ ---
33
+
34
+ ## Playwright Errors
35
+
36
+ ### Selector errors → fix in `selectors.yaml`
37
+
38
+ | Error | Diagnosis | Fix |
39
+ |---|---|---|
40
+ | strict mode violation | Multiple elements match | Add `nth: 0`, `exact: true`, or more specific `name` |
41
+ | Timeout / not found | Element doesn't exist or name wrong | Re-snapshot → copy exact accessible name. Check iframe/dialog scope |
42
+ | Element is not an input | Wrong element type targeted | Change `type` or `value` to match actual element |
43
+ | not a select | Custom dropdown, not native `<select>` | Set `variant: 'custom'` |
44
+ | Frame not found | iframe selector wrong or doesn't exist | Fix `frame` value, verify iframe in snapshot |
45
+
46
+ ### Assertion errors → fix in `test-data.yaml` or `.feature`
47
+
48
+ | Error | Diagnosis | Fix |
49
+ |---|---|---|
50
+ | toHaveText mismatch | Expected text differs from actual | Fix value in test-data. If element is input type → change Gherkin type to `field`/`textarea` (triggers `toHaveValue` instead) |
51
+ | toHaveValue mismatch | Expected value differs from actual | Fix value in test-data |
52
+ | toContainText mismatch | Partial text not found | Fix expected partial text in test-data |
53
+ | toBeVisible timeout | Element exists but hidden, or name wrong | Check: is element conditionally visible? Wrong name? Inside dialog? |
54
+ | toHaveCount mismatch | Row count differs | Fix expected count in test-data. Verify: is table loaded? Filtered? |
41
55
 
42
- **Never run `sungen makeauth` yourself.** Always tell the user to run it manually.
56
+ ### Assertion type rule
43
57
 
44
- ## Performance Errors Fix (specs/base.ts)
58
+ Sungen picks assertion based on element type:
59
+ - **Input** (`field`, `textarea`, `search`, `dropdown`, `slider`) → `toHaveValue()`
60
+ - **Text** (everything else: `message`, `heading`, `label`, `row`) → `toHaveText()`
61
+ - **Partial** (`contains` keyword) → `toContainText()`
45
62
 
46
- All generated `.spec.ts` files import from `specs/base.ts`. This shared file handles context caching, navigation skip, and overlay cleanup.
63
+ If `toHaveText` fails on an input the Gherkin step has wrong target type. Fix: change type in `.feature`.
47
64
 
48
- | Symptom | Fix in `specs/base.ts` |
65
+ ---
66
+
67
+ ## Table-Specific Errors
68
+
69
+ | Error | Diagnosis | Fix |
70
+ |---|---|---|
71
+ | `tableRow is not defined` | Column assertion without preceding row scope step | Add `User see [Ref] row in [Table] table with {{v}}` before `User see [Col] column with {{v}}` |
72
+ | `toHaveText` on cell fails (with columns) | Wrong column index in `columns` config | Re-count columns in snapshot (0-indexed). Fix `index` in selectors.yaml |
73
+ | `toBeVisible` on cell fails (no columns) | `filter({ hasText })` didn't match | Check exact cell text in snapshot. Fix value in test-data |
74
+ | Row filter matches 0 rows | Filter text doesn't match any row content | Re-snapshot → find actual row text. Fix filter value in test-data |
75
+ | Row filter matches multiple rows | Filter text is too generic (matches multiple rows) | Use more specific filter text (unique identifier like email, ID) |
76
+ | Table not found | Wrong table name or table not rendered | Re-snapshot → copy exact table accessible name |
77
+
78
+ ---
79
+
80
+ ## Auth Errors
81
+
82
+ | Symptom | Fix |
49
83
  |---|---|
50
- | Server returns 429 (rate limited) | Context caching broken check `contextCache` Map reuses sessions per `storageState` |
51
- | Tests slow even with `--workers=1` | Navigation skip not working check `goto` patch compares `currentPath === url` |
52
- | Previous test's modal blocks next test | Overlay cleanup insufficient improve Escape/click dismiss logic |
53
- | All tests fail on first navigation | `page.url()` is `about:blank` check try/catch in goto patch |
54
- | Multiple session invalidation errors | Too many contexts created — ensure `contextCache.get(cacheKey)` returns existing context |
84
+ | Redirect to login page | Auth expired. Tell user: `sungen makeauth <role> --url <baseURL>` |
85
+ | `storageState` file not found | Run `sungen makeauth <role> --url <baseURL>` |
86
+ | Most tests timeout on first step | Auth expiredre-run makeauth |
87
+ | Page shows home instead of target | SPA + expired auth. Re-run makeauth + add `wait for` step |
55
88
 
56
- **Rules:**
57
- - `base.ts` changes affect ALL tests — edit carefully
58
- - Small project-specific tweaks are OK (e.g., custom overlay dismiss, timeout tuning, extra wait logic)
59
- - Do NOT move individual selector/data fixes into `base.ts` — those belong in `selectors.yaml` / `test-data.yaml`
60
- - If unsure whether a fix belongs in `base.ts` or selectors, ask the user
89
+ **Never run `sungen makeauth` yourself.** Tell the user.
61
90
 
62
- ## Sungen Errors → Fix
91
+ ---
92
+
93
+ ## Sungen Compile Errors
63
94
 
64
95
  | Error | Fix |
65
96
  |---|---|
66
- | Unknown step pattern | rewrite `.feature` step to match supported pattern |
67
- | Missing selector | add key to `selectors.yaml` |
68
- | Missing variable | add key to `test-data.yaml` |
69
- | Invalid selector type | use: role/testid/placeholder/label/text/locator/page/upload/frame |
97
+ | Unknown step pattern | Rewrite step to match `sungen-gherkin-syntax` patterns |
98
+ | Missing selector | Add key to `selectors.yaml` |
99
+ | Missing variable | Add key to `test-data.yaml` |
100
+ | Invalid selector type | Use: role/testid/placeholder/label/text/locator/page/upload/frame |
101
+
102
+ ---
103
+
104
+ ## Performance & Infrastructure Errors → Fix in `specs/base.ts`
105
+
106
+ All generated `.spec.ts` import from `specs/base.ts` — shared context caching, navigation, overlay cleanup. AI **can and should** tune `base.ts` to match the project.
107
+
108
+ | Symptom | Root cause | Fix |
109
+ |---|---|---|
110
+ | Server 429 (rate limited) | Too many browser contexts | Fix `contextCache` to reuse sessions per `storageState` |
111
+ | Tests slow with `--workers=1` | Redundant navigation | Fix goto patch: skip if `currentPath === url` |
112
+ | Previous test's modal blocks next | Overlay not cleaned up | Add/improve Escape + backdrop click in cleanup hook |
113
+ | All tests fail on first navigation | `page.url()` is `about:blank` | Add try/catch in goto patch |
114
+ | Flaky timeouts on SPA pages | Default timeout too short for app | Increase `actionTimeout` / `navigationTimeout` |
115
+ | Tests pass individually but fail in batch | Shared state leaking between tests | Isolate context per test or reset state in `beforeEach` |
116
+
117
+ ### What AI CAN fix in `base.ts`
118
+
119
+ - Timeout tuning (increase for slow APIs, decrease for fast apps)
120
+ - Custom overlay/modal dismiss logic (project has unique close patterns)
121
+ - Navigation wait strategy (`networkidle` vs `domcontentloaded` vs custom)
122
+ - Context caching strategy (per-role, per-session)
123
+ - Cookie/localStorage cleanup between tests
124
+
125
+ ### What AI should NOT move into `base.ts`
126
+
127
+ - Individual selector fixes → belong in `selectors.yaml`
128
+ - Test data values → belong in `test-data.yaml`
129
+ - Single-test workarounds → belong in the `.spec.ts` directly
130
+
131
+ ---
132
+
133
+ ## Smart Fix Strategy
134
+
135
+ ### When multiple tests fail on the same element
136
+ Fix the **root cause once** — don't fix each test individually. Group by selector key, fix selectors.yaml, recompile, re-run batch.
137
+
138
+ ### When you're unsure about the fix
139
+ **Re-snapshot** — `browser_navigate` to the failing page, take `browser_snapshot`, compare the live state against what the test expects. This is faster than guessing.
140
+
141
+ ### When the fix might break other tests
142
+ After fixing selectors.yaml, run ALL tests (not just the failing one) to catch regressions from renamed/changed selectors.
143
+
144
+ ### When nothing works after 3 attempts
145
+ Ask the user. The issue might be: dynamic content, race condition, environment-specific state, or a real application bug.
@@ -33,7 +33,6 @@ AND → inherits from preceding keyword
33
33
  User is on [T] page # navigate to page
34
34
  User is on [T] page with {{v}} # navigate with data (e.g., detail page with ID)
35
35
  User is on [T] dialog # enter dialog scope
36
- User is logged in | is not logged in # auth state
37
36
  ```
38
37
 
39
38
  ### Form
@@ -155,29 +154,16 @@ User see [T] page # URL assertion
155
154
  ### Table
156
155
 
157
156
  ```
158
- User see [Col] column in [Table] table # column exists (parent scoping)
159
- User see [Table] table row with {{f}} # row exists
160
- User see [Table] table has no row with {{f}} # row not exists
161
- User see [Table] table with {{count}} rows # row count
162
- User see [Table] table is empty # empty table
163
- User see [Table] table row with {{f}} has [Col] with {{v}} # cell by filter
164
- User see [Table] table row 1 [Col] cell with {{v}} # cell by index
165
- User click [Act] in [Table] table row with {{f}} # action in row
157
+ User see [Col] column in [Table] table # column exists
158
+ User see [Ref] row in [Table] table with {{v}} # row exists (enters row scope)
159
+ User see [Ref] row in [Table] table with {{v}} is hidden # row hidden
160
+ User see [Table] table with {{count}} # row count
161
+ User see [Table] table is empty # empty table
162
+ User see [Col] column with {{v}} # cell value (row scoped)
163
+ User click [Act] button in [Table] table with {{v}} # action in row
166
164
  ```
167
165
 
168
- ### Parent Scoping (disambiguation)
169
-
170
- ```
171
- User click [Submit] button in [User Info] form # button inside specific form
172
- User fill [Email] field in [Registration] form with {{v}} # field inside specific form
173
- User see [Total] text in [Summary] section with {{v}} # text inside specific section
174
- User click [Delete] button in [Active Users] table # button inside specific table
175
- ```
176
-
177
- - **Optional** — only use when page has 2+ similar UI blocks
178
- - **Valid parent types**: `table`, `list`, `section`, `dialog`, `form`
179
- - **Max 2 levels**: `[Target] in [Parent]`. **NEVER** nest 3 levels: `[A] in [B] in [C]`
180
- - Parent resolves from selectors YAML first, falls back to auto-infer `getByRole(parentType, { name })`
166
+ Row scope: `see [Ref] row in [Table] table with {{v}}` defines `tableRow`. Subsequent `see [Col] column with {{v}}` checks the cell in that row. With `columns` config → exact `nth(index)`, without → `filter({ hasText })`.
181
167
 
182
168
  ### States
183
169
 
@@ -196,24 +182,7 @@ User click [Delete] button in [Active Users] table # button inside specifi
196
182
 
197
183
  ### Auto-infer (no YAML entry needed)
198
184
 
199
- | Gherkin | Playwright locator |
200
- |---|---|
201
- | `[Submit] button` | `getByRole('button', { name: 'Submit' })` |
202
- | `[Home] link` | `getByRole('link', { name: 'Home' })` |
203
- | `[Welcome] heading` / `header` | `getByRole('heading', { name: 'Welcome' })` |
204
- | `[Email] field` | `getByPlaceholder('Email')` |
205
- | `[Success] text` / `message` / `label` | `getByText('Success')` |
206
- | `[Terms] checkbox` | `getByRole('checkbox', { name: 'Terms' })` |
207
- | `[Male] radio` | `getByRole('radio', { name: 'Male' })` |
208
- | `[Global] search` | `getByRole('searchbox', { name: 'Global' })` |
209
- | `[Vietnam] option` | `getByRole('option', { name: 'Vietnam' })` |
210
- | `[Price] slider` | `getByRole('slider', { name: 'Price' })` |
211
- | `[Notification] toggle` | `getByRole('switch', { name: 'Notification' })` |
212
- | `[Profile] tab` | `getByRole('tab', { name: 'Profile' })` |
213
- | `[Orders] table` | `getByRole('table', { name: 'Orders' })` |
214
- | `[Products] list` | `getByRole('list', { name: 'Products' })` |
215
- | `[Name] column` | `getByRole('columnheader', { name: 'Name' })` |
216
- | `[Confirm] dialog` / `modal` / `drawer` | `getByRole('dialog', { name: 'Confirm' })` |
185
+ Most elements auto-infer from `[Label] type` → `getByRole(type, { name: 'Label' })`. Only add YAML when the accessible name differs, needs `nth`, or needs `testid`. Full auto-infer table → see `sungen-selector-keys` skill.
217
186
 
218
187
  ## YAML Keys
219
188
 
@@ -1,312 +1,140 @@
1
1
  ---
2
2
  name: sungen-selector-fix
3
- description: 'Selector generation and fixing strategy for make-test explores live page, generates selectors.yaml, auto-fixes on test failure. Auto-loaded by make-test command.'
3
+ description: 'Selector generation and fixing strategy — explore live page, generate selectors.yaml, validate, auto-fix. Auto-loaded by run-test command.'
4
4
  user-invocable: false
5
5
  ---
6
6
 
7
- ## Selector Generation & Fix Strategy
7
+ ## When to Generate
8
8
 
9
- ### When to Generate Selectors
10
-
11
- Selectors are generated during `/sungen:make-test`, NOT during `/sungen:make-tc`. This ensures selectors are always validated against the live page.
9
+ Selectors are generated during `/sungen:run-test`, NOT during `/sungen:create-test`.
12
10
 
13
11
  ---
14
12
 
15
- ### Step 1: Authenticate
13
+ ## Step 1: Authenticate & Snapshot
16
14
 
17
15
  1. Read `baseURL` from `playwright.config.ts`
18
16
  2. `browser_navigate` to `baseURL`
19
- 3. If redirected to login, ask the user to log in manually:
20
- > "This page requires login. Please log in using the browser. Confirm when you're done."
21
- 4. Once confirmed, `browser_navigate` to the target page and take `browser_snapshot`
17
+ 3. If redirected to login ask user to log in manually via MCP browser
18
+ 4. Navigate to target page `browser_snapshot`
22
19
 
23
- **Never use `sungen makeauth`.** Always let the user log in manually via the MCP browser.
24
- **Never use `browser_evaluate` to inject cookies or localStorage.** Use `browser_evaluate` only to read DOM info (e.g. detect `data-testid` attributes, check element properties).
20
+ **Never use `sungen makeauth`.** Never use `browser_evaluate` to inject cookies.
25
21
 
26
22
  ---
27
23
 
28
- ### Step 2: Build an Element Catalog from the Snapshot
29
-
30
- **Never guess or infer selector names from Gherkin labels. Always copy exact values from the snapshot.**
24
+ ## Step 2: Build Element Catalog
31
25
 
32
- After taking the snapshot, mentally catalog ALL accessible elements:
26
+ **Never guess names from Gherkin labels. Copy exact values from snapshot.**
33
27
 
34
- - **Buttons**: list each `button "..."` with its exact accessible name
35
- - **Links**: list each `link "..."` with its exact accessible name
36
- - **Headings**: list each `heading "..."` with level and exact name
37
- - **Inputs**: list each `textbox`, `searchbox`, `combobox` — note if it has an accessible name or not
38
- - **Regions/landmarks**: `region "..."`, `navigation "..."`, `dialog "..."`
39
- - **Images**: `img "..."` with accessible name (alt text)
28
+ Catalog all accessible elements from snapshot: buttons, links, headings, inputs, regions, images.
40
29
 
41
- Then check for `data-testid` attributes (more stable than accessible names):
30
+ Then check for `data-testid` attributes:
42
31
  ```js
43
- // browser_evaluate — read-only, safe:
44
32
  Array.from(document.querySelectorAll('[data-testid]'))
45
33
  .map(e => ({ testid: e.dataset.testid, tag: e.tagName, text: e.textContent.trim().slice(0, 60) }))
46
34
  ```
47
35
 
48
36
  ---
49
37
 
50
- ### Step 3: Map Gherkin Labels → Selectors
51
-
52
- For each `[Reference]` in the `.feature` file, find the best match in the element catalog.
38
+ ## Step 3: Map Gherkin Labels → Selectors
53
39
 
54
- #### Selector priority (use first applicable)
40
+ Selector priority (use first applicable):
55
41
 
56
- | Priority | type | When to use |
42
+ | Priority | type | When |
57
43
  |---|---|---|
58
- | 1 | `testid` | `data-testid` attribute exists |
59
- | 2 | `role` + exact name | Interactive elements (button, link, heading) |
60
- | 3 | `placeholder` | Input with visible placeholder text |
61
- | 4 | `label` | Form field with `<label>` association |
62
- | 5 | `locator` (CSS / aria attr) | No accessible name (e.g. bare searchbox, icon button) |
63
- | 6 | `text` | Static text only — avoid for dynamic/data-driven content |
44
+ | 1 | `testid` | `data-testid` exists |
45
+ | 2 | `role` + exact name | Interactive elements |
46
+ | 3 | `placeholder` | Input with placeholder |
47
+ | 4 | `label` | Form field with `<label>` |
48
+ | 5 | `locator` (CSS) | No accessible name |
49
+ | 6 | `text` | Static text only |
64
50
 
65
- #### The exact name rule
66
-
67
- Copy the accessible name **character-for-character** from the snapshot. Do not transform, translate, or infer from the Gherkin label.
68
-
69
- ```yaml
70
- # BAD — name inferred from Gherkin label, not from snapshot
71
- notification bell:
72
- type: 'role'
73
- value: 'button'
74
- name: 'Notification Bell'
75
-
76
- # GOOD — exact name from snapshot: button "Notifications"
77
- notification bell:
78
- type: 'role'
79
- value: 'button'
80
- name: 'Notifications'
81
- ```
82
-
83
- #### When there is no accessible name
84
-
85
- Use a `locator` with a CSS attribute selector or ARIA role:
51
+ **Exact name rule**: copy name character-for-character from snapshot. Never infer from Gherkin label.
86
52
 
53
+ **No accessible name**: use `locator` with CSS/ARIA attribute:
87
54
  ```yaml
88
55
  search field:
89
56
  type: 'locator'
90
57
  value: '[role="searchbox"]'
91
-
92
- icon button:
93
- type: 'locator'
94
- value: '[aria-label="Close"]'
95
58
  ```
96
59
 
97
- ---
98
-
99
- ### Step 4: Handle Dynamic Content
60
+ **Dynamic content**: never use `type: text` with specific value. Use `testid` or structural `role` + `nth`.
100
61
 
101
- For elements whose text changes between test runs (list items, card data, user-generated content, timestamps):
62
+ **Multiple matches**: add `nth: 0` for first occurrence.
102
63
 
103
- **Never use `type: text` with a specific value** — it breaks when data changes.
104
-
105
- ```yaml
106
- # BAD — specific text breaks when data changes
107
- card title:
108
- type: 'text'
109
- value: 'Some specific title'
110
-
111
- # GOOD — structural: first heading inside a card container
112
- card title:
113
- type: 'role'
114
- value: 'heading'
115
- nth: 1 # skip page-level headings, get first card heading
116
-
117
- # BEST — if the app has data-testid
118
- card title:
119
- type: 'testid'
120
- value: 'card-title'
121
- ```
122
-
123
- To determine the correct `nth` offset, count how many matching elements appear before the target in the snapshot (e.g. how many headings come before the first card heading).
64
+ Auto-infer rules and lookup priority see `sungen-selector-keys` skill.
124
65
 
125
66
  ---
126
67
 
127
- ### Step 4b: Handle Detail Screens with Dynamic IDs
68
+ ## Step 4: Table Selectors
128
69
 
129
- For screens like `/admin/users/:id` or `/products/:slug`:
130
- 1. Navigate to the **list page** first via MCP browser to find a real record ID
131
- 2. Use that ID in the page selector value
132
- 3. Use `User is on [X] page` — sungen resolves the path from the selector
70
+ For table patterns using the v2.3 syntax, generate table selectors with `columns` config:
133
71
 
134
72
  ```yaml
135
- # selectors.yaml — full path with real ID
136
- user detail:
137
- type: 'page'
138
- value: '/admin/users/de42d800-0f5a-490e-9dcf-344fedbd34a5'
73
+ users:
74
+ type: 'role'
75
+ value: 'table'
76
+ name: 'Users'
77
+ columns:
78
+ username:
79
+ index: 0
80
+ header: 'Username'
81
+ email:
82
+ index: 1
83
+ header: 'Email'
84
+ status:
85
+ index: 2
86
+ header: 'Status'
139
87
  ```
140
88
 
141
- Note: the selector uses a hardcoded ID from the live page. If the record is deleted, update the ID in `selectors.yaml`.
142
-
143
- ---
144
-
145
- ### Step 5: Handle SPA / Client-side Routing
146
-
147
- Many modern apps (Next.js, Nuxt, SvelteKit, etc.) return the same HTML shell for all URLs and route client-side. `domcontentloaded` fires on the shell before the target page renders.
89
+ **How to build `columns`**: count column headers in snapshot (left to right, 0-indexed). Map each `[Col] column` reference from feature file to its index.
148
90
 
149
- **Symptom**: Tests see a different page than expected (e.g. home page instead of `/dashboard`).
150
-
151
- **Fix in the feature file**: Add an explicit assertion after navigation to confirm the correct page has loaded before proceeding:
91
+ **Why**: enables exact cell assertion via `cell.nth(index).toHaveText(value)` instead of approximate `cell.filter({ hasText })`.
152
92
 
93
+ **Row scope Gherkin** — these patterns use the table selector:
153
94
  ```gherkin
154
- Scenario: VP-UI-001 Some test
155
- Given User is on [Dashboard] page
156
- Then User see [Dashboard Title] heading is visible ← confirms page loaded
157
- When User click [Some Button] button
158
- ...
95
+ Then User see [Username] row in [Users] table with {{user_name}}
96
+ Then User see [Status] column with {{expected_status}}
97
+ When User click [Edit] button in [Users] table with {{user_name}}
159
98
  ```
160
99
 
161
- The "page loaded" check should use a **stable, always-visible element** on that specific page — typically the main page heading.
162
-
163
100
  ---
164
101
 
165
- ### Step 6: Handle Multiple Matches (strict mode)
102
+ ## Step 5: Detail Screens with Dynamic IDs
166
103
 
167
- When an element appears more than once (e.g. desktop + mobile nav, repeated list items), add `nth`:
104
+ For screens like `/admin/users/:id`:
105
+ 1. Navigate to list page via MCP to find a real record ID
106
+ 2. Hardcode the ID in page selector
168
107
 
169
108
  ```yaml
170
- nav link:
171
- type: 'role'
172
- value: 'link'
173
- name: 'Dashboard'
174
- nth: 0 # first occurrence (desktop nav)
109
+ user detail:
110
+ type: 'page'
111
+ value: '/admin/users/de42d800-0f5a-490e-9dcf-344fedbd34a5'
175
112
  ```
176
113
 
177
- Use `nth: 0` for the first match of dynamic/repeated elements.
178
-
179
- ---
180
-
181
- ### Auto-Infer (skip YAML entry)
182
-
183
- Many elements don't need a YAML entry — sungen auto-infers from the Gherkin label:
184
-
185
- | Gherkin | Auto-inferred locator |
186
- |---|---|
187
- | `[Submit] button` | `getByRole('button', { name: 'Submit' })` |
188
- | `[Home] link` | `getByRole('link', { name: 'Home' })` |
189
- | `[Welcome] heading` / `header` | `getByRole('heading', { name: 'Welcome' })` |
190
- | `[Email] field` | `getByPlaceholder('Email')` |
191
- | `[Success] text` / `message` / `label` | `getByText('Success')` |
192
- | `[Terms] checkbox` | `getByRole('checkbox', { name: 'Terms' })` |
193
- | `[Global] search` | `getByRole('searchbox', { name: 'Global' })` |
194
- | `[Vietnam] option` | `getByRole('option', { name: 'Vietnam' })` |
195
- | `[Price] slider` | `getByRole('slider', { name: 'Price' })` |
196
- | `[Notification] toggle` | `getByRole('switch', { name: 'Notification' })` |
197
- | `[Profile] tab` | `getByRole('tab', { name: 'Profile' })` |
198
- | `[Confirm] dialog` / `modal` / `drawer` | `getByRole('dialog', { name: 'Confirm' })` |
199
-
200
- **Only add a YAML entry when auto-infer won't work:**
201
- - The accessible name differs from the Gherkin label
202
- - Needs `nth` to disambiguate
203
- - Needs `testid` or CSS locator
204
- - Is dynamic content (needs structural selector)
205
- - Is a bare input with no label or placeholder
206
-
207
114
  ---
208
115
 
209
- ### Proactive Selector Validation (before running tests)
210
-
211
- **Most failures are selector mismatches.** Validate selectors against the live page BEFORE running any test — this eliminates slow compile→run→read→fix cycles.
212
-
213
- After generating `selectors.yaml` (Step 3), verify each entry:
214
-
215
- #### How to validate
216
-
217
- Use `browser_evaluate` to check if each selector actually finds an element on the page:
218
-
219
- ```js
220
- // Validate a role selector
221
- document.querySelectorAll('[role="button"]').length;
222
- // or check accessible name exists
223
- Array.from(document.querySelectorAll('button'))
224
- .filter(el => el.textContent.includes('Submit') || el.getAttribute('aria-label')?.includes('Submit'))
225
- .length;
226
- ```
227
-
228
- Or use `browser_snapshot` and cross-check:
229
- 1. Read all `[Reference]` entries from `selectors.yaml`
230
- 2. Take a `browser_snapshot`
231
- 3. For each entry, verify:
232
- - **role + name**: does the snapshot contain `button "Submit"` or `link "Home"`?
233
- - **testid**: does `browser_evaluate` find `[data-testid="xxx"]`?
234
- - **placeholder**: does the snapshot contain a textbox with that placeholder?
235
- - **locator**: does `browser_evaluate` find `document.querySelector('xxx')`?
236
- 4. Fix mismatches immediately — no need to run tests
116
+ ## Step 6: Validate Before Running
237
117
 
238
- #### What to check
118
+ **Fix 80%+ of selector issues before first test run.**
239
119
 
240
- | Selector type | Validation method |
241
- |---|---|
242
- | `role` + `name` | Search snapshot for `role "name"` text |
243
- | `testid` | `browser_evaluate`: `document.querySelector('[data-testid="xxx"]')` |
244
- | `placeholder` | Search snapshot for textbox with placeholder |
245
- | `locator` (CSS) | `browser_evaluate`: `document.querySelector('xxx')` |
246
- | `page` | Verify URL path exists (navigate and check) |
120
+ After generating `selectors.yaml`, verify each entry against the live snapshot:
121
+ - `role` + `name` → search snapshot for `role "name"`
122
+ - `testid` `browser_evaluate`: `document.querySelector('[data-testid="xxx"]')`
123
+ - `placeholder` search snapshot for textbox with placeholder
124
+ - `page` verify URL path exists
247
125
 
248
- #### Common proactive fixes
249
-
250
- - Name mismatch → copy exact name from snapshot
251
- - Element not found → check if element is inside iframe/dialog (needs `frame` or scope)
252
- - Multiple matches → add `nth: 0` or `exact: true`
253
- - No accessible name → switch to `testid` or `locator`
254
-
255
- **Target: fix 80%+ of selector issues before the first test run.**
126
+ Common fixes: name mismatch → copy from snapshot, element in iframe → add `frame`, multiple matches → add `nth` or `exact: true`.
256
127
 
257
128
  ---
258
129
 
259
- ### Batched Test Execution
260
-
261
- After proactive validation, run tests in **batches of 20** for faster feedback:
262
-
263
- ```
264
- 1. COMPILE — sungen generate --screen <screen>
265
-
266
- 2. BATCH RUN — run 20 tests at a time:
267
- npx playwright test <spec> --grep "VP-UI-001|VP-UI-002|...|VP-UI-020" --reporter=line
268
-
269
- 3. IF FAILURES in batch:
270
- a. Group failures by root cause (same selector, same error type)
271
- b. Fix selectors.yaml or test-data.yaml
272
- c. Recompile: sungen generate --screen <screen>
273
- d. Re-run ONLY the failing tests from this batch
274
- e. If fixed → move to next batch
275
-
276
- 4. NEXT BATCH — repeat with next 20 tests (VP-UI-021...VP-UI-040)
130
+ ## Step 7: Run & Fix Loop
277
131
 
278
- 5. FINAL CONFIRMATION after all batches pass, run ALL tests once:
279
- npx playwright test <spec> --reporter=line
280
- This catches regressions from selector changes.
132
+ Run tests in batches of 20. Group failures by root cause (same selector, same error type). Fix once, verify batch, move to next.
281
133
 
282
- 6. If still failing after 5 fix attempts per batch → ask user about direct .spec.ts fix
283
134
  ```
284
-
285
- #### Building batch `--grep` patterns
286
-
287
- Extract scenario names and batch them:
288
- ```bash
289
- # Batch 1: first 20
290
- npx playwright test <spec> --grep "VP-UI-001|VP-UI-002|...|VP-UI-020" --reporter=line
291
-
292
- # Batch 2: next 20
293
- npx playwright test <spec> --grep "VP-VAL-001|VP-VAL-002|...|VP-VAL-020" --reporter=line
135
+ compile → batch run 20 → fix root cause → recompile → re-run failures → next batch
294
136
  ```
295
137
 
296
- #### Grouping failures by root cause
297
-
298
- Common patterns where 1 fix resolves many failures:
299
- - **Same selector** — e.g., all `[Email Error]` tests fail → fix `email error` in selectors.yaml once
300
- - **Same error type** — e.g., all `strict mode violation` → add `exact: true` or `nth`
301
- - **Same assertion** — e.g., all `toHaveText` on inputs fail → change Gherkin pattern
302
-
303
- Fix the root cause first, verify with the batch, then move on.
304
-
305
- ---
306
-
307
- ### Key Rules (from sungen-selector-keys)
138
+ After all batches pass run ALL tests once for regression check.
308
139
 
309
- - `[Reference]` lowercase, keep Unicode: `[Search Content]``search content:`, `[Thời gian]` `thời gian:`
310
- - Keys use **spaces** (not dots) as word separators
311
- - Same label, different element types → add `--type` suffix
312
- - Same label, nth occurrence → add `--N` suffix
140
+ If still failing after 5 attempts per batchask user about direct `.spec.ts` fix.