@sun-asterisk/sungen 2.4.2 → 2.4.5

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (73) hide show
  1. package/dist/cli/commands/add.d.ts.map +1 -1
  2. package/dist/cli/commands/add.js +7 -1
  3. package/dist/cli/commands/add.js.map +1 -1
  4. package/dist/cli/index.js +1 -1
  5. package/dist/generators/test-generator/code-generator.d.ts.map +1 -1
  6. package/dist/generators/test-generator/code-generator.js +27 -4
  7. package/dist/generators/test-generator/code-generator.js.map +1 -1
  8. package/dist/orchestrator/ai-rules-updater.d.ts.map +1 -1
  9. package/dist/orchestrator/ai-rules-updater.js +2 -0
  10. package/dist/orchestrator/ai-rules-updater.js.map +1 -1
  11. package/dist/orchestrator/project-initializer.d.ts +4 -0
  12. package/dist/orchestrator/project-initializer.d.ts.map +1 -1
  13. package/dist/orchestrator/project-initializer.js +20 -3
  14. package/dist/orchestrator/project-initializer.js.map +1 -1
  15. package/dist/orchestrator/screen-manager.d.ts +9 -0
  16. package/dist/orchestrator/screen-manager.d.ts.map +1 -1
  17. package/dist/orchestrator/screen-manager.js +120 -0
  18. package/dist/orchestrator/screen-manager.js.map +1 -1
  19. package/dist/orchestrator/templates/ai-instructions/claude-cmd-add-screen.md +22 -19
  20. package/dist/orchestrator/templates/ai-instructions/claude-cmd-create-test.md +10 -2
  21. package/dist/orchestrator/templates/ai-instructions/claude-cmd-review.md +5 -0
  22. package/dist/orchestrator/templates/ai-instructions/claude-cmd-run-test.md +25 -16
  23. package/dist/orchestrator/templates/ai-instructions/claude-config.md +4 -97
  24. package/dist/orchestrator/templates/ai-instructions/claude-skill-gherkin-syntax.md +48 -122
  25. package/dist/orchestrator/templates/ai-instructions/claude-skill-selector-fix.md +172 -25
  26. package/dist/orchestrator/templates/ai-instructions/claude-skill-tc-generation.md +62 -34
  27. package/dist/orchestrator/templates/ai-instructions/claude-skill-tc-review.md +19 -14
  28. package/dist/orchestrator/templates/ai-instructions/claude-skill-test-design-techniques.md +99 -0
  29. package/dist/orchestrator/templates/ai-instructions/claude-skill-viewpoint.md +151 -64
  30. package/dist/orchestrator/templates/ai-instructions/copilot-cmd-add-screen.md +21 -15
  31. package/dist/orchestrator/templates/ai-instructions/copilot-cmd-create-test.md +10 -3
  32. package/dist/orchestrator/templates/ai-instructions/copilot-cmd-review.md +5 -0
  33. package/dist/orchestrator/templates/ai-instructions/copilot-cmd-run-test.md +24 -15
  34. package/dist/orchestrator/templates/ai-instructions/copilot-config.md +4 -97
  35. package/dist/orchestrator/templates/ai-instructions/github-skill-sungen-gherkin-syntax.md +48 -122
  36. package/dist/orchestrator/templates/ai-instructions/github-skill-sungen-selector-fix.md +172 -25
  37. package/dist/orchestrator/templates/ai-instructions/github-skill-sungen-tc-generation.md +63 -29
  38. package/dist/orchestrator/templates/ai-instructions/github-skill-sungen-tc-review.md +19 -14
  39. package/dist/orchestrator/templates/ai-instructions/github-skill-sungen-test-design-techniques.md +99 -0
  40. package/dist/orchestrator/templates/ai-instructions/github-skill-sungen-viewpoint.md +151 -64
  41. package/dist/orchestrator/templates/readme.md +1 -1
  42. package/dist/orchestrator/templates/tsconfig.json +21 -0
  43. package/package.json +1 -1
  44. package/src/cli/commands/add.ts +8 -1
  45. package/src/cli/index.ts +1 -1
  46. package/src/generators/test-generator/code-generator.ts +29 -4
  47. package/src/orchestrator/ai-rules-updater.ts +2 -0
  48. package/src/orchestrator/project-initializer.ts +24 -3
  49. package/src/orchestrator/screen-manager.ts +125 -0
  50. package/src/orchestrator/templates/ai-instructions/claude-cmd-add-screen.md +22 -19
  51. package/src/orchestrator/templates/ai-instructions/claude-cmd-create-test.md +10 -2
  52. package/src/orchestrator/templates/ai-instructions/claude-cmd-review.md +5 -0
  53. package/src/orchestrator/templates/ai-instructions/claude-cmd-run-test.md +25 -16
  54. package/src/orchestrator/templates/ai-instructions/claude-config.md +4 -97
  55. package/src/orchestrator/templates/ai-instructions/claude-skill-gherkin-syntax.md +48 -122
  56. package/src/orchestrator/templates/ai-instructions/claude-skill-selector-fix.md +172 -25
  57. package/src/orchestrator/templates/ai-instructions/claude-skill-tc-generation.md +62 -34
  58. package/src/orchestrator/templates/ai-instructions/claude-skill-tc-review.md +19 -14
  59. package/src/orchestrator/templates/ai-instructions/claude-skill-test-design-techniques.md +99 -0
  60. package/src/orchestrator/templates/ai-instructions/claude-skill-viewpoint.md +151 -64
  61. package/src/orchestrator/templates/ai-instructions/copilot-cmd-add-screen.md +21 -15
  62. package/src/orchestrator/templates/ai-instructions/copilot-cmd-create-test.md +10 -3
  63. package/src/orchestrator/templates/ai-instructions/copilot-cmd-review.md +5 -0
  64. package/src/orchestrator/templates/ai-instructions/copilot-cmd-run-test.md +24 -15
  65. package/src/orchestrator/templates/ai-instructions/copilot-config.md +4 -97
  66. package/src/orchestrator/templates/ai-instructions/github-skill-sungen-gherkin-syntax.md +48 -122
  67. package/src/orchestrator/templates/ai-instructions/github-skill-sungen-selector-fix.md +172 -25
  68. package/src/orchestrator/templates/ai-instructions/github-skill-sungen-tc-generation.md +63 -29
  69. package/src/orchestrator/templates/ai-instructions/github-skill-sungen-tc-review.md +19 -14
  70. package/src/orchestrator/templates/ai-instructions/github-skill-sungen-test-design-techniques.md +99 -0
  71. package/src/orchestrator/templates/ai-instructions/github-skill-sungen-viewpoint.md +151 -64
  72. package/src/orchestrator/templates/readme.md +1 -1
  73. package/src/orchestrator/templates/tsconfig.json +21 -0
@@ -1,6 +1,6 @@
1
1
  ---
2
2
  name: sungen-run-test
3
- description: 'Compile and run Playwright tests — auto-fixes selectors on failure. Uses sungen-selector-fix, sungen-selector-keys, and sungen-error-mapping skills.'
3
+ description: 'Generate selectors + auth state via Playwright MCP, compile, and run Playwright tests — auto-fixes selectors on failure. Uses sungen-selector-fix, sungen-selector-keys, and sungen-error-mapping skills.'
4
4
  argument-hint: '[screen-name]'
5
5
  agent: 'agent'
6
6
  tools: [vscode, execute, read, agent, edit, search, web, browser, todo, 'playwright/*']
@@ -16,20 +16,29 @@ You are a **Senior Developer**. Use `sungen-selector-fix`, `sungen-selector-keys
16
16
 
17
17
  - **screen** — ${input:screen:screen name (e.g., login, dashboard)}
18
18
 
19
- ## Phase 1: Compile & Run (no MCP)
19
+ ## Pre-run (phased per `sungen-selector-fix` skill)
20
20
 
21
- 1. Verify `qa/screens/${input:screen}/` has `.feature` + `test-data.yaml`
22
- 2. Ensure `selectors.yaml` has page selector. If missing, ask user for URL path
23
- 3. `sungen generate --screen ${input:screen}`
24
- 4. `npx playwright test specs/generated/${input:screen}/*.spec.ts --reporter=line`
25
- 5. If all pass → done
21
+ 1. Verify `qa/screens/${input:screen}/` has `.feature` + `test-data.yaml`.
22
+ 2. **Phase 0 — Selector Pre-gen**: if `selectors.yaml` is missing/empty or doesn't cover the feature file's `[Reference]`s, run Phase 0 from `sungen-selector-fix` — confirm with user, `browser_navigate` one `browser_snapshot` → merge YAML entries.
23
+ 3. **Phase 0.5 — Auth Persistence**: if the feature has `@auth:<role>` tags and `specs/.auth/<role>.json` is missing/expired, run Phase 0.5 from `sungen-selector-fix` user logs in manually in MCP browser → `browser_storage_state` → `specs/.auth/<role>.json`. Offer `sungen makeauth <role>` as CLI fallback only if `browser_storage_state` isn't available in this MCP version.
24
+ 4. Compile: `sungen generate --screen ${input:screen}`.
26
25
 
27
- ## Phase 2: Targeted Fix (only if failures)
26
+ ## Run & Fix (phased per `sungen-selector-fix` skill)
28
27
 
29
- 6. Parse failures → group by root cause
30
- 7. Navigate to page ONCE via #tool:playwright ONE `browser_snapshot`
31
- 8. Fix broken selectors per `sungen-selector-fix` skill
32
- 9. Recompile re-run only failing tests
33
- 10. Repeat up to 3 attempts
34
- 11. Final full run for regression check
35
- 12. Still failing → ask user
28
+ 5. **Phase 1 — Smoke Check**: Run first 5 `@critical`/`@high` scenarios only. If failures → diagnose, fix fundamentals (page selector, auth, base @steps), re-run. Max 2 attempts. If still broken → ask user.
29
+ 6. **Phase 2 Priority Wave**: Run all `@critical` + `@high` scenarios. Fix only failures from this wave. Max 2 attempts. Shared selectors fixed here cascade to later phases.
30
+ 7. **Phase 3 — Full Run**: Run all tests. Fix only **new** failures (elements unique to `@normal`/`@low`). Max 1 attempt. Don't loop on low-priority failures.
31
+ 8. **Phase 4 — Regression**: One final full run. Report results. No more fix loops.
32
+
33
+ ## Next steps
34
+
35
+ After showing results, offer next steps:
36
+
37
+ If all tests **passed**:
38
+ - **`/sungen-create-test ${input:screen}`** — Add more test cases (Recommended)
39
+ - **Done** — All tests passed, I'm finished
40
+
41
+ If tests **failed** (after retries):
42
+ - **`/sungen-run-test ${input:screen}`** — Re-run after manual fixes
43
+ - **`/sungen-create-test ${input:screen}`** — Revise test cases
44
+ - **Done for now** — I'll fix manually later
@@ -10,8 +10,9 @@ You generate 3 files for sungen — a Gherkin compiler that produces Playwright
10
10
  | `sungen-gherkin-syntax` | All 70+ step patterns, selector types, YAML key rules, tags, element types |
11
11
  | `sungen-error-mapping` | Playwright & sungen error → fix mapping |
12
12
  | `sungen-tc-generation` | Test case generation strategy, output format |
13
+ | `sungen-test-design-techniques` | EP, BVA, Decision Table, State Transition — systematic scenario generation |
13
14
  | `sungen-tc-review` | Review scoring, quality rules, checklist |
14
- | `sungen-viewpoint` | 7 UI patterns x 4 viewpoints — coverage checklists |
15
+ | `sungen-viewpoint` | 10 UI patterns x 4 viewpoints — coverage checklists |
15
16
  | `sungen-selector-keys` | YAML key generation from `[Reference]` names, suffixes, lookup priority |
16
17
  | `sungen-selector-fix` | Selector generation from live page, auto-fix strategy |
17
18
 
@@ -26,6 +27,8 @@ You generate 3 files for sungen — a Gherkin compiler that produces Playwright
26
27
 
27
28
  **Order:** add-screen → create-test → review → run-test.
28
29
 
30
+ After each command completes, present the next actions as selectable options. Never just print text — always give clickable choices so the user can continue the workflow seamlessly.
31
+
29
32
  ## File Structure
30
33
 
31
34
  ```
@@ -38,102 +41,6 @@ qa/screens/<screen-name>/
38
41
  └── ui/ # Screenshots, mockups
39
42
  ```
40
43
 
41
- ## Complete Example
42
-
43
- Given a login page, here are the 3 files you generate:
44
-
45
- **qa/screens/login/features/login.feature**
46
- ```gherkin
47
- @no-auth
48
- Feature: Login Screen
49
-
50
- Scenario: VP-LOGIC-001 Successful login
51
- Given User is on [login] page
52
- When User fill [Email] field with {{email}}
53
- And User fill [Password] field with {{password}}
54
- And User click [Submit] button
55
- Then User see [Welcome] heading with {{welcome_text}}
56
- And User see [Dashboard] link
57
- ```
58
-
59
- **qa/screens/login/selectors/login.yaml**
60
- ```yaml
61
- login:
62
- type: 'page'
63
- value: '/login'
64
-
65
- email:
66
- type: 'placeholder'
67
- value: 'Enter your email'
68
-
69
- password:
70
- type: 'placeholder'
71
- value: 'Enter your password'
72
-
73
- submit:
74
- type: 'role'
75
- value: 'button'
76
- name: 'Submit'
77
-
78
- welcome:
79
- type: 'role'
80
- value: 'heading'
81
- name: 'Welcome'
82
-
83
- dashboard:
84
- type: 'role'
85
- value: 'link'
86
- name: 'Dashboard'
87
- ```
88
-
89
- **qa/screens/login/test-data/login.yaml**
90
- ```yaml
91
- email: 'admin@example.com'
92
- password: 'password123'
93
- welcome_text: 'Welcome back'
94
- ```
95
-
96
- ## Critical Gherkin Rules (quick reference)
97
-
98
- 1. **NEVER write `is visible`** — `User see [T] type` already asserts visibility. Only use `is hidden` to assert absence.
99
- 2. **Use `@no-auth` for pre-login pages** (login, register, forgot-password). Use `@auth:role` for authenticated pages.
100
- 3. **Scenario names use VP prefix** — `VP-UI-001`, `VP-VAL-001`, `VP-LOGIC-001`, `VP-SEC-001`.
101
- 4. **Values use `{{snake_case}}`** — never hardcode data in `.feature`. All values go in `test-data.yaml`.
102
- 5. **Selectors are deferred** — `selectors.yaml` is generated during `/sungen-run-test` from the live page, NOT during `/sungen-create-test`.
103
- 6. **Every `{{variable}}` must exist in `test-data.yaml`** — missing variables cause compile failures.
104
- 7. **Assertion type is determined by element type** — input types (`field`, `textarea`, `search`, `dropdown`, `slider`, `date-picker`) → `toHaveValue()`. Everything else → `toHaveText()`. Wrong type = wrong assertion = test failure.
105
-
106
- ## Using Playwright MCP to explore pages
107
-
108
- When exploring a page to generate test files:
109
- 1. Read `playwright.config.ts` for `baseURL`
110
- 2. Use `browser_navigate` to open `baseURL + /path`
111
- 3. Use `browser_snapshot` to see all elements
112
- 4. Generate the 3 files from the snapshot
113
-
114
- ### Allowed MCP tools
115
-
116
- | Tool | Use for |
117
- |---|---|
118
- | `browser_navigate` | Open pages |
119
- | `browser_snapshot` | Capture all accessible elements |
120
- | `browser_click` | Interact with elements (open dialogs, dropdowns) |
121
- | `browser_fill_form` | Fill form fields |
122
- | `browser_evaluate` | **Read-only DOM queries only** (e.g., detect `data-testid` attributes) |
123
-
124
- **NEVER use `browser_run_code`** — fails with `require is not defined`.
125
-
126
- ### Authentication via MCP
127
-
128
- 1. `browser_navigate` to `baseURL`
129
- 2. If redirected to login, ask the user to log in manually:
130
- > "This page requires login. Please log in using the browser. Confirm when you're done."
131
- 3. Once confirmed, `browser_navigate` to the target page and take `browser_snapshot`
132
-
133
- **Never use `sungen makeauth`.** Always let the user log in manually via the MCP browser.
134
- **NEVER use `browser_evaluate` to inject cookies or localStorage** — causes size limit issues and server 500 errors. Use `browser_evaluate` ONLY for read-only DOM queries.
135
- **Minimize navigations** — take one snapshot per page, do not navigate repeatedly.
136
-
137
44
  ## CLI Commands
138
45
 
139
46
  ```bash
@@ -27,150 +27,68 @@ AND → inherits from preceding keyword
27
27
 
28
28
  ## Step Patterns (70 patterns)
29
29
 
30
- ### Setup
30
+ ### Setup / Form / Interaction
31
31
 
32
32
  ```
33
- User is on [T] page # navigate to page
34
- User is on [T] page with {{v}} # navigate with data (e.g., detail page with ID)
35
- User is on [T] dialog # enter dialog scope
33
+ User is on [T] page | page with {{v}} | dialog
34
+ User fill [T] field | textarea | search | slider | date-picker with {{v}}
35
+ User fill [T] uploader with {{f}}
36
+ User clear [T] field
37
+ User check [T] checkbox | toggle | radio
38
+ User uncheck [T] checkbox | toggle
39
+ User select [T] dropdown with {{v}}
40
+ User click [T] button | tab | column | breadcrumb
41
+ User click [T] row with {{v}}
42
+ User double click [T] element
43
+ User hover [T] icon | row
44
+ User drag [T] to [T2]
45
+ User expand | collapse [T] row
36
46
  ```
37
47
 
38
- ### Form
48
+ **click + with {{Value}} rule**: NO value for static (`button`, `link`, `icon`, `tab`). WITH value only for dynamic lists (`row`, `item`, `card`, `option`).
39
49
 
40
- ```
41
- User fill [T] field with {{v}} # text input
42
- User fill [T] textarea with {{v}} # multiline input
43
- User fill [T] search with {{v}} # search input
44
- User fill [T] slider with {{v}} # set slider value
45
- User fill [T] date picker with {{v}} # date input
46
- User clear [T] field # clear input
47
- User check [T] checkbox # check
48
- User check [T] toggle # toggle on (same as check)
49
- User check [T] radio # select radio
50
- User uncheck [T] checkbox # uncheck
51
- User uncheck [T] toggle # toggle off
52
- User select [T] dropdown with {{v}} # select option
53
- User fill [T] uploader with {{f}} # file upload
54
- ```
55
-
56
- ### Interaction
57
-
58
- ```
59
- User click [T] button # click (static element, no value)
60
- User click [T] row with {{v}} # click dynamic list item (row/item/card/option)
61
- User click [T] tab # switch tab
62
- User click [T] column # sort column
63
- User click [T] breadcrumb # navigate breadcrumb
64
- User double click [T] element # double click
65
- User hover [T] icon # hover (must have see/click after)
66
- User hover [T] row # hover row for hidden actions
67
- User drag [T] to [T2] # drag and drop
68
- User expand [T] row # expand (aria-expanded)
69
- User collapse [T] row # collapse (aria-expanded)
70
- ```
71
-
72
- #### click + with {{Value}} rule
73
-
74
- - **NO value** for static elements: `button`, `link`, `icon`, `tab`, `toggle`
75
- - **WITH value** only for dynamic lists: `row`, `item`, `card`, `option`
76
-
77
- ### Browser Alert (system dialog)
50
+ ### Alert / Keyboard / Wait / Scroll
78
51
 
79
52
  ```
80
- User click [OK] alert # accept (OK/Accept/Yes/Confirm)
81
- User click [Cancel] alert # dismiss (Cancel/Dismiss/No)
82
- User fill [T] alert with {{v}} # fill prompt + accept
83
- User see [message text] alert # assert dialog message
53
+ User click [OK | Cancel] alert
54
+ User fill [T] alert with {{v}}
55
+ User see [message text] alert
56
+ User press Escape key | Enter on [T] field
57
+ User wait for N seconds | [T] dialog | [T] dialog is STATE | [T] page
58
+ User scroll to [T] section
59
+ User switch to [T] frame | [main] frame
84
60
  ```
85
61
 
86
62
  > Alert steps must appear BEFORE the action that triggers the dialog.
87
63
 
88
- ### Keyboard
64
+ ### Assertions (8 patterns → determines Playwright assertion)
89
65
 
90
66
  ```
91
- User press Escape key # global key press
92
- User press Enter on [T] field # key on element
93
- ```
94
-
95
- ### Wait
96
-
97
- ```
98
- User wait for N seconds # wait timeout
99
- User wait for [T] dialog # wait for element
100
- User wait for [T] dialog is STATE # wait for state
101
- User wait for [T] dialog with {{v}} # wait for element with text
102
- User wait for [T] page # wait for navigation
103
- User wait for [T] message # wait for toast/feedback
104
- User wait for [T] button # wait for button to appear
105
- ```
106
-
107
- ### Scroll & Frame
108
-
109
- ```
110
- User scroll to [T] section # scroll into view
111
- User switch to [T] frame # enter iframe
112
- User switch to [main] frame # exit iframe
113
- ```
114
-
115
- ### Assertions (8 verify patterns)
116
-
117
- ```
118
- # 1. Visibility
119
- User see [T] message # visible (default — NEVER add "is visible")
120
- User see [T] modal is hidden # hidden
121
-
122
- # 2. Text Content (exact full match — toHaveText)
123
- User see [T] message with {{v}} # exact text match
124
- User see [T] header with {{v}} # heading text
125
- User see [T] label with {{v}} # label text
126
-
127
- # 3. Partial Text Match (toContainText)
128
- User see [T] text contains {{v}} # partial text match
129
-
130
- # 4. Input Value (toHaveValue)
131
- User see [T] field with {{v}} # input value
132
- User see [T] dropdown with {{v}} # selected value
133
- User see [T] date picker with {{v}} # date value
134
- User see [T] search with {{v}} # search input value
135
- User see [T] slider with {{v}} # slider value
136
-
137
- # 5. Component State
138
- User see [T] button is disabled # state assertion
139
- User see [T] checkbox is checked # checked state
140
- User see [T] toggle is unchecked # unchecked state
141
- User see [T] dialog with {{v}} is hidden # text + state combined
142
-
143
- # 6. Attribute (toHaveAttribute — when selector YAML has `attribute` field)
144
- User see [T] image with {{v}} # image src
145
- User see [T] link with {{v}} # link href
146
-
147
- # 7. Count
148
- User see [T] row with {{count}} # element count
149
-
150
- # 8. Page Context
151
- User see [T] page # URL assertion
67
+ # 1. Visibility: User see [T] type (NEVER add "is visible") | is hidden
68
+ # 2. Text (toHaveText): User see [T] message | header | label with {{v}}
69
+ # 3. Partial (toContainText): User see [T] text contains {{v}}
70
+ # 4. Input (toHaveValue): User see [T] field | dropdown | date-picker | search | slider with {{v}}
71
+ # 5. State: User see [T] button is disabled | checkbox is checked | dialog with {{v}} is hidden
72
+ # 6. Attribute (toHaveAttribute): User see [T] image | link with {{v}}
73
+ # 7. Count: User see [T] row with {{count}}
74
+ # 8. Page: User see [T] page
152
75
  ```
153
76
 
154
77
  ### Table
155
78
 
156
79
  ```
157
- User see [Col] column in [Table] table # column exists
158
- User see [Ref] row in [Table] table with {{v}} # row exists (enters row scope)
159
- User see [Ref] row in [Table] table with {{v}} is hidden # row hidden
160
- User see [Table] table with {{count}} # row count
161
- User see [Table] table is empty # empty table
162
- User see [Col] column with {{v}} # cell value (row scoped)
163
- User click [Act] button in [Table] table with {{v}} # action in row
164
- User see [Table] table match data: # exact table match (inline DataTable)
80
+ User see [Col] column in [Table] table
81
+ User see [Ref] row in [Table] table with {{v}}
82
+ User see [Ref] row in [Table] table with {{v}} is hidden
83
+ User see [Table] table with {{count}} | is empty
84
+ User see [Col] column with {{v}}
85
+ User click [Act] button in [Table] table with {{v}}
86
+ User see [Table] table match data:
165
87
  | Header1 | Header2 |
166
88
  | {{value1}} | {{value2}} |
167
89
  ```
168
90
 
169
- **Priority**: Use row scope patterns for single-row verification + actions. Use `table match data` only when verifying multiple rows at once.
170
-
171
- Row scope: `see [Ref] row in [Table] table with {{v}}` defines `tableRow`. Subsequent `see [Col] column with {{v}}` checks the cell in that row. With `columns` config → exact `nth(index)`, without → `filter({ hasText })`.
172
-
173
- Table match data: `see [Table] table match data:` followed by a Cucumber DataTable. First row = headers, remaining rows = expected data. Uses **filter-based matching** — verifies rows containing expected values exist, resilient to data changes, extra rows, and row reordering. Use `{{variable}}` for cell values.
91
+ Row scope: `see [Ref] row in [Table] table with {{v}}` enters scope. Subsequent `see [Col] column with {{v}}` checks cell in that row. Use `table match data:` for multi-row verification.
174
92
 
175
93
  ### States
176
94
 
@@ -223,6 +141,10 @@ Options: `nth` `exact` `scope` `match` `variant` `frame` `contenteditable` `colu
223
141
  | `@auto` | Standard scenario, ready for automation |
224
142
  | `@manual` | Skip in generation |
225
143
  | `@smoke` / `@regression` | Test suite grouping |
144
+ | `@critical` | Priority: system unusable if fails (login, auth, core CRUD) |
145
+ | `@high` | Priority: major feature broken (required validation, key business rules) |
146
+ | `@normal` | Priority: degraded experience (UI layout, secondary flows) |
147
+ | `@low` | Priority: minor/cosmetic (tooltips, hover states, empty states) |
226
148
  | `@auth:role` | Use auth storage state for role |
227
149
  | `@no-auth` | Disable inherited auth |
228
150
  | `@steps:name` | Define reusable step block (base scenario) |
@@ -232,6 +154,8 @@ Options: `nth` `exact` `scope` `match` `variant` `frame` `contenteditable` `colu
232
154
 
233
155
  - Tool executes **only Given→When** of `@steps` scenario (skips Then)
234
156
  - The `Given` in `@extend` scenario is the **entry assertion** (confirms state after base steps)
157
+ - **Entry assertion MUST use `Given User is on [X] type`** — NEVER `Given User see [X] type`
158
+ - `Given` keyword ONLY allows `is on` action. `see` = `Then` only.
235
159
  - If `@steps` scenario fails, `@extend` scenario is **skipped**
236
160
  - Name format: `snake_case` or `kebab-case` with module prefix: `@steps:kudos__open_modal`
237
161
 
@@ -240,6 +164,8 @@ Options: `nth` `exact` `scope` `match` `variant` `frame` `contenteditable` `colu
240
164
  | Error | Wrong | Correct |
241
165
  |---|---|---|
242
166
  | Wrong keyword | `Given User click [T] button` | `When User click [T] button` |
167
+ | `see` after Given | `Given User see [T] heading with {{v}}` | `Then User see [T] heading with {{v}}` (or `Given User is on [T] page` for entry assertion) |
168
+ | Name ≠ step type | Scenario name says "modal" but step uses `dialog` | Use the **same element type** in both: "...dialog opens" + `[X] dialog` |
243
169
  | Wrong action for type | `When User click [T] checkbox` | `When User check [T] checkbox` |
244
170
  | press wrong target | `When User press [Submit] button` | `When User press Enter key` |
245
171
  | uncheck radio | `When User uncheck [Male] radio` | `When User check [Female] radio` |
@@ -1,20 +1,161 @@
1
1
  ---
2
2
  name: sungen-selector-fix
3
- description: 'Selector fixing strategy — diagnose failures, explore live page targeted, fix broken selectors. Auto-loaded by run-test command.'
3
+ description: 'Selector fixing strategy — phased execution, priority-first diagnosis, targeted MCP exploration. Auto-loaded by run-test command.'
4
4
  user-invocable: false
5
5
  ---
6
6
 
7
- ## Strategy: Fix Only What Breaks
7
+ ## Strategy: Phased Execution
8
8
 
9
- Most selectors auto-infer from Gherkin element types (see `sungen-selector-keys` skill). Only add YAML entries for selectors that fail during test execution.
9
+ Run tests in priority waves catch fundamental issues early, fix critical paths first, let shared fixes cascade to lower-priority tests.
10
10
 
11
- **Minimal `selectors.yaml`**: page selectors (required), overrides for elements where auto-infer doesn't work. Never generate a full page catalog upfront.
11
+ **Never run all tests blindly.** Always start with selector pre-generation, then a smoke check.
12
12
 
13
13
  ---
14
14
 
15
- ## Step 1: Diagnose Failures
15
+ ## Phase 0: Pre-run Selector Generation (Playwright MCP)
16
16
 
17
- Parse Playwright error output to categorize failures:
17
+ **Before any `sungen generate` or test run**, populate `selectors.yaml` from the live page so tests don't fail on missing keys in Phase 1.
18
+
19
+ ### When to run Phase 0
20
+
21
+ - `selectors.yaml` missing, empty, or contains only the page selector
22
+ - The `.feature` file has `[Reference]` keys without corresponding YAML entries and the referenced element can't be auto-inferred (see `sungen-selector-keys` § Auto-Infer)
23
+ - User explicitly re-scans after UI changes
24
+
25
+ If existing selectors already cover the feature file, **skip Phase 0** and go straight to compile + Phase 1.
26
+
27
+ ### Steps
28
+
29
+ 1. **Confirm with the user**: *"Generate selectors from the live page via Playwright MCP now?"* — offer **Yes, scan live page** / **Skip (use existing selectors.yaml)** / **Cancel**.
30
+ 2. **Collect references**: parse the `.feature` file for every `[Reference]` element + its type (e.g. `[Submit] button`, `[Email] field`). Deduplicate.
31
+ 3. **Ensure page selector**: if missing, ask user for URL path and write it first.
32
+ 4. **Navigate**:
33
+ - Read `baseURL` from `playwright.config.ts`.
34
+ - `browser_navigate` to the page URL.
35
+ - If redirected to login → **ask the user to log in manually in the MCP browser**, wait for confirmation, then continue. Never use `sungen makeauth`. Never inject cookies via `browser_evaluate`.
36
+ 5. **Snapshot**: take **ONE** `browser_snapshot`. All Phase 0 selectors come from this single snapshot.
37
+ 6. **Generate YAML entries**:
38
+ - Keys: follow `sungen-selector-keys` (lowercase, Unicode preserved, `--type` / `--N` suffixes).
39
+ - Selector priority: follow the table in **Diagnosis & Fix § Step 3** (`testid` > `role`+name > `placeholder` > `label` > `locator` > `text`).
40
+ - Copy names **character-for-character** from the snapshot. Never infer from the Gherkin label.
41
+ - If an element is auto-inferable per `sungen-selector-keys` § Auto-Infer, **omit it** from YAML — keep the file minimal.
42
+ 7. **Merge, don't overwrite**: preserve the page selector and any user-authored entries in `selectors.yaml`. Only add missing keys.
43
+ 8. **Show summary + confirm**: list the keys that will be added, ask the user to approve, then write the file.
44
+ 9. **Compile**: `sungen generate --screen <screen>` — then proceed to Phase 1.
45
+
46
+ ### Common Phase 0 pitfalls
47
+
48
+ - Writing keys inferred from the Gherkin label instead of the snapshot name → Phase 1 will fail with "no element found".
49
+ - Skipping Phase 0.5 when an auth redirect happened → snapshot captures the login page, all selectors wrong.
50
+ - Using `browser_evaluate` alone to scrape cookies → misses httpOnly session cookies. Always use `browser_storage_state` (or the `browser_run_code` fallback).
51
+ - Overwriting user-authored selectors → always merge.
52
+
53
+ ---
54
+
55
+ ## Phase 0.5: Auth Persistence (MCP alternative to `sungen makeauth`)
56
+
57
+ Capture an authenticated session from the MCP browser into `specs/.auth/<role>.json` — the same path `sungen makeauth` writes to, which compiled tests already reference via `test.use({ storageState })` based on `@auth:<role>` tags. No `playwright.config.ts` edits needed. Run once per auth lifetime, not on every selector fix.
58
+
59
+ ### When to run Phase 0.5
60
+
61
+ - Phase 0 navigation hit a login redirect and `specs/.auth/<role>.json` is missing or expired
62
+ - A scenario tagged `@auth:<role>` is about to run and its auth file is absent
63
+ - User asks to refresh auth
64
+
65
+ Skip if `specs/.auth/<role>.json` already exists and a probe navigation reaches an authenticated page without redirecting to login.
66
+
67
+ ### Steps
68
+
69
+ 1. **Resolve the role**:
70
+ - Look at the `.feature` file for `@auth:<role>` tags (feature-level or scenario-level). Pick the role for the scenario being run. If no tag exists, default to `user`.
71
+ - Target file: `specs/.auth/<role>.json`. Create `specs/.auth/` if missing.
72
+ - If the file already exists → confirm overwrite with the user (mirrors the `(y/N)` prompt in `sungen makeauth`).
73
+ 2. **Navigate to login**:
74
+ - Read `baseURL` from `playwright.config.ts` (fall back to `APP_BASE_URL` env, then `http://localhost:3000` — same resolution order as `sungen makeauth`).
75
+ - `browser_navigate` to `<baseURL>/login`. If the app uses a different login path, ask the user.
76
+ - If the URL doesn't stay on `/login` after load → user is already signed in. Skip step 3.
77
+ 3. **Ask the user to log in manually** in the MCP browser (username, password, MFA, SSO — whatever the app needs). Never type credentials via `browser_type` or script the login. Wait for the user to confirm in chat that they're signed in.
78
+ 4. **Verify login** — check the current URL or take a `browser_snapshot`; confirm the page is no longer on `/login`.
79
+ 5. **Export storage state** (preferred → fallback):
80
+ - **Preferred** — `browser_storage_state` with `filename: "specs/.auth/<role>.json"` (native Playwright MCP tool; captures cookies including httpOnly + localStorage + sessionStorage via the Playwright context — same output format as `context.storageState({ path })` used by `sungen makeauth`).
81
+ - **Fallback** — if `browser_storage_state` isn't available in this MCP version, use `browser_run_code` to execute `await context.storageState({ path: 'specs/.auth/<role>.json' })`.
82
+ - **Do NOT** use `browser_evaluate` for auth export — it misses httpOnly cookies and session auth will fail silently.
83
+ 6. **Gitignore** — ensure `specs/.auth/` (or `specs/.auth/*.json`) is in `.gitignore`. Add it if missing.
84
+ 7. **Return to Phase 0 step 4** — re-`browser_navigate` to the target page; the session is now active.
85
+
86
+ ### Phase 0.5 pitfalls
87
+
88
+ - Writing to a path other than `specs/.auth/<role>.json` → compiled tests won't find the file. Always match `sungen makeauth`'s convention.
89
+ - Committing `specs/.auth/*.json` → leaks a live session. Always gitignore.
90
+ - Scripting the login with `browser_type` → bypasses MFA/CAPTCHA and risks account lockout. Always manual.
91
+ - Running Phase 0.5 on every `run-test` invocation → unnecessary; reuse the file until tests start redirecting to login.
92
+ - Mismatch between `<role>` in the auth file and `@auth:<role>` tag → compiled tests reference a nonexistent file.
93
+
94
+ ---
95
+
96
+ ## Phase 1: Smoke Check (catch fundamentals)
97
+
98
+ Run **up to 5 scenarios** — pick the first `@critical` or `@high` scenarios in the feature file.
99
+
100
+ ```bash
101
+ npx playwright test --grep "VP-.*-001|VP-.*-002|VP-.*-003|VP-.*-004|VP-.*-005" --reporter=line
102
+ ```
103
+
104
+ **Purpose:** Detect broken fundamentals before running 50+ tests:
105
+ - Page selector wrong → ALL tests would fail (1 fix, not 50 diagnoses)
106
+ - Auth redirect → need `@no-auth` or user login
107
+ - Base `@steps:` scenario broken → all `@extend:` scenarios would fail
108
+
109
+ **If all 5 pass** → skip to Phase 2.
110
+ **If failures** → diagnose and fix (see Diagnosis & Fix below), then re-run smoke. Max 2 attempts here.
111
+
112
+ ---
113
+
114
+ ## Phase 2: Priority Wave (@critical + @high)
115
+
116
+ Run all `@critical` and `@high` scenarios:
117
+
118
+ ```bash
119
+ npx playwright test --grep "@critical|@high" --reporter=line
120
+ ```
121
+
122
+ If your Playwright config doesn't support tag grep, use scenario name grep from the feature file — collect VP IDs of `@critical` and `@high` scenarios.
123
+
124
+ **Fix only failures from this wave.** Most shared selectors (buttons, headings, navigation) get fixed here because critical/high scenarios exercise them.
125
+
126
+ Max 2 fix attempts in this phase.
127
+
128
+ ---
129
+
130
+ ## Phase 3: Full Run (@normal + @low)
131
+
132
+ Run remaining scenarios:
133
+
134
+ ```bash
135
+ npx playwright test --reporter=line
136
+ ```
137
+
138
+ Many selectors already fixed from Phase 2 (shared elements). Only diagnose **new** failures — selectors that only appear in lower-priority scenarios.
139
+
140
+ Max 1 fix attempt. If `@low` scenarios still fail after fix → **report and move on**, don't loop.
141
+
142
+ ---
143
+
144
+ ## Phase 4: Regression
145
+
146
+ One final full run to confirm all phases together:
147
+
148
+ ```bash
149
+ npx playwright test --reporter=line
150
+ ```
151
+
152
+ Report results. Do NOT enter another fix loop here.
153
+
154
+ ---
155
+
156
+ ## Diagnosis & Fix (used in each phase)
157
+
158
+ ### Step 1: Parse Failures
18
159
 
19
160
  | Error pattern | Root cause | Fix target |
20
161
  |---|---|---|
@@ -28,24 +169,18 @@ Parse Playwright error output to categorize failures:
28
169
 
29
170
  **Check `test-results/` first** — Playwright captures failure screenshots automatically. Use these to diagnose before any MCP exploration.
30
171
 
31
- ---
32
-
33
- ## Step 2: Targeted MCP Exploration
172
+ ### Step 2: Targeted MCP Exploration
34
173
 
35
174
  Only when `test-results/` screenshots are insufficient:
36
175
 
37
176
  1. Read `baseURL` from `playwright.config.ts`
38
177
  2. `browser_navigate` to target page
39
- 3. If redirected to login → ask user to log in manually via MCP browser
178
+ 3. If redirected to login → run **Phase 0.5: Auth Persistence**, then re-navigate
40
179
  4. Take **ONE** `browser_snapshot` — fix all broken selectors from this single snapshot
41
180
 
42
- **Never use `sungen makeauth`.** Never use `browser_evaluate` to inject cookies.
181
+ Never use `browser_evaluate` to inject or read cookies (misses httpOnly). For auth, use Phase 0.5 or `sungen makeauth`.
43
182
 
44
- ---
45
-
46
- ## Step 3: Fix Broken Selectors
47
-
48
- For each failed selector, find the correct locator from the snapshot:
183
+ ### Step 3: Fix Broken Selectors
49
184
 
50
185
  Selector priority (use first applicable):
51
186
 
@@ -73,9 +208,18 @@ Common fixes:
73
208
  - Element in iframe → add `frame` field
74
209
  - Dynamic content → use `testid` or structural `role` + `nth`
75
210
 
211
+ ### Step 4: Recompile After Fix
212
+
213
+ Always recompile before re-running:
214
+ ```bash
215
+ sungen generate --screen <screen>
216
+ ```
217
+
218
+ Then re-run only the current phase's failing tests, not all tests.
219
+
76
220
  ---
77
221
 
78
- ## Step 4: Table Selectors
222
+ ## Table Selectors
79
223
 
80
224
  For table patterns, add table selectors with `columns` config:
81
225
 
@@ -100,7 +244,7 @@ users:
100
244
 
101
245
  ---
102
246
 
103
- ## Step 5: Detail Screens with Dynamic IDs
247
+ ## Detail Screens with Dynamic IDs
104
248
 
105
249
  For screens like `/admin/users/:id`:
106
250
  1. Navigate to list page via MCP to find a real record ID
@@ -114,12 +258,15 @@ user detail:
114
258
 
115
259
  ---
116
260
 
117
- ## Step 6: Fix Loop
261
+ ## Attempt Budget Summary
118
262
 
119
- After fixing selectors:
120
- 1. Recompile: `sungen generate --screen <screen>`
121
- 2. Re-run only failing tests: `npx playwright test --grep "VP-XXX-001|VP-XXX-002" --reporter=line`
122
- 3. If new failuresrepeat (max 3 attempts)
123
- 4. After all fixes run ALL tests once for regression check
263
+ | Phase | What runs | Max fix attempts | On failure after max |
264
+ |---|---|---|---|
265
+ | 0. Pre-gen | Playwright MCP snapshot write selectors.yaml | 1 snapshot | Ask user — skip or retry navigation |
266
+ | 0.5. Auth | Manual login in MCP browser `browser_storage_state` `specs/.auth/<role>.json` | 1 login | Ask user — retry login or fall back to `sungen makeauth` |
267
+ | 1. Smoke | First 5 @critical/@high | 2 | Ask user — fundamentals broken |
268
+ | 2. Priority | All @critical + @high | 2 | Report failures, continue to Phase 3 |
269
+ | 3. Full | All tests | 1 | Report @low/@normal failures, continue |
270
+ | 4. Regression | All tests | 0 | Report final results |
124
271
 
125
- If still failing after 3 attempts ask user for guidance.
272
+ **Total worst case: 5 fix attempts** (2+2+1), not unbounded loops. Phases 0 and 0.5 don't count toward fix budget.