@sun-asterisk/sungen 2.4.1 → 2.4.3
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/dist/cli/commands/add.js +2 -2
- package/dist/cli/commands/add.js.map +1 -1
- package/dist/cli/index.js +1 -1
- package/dist/generators/test-generator/code-generator.d.ts.map +1 -1
- package/dist/generators/test-generator/code-generator.js +27 -4
- package/dist/generators/test-generator/code-generator.js.map +1 -1
- package/dist/orchestrator/ai-rules-updater.d.ts.map +1 -1
- package/dist/orchestrator/ai-rules-updater.js +2 -0
- package/dist/orchestrator/ai-rules-updater.js.map +1 -1
- package/dist/orchestrator/project-initializer.d.ts +4 -0
- package/dist/orchestrator/project-initializer.d.ts.map +1 -1
- package/dist/orchestrator/project-initializer.js +20 -3
- package/dist/orchestrator/project-initializer.js.map +1 -1
- package/dist/orchestrator/screen-manager.d.ts +8 -0
- package/dist/orchestrator/screen-manager.d.ts.map +1 -1
- package/dist/orchestrator/screen-manager.js +120 -0
- package/dist/orchestrator/screen-manager.js.map +1 -1
- package/dist/orchestrator/templates/ai-instructions/claude-cmd-add-screen.md +6 -14
- package/dist/orchestrator/templates/ai-instructions/claude-cmd-create-test.md +10 -2
- package/dist/orchestrator/templates/ai-instructions/claude-cmd-review.md +5 -0
- package/dist/orchestrator/templates/ai-instructions/claude-cmd-run-test.md +21 -13
- package/dist/orchestrator/templates/ai-instructions/claude-config.md +4 -97
- package/dist/orchestrator/templates/ai-instructions/claude-skill-gherkin-syntax.md +84 -122
- package/dist/orchestrator/templates/ai-instructions/claude-skill-selector-fix.md +87 -23
- package/dist/orchestrator/templates/ai-instructions/claude-skill-tc-generation.md +71 -30
- package/dist/orchestrator/templates/ai-instructions/claude-skill-tc-review.md +19 -14
- package/dist/orchestrator/templates/ai-instructions/claude-skill-test-design-techniques.md +99 -0
- package/dist/orchestrator/templates/ai-instructions/claude-skill-viewpoint.md +151 -64
- package/dist/orchestrator/templates/ai-instructions/copilot-cmd-add-screen.md +5 -10
- package/dist/orchestrator/templates/ai-instructions/copilot-cmd-create-test.md +10 -3
- package/dist/orchestrator/templates/ai-instructions/copilot-cmd-review.md +5 -0
- package/dist/orchestrator/templates/ai-instructions/copilot-cmd-run-test.md +21 -13
- package/dist/orchestrator/templates/ai-instructions/copilot-config.md +4 -97
- package/dist/orchestrator/templates/ai-instructions/github-skill-sungen-gherkin-syntax.md +85 -123
- package/dist/orchestrator/templates/ai-instructions/github-skill-sungen-selector-fix.md +87 -23
- package/dist/orchestrator/templates/ai-instructions/github-skill-sungen-tc-generation.md +72 -25
- package/dist/orchestrator/templates/ai-instructions/github-skill-sungen-tc-review.md +19 -14
- package/dist/orchestrator/templates/ai-instructions/github-skill-sungen-test-design-techniques.md +99 -0
- package/dist/orchestrator/templates/ai-instructions/github-skill-sungen-viewpoint.md +151 -64
- package/dist/orchestrator/templates/readme.md +1 -1
- package/dist/orchestrator/templates/tsconfig.json +21 -0
- package/package.json +1 -1
- package/src/cli/commands/add.ts +2 -2
- package/src/cli/index.ts +1 -1
- package/src/generators/test-generator/code-generator.ts +29 -4
- package/src/orchestrator/ai-rules-updater.ts +2 -0
- package/src/orchestrator/project-initializer.ts +24 -3
- package/src/orchestrator/screen-manager.ts +124 -0
- package/src/orchestrator/templates/ai-instructions/claude-cmd-add-screen.md +6 -14
- package/src/orchestrator/templates/ai-instructions/claude-cmd-create-test.md +10 -2
- package/src/orchestrator/templates/ai-instructions/claude-cmd-review.md +5 -0
- package/src/orchestrator/templates/ai-instructions/claude-cmd-run-test.md +21 -13
- package/src/orchestrator/templates/ai-instructions/claude-config.md +4 -97
- package/src/orchestrator/templates/ai-instructions/claude-skill-gherkin-syntax.md +84 -122
- package/src/orchestrator/templates/ai-instructions/claude-skill-selector-fix.md +87 -23
- package/src/orchestrator/templates/ai-instructions/claude-skill-tc-generation.md +71 -30
- package/src/orchestrator/templates/ai-instructions/claude-skill-tc-review.md +19 -14
- package/src/orchestrator/templates/ai-instructions/claude-skill-test-design-techniques.md +99 -0
- package/src/orchestrator/templates/ai-instructions/claude-skill-viewpoint.md +151 -64
- package/src/orchestrator/templates/ai-instructions/copilot-cmd-add-screen.md +5 -10
- package/src/orchestrator/templates/ai-instructions/copilot-cmd-create-test.md +10 -3
- package/src/orchestrator/templates/ai-instructions/copilot-cmd-review.md +5 -0
- package/src/orchestrator/templates/ai-instructions/copilot-cmd-run-test.md +21 -13
- package/src/orchestrator/templates/ai-instructions/copilot-config.md +4 -97
- package/src/orchestrator/templates/ai-instructions/github-skill-sungen-gherkin-syntax.md +85 -123
- package/src/orchestrator/templates/ai-instructions/github-skill-sungen-selector-fix.md +87 -23
- package/src/orchestrator/templates/ai-instructions/github-skill-sungen-tc-generation.md +72 -25
- package/src/orchestrator/templates/ai-instructions/github-skill-sungen-tc-review.md +19 -14
- package/src/orchestrator/templates/ai-instructions/github-skill-sungen-test-design-techniques.md +99 -0
- package/src/orchestrator/templates/ai-instructions/github-skill-sungen-viewpoint.md +151 -64
- package/src/orchestrator/templates/readme.md +1 -1
- package/src/orchestrator/templates/tsconfig.json +21 -0
|
@@ -27,150 +27,68 @@ AND → inherits from preceding keyword
|
|
|
27
27
|
|
|
28
28
|
## Step Patterns (70 patterns)
|
|
29
29
|
|
|
30
|
-
### Setup
|
|
30
|
+
### Setup / Form / Interaction
|
|
31
31
|
|
|
32
32
|
```
|
|
33
|
-
User is on [T] page
|
|
34
|
-
User
|
|
35
|
-
User
|
|
33
|
+
User is on [T] page | page with {{v}} | dialog
|
|
34
|
+
User fill [T] field | textarea | search | slider | date-picker with {{v}}
|
|
35
|
+
User fill [T] uploader with {{f}}
|
|
36
|
+
User clear [T] field
|
|
37
|
+
User check [T] checkbox | toggle | radio
|
|
38
|
+
User uncheck [T] checkbox | toggle
|
|
39
|
+
User select [T] dropdown with {{v}}
|
|
40
|
+
User click [T] button | tab | column | breadcrumb
|
|
41
|
+
User click [T] row with {{v}}
|
|
42
|
+
User double click [T] element
|
|
43
|
+
User hover [T] icon | row
|
|
44
|
+
User drag [T] to [T2]
|
|
45
|
+
User expand | collapse [T] row
|
|
36
46
|
```
|
|
37
47
|
|
|
38
|
-
|
|
48
|
+
**click + with {{Value}} rule**: NO value for static (`button`, `link`, `icon`, `tab`). WITH value only for dynamic lists (`row`, `item`, `card`, `option`).
|
|
39
49
|
|
|
40
|
-
|
|
41
|
-
User fill [T] field with {{v}} # text input
|
|
42
|
-
User fill [T] textarea with {{v}} # multiline input
|
|
43
|
-
User fill [T] search with {{v}} # search input
|
|
44
|
-
User fill [T] slider with {{v}} # set slider value
|
|
45
|
-
User fill [T] date picker with {{v}} # date input
|
|
46
|
-
User clear [T] field # clear input
|
|
47
|
-
User check [T] checkbox # check
|
|
48
|
-
User check [T] toggle # toggle on (same as check)
|
|
49
|
-
User check [T] radio # select radio
|
|
50
|
-
User uncheck [T] checkbox # uncheck
|
|
51
|
-
User uncheck [T] toggle # toggle off
|
|
52
|
-
User select [T] dropdown with {{v}} # select option
|
|
53
|
-
User fill [T] uploader with {{f}} # file upload
|
|
54
|
-
```
|
|
55
|
-
|
|
56
|
-
### Interaction
|
|
57
|
-
|
|
58
|
-
```
|
|
59
|
-
User click [T] button # click (static element, no value)
|
|
60
|
-
User click [T] row with {{v}} # click dynamic list item (row/item/card/option)
|
|
61
|
-
User click [T] tab # switch tab
|
|
62
|
-
User click [T] column # sort column
|
|
63
|
-
User click [T] breadcrumb # navigate breadcrumb
|
|
64
|
-
User double click [T] element # double click
|
|
65
|
-
User hover [T] icon # hover (must have see/click after)
|
|
66
|
-
User hover [T] row # hover row for hidden actions
|
|
67
|
-
User drag [T] to [T2] # drag and drop
|
|
68
|
-
User expand [T] row # expand (aria-expanded)
|
|
69
|
-
User collapse [T] row # collapse (aria-expanded)
|
|
70
|
-
```
|
|
71
|
-
|
|
72
|
-
#### click + with {{Value}} rule
|
|
73
|
-
|
|
74
|
-
- **NO value** for static elements: `button`, `link`, `icon`, `tab`, `toggle`
|
|
75
|
-
- **WITH value** only for dynamic lists: `row`, `item`, `card`, `option`
|
|
76
|
-
|
|
77
|
-
### Browser Alert (system dialog)
|
|
50
|
+
### Alert / Keyboard / Wait / Scroll
|
|
78
51
|
|
|
79
52
|
```
|
|
80
|
-
User click [OK] alert
|
|
81
|
-
User
|
|
82
|
-
User
|
|
83
|
-
User
|
|
53
|
+
User click [OK | Cancel] alert
|
|
54
|
+
User fill [T] alert with {{v}}
|
|
55
|
+
User see [message text] alert
|
|
56
|
+
User press Escape key | Enter on [T] field
|
|
57
|
+
User wait for N seconds | [T] dialog | [T] dialog is STATE | [T] page
|
|
58
|
+
User scroll to [T] section
|
|
59
|
+
User switch to [T] frame | [main] frame
|
|
84
60
|
```
|
|
85
61
|
|
|
86
62
|
> Alert steps must appear BEFORE the action that triggers the dialog.
|
|
87
63
|
|
|
88
|
-
###
|
|
89
|
-
|
|
90
|
-
```
|
|
91
|
-
User press Escape key # global key press
|
|
92
|
-
User press Enter on [T] field # key on element
|
|
93
|
-
```
|
|
94
|
-
|
|
95
|
-
### Wait
|
|
96
|
-
|
|
97
|
-
```
|
|
98
|
-
User wait for N seconds # wait timeout
|
|
99
|
-
User wait for [T] dialog # wait for element
|
|
100
|
-
User wait for [T] dialog is STATE # wait for state
|
|
101
|
-
User wait for [T] dialog with {{v}} # wait for element with text
|
|
102
|
-
User wait for [T] page # wait for navigation
|
|
103
|
-
User wait for [T] message # wait for toast/feedback
|
|
104
|
-
User wait for [T] button # wait for button to appear
|
|
105
|
-
```
|
|
106
|
-
|
|
107
|
-
### Scroll & Frame
|
|
108
|
-
|
|
109
|
-
```
|
|
110
|
-
User scroll to [T] section # scroll into view
|
|
111
|
-
User switch to [T] frame # enter iframe
|
|
112
|
-
User switch to [main] frame # exit iframe
|
|
113
|
-
```
|
|
114
|
-
|
|
115
|
-
### Assertions (8 verify patterns)
|
|
64
|
+
### Assertions (8 patterns → determines Playwright assertion)
|
|
116
65
|
|
|
117
66
|
```
|
|
118
|
-
# 1. Visibility
|
|
119
|
-
User see [T] message
|
|
120
|
-
User see [T]
|
|
121
|
-
|
|
122
|
-
#
|
|
123
|
-
User see [T]
|
|
124
|
-
User see [T]
|
|
125
|
-
User see [T]
|
|
126
|
-
|
|
127
|
-
# 3. Partial Text Match (toContainText)
|
|
128
|
-
User see [T] text contains {{v}} # partial text match
|
|
129
|
-
|
|
130
|
-
# 4. Input Value (toHaveValue)
|
|
131
|
-
User see [T] field with {{v}} # input value
|
|
132
|
-
User see [T] dropdown with {{v}} # selected value
|
|
133
|
-
User see [T] date picker with {{v}} # date value
|
|
134
|
-
User see [T] search with {{v}} # search input value
|
|
135
|
-
User see [T] slider with {{v}} # slider value
|
|
136
|
-
|
|
137
|
-
# 5. Component State
|
|
138
|
-
User see [T] button is disabled # state assertion
|
|
139
|
-
User see [T] checkbox is checked # checked state
|
|
140
|
-
User see [T] toggle is unchecked # unchecked state
|
|
141
|
-
User see [T] dialog with {{v}} is hidden # text + state combined
|
|
142
|
-
|
|
143
|
-
# 6. Attribute (toHaveAttribute — when selector YAML has `attribute` field)
|
|
144
|
-
User see [T] image with {{v}} # image src
|
|
145
|
-
User see [T] link with {{v}} # link href
|
|
146
|
-
|
|
147
|
-
# 7. Count
|
|
148
|
-
User see [T] row with {{count}} # element count
|
|
149
|
-
|
|
150
|
-
# 8. Page Context
|
|
151
|
-
User see [T] page # URL assertion
|
|
67
|
+
# 1. Visibility: User see [T] type (NEVER add "is visible") | is hidden
|
|
68
|
+
# 2. Text (toHaveText): User see [T] message | header | label with {{v}}
|
|
69
|
+
# 3. Partial (toContainText): User see [T] text contains {{v}}
|
|
70
|
+
# 4. Input (toHaveValue): User see [T] field | dropdown | date-picker | search | slider with {{v}}
|
|
71
|
+
# 5. State: User see [T] button is disabled | checkbox is checked | dialog with {{v}} is hidden
|
|
72
|
+
# 6. Attribute (toHaveAttribute): User see [T] image | link with {{v}}
|
|
73
|
+
# 7. Count: User see [T] row with {{count}}
|
|
74
|
+
# 8. Page: User see [T] page
|
|
152
75
|
```
|
|
153
76
|
|
|
154
77
|
### Table
|
|
155
78
|
|
|
156
79
|
```
|
|
157
|
-
User see [Col] column in [Table] table
|
|
158
|
-
User see [Ref] row in [Table] table with {{v}}
|
|
159
|
-
User see [Ref] row in [Table] table with {{v}} is hidden
|
|
160
|
-
User see [Table] table with {{count}}
|
|
161
|
-
User see [
|
|
162
|
-
User
|
|
163
|
-
User
|
|
164
|
-
User see [Table] table match data: # exact table match (inline DataTable)
|
|
80
|
+
User see [Col] column in [Table] table
|
|
81
|
+
User see [Ref] row in [Table] table with {{v}}
|
|
82
|
+
User see [Ref] row in [Table] table with {{v}} is hidden
|
|
83
|
+
User see [Table] table with {{count}} | is empty
|
|
84
|
+
User see [Col] column with {{v}}
|
|
85
|
+
User click [Act] button in [Table] table with {{v}}
|
|
86
|
+
User see [Table] table match data:
|
|
165
87
|
| Header1 | Header2 |
|
|
166
88
|
| {{value1}} | {{value2}} |
|
|
167
89
|
```
|
|
168
90
|
|
|
169
|
-
|
|
170
|
-
|
|
171
|
-
Row scope: `see [Ref] row in [Table] table with {{v}}` defines `tableRow`. Subsequent `see [Col] column with {{v}}` checks the cell in that row. With `columns` config → exact `nth(index)`, without → `filter({ hasText })`.
|
|
172
|
-
|
|
173
|
-
Table match data: `see [Table] table match data:` followed by a Cucumber DataTable. First row = headers, remaining rows = expected data. Uses **filter-based matching** — verifies rows containing expected values exist, resilient to data changes, extra rows, and row reordering. Use `{{variable}}` for cell values.
|
|
91
|
+
Row scope: `see [Ref] row in [Table] table with {{v}}` enters scope. Subsequent `see [Col] column with {{v}}` checks cell in that row. Use `table match data:` for multi-row verification.
|
|
174
92
|
|
|
175
93
|
### States
|
|
176
94
|
|
|
@@ -223,6 +141,10 @@ Options: `nth` `exact` `scope` `match` `variant` `frame` `contenteditable` `colu
|
|
|
223
141
|
| `@auto` | Standard scenario, ready for automation |
|
|
224
142
|
| `@manual` | Skip in generation |
|
|
225
143
|
| `@smoke` / `@regression` | Test suite grouping |
|
|
144
|
+
| `@critical` | Priority: system unusable if fails (login, auth, core CRUD) |
|
|
145
|
+
| `@high` | Priority: major feature broken (required validation, key business rules) |
|
|
146
|
+
| `@normal` | Priority: degraded experience (UI layout, secondary flows) |
|
|
147
|
+
| `@low` | Priority: minor/cosmetic (tooltips, hover states, empty states) |
|
|
226
148
|
| `@auth:role` | Use auth storage state for role |
|
|
227
149
|
| `@no-auth` | Disable inherited auth |
|
|
228
150
|
| `@steps:name` | Define reusable step block (base scenario) |
|
|
@@ -232,6 +154,8 @@ Options: `nth` `exact` `scope` `match` `variant` `frame` `contenteditable` `colu
|
|
|
232
154
|
|
|
233
155
|
- Tool executes **only Given→When** of `@steps` scenario (skips Then)
|
|
234
156
|
- The `Given` in `@extend` scenario is the **entry assertion** (confirms state after base steps)
|
|
157
|
+
- **Entry assertion MUST use `Given User is on [X] type`** — NEVER `Given User see [X] type`
|
|
158
|
+
- `Given` keyword ONLY allows `is on` action. `see` = `Then` only.
|
|
235
159
|
- If `@steps` scenario fails, `@extend` scenario is **skipped**
|
|
236
160
|
- Name format: `snake_case` or `kebab-case` with module prefix: `@steps:kudos__open_modal`
|
|
237
161
|
|
|
@@ -240,6 +164,8 @@ Options: `nth` `exact` `scope` `match` `variant` `frame` `contenteditable` `colu
|
|
|
240
164
|
| Error | Wrong | Correct |
|
|
241
165
|
|---|---|---|
|
|
242
166
|
| Wrong keyword | `Given User click [T] button` | `When User click [T] button` |
|
|
167
|
+
| `see` after Given | `Given User see [T] heading with {{v}}` | `Then User see [T] heading with {{v}}` (or `Given User is on [T] page` for entry assertion) |
|
|
168
|
+
| Name ≠ step type | Scenario name says "modal" but step uses `dialog` | Use the **same element type** in both: "...dialog opens" + `[X] dialog` |
|
|
243
169
|
| Wrong action for type | `When User click [T] checkbox` | `When User check [T] checkbox` |
|
|
244
170
|
| press wrong target | `When User press [Submit] button` | `When User press Enter key` |
|
|
245
171
|
| uncheck radio | `When User uncheck [Male] radio` | `When User check [Female] radio` |
|
|
@@ -247,3 +173,39 @@ Options: `nth` `exact` `scope` `match` `variant` `frame` `contenteditable` `colu
|
|
|
247
173
|
| Missing `is` for state | `with {{text}} hidden` | `with {{text}} is hidden` |
|
|
248
174
|
| State as value | `with {{disabled}}` | `is disabled` |
|
|
249
175
|
| Missing target type | `fill [email] with {{v}}` | `fill [email] field with {{v}}` |
|
|
176
|
+
| Using Background | `Background: Given User is on...` | Use `@steps` + `@extend` instead |
|
|
177
|
+
| `is on` after When | `When ... And User is on [X] dialog` | `And User see [X] dialog` or separate Given |
|
|
178
|
+
|
|
179
|
+
## Background vs @steps/@extend
|
|
180
|
+
|
|
181
|
+
**Do NOT use `Background:` block.** Use `@steps` + `@extend` instead.
|
|
182
|
+
|
|
183
|
+
**Why:**
|
|
184
|
+
- `Background` runs before EVERY scenario but cannot set dialog/frame scope correctly
|
|
185
|
+
- `Background` with `When` + `And User is on [X] dialog` creates keyword mismatch (`is on` = Given, not When)
|
|
186
|
+
- `@steps` + `@extend` gives explicit control: base steps run Given→When, extending scenario starts with `Given User is on [X] dialog` (correct scope setup)
|
|
187
|
+
|
|
188
|
+
**Wrong:**
|
|
189
|
+
```gherkin
|
|
190
|
+
Background:
|
|
191
|
+
Given User is on [Kudos] page
|
|
192
|
+
When User click [Open] button
|
|
193
|
+
And User is on [Modal] dialog ← keyword mismatch
|
|
194
|
+
|
|
195
|
+
Scenario: VP-UI-001 Title visible
|
|
196
|
+
Then User see [Title] heading
|
|
197
|
+
```
|
|
198
|
+
|
|
199
|
+
**Correct:**
|
|
200
|
+
```gherkin
|
|
201
|
+
@steps:open_modal
|
|
202
|
+
Scenario: Open modal
|
|
203
|
+
Given User is on [Kudos] page
|
|
204
|
+
When User click [Open] button
|
|
205
|
+
Then User see [Modal] dialog
|
|
206
|
+
|
|
207
|
+
@extend:open_modal
|
|
208
|
+
Scenario: VP-UI-001 Title visible
|
|
209
|
+
Given User is on [Modal] dialog
|
|
210
|
+
Then User see [Title] heading
|
|
211
|
+
```
|
|
@@ -1,20 +1,80 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: sungen-selector-fix
|
|
3
|
-
description: 'Selector fixing strategy —
|
|
3
|
+
description: 'Selector fixing strategy — phased execution, priority-first diagnosis, targeted MCP exploration. Auto-loaded by run-test command.'
|
|
4
4
|
user-invocable: false
|
|
5
5
|
---
|
|
6
6
|
|
|
7
|
-
## Strategy:
|
|
7
|
+
## Strategy: Phased Execution
|
|
8
8
|
|
|
9
|
-
|
|
9
|
+
Run tests in priority waves — catch fundamental issues early, fix critical paths first, let shared fixes cascade to lower-priority tests.
|
|
10
10
|
|
|
11
|
-
**
|
|
11
|
+
**Never run all tests blindly.** Always start with a smoke check.
|
|
12
12
|
|
|
13
13
|
---
|
|
14
14
|
|
|
15
|
-
##
|
|
15
|
+
## Phase 1: Smoke Check (catch fundamentals)
|
|
16
16
|
|
|
17
|
-
|
|
17
|
+
Run **up to 5 scenarios** — pick the first `@critical` or `@high` scenarios in the feature file.
|
|
18
|
+
|
|
19
|
+
```bash
|
|
20
|
+
npx playwright test --grep "VP-.*-001|VP-.*-002|VP-.*-003|VP-.*-004|VP-.*-005" --reporter=line
|
|
21
|
+
```
|
|
22
|
+
|
|
23
|
+
**Purpose:** Detect broken fundamentals before running 50+ tests:
|
|
24
|
+
- Page selector wrong → ALL tests would fail (1 fix, not 50 diagnoses)
|
|
25
|
+
- Auth redirect → need `@no-auth` or user login
|
|
26
|
+
- Base `@steps:` scenario broken → all `@extend:` scenarios would fail
|
|
27
|
+
|
|
28
|
+
**If all 5 pass** → skip to Phase 2.
|
|
29
|
+
**If failures** → diagnose and fix (see Diagnosis & Fix below), then re-run smoke. Max 2 attempts here.
|
|
30
|
+
|
|
31
|
+
---
|
|
32
|
+
|
|
33
|
+
## Phase 2: Priority Wave (@critical + @high)
|
|
34
|
+
|
|
35
|
+
Run all `@critical` and `@high` scenarios:
|
|
36
|
+
|
|
37
|
+
```bash
|
|
38
|
+
npx playwright test --grep "@critical|@high" --reporter=line
|
|
39
|
+
```
|
|
40
|
+
|
|
41
|
+
If your Playwright config doesn't support tag grep, use scenario name grep from the feature file — collect VP IDs of `@critical` and `@high` scenarios.
|
|
42
|
+
|
|
43
|
+
**Fix only failures from this wave.** Most shared selectors (buttons, headings, navigation) get fixed here because critical/high scenarios exercise them.
|
|
44
|
+
|
|
45
|
+
Max 2 fix attempts in this phase.
|
|
46
|
+
|
|
47
|
+
---
|
|
48
|
+
|
|
49
|
+
## Phase 3: Full Run (@normal + @low)
|
|
50
|
+
|
|
51
|
+
Run remaining scenarios:
|
|
52
|
+
|
|
53
|
+
```bash
|
|
54
|
+
npx playwright test --reporter=line
|
|
55
|
+
```
|
|
56
|
+
|
|
57
|
+
Many selectors already fixed from Phase 2 (shared elements). Only diagnose **new** failures — selectors that only appear in lower-priority scenarios.
|
|
58
|
+
|
|
59
|
+
Max 1 fix attempt. If `@low` scenarios still fail after fix → **report and move on**, don't loop.
|
|
60
|
+
|
|
61
|
+
---
|
|
62
|
+
|
|
63
|
+
## Phase 4: Regression
|
|
64
|
+
|
|
65
|
+
One final full run to confirm all phases together:
|
|
66
|
+
|
|
67
|
+
```bash
|
|
68
|
+
npx playwright test --reporter=line
|
|
69
|
+
```
|
|
70
|
+
|
|
71
|
+
Report results. Do NOT enter another fix loop here.
|
|
72
|
+
|
|
73
|
+
---
|
|
74
|
+
|
|
75
|
+
## Diagnosis & Fix (used in each phase)
|
|
76
|
+
|
|
77
|
+
### Step 1: Parse Failures
|
|
18
78
|
|
|
19
79
|
| Error pattern | Root cause | Fix target |
|
|
20
80
|
|---|---|---|
|
|
@@ -28,9 +88,7 @@ Parse Playwright error output to categorize failures:
|
|
|
28
88
|
|
|
29
89
|
**Check `test-results/` first** — Playwright captures failure screenshots automatically. Use these to diagnose before any MCP exploration.
|
|
30
90
|
|
|
31
|
-
|
|
32
|
-
|
|
33
|
-
## Step 2: Targeted MCP Exploration
|
|
91
|
+
### Step 2: Targeted MCP Exploration
|
|
34
92
|
|
|
35
93
|
Only when `test-results/` screenshots are insufficient:
|
|
36
94
|
|
|
@@ -41,11 +99,7 @@ Only when `test-results/` screenshots are insufficient:
|
|
|
41
99
|
|
|
42
100
|
**Never use `sungen makeauth`.** Never use `browser_evaluate` to inject cookies.
|
|
43
101
|
|
|
44
|
-
|
|
45
|
-
|
|
46
|
-
## Step 3: Fix Broken Selectors
|
|
47
|
-
|
|
48
|
-
For each failed selector, find the correct locator from the snapshot:
|
|
102
|
+
### Step 3: Fix Broken Selectors
|
|
49
103
|
|
|
50
104
|
Selector priority (use first applicable):
|
|
51
105
|
|
|
@@ -73,9 +127,18 @@ Common fixes:
|
|
|
73
127
|
- Element in iframe → add `frame` field
|
|
74
128
|
- Dynamic content → use `testid` or structural `role` + `nth`
|
|
75
129
|
|
|
130
|
+
### Step 4: Recompile After Fix
|
|
131
|
+
|
|
132
|
+
Always recompile before re-running:
|
|
133
|
+
```bash
|
|
134
|
+
sungen generate --screen <screen>
|
|
135
|
+
```
|
|
136
|
+
|
|
137
|
+
Then re-run only the current phase's failing tests, not all tests.
|
|
138
|
+
|
|
76
139
|
---
|
|
77
140
|
|
|
78
|
-
##
|
|
141
|
+
## Table Selectors
|
|
79
142
|
|
|
80
143
|
For table patterns, add table selectors with `columns` config:
|
|
81
144
|
|
|
@@ -100,7 +163,7 @@ users:
|
|
|
100
163
|
|
|
101
164
|
---
|
|
102
165
|
|
|
103
|
-
##
|
|
166
|
+
## Detail Screens with Dynamic IDs
|
|
104
167
|
|
|
105
168
|
For screens like `/admin/users/:id`:
|
|
106
169
|
1. Navigate to list page via MCP to find a real record ID
|
|
@@ -114,12 +177,13 @@ user detail:
|
|
|
114
177
|
|
|
115
178
|
---
|
|
116
179
|
|
|
117
|
-
##
|
|
180
|
+
## Attempt Budget Summary
|
|
118
181
|
|
|
119
|
-
|
|
120
|
-
|
|
121
|
-
|
|
122
|
-
|
|
123
|
-
|
|
182
|
+
| Phase | What runs | Max fix attempts | On failure after max |
|
|
183
|
+
|---|---|---|---|
|
|
184
|
+
| 1. Smoke | First 5 @critical/@high | 2 | Ask user — fundamentals broken |
|
|
185
|
+
| 2. Priority | All @critical + @high | 2 | Report failures, continue to Phase 3 |
|
|
186
|
+
| 3. Full | All tests | 1 | Report @low/@normal failures, continue |
|
|
187
|
+
| 4. Regression | All tests | 0 | Report final results |
|
|
124
188
|
|
|
125
|
-
|
|
189
|
+
**Total worst case: 5 fix attempts** (2+2+1), not unbounded loops.
|
|
@@ -22,7 +22,7 @@ For append: read highest `VP-<CAT>-<NNN>`, continue from next number. Never modi
|
|
|
22
22
|
When `qa/screens/<screen>/requirements/` exists:
|
|
23
23
|
- **`spec.md`** — primary: sections, field constraints, validation messages, business rules, states
|
|
24
24
|
- **`ui/`** — supplementary: screenshots for layout/visual context
|
|
25
|
-
- **`
|
|
25
|
+
- **`test-viewpoint.md`** — supplementary: edge cases, known issues
|
|
26
26
|
|
|
27
27
|
Requirements improve every viewpoint: exact error messages for VAL, business rules for LOGIC, role permissions for SEC.
|
|
28
28
|
|
|
@@ -47,16 +47,64 @@ When exploring live page or reading Figma designs, actively collect to hardcode
|
|
|
47
47
|
|
|
48
48
|
## Section Identification
|
|
49
49
|
|
|
50
|
-
Identify sections from page patterns. Use `sungen-viewpoint` skill for the
|
|
50
|
+
Identify sections from page patterns. Use `sungen-viewpoint` skill for the 10 pattern types (Form & Inputs, Data Table, Create/Add, Update/Edit, Delete, Search, Filter, Pagination, Modal/Dialog, List/Card). Present sections and ask user which to focus on.
|
|
51
51
|
|
|
52
|
-
##
|
|
52
|
+
## Test Generation Strategy
|
|
53
|
+
|
|
54
|
+
### Step 1 — Spec-first extraction (always do this first)
|
|
55
|
+
|
|
56
|
+
Before applying any checklist, extract test conditions from `spec.md` (and `test-viewpoint.md` if present):
|
|
57
|
+
- **Validation rules**: field constraints, error messages, required/optional
|
|
58
|
+
- **Business rules**: eligibility, calculation logic, permission-based behavior
|
|
59
|
+
- **State lifecycle**: allowed transitions, blocked transitions
|
|
60
|
+
- **Edge cases**: boundary values, empty states, concurrent conditions
|
|
61
|
+
|
|
62
|
+
These spec-extracted conditions drive **which scenarios exist** — `sungen-viewpoint` only supplements with generic web UI coverage that spec doesn't explicitly state.
|
|
63
|
+
|
|
64
|
+
### Step 2 — Apply test design techniques
|
|
65
|
+
|
|
66
|
+
Apply `sungen-test-design-techniques` to spec-extracted conditions:
|
|
67
|
+
|
|
68
|
+
| Technique | Apply when spec mentions |
|
|
69
|
+
|---|---|
|
|
70
|
+
| EP | Valid/invalid ranges, categories → **one** scenario per class, not per value |
|
|
71
|
+
| BVA | Numeric range, string length → `min-1`, `min`, `max`, `max+1` (compact 4-point default) |
|
|
72
|
+
| Decision Table | 2+ dependent conditions → one scenario per combination (cap at distinct outcomes if >3 conditions) |
|
|
73
|
+
| State Transition | Entity lifecycle → one scenario per valid transition + key invalid transitions |
|
|
74
|
+
|
|
75
|
+
### Step 3 — Fill coverage gaps with viewpoint checklists
|
|
53
76
|
|
|
54
77
|
Use `sungen-viewpoint` skill for per-pattern checklists across 4 viewpoints: UI/UX, Data & Validate, Logic, Security.
|
|
55
78
|
|
|
56
|
-
|
|
79
|
+
Add scenarios for generic UI coverage that spec didn't explicitly state (empty states, loading states, keyboard nav, hover effects). Skip viewpoints truly N/A.
|
|
57
80
|
|
|
58
81
|
**Validation rule**: capture actual error messages from live page or spec.md. Use `User see {{error_var}}` — never assert just "is visible".
|
|
59
82
|
|
|
83
|
+
## Priority Tags (auto-assign)
|
|
84
|
+
|
|
85
|
+
Every scenario **MUST** have exactly one priority tag. Add it before the scenario line (after `@extend:` if present).
|
|
86
|
+
|
|
87
|
+
| Tag | When to use |
|
|
88
|
+
|---|---|
|
|
89
|
+
| `@critical` | System unusable if fails — login/logout, authentication redirect, main create/submit/delete, permission denied |
|
|
90
|
+
| `@high` | Major feature broken — required field validation, core business rules, data displays correctly, key navigation |
|
|
91
|
+
| `@normal` | Degraded experience — UI layout/element presence, secondary flows, optional field validation, search/filter |
|
|
92
|
+
| `@low` | Minor/cosmetic — tooltips, hover states, empty states, default sort, placeholder text |
|
|
93
|
+
|
|
94
|
+
### Auto-assign heuristics
|
|
95
|
+
|
|
96
|
+
| Viewpoint + Pattern | Default priority |
|
|
97
|
+
|---|---|
|
|
98
|
+
| VP-SEC-* (all security scenarios) | `@critical` |
|
|
99
|
+
| VP-LOGIC-* with create/submit/delete/login | `@critical` |
|
|
100
|
+
| VP-LOGIC-* other state changes | `@high` |
|
|
101
|
+
| VP-VAL-* required field / submit empty | `@high` |
|
|
102
|
+
| VP-VAL-* format, boundary, optional fields | `@normal` |
|
|
103
|
+
| VP-UI-* form fields present, table columns | `@normal` |
|
|
104
|
+
| VP-UI-* hover, tooltip, empty state, placeholder | `@low` |
|
|
105
|
+
|
|
106
|
+
**`@steps:` scenarios** do NOT get a priority tag (they are setup blocks, not test cases).
|
|
107
|
+
|
|
60
108
|
## SPA Wait-For Steps
|
|
61
109
|
|
|
62
110
|
```gherkin
|
|
@@ -68,51 +116,44 @@ And User wait for [Page Title] heading is visible
|
|
|
68
116
|
|
|
69
117
|
**Feature file** — `qa/screens/<screen>/features/<screen>.feature`
|
|
70
118
|
|
|
119
|
+
**Never use `Background:`.** Use `@steps` + `@extend` for shared setup (see `sungen-gherkin-syntax` skill).
|
|
120
|
+
|
|
71
121
|
```gherkin
|
|
72
122
|
@auth:role
|
|
73
123
|
Feature: <Screen> Screen
|
|
74
124
|
|
|
75
|
-
#
|
|
76
|
-
|
|
77
|
-
|
|
125
|
+
# Shared setup — NO priority tag on @steps
|
|
126
|
+
@steps:open_form
|
|
127
|
+
Scenario: Open form
|
|
128
|
+
Given User is on [Screen] page
|
|
129
|
+
When User click [Create] button
|
|
130
|
+
Then User see [Form] dialog
|
|
78
131
|
|
|
79
|
-
# ---
|
|
132
|
+
# --- Section: Create User Form ---
|
|
80
133
|
|
|
134
|
+
@normal @extend:open_form
|
|
81
135
|
Scenario: VP-UI-001 Form displays all fields with correct defaults
|
|
82
|
-
Given User is on [
|
|
136
|
+
Given User is on [Form] dialog
|
|
83
137
|
Then User see [Name] field
|
|
84
|
-
And User see [Email] field
|
|
85
138
|
And User see [Submit] button is disabled
|
|
86
139
|
|
|
87
|
-
|
|
88
|
-
|
|
140
|
+
@high @extend:open_form
|
|
89
141
|
Scenario: VP-VAL-001 Submit with all empty fields shows errors
|
|
142
|
+
Given User is on [Form] dialog
|
|
90
143
|
When User click [Submit] button
|
|
91
144
|
Then User see [Name error] message with {{name_required_error}}
|
|
92
145
|
|
|
93
|
-
#
|
|
94
|
-
# Section: User Table
|
|
95
|
-
# ============================================================
|
|
96
|
-
|
|
97
|
-
# --- UI/UX ---
|
|
146
|
+
# --- Section: User Table ---
|
|
98
147
|
|
|
148
|
+
@normal
|
|
99
149
|
Scenario: VP-UI-010 Table displays all columns
|
|
100
150
|
Then User see [Name] column in [Users] table
|
|
101
|
-
And User see [Email] column in [Users] table
|
|
102
|
-
And User see [Status] column in [Users] table
|
|
103
|
-
|
|
104
|
-
# --- Data & Validate ---
|
|
105
151
|
|
|
152
|
+
@high
|
|
106
153
|
Scenario: VP-VAL-010 Table displays correct data
|
|
107
154
|
Then User see [Users] table match data:
|
|
108
|
-
| Name
|
|
109
|
-
| {{name_1}}
|
|
110
|
-
| {{name_2}} | {{email_2}} | {{status_2}} |
|
|
111
|
-
|
|
112
|
-
Scenario: VP-VAL-011 Edit button targets correct row
|
|
113
|
-
Given User see [Target] row in [Users] table with {{name_1}}
|
|
114
|
-
When User click [Edit] button in [Users] table with {{name_1}}
|
|
115
|
-
Then User see [Name] field with {{name_1}}
|
|
155
|
+
| Name | Email |
|
|
156
|
+
| {{name_1}} | {{email_1}} |
|
|
116
157
|
```
|
|
117
158
|
|
|
118
159
|
### When to use DataTable vs Row Scope
|
|
@@ -122,7 +163,7 @@ Feature: <Screen> Screen
|
|
|
122
163
|
| `table match data:` + DataTable | Verifying **multiple rows** exist with expected values |
|
|
123
164
|
| `row in [Table] table with {{v}}` + `column with {{v}}` | Checking **single row** details or **acting** on a row (click, edit) |
|
|
124
165
|
|
|
125
|
-
**Naming**: `VP-<CATEGORY>-<NNN>` prefix.
|
|
166
|
+
**Naming**: `VP-<CATEGORY>-<NNN>` prefix. Scenario name must use the **same element type** as the steps — e.g., if the step uses `dialog`, write "dialog opens" not "modal opens".
|
|
126
167
|
|
|
127
168
|
**Test data** — `qa/screens/<screen>/test-data/<screen>.yaml`, grouped by section.
|
|
128
169
|
|
|
@@ -20,16 +20,16 @@ user-invocable: false
|
|
|
20
20
|
|
|
21
21
|
### Coverage (40 pts)
|
|
22
22
|
|
|
23
|
-
| Dimension | Pts |
|
|
24
|
-
|
|
25
|
-
| Happy paths | 8 |
|
|
26
|
-
| Negative cases | 8 |
|
|
27
|
-
| Edge cases |
|
|
28
|
-
| Boundary values |
|
|
29
|
-
| State transitions | 5 |
|
|
30
|
-
|
|
|
23
|
+
| Dimension | Technique | Pts | What to check |
|
|
24
|
+
|---|---|---|---|
|
|
25
|
+
| Happy paths | — | 8 | Core success flows exist |
|
|
26
|
+
| Negative cases | EP | 8 | One scenario per invalid class, no redundant same-class scenarios |
|
|
27
|
+
| Edge cases | EP | 6 | Empty, null, whitespace, special chars covered |
|
|
28
|
+
| Boundary values | BVA | 8 | `min-1`, `min`, `max`, `max+1` for each spec range |
|
|
29
|
+
| State transitions | ST | 5 | Valid transitions + key blocked paths from spec |
|
|
30
|
+
| Condition combos | DT | 5 | Dependent conditions covered, distinct outcomes tested |
|
|
31
31
|
|
|
32
|
-
Score: `(dimensions_covered / 6) * 40`. Per-pattern checklists → `sungen-viewpoint` skill.
|
|
32
|
+
Score: `(dimensions_covered / 6) * 40`. Validate technique application with `sungen-test-design-techniques`. Per-pattern checklists → `sungen-viewpoint` skill.
|
|
33
33
|
|
|
34
34
|
### Viewpoint (30 pts)
|
|
35
35
|
|
|
@@ -37,7 +37,8 @@ Score: `(dimensions_covered / 6) * 40`. Per-pattern checklists → `sungen-viewp
|
|
|
37
37
|
|---|---|
|
|
38
38
|
| All applicable VP present (UI/VAL/LOGIC/SEC) | 10 |
|
|
39
39
|
| Correct classification | 8 |
|
|
40
|
-
| `VP-<CAT>-<NNN>` naming + section grouping |
|
|
40
|
+
| `VP-<CAT>-<NNN>` naming + section grouping | 4 |
|
|
41
|
+
| Priority tag present and correct (`@critical`/`@high`/`@normal`/`@low`) | 4 |
|
|
41
42
|
| Assertion quality (see rules below) | 4 |
|
|
42
43
|
|
|
43
44
|
**Classification**: UI = static/always-same appearance. VAL = input validation/errors. LOGIC = behavior/state changes (includes persisted state without When). SEC = auth/permissions.
|
|
@@ -58,6 +59,7 @@ Score: `(dimensions_covered / 6) * 40`. Per-pattern checklists → `sungen-viewp
|
|
|
58
59
|
2. **`When fill [X]`** → Then must assert the **visible result** (search results, validation error). Don't re-assert the field value.
|
|
59
60
|
3. **UI-only scenarios** (no action needed) → use Given + Then without When.
|
|
60
61
|
4. **Scenario name must match the assertion**, not the action.
|
|
62
|
+
5. **Scenario name must use the same element type as the steps** — e.g., "dialog opens" + `[X] dialog`, never "modal opens" + `[X] dialog`.
|
|
61
63
|
|
|
62
64
|
### @manual Rules
|
|
63
65
|
|
|
@@ -72,13 +74,16 @@ Do NOT mark `@manual` when data is visible in snapshot or documented in spec —
|
|
|
72
74
|
|
|
73
75
|
## Checklist (auto-fix on detection)
|
|
74
76
|
|
|
75
|
-
1. **Redundant scenarios** —
|
|
77
|
+
1. **Redundant scenarios (EP violation)** — multiple scenarios testing same equivalence class? Keep one representative, remove rest
|
|
76
78
|
2. **Misclassified VP** — UI tests behavior? Move to LOGIC. Logic tests appearance? Move to UI
|
|
77
79
|
3. **Dynamic content** — exact match on counters/timestamps? Use `contains` instead
|
|
78
80
|
4. **Duplicate across sections** — SEC scenario identical to UI? Remove duplicate
|
|
79
|
-
5. **
|
|
80
|
-
6. **
|
|
81
|
-
7. **
|
|
81
|
+
5. **Missing/wrong priority tag** — every non-`@steps` scenario needs exactly one of `@critical`/`@high`/`@normal`/`@low`. SEC→`@critical`, VAL required→`@high`, UI layout→`@normal`, hover/tooltip→`@low`
|
|
82
|
+
6. **Always-enabled elements** — `is enabled` on never-disabled element? Remove
|
|
83
|
+
7. **Test-data completeness** — every `{{var}}` must exist in test-data.yaml
|
|
84
|
+
8. **Missing BVA boundaries** — spec defines min/max range but scenarios only test midpoint? Add `min-1`, `min`, `max`, `max+1`
|
|
85
|
+
9. **Missing state transitions** — spec defines lifecycle states but only happy path tested? Add blocked transitions
|
|
86
|
+
10. **Uncovered condition combos** — spec has 2+ dependent conditions but only partial combinations tested? Build decision table
|
|
82
87
|
|
|
83
88
|
---
|
|
84
89
|
|