@sun-asterisk/sungen 2.3.1 → 2.4.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (90) hide show
  1. package/README.md +10 -11
  2. package/dist/cli/index.js +1 -1
  3. package/dist/generators/test-generator/adapters/playwright/templates/steps/actions/table-action-in-row.hbs +1 -1
  4. package/dist/generators/test-generator/adapters/playwright/templates/steps/assertions/table-column-exists.hbs +5 -1
  5. package/dist/generators/test-generator/adapters/playwright/templates/steps/assertions/table-row-exists.hbs +7 -1
  6. package/dist/generators/test-generator/adapters/playwright/templates/steps/assertions/table-row-not-exists.hbs +5 -1
  7. package/dist/generators/test-generator/patterns/table-patterns.d.ts +9 -0
  8. package/dist/generators/test-generator/patterns/table-patterns.d.ts.map +1 -1
  9. package/dist/generators/test-generator/patterns/table-patterns.js +94 -98
  10. package/dist/generators/test-generator/patterns/table-patterns.js.map +1 -1
  11. package/dist/generators/test-generator/patterns/types.d.ts +2 -0
  12. package/dist/generators/test-generator/patterns/types.d.ts.map +1 -1
  13. package/dist/generators/test-generator/step-mapper.d.ts +13 -0
  14. package/dist/generators/test-generator/step-mapper.d.ts.map +1 -1
  15. package/dist/generators/test-generator/step-mapper.js +80 -0
  16. package/dist/generators/test-generator/step-mapper.js.map +1 -1
  17. package/dist/orchestrator/ai-rules-updater.d.ts.map +1 -1
  18. package/dist/orchestrator/ai-rules-updater.js +8 -6
  19. package/dist/orchestrator/ai-rules-updater.js.map +1 -1
  20. package/dist/orchestrator/project-initializer.d.ts.map +1 -1
  21. package/dist/orchestrator/project-initializer.js +33 -2
  22. package/dist/orchestrator/project-initializer.js.map +1 -1
  23. package/dist/orchestrator/screen-manager.js +1 -1
  24. package/dist/orchestrator/screen-manager.js.map +1 -1
  25. package/dist/orchestrator/templates/ai-instructions/claude-cmd-add-screen.md +5 -5
  26. package/{src/orchestrator/templates/ai-instructions/claude-cmd-make-tc.md → dist/orchestrator/templates/ai-instructions/claude-cmd-create-test.md} +7 -7
  27. package/dist/orchestrator/templates/ai-instructions/claude-cmd-review.md +21 -0
  28. package/{src/orchestrator/templates/ai-instructions/claude-cmd-make-test.md → dist/orchestrator/templates/ai-instructions/claude-cmd-run-test.md} +2 -2
  29. package/dist/orchestrator/templates/ai-instructions/claude-config.md +65 -12
  30. package/dist/orchestrator/templates/ai-instructions/claude-skill-error-mapping.md +128 -52
  31. package/dist/orchestrator/templates/ai-instructions/claude-skill-gherkin-syntax.md +9 -40
  32. package/dist/orchestrator/templates/ai-instructions/claude-skill-selector-fix.md +67 -239
  33. package/dist/orchestrator/templates/ai-instructions/claude-skill-tc-generation.md +40 -208
  34. package/dist/orchestrator/templates/ai-instructions/claude-skill-tc-review.md +104 -0
  35. package/dist/orchestrator/templates/ai-instructions/claude-skill-viewpoint.md +186 -0
  36. package/dist/orchestrator/templates/ai-instructions/copilot-cmd-add-screen.md +4 -4
  37. package/dist/orchestrator/templates/ai-instructions/{copilot-cmd-make-tc.md → copilot-cmd-create-test.md} +8 -8
  38. package/dist/orchestrator/templates/ai-instructions/copilot-cmd-review.md +24 -0
  39. package/{src/orchestrator/templates/ai-instructions/copilot-cmd-make-test.md → dist/orchestrator/templates/ai-instructions/copilot-cmd-run-test.md} +3 -3
  40. package/dist/orchestrator/templates/ai-instructions/copilot-config.md +66 -13
  41. package/dist/orchestrator/templates/ai-instructions/github-skill-sungen-error-mapping.md +128 -52
  42. package/dist/orchestrator/templates/ai-instructions/github-skill-sungen-gherkin-syntax.md +9 -40
  43. package/dist/orchestrator/templates/ai-instructions/github-skill-sungen-selector-fix.md +67 -214
  44. package/dist/orchestrator/templates/ai-instructions/github-skill-sungen-tc-generation.md +40 -208
  45. package/dist/orchestrator/templates/ai-instructions/github-skill-sungen-tc-review.md +104 -0
  46. package/dist/orchestrator/templates/ai-instructions/github-skill-sungen-viewpoint.md +186 -0
  47. package/dist/orchestrator/templates/readme.md +4 -4
  48. package/package.json +1 -1
  49. package/src/cli/index.ts +1 -1
  50. package/src/generators/test-generator/adapters/playwright/templates/steps/actions/table-action-in-row.hbs +1 -1
  51. package/src/generators/test-generator/adapters/playwright/templates/steps/assertions/table-column-exists.hbs +5 -1
  52. package/src/generators/test-generator/adapters/playwright/templates/steps/assertions/table-row-exists.hbs +7 -1
  53. package/src/generators/test-generator/adapters/playwright/templates/steps/assertions/table-row-not-exists.hbs +5 -1
  54. package/src/generators/test-generator/patterns/table-patterns.ts +98 -111
  55. package/src/generators/test-generator/patterns/types.ts +2 -0
  56. package/src/generators/test-generator/step-mapper.ts +87 -1
  57. package/src/orchestrator/ai-rules-updater.ts +8 -6
  58. package/src/orchestrator/project-initializer.ts +38 -2
  59. package/src/orchestrator/screen-manager.ts +1 -1
  60. package/src/orchestrator/templates/ai-instructions/claude-cmd-add-screen.md +5 -5
  61. package/{dist/orchestrator/templates/ai-instructions/claude-cmd-make-tc.md → src/orchestrator/templates/ai-instructions/claude-cmd-create-test.md} +7 -7
  62. package/src/orchestrator/templates/ai-instructions/claude-cmd-review.md +21 -0
  63. package/{dist/orchestrator/templates/ai-instructions/claude-cmd-make-test.md → src/orchestrator/templates/ai-instructions/claude-cmd-run-test.md} +2 -2
  64. package/src/orchestrator/templates/ai-instructions/claude-config.md +65 -12
  65. package/src/orchestrator/templates/ai-instructions/claude-skill-error-mapping.md +128 -52
  66. package/src/orchestrator/templates/ai-instructions/claude-skill-gherkin-syntax.md +9 -40
  67. package/src/orchestrator/templates/ai-instructions/claude-skill-selector-fix.md +67 -239
  68. package/src/orchestrator/templates/ai-instructions/claude-skill-tc-generation.md +40 -208
  69. package/src/orchestrator/templates/ai-instructions/claude-skill-tc-review.md +104 -0
  70. package/src/orchestrator/templates/ai-instructions/claude-skill-viewpoint.md +186 -0
  71. package/src/orchestrator/templates/ai-instructions/copilot-cmd-add-screen.md +4 -4
  72. package/src/orchestrator/templates/ai-instructions/{copilot-cmd-make-tc.md → copilot-cmd-create-test.md} +8 -8
  73. package/src/orchestrator/templates/ai-instructions/copilot-cmd-review.md +24 -0
  74. package/{dist/orchestrator/templates/ai-instructions/copilot-cmd-make-test.md → src/orchestrator/templates/ai-instructions/copilot-cmd-run-test.md} +3 -3
  75. package/src/orchestrator/templates/ai-instructions/copilot-config.md +66 -13
  76. package/src/orchestrator/templates/ai-instructions/github-skill-sungen-error-mapping.md +128 -52
  77. package/src/orchestrator/templates/ai-instructions/github-skill-sungen-gherkin-syntax.md +9 -40
  78. package/src/orchestrator/templates/ai-instructions/github-skill-sungen-selector-fix.md +67 -214
  79. package/src/orchestrator/templates/ai-instructions/github-skill-sungen-tc-generation.md +40 -208
  80. package/src/orchestrator/templates/ai-instructions/github-skill-sungen-tc-review.md +104 -0
  81. package/src/orchestrator/templates/ai-instructions/github-skill-sungen-viewpoint.md +186 -0
  82. package/src/orchestrator/templates/readme.md +4 -4
  83. package/dist/generators/test-generator/adapters/playwright/templates/steps/assertions/table-cell-by-filter.hbs +0 -2
  84. package/dist/generators/test-generator/adapters/playwright/templates/steps/assertions/table-cell-by-index.hbs +0 -2
  85. package/dist/orchestrator/templates/ai-instructions/claude-skill-gherkin-review.md +0 -228
  86. package/dist/orchestrator/templates/ai-instructions/github-skill-sungen-gherkin-review.md +0 -228
  87. package/src/generators/test-generator/adapters/playwright/templates/steps/assertions/table-cell-by-filter.hbs +0 -2
  88. package/src/generators/test-generator/adapters/playwright/templates/steps/assertions/table-cell-by-index.hbs +0 -2
  89. package/src/orchestrator/templates/ai-instructions/claude-skill-gherkin-review.md +0 -228
  90. package/src/orchestrator/templates/ai-instructions/github-skill-sungen-gherkin-review.md +0 -228
@@ -1,228 +0,0 @@
1
- ---
2
- name: sungen-gherkin-review
3
- description: 'Quality review checklist for Gherkin test cases — assertion quality rules, action-result coherence, and 9-point checklist. Auto-loaded during self-review step of make-tc.'
4
- user-invocable: false
5
- ---
6
-
7
- ## Assertion Quality Rules
8
-
9
- **CRITICAL** — these rules prevent shallow, low-value test cases:
10
-
11
- 1. **NEVER use `is visible`** — `User see [T] type` already asserts visibility. Writing `is visible` is redundant noise. Only use `is hidden` to assert something is NOT shown.
12
- 2. **Group related assertions** — one scenario can have 3-7 `Then/And` steps. Don't waste a whole scenario on one element. Group elements that verify the same concern.
13
- 3. **Assert content, not just existence** — verify values, text, states, not just "it's there". Every assertion should answer: what EXACTLY should the user see? Add `with {{value}}` or `is state` whenever the expected content/state is known.
14
- - `User see [Title] heading with {{page_title}}` — verify text
15
- - `User see [Email] field with {{default_email}}` — verify default value
16
- - `User see [Submit] button is disabled` — verify state
17
- - `User see [Error] message with {{error_text}}` — verify exact error
18
- 4. **Don't assume element roles** — never guess the element type (`button`, `option`, `link`, etc.) based on the label or expected behavior. Developers create custom components that don't match standard HTML roles (e.g., a `<div>` styled as a button, a custom dropdown built from `<li>` elements). Always use the **live-page selector scan result** or the selector YAML to determine the correct type. If you haven't scanned the page and the type is uncertain, use a generic type (`text`) or mark the scenario `@manual` until selectors are confirmed.
19
-
20
- **Bad** (shallow — just checks existence):
21
- ```gherkin
22
- Scenario: VP-UI-001 Email field is visible
23
- Given User is on [Login] page
24
- Then User see [Email] field is visible
25
-
26
- Scenario: VP-UI-002 Password field is visible
27
- Given User is on [Login] page
28
- Then User see [Password] field is visible
29
- ```
30
-
31
- **Good** (rich — verifies content, states, groups related checks):
32
- ```gherkin
33
- Scenario: VP-UI-001 Login form displays all fields with correct defaults
34
- Given User is on [Login] page
35
- Then User see [Login] heading with {{login_title}}
36
- And User see [Email] field
37
- And User see [Password] field
38
- And User see [Remember me] checkbox is unchecked
39
- And User see [Submit] button is enabled
40
- And User see [Forgot password] link
41
- ```
42
-
43
- ## Action-Result Coherence Rules
44
-
45
- Every scenario with a `When` action **must** assert the *result* of that action in `Then` — not just re-assert something already visible.
46
-
47
- **Anti-patterns (WRONG):**
48
- ```gherkin
49
- # BAD: click then assert the same element unchanged
50
- When User click [Select language] button
51
- Then User see [Select language] button
52
-
53
- # BAD: fill then assert unrelated page (fill doesn't navigate)
54
- When User fill [Search] field with {{term}}
55
- Then User see [Home] page
56
-
57
- # BAD: action with no meaningful result check
58
- When User click [Submit] button
59
- Then User see [Submit] button
60
- ```
61
-
62
- **Correct patterns:**
63
- ```gherkin
64
- # GOOD: click opens something new (dropdown, dialog, page)
65
- When User click [Select language] button
66
- Then User see [VN] button
67
-
68
- # GOOD: click changes state on the element itself
69
- When User click [Submit] button
70
- Then User see [Submit] button is disabled
71
-
72
- # GOOD: click navigates to different page
73
- When User click [Login] button
74
- Then User see [Dashboard] page
75
-
76
- # GOOD: fill triggers visible result (search results, validation)
77
- When User fill [Search] field with {{term}}
78
- Then User see [search result] text with {{result_name}}
79
-
80
- # GOOD: fill then submit, assert result of submission
81
- When User fill [Email] field with {{invalid_email}}
82
- And User click [Submit] button
83
- Then User see [email error] message with {{email_error_text}}
84
-
85
- # GOOD: click opens dialog
86
- When User click [Submit] button
87
- Then User see [Confirm] dialog
88
-
89
- # GOOD: visibility-only scenario has no When (just Given + Then)
90
- Given User is on [Login] page
91
- Then User see [Email] field
92
- ```
93
-
94
- **Rules:**
95
- 1. `When click [X]` → `Then` must assert either a **new element appears** (dialog, dropdown, page change, new content) or a **state change on `[X]` itself** (e.g., `is disabled`, `is checked`). Never assert `[X]` unchanged. Asserting something already visible before the click is a pre-existing state error — if the result is uncertain, mark `@manual`.
96
- 2. `When fill [X] field` → `Then` must assert the **visible result of the input** (search results appear, validation message shows, dropdown filters). Do NOT just assert the field has the value — `see [X] field with {{v}}` after fill is a **weak test** (Playwright fill already guarantees the value is set). Only use field value assertion when the field transforms the input (e.g., auto-format phone number, currency mask).
97
- - **Search fields need a wait step** — search inputs typically have debounce/delay before results appear. After filling a search field, add `User wait for {{search_delay}}` before asserting results. Without this, the assertion fires before results render and the test flakes.
98
- ```gherkin
99
- # GOOD: wait for debounce before asserting search results
100
- When User fill [Search] field with {{term}}
101
- And User wait for {{search_delay}}
102
- Then User see [search result] text with {{result_name}}
103
- ```
104
- 3. If you only want to verify an element **exists/is visible** — use a UI/UX scenario with **no `When`** (just `Given` + `Then`)
105
- 4. If the result of an action is **unknown or uncertain** (e.g., what appears after filling a search, what dialog opens after click), either **explore via MCP first** to see the actual result, or **mark the scenario `@manual`**. Never guess the result — wrong assertions cause test failures that waste fix cycles.
106
- 5. Scenario **name must match the actual assertion**, not the action. "Fill searchbox shows search results" must assert search results — not field value.
107
-
108
- ## Quality Review Checklist (9 checks, auto-fix on detection)
109
-
110
- After generating scenarios, review every scenario against these checks. **If an issue is detected, fix it immediately** — do not just flag it.
111
-
112
- ### 1. Redundant scenarios
113
-
114
- **Problem:** Two scenarios test the same element with overlapping assertions.
115
- ```gherkin
116
- # REDUNDANT: VP-UI-009 and VP-UI-010 both test CTA button
117
- Scenario: CTA button is visible → Then see [cta] button
118
- Scenario: CTA button shows text → Then see [cta] button with {{text}}
119
- ```
120
- **Fix:** Keep only the stronger assertion (`with {{text}}` implies visibility). Remove the weaker one.
121
-
122
- ### 2. Misclassified viewpoint
123
-
124
- **Problem:** A scenario classified in the wrong viewpoint category.
125
-
126
- **Classification rules:**
127
- - **UI/UX** = Given + Then asserting **static, always-the-same** defaults (layout, placeholder text, initial state on first-ever load)
128
- - **Logic** = Given + When + Then (action causes a result), OR Given + Then asserting **behavior-dependent state** (persisted values, dynamic content, conditional defaults)
129
-
130
- **Key distinction:** If the element's state depends on **previous user interaction, saved preferences, or business rules** — it's Logic even without a `When`, because it verifies behavior, not layout.
131
-
132
- ```gherkin
133
- # UI/UX — static default, always the same for every user
134
- Given User is on [Settings] page
135
- Then User see [Language] dropdown with {{default_language}}
136
-
137
- # Logic — checkbox remembers last user choice (persisted state)
138
- Given User is on [Settings] page
139
- Then User see [Newsletter] checkbox is checked
140
-
141
- # Logic — empty state text depends on business rule (no data yet)
142
- Given User is on [Orders] page
143
- Then User see [empty state] text with {{no_orders_message}}
144
- ```
145
-
146
- **Fix:** Check whether the asserted state is truly static (same for all users, all times) or depends on data/behavior. Move accordingly.
147
-
148
- ### 3. Dynamic/data-dependent content
149
-
150
- **Problem:** Asserting content that depends on live data which may change.
151
- ```gherkin
152
- # FRAGILE: "Attachment 1" only exists if first card has an attachment
153
- Scenario: Card shows attachment image
154
- Then User see [Attachment 1] image
155
- ```
156
- **Fix:** Mark as `@manual` with a comment explaining the data dependency.
157
-
158
- ### 4. Duplicate across sections
159
-
160
- **Problem:** Security scenario has identical steps to a UI scenario.
161
- ```gherkin
162
- # VP-UI-006 and VP-SEC-002 are identical
163
- Scenario: VP-UI-006 User profile button is visible
164
- Scenario: VP-SEC-002 Authenticated user sees profile button
165
- ```
166
- **Fix:** Remove the duplicate. Security scenarios should test **auth boundaries** (e.g., `@no-auth` → redirect), not re-test visibility.
167
-
168
- ### 5. "Enabled" state on always-enabled elements
169
-
170
- **Problem:** Testing `is enabled` on elements that have no disabled state in the application.
171
- ```gherkin
172
- # BAD: this button is never disabled — the assertion adds no value
173
- Then User see [Home] button is enabled
174
- ```
175
- **Fix:** Remove. Only assert `is enabled` or `is disabled` when the element **actually toggles** between states based on conditions (e.g., form validity, permissions).
176
-
177
- ### 6. Exact match on dynamic counters/numbers
178
-
179
- **Problem:** Using exact match (`with {{value}}`) on data that changes (counters, totals, timestamps).
180
- ```gherkin
181
- # FRAGILE: "+61 KUDOS" changes when new kudos are sent
182
- Then User see [kudos count] text with {{kudos_count}}
183
- ```
184
- **Fix:** Use partial match (`text contains`) with a stable keyword:
185
- ```gherkin
186
- Then User see [kudos count] text contains {{kudos_count_keyword}}
187
- ```
188
-
189
- ### 7. Current active page link
190
-
191
- **Problem:** Testing visibility of a nav link pointing to the current page.
192
- ```gherkin
193
- # LOW VALUE: we are ON /kudos, Sun* Kudos link is always there
194
- Scenario: Nav link Sun Kudos is visible
195
- Given User is on [kudos] page
196
- Then User see [Sun* Kudos] link
197
- ```
198
- **Fix:** Remove or replace with active state test.
199
-
200
- ### 8. Test-data completeness
201
-
202
- **Problem:** Feature file references `{{variable}}` but the corresponding key is missing in `test-data.yaml`.
203
- ```gherkin
204
- # Feature uses {{login_error}} but test-data.yaml has no "login_error" key
205
- When User fill [Email] field with {{invalid_email}}
206
- And User click [Submit] button
207
- Then User see [error] message with {{login_error}}
208
- ```
209
- **Fix:** After generation, verify every `{{variable}}` in the `.feature` file has a matching entry in `test-data.yaml`. Add missing entries with realistic values. This is a **mandatory post-generation check** — missing test data causes compile failures.
210
-
211
- ### 9. Negative/boundary coverage
212
-
213
- **Problem:** All scenarios are happy-path. No validation, error, or boundary cases exist.
214
- ```gherkin
215
- # BAD: only tests successful login — what about invalid credentials?
216
- Scenario: VP-LOGIC-001 User logs in successfully
217
- Given User is on [Login] page
218
- When User fill [Email] field with {{valid_email}}
219
- And User fill [Password] field with {{valid_password}}
220
- And User click [Submit] button
221
- Then User see [Dashboard] page
222
- ```
223
- **Fix:** For every form/input section, ensure at least one scenario covers:
224
- - **Required field empty** → validation message appears
225
- - **Invalid format** (email, phone, etc.) → format error appears
226
- - **Boundary values** (max length, min value) → when relevant
227
-
228
- If specific error messages are unknown, mark validation scenarios `@manual` rather than omitting them entirely.
@@ -1,228 +0,0 @@
1
- ---
2
- name: sungen-gherkin-review
3
- description: 'Quality review checklist for Gherkin test cases — assertion quality rules, action-result coherence, and 9-point checklist. Auto-loaded during self-review step of make-tc.'
4
- user-invocable: false
5
- ---
6
-
7
- ## Assertion Quality Rules
8
-
9
- **CRITICAL** — these rules prevent shallow, low-value test cases:
10
-
11
- 1. **NEVER use `is visible`** — `User see [T] type` already asserts visibility. Writing `is visible` is redundant noise. Only use `is hidden` to assert something is NOT shown.
12
- 2. **Group related assertions** — one scenario can have 3-7 `Then/And` steps. Don't waste a whole scenario on one element. Group elements that verify the same concern.
13
- 3. **Assert content, not just existence** — verify values, text, states, not just "it's there". Every assertion should answer: what EXACTLY should the user see? Add `with {{value}}` or `is state` whenever the expected content/state is known.
14
- - `User see [Title] heading with {{page_title}}` — verify text
15
- - `User see [Email] field with {{default_email}}` — verify default value
16
- - `User see [Submit] button is disabled` — verify state
17
- - `User see [Error] message with {{error_text}}` — verify exact error
18
- 4. **Don't assume element roles** — never guess the element type (`button`, `option`, `link`, etc.) based on the label or expected behavior. Developers create custom components that don't match standard HTML roles (e.g., a `<div>` styled as a button, a custom dropdown built from `<li>` elements). Always use the **live-page selector scan result** or the selector YAML to determine the correct type. If you haven't scanned the page and the type is uncertain, use a generic type (`text`) or mark the scenario `@manual` until selectors are confirmed.
19
-
20
- **Bad** (shallow — just checks existence):
21
- ```gherkin
22
- Scenario: VP-UI-001 Email field is visible
23
- Given User is on [Login] page
24
- Then User see [Email] field is visible
25
-
26
- Scenario: VP-UI-002 Password field is visible
27
- Given User is on [Login] page
28
- Then User see [Password] field is visible
29
- ```
30
-
31
- **Good** (rich — verifies content, states, groups related checks):
32
- ```gherkin
33
- Scenario: VP-UI-001 Login form displays all fields with correct defaults
34
- Given User is on [Login] page
35
- Then User see [Login] heading with {{login_title}}
36
- And User see [Email] field
37
- And User see [Password] field
38
- And User see [Remember me] checkbox is unchecked
39
- And User see [Submit] button is enabled
40
- And User see [Forgot password] link
41
- ```
42
-
43
- ## Action-Result Coherence Rules
44
-
45
- Every scenario with a `When` action **must** assert the *result* of that action in `Then` — not just re-assert something already visible.
46
-
47
- **Anti-patterns (WRONG):**
48
- ```gherkin
49
- # BAD: click then assert the same element unchanged
50
- When User click [Select language] button
51
- Then User see [Select language] button
52
-
53
- # BAD: fill then assert unrelated page (fill doesn't navigate)
54
- When User fill [Search] field with {{term}}
55
- Then User see [Home] page
56
-
57
- # BAD: action with no meaningful result check
58
- When User click [Submit] button
59
- Then User see [Submit] button
60
- ```
61
-
62
- **Correct patterns:**
63
- ```gherkin
64
- # GOOD: click opens something new (dropdown, dialog, page)
65
- When User click [Select language] button
66
- Then User see [VN] button
67
-
68
- # GOOD: click changes state on the element itself
69
- When User click [Submit] button
70
- Then User see [Submit] button is disabled
71
-
72
- # GOOD: click navigates to different page
73
- When User click [Login] button
74
- Then User see [Dashboard] page
75
-
76
- # GOOD: fill triggers visible result (search results, validation)
77
- When User fill [Search] field with {{term}}
78
- Then User see [search result] text with {{result_name}}
79
-
80
- # GOOD: fill then submit, assert result of submission
81
- When User fill [Email] field with {{invalid_email}}
82
- And User click [Submit] button
83
- Then User see [email error] message with {{email_error_text}}
84
-
85
- # GOOD: click opens dialog
86
- When User click [Submit] button
87
- Then User see [Confirm] dialog
88
-
89
- # GOOD: visibility-only scenario has no When (just Given + Then)
90
- Given User is on [Login] page
91
- Then User see [Email] field
92
- ```
93
-
94
- **Rules:**
95
- 1. `When click [X]` → `Then` must assert either a **new element appears** (dialog, dropdown, page change, new content) or a **state change on `[X]` itself** (e.g., `is disabled`, `is checked`). Never assert `[X]` unchanged. Asserting something already visible before the click is a pre-existing state error — if the result is uncertain, mark `@manual`.
96
- 2. `When fill [X] field` → `Then` must assert the **visible result of the input** (search results appear, validation message shows, dropdown filters). Do NOT just assert the field has the value — `see [X] field with {{v}}` after fill is a **weak test** (Playwright fill already guarantees the value is set). Only use field value assertion when the field transforms the input (e.g., auto-format phone number, currency mask).
97
- - **Search fields need a wait step** — search inputs typically have debounce/delay before results appear. After filling a search field, add `User wait for {{search_delay}}` before asserting results. Without this, the assertion fires before results render and the test flakes.
98
- ```gherkin
99
- # GOOD: wait for debounce before asserting search results
100
- When User fill [Search] field with {{term}}
101
- And User wait for {{search_delay}}
102
- Then User see [search result] text with {{result_name}}
103
- ```
104
- 3. If you only want to verify an element **exists/is visible** — use a UI/UX scenario with **no `When`** (just `Given` + `Then`)
105
- 4. If the result of an action is **unknown or uncertain** (e.g., what appears after filling a search, what dialog opens after click), either **explore via MCP first** to see the actual result, or **mark the scenario `@manual`**. Never guess the result — wrong assertions cause test failures that waste fix cycles.
106
- 5. Scenario **name must match the actual assertion**, not the action. "Fill searchbox shows search results" must assert search results — not field value.
107
-
108
- ## Quality Review Checklist (9 checks, auto-fix on detection)
109
-
110
- After generating scenarios, review every scenario against these checks. **If an issue is detected, fix it immediately** — do not just flag it.
111
-
112
- ### 1. Redundant scenarios
113
-
114
- **Problem:** Two scenarios test the same element with overlapping assertions.
115
- ```gherkin
116
- # REDUNDANT: VP-UI-009 and VP-UI-010 both test CTA button
117
- Scenario: CTA button is visible → Then see [cta] button
118
- Scenario: CTA button shows text → Then see [cta] button with {{text}}
119
- ```
120
- **Fix:** Keep only the stronger assertion (`with {{text}}` implies visibility). Remove the weaker one.
121
-
122
- ### 2. Misclassified viewpoint
123
-
124
- **Problem:** A scenario classified in the wrong viewpoint category.
125
-
126
- **Classification rules:**
127
- - **UI/UX** = Given + Then asserting **static, always-the-same** defaults (layout, placeholder text, initial state on first-ever load)
128
- - **Logic** = Given + When + Then (action causes a result), OR Given + Then asserting **behavior-dependent state** (persisted values, dynamic content, conditional defaults)
129
-
130
- **Key distinction:** If the element's state depends on **previous user interaction, saved preferences, or business rules** — it's Logic even without a `When`, because it verifies behavior, not layout.
131
-
132
- ```gherkin
133
- # UI/UX — static default, always the same for every user
134
- Given User is on [Settings] page
135
- Then User see [Language] dropdown with {{default_language}}
136
-
137
- # Logic — checkbox remembers last user choice (persisted state)
138
- Given User is on [Settings] page
139
- Then User see [Newsletter] checkbox is checked
140
-
141
- # Logic — empty state text depends on business rule (no data yet)
142
- Given User is on [Orders] page
143
- Then User see [empty state] text with {{no_orders_message}}
144
- ```
145
-
146
- **Fix:** Check whether the asserted state is truly static (same for all users, all times) or depends on data/behavior. Move accordingly.
147
-
148
- ### 3. Dynamic/data-dependent content
149
-
150
- **Problem:** Asserting content that depends on live data which may change.
151
- ```gherkin
152
- # FRAGILE: "Attachment 1" only exists if first card has an attachment
153
- Scenario: Card shows attachment image
154
- Then User see [Attachment 1] image
155
- ```
156
- **Fix:** Mark as `@manual` with a comment explaining the data dependency.
157
-
158
- ### 4. Duplicate across sections
159
-
160
- **Problem:** Security scenario has identical steps to a UI scenario.
161
- ```gherkin
162
- # VP-UI-006 and VP-SEC-002 are identical
163
- Scenario: VP-UI-006 User profile button is visible
164
- Scenario: VP-SEC-002 Authenticated user sees profile button
165
- ```
166
- **Fix:** Remove the duplicate. Security scenarios should test **auth boundaries** (e.g., `@no-auth` → redirect), not re-test visibility.
167
-
168
- ### 5. "Enabled" state on always-enabled elements
169
-
170
- **Problem:** Testing `is enabled` on elements that have no disabled state in the application.
171
- ```gherkin
172
- # BAD: this button is never disabled — the assertion adds no value
173
- Then User see [Home] button is enabled
174
- ```
175
- **Fix:** Remove. Only assert `is enabled` or `is disabled` when the element **actually toggles** between states based on conditions (e.g., form validity, permissions).
176
-
177
- ### 6. Exact match on dynamic counters/numbers
178
-
179
- **Problem:** Using exact match (`with {{value}}`) on data that changes (counters, totals, timestamps).
180
- ```gherkin
181
- # FRAGILE: "+61 KUDOS" changes when new kudos are sent
182
- Then User see [kudos count] text with {{kudos_count}}
183
- ```
184
- **Fix:** Use partial match (`text contains`) with a stable keyword:
185
- ```gherkin
186
- Then User see [kudos count] text contains {{kudos_count_keyword}}
187
- ```
188
-
189
- ### 7. Current active page link
190
-
191
- **Problem:** Testing visibility of a nav link pointing to the current page.
192
- ```gherkin
193
- # LOW VALUE: we are ON /kudos, Sun* Kudos link is always there
194
- Scenario: Nav link Sun Kudos is visible
195
- Given User is on [kudos] page
196
- Then User see [Sun* Kudos] link
197
- ```
198
- **Fix:** Remove or replace with active state test.
199
-
200
- ### 8. Test-data completeness
201
-
202
- **Problem:** Feature file references `{{variable}}` but the corresponding key is missing in `test-data.yaml`.
203
- ```gherkin
204
- # Feature uses {{login_error}} but test-data.yaml has no "login_error" key
205
- When User fill [Email] field with {{invalid_email}}
206
- And User click [Submit] button
207
- Then User see [error] message with {{login_error}}
208
- ```
209
- **Fix:** After generation, verify every `{{variable}}` in the `.feature` file has a matching entry in `test-data.yaml`. Add missing entries with realistic values. This is a **mandatory post-generation check** — missing test data causes compile failures.
210
-
211
- ### 9. Negative/boundary coverage
212
-
213
- **Problem:** All scenarios are happy-path. No validation, error, or boundary cases exist.
214
- ```gherkin
215
- # BAD: only tests successful login — what about invalid credentials?
216
- Scenario: VP-LOGIC-001 User logs in successfully
217
- Given User is on [Login] page
218
- When User fill [Email] field with {{valid_email}}
219
- And User fill [Password] field with {{valid_password}}
220
- And User click [Submit] button
221
- Then User see [Dashboard] page
222
- ```
223
- **Fix:** For every form/input section, ensure at least one scenario covers:
224
- - **Required field empty** → validation message appears
225
- - **Invalid format** (email, phone, etc.) → format error appears
226
- - **Boundary values** (max length, min value) → when relevant
227
-
228
- If specific error messages are unknown, mark validation scenarios `@manual` rather than omitting them entirely.
@@ -1,2 +0,0 @@
1
- { const tableRow = {{> locator}}.getByRole('row').filter({ hasText: '{{escapeQuotes filterValue}}' });
2
- await expect(tableRow.getByRole('cell').filter({ hasText: '{{escapeQuotes cellValue}}' })).toBeVisible(); }
@@ -1,2 +0,0 @@
1
- { const tableRow = {{> locator}}.getByRole('row').nth({{rowIndex}});
2
- await expect(tableRow.getByRole('cell').filter({ hasText: '{{escapeQuotes cellValue}}' })).toBeVisible(); }