@sun-asterisk/sungen 2.6.5 → 2.6.7

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (59) hide show
  1. package/dist/cli/index.js +1 -1
  2. package/dist/dashboard/templates/index.html +2 -2
  3. package/dist/dashboard/types.d.ts +1 -1
  4. package/dist/dashboard/types.d.ts.map +1 -1
  5. package/dist/exporters/csv-exporter.js +1 -1
  6. package/dist/exporters/csv-exporter.js.map +1 -1
  7. package/dist/exporters/feature-parser.d.ts +5 -2
  8. package/dist/exporters/feature-parser.d.ts.map +1 -1
  9. package/dist/exporters/feature-parser.js +11 -9
  10. package/dist/exporters/feature-parser.js.map +1 -1
  11. package/dist/exporters/json-exporter.js +1 -1
  12. package/dist/exporters/json-exporter.js.map +1 -1
  13. package/dist/exporters/spec-parser.js +1 -1
  14. package/dist/exporters/spec-parser.js.map +1 -1
  15. package/dist/orchestrator/templates/ai-instructions/claude-cmd-create-test.md +23 -7
  16. package/dist/orchestrator/templates/ai-instructions/claude-cmd-run-test.md +2 -2
  17. package/dist/orchestrator/templates/ai-instructions/claude-skill-delivery.md +1 -1
  18. package/dist/orchestrator/templates/ai-instructions/claude-skill-gherkin-syntax.md +5 -6
  19. package/dist/orchestrator/templates/ai-instructions/claude-skill-selector-fix.md +7 -7
  20. package/dist/orchestrator/templates/ai-instructions/claude-skill-tc-generation.md +60 -26
  21. package/dist/orchestrator/templates/ai-instructions/claude-skill-tc-review.md +3 -3
  22. package/dist/orchestrator/templates/ai-instructions/claude-skill-test-design-techniques.md +2 -2
  23. package/dist/orchestrator/templates/ai-instructions/claude-skill-viewpoint.md +1 -1
  24. package/dist/orchestrator/templates/ai-instructions/copilot-cmd-create-test.md +22 -6
  25. package/dist/orchestrator/templates/ai-instructions/copilot-cmd-run-test.md +2 -2
  26. package/dist/orchestrator/templates/ai-instructions/github-skill-sungen-delivery.md +1 -1
  27. package/dist/orchestrator/templates/ai-instructions/github-skill-sungen-gherkin-syntax.md +5 -6
  28. package/dist/orchestrator/templates/ai-instructions/github-skill-sungen-selector-fix.md +7 -7
  29. package/dist/orchestrator/templates/ai-instructions/github-skill-sungen-tc-generation.md +60 -26
  30. package/dist/orchestrator/templates/ai-instructions/github-skill-sungen-tc-review.md +3 -3
  31. package/dist/orchestrator/templates/ai-instructions/github-skill-sungen-test-design-techniques.md +2 -2
  32. package/dist/orchestrator/templates/ai-instructions/github-skill-sungen-viewpoint.md +1 -1
  33. package/package.json +1 -1
  34. package/src/cli/index.ts +1 -1
  35. package/src/dashboard/templates/index.html +2 -2
  36. package/src/dashboard/types.ts +1 -1
  37. package/src/exporters/csv-exporter.ts +1 -1
  38. package/src/exporters/feature-parser.ts +11 -8
  39. package/src/exporters/json-exporter.ts +1 -1
  40. package/src/exporters/spec-parser.ts +1 -1
  41. package/src/generators/test-generator/adapters/adapter-interface.ts +1 -1
  42. package/src/orchestrator/templates/ai-instructions/claude-cmd-create-test.md +23 -7
  43. package/src/orchestrator/templates/ai-instructions/claude-cmd-run-test.md +2 -2
  44. package/src/orchestrator/templates/ai-instructions/claude-skill-delivery.md +1 -1
  45. package/src/orchestrator/templates/ai-instructions/claude-skill-gherkin-syntax.md +5 -6
  46. package/src/orchestrator/templates/ai-instructions/claude-skill-selector-fix.md +7 -7
  47. package/src/orchestrator/templates/ai-instructions/claude-skill-tc-generation.md +60 -26
  48. package/src/orchestrator/templates/ai-instructions/claude-skill-tc-review.md +3 -3
  49. package/src/orchestrator/templates/ai-instructions/claude-skill-test-design-techniques.md +2 -2
  50. package/src/orchestrator/templates/ai-instructions/claude-skill-viewpoint.md +1 -1
  51. package/src/orchestrator/templates/ai-instructions/copilot-cmd-create-test.md +22 -6
  52. package/src/orchestrator/templates/ai-instructions/copilot-cmd-run-test.md +2 -2
  53. package/src/orchestrator/templates/ai-instructions/github-skill-sungen-delivery.md +1 -1
  54. package/src/orchestrator/templates/ai-instructions/github-skill-sungen-gherkin-syntax.md +5 -6
  55. package/src/orchestrator/templates/ai-instructions/github-skill-sungen-selector-fix.md +7 -7
  56. package/src/orchestrator/templates/ai-instructions/github-skill-sungen-tc-generation.md +60 -26
  57. package/src/orchestrator/templates/ai-instructions/github-skill-sungen-tc-review.md +3 -3
  58. package/src/orchestrator/templates/ai-instructions/github-skill-sungen-test-design-techniques.md +2 -2
  59. package/src/orchestrator/templates/ai-instructions/github-skill-sungen-viewpoint.md +1 -1
@@ -12,19 +12,39 @@ Generate **focused test cases per screen section** using a **tier-based approach
12
12
 
13
13
  | Tier | Priority | What to generate | When |
14
14
  |---|---|---|---|
15
- | **Tier 1** (default) | `@critical` + `@high` | Happy paths, required validation, core business rules, security basics | First run of `create-test` |
15
+ | **Tier 1** (default) | `@high` | Happy paths, required validation, core business rules, security basics | First run of `create-test` |
16
16
  | **Tier 2** (expand) | `@normal` + `@low` | UI presence, optional validation, edge cases, cosmetic checks | User runs `create-test` again with "Add viewpoints" mode |
17
+ | **Tier 3** (deep) | `@high` + `@normal` | Extra BVA combinations, cross-field validation, negative/destructive inputs, concurrent/race conditions, complex state transitions | Recommended after Tier 2 completes |
18
+ | **Full** (all-at-once) | All | Tier 1 + 2 + 3 combined in one run | Option at first run, **not recommended** — large output |
17
19
 
18
- **Round 1 (Tier 1)** targets **~10-15 scenarios per section** — enough to cover critical flows and catch real bugs. This is the default behavior.
20
+ **Round 1 (Tier 1)** targets **~10-15 scenarios per section** — enough to cover critical flows and catch real bugs. This is the default behavior. A **Full** option is available but not recommended — it generates all tiers at once, producing very large output (~40-60 scenarios/section).
19
21
 
20
- **Round 2 (Tier 2)** expands to full coverage when the user explicitly chooses "Add viewpoints" or "Add new sections" update mode. Only then generate `@normal` + `@low` scenarios to fill coverage gaps.
22
+ **Round 2 (Tier 2)** expands coverage when the user explicitly chooses "Add viewpoints" or "Add new sections" update mode. Only then generate `@normal` + `@low` scenarios to fill coverage gaps.
23
+
24
+ **Round 3 (Tier 3)** deepens coverage with advanced test design techniques after Tier 2 is complete. This tier focuses on scenarios that Tier 1+2 didn't cover:
25
+ - **Extra BVA combinations**: additional boundary points not covered in Tier 1 (e.g., `min+1`, `max-1`, typical values between boundaries)
26
+ - **Cross-field validation**: field A value affects field B behavior (dependent fields, conditional required, cascading dropdowns)
27
+ - **Negative/destructive inputs**: SQL injection, XSS payloads, special characters, excessively long strings, unicode edge cases
28
+ - **Concurrent/race conditions**: double submit, back button after submit, multiple tabs, session expiry mid-action
29
+ - **Complex state transitions**: multi-step state changes, undo/redo, conflicting state updates, transition from every non-obvious state
21
30
 
22
31
  ## Update Mode
23
32
 
24
- When `.feature` already has scenarios, summarize and ask:
33
+ When `.feature` already has scenarios, summarize existing coverage and ask:
34
+
35
+ **If only Tier 1 exists** (only `@high` scenarios):
25
36
  1. **Add new sections** — append new sections with Tier 2 (`@normal` + `@low`) scenarios, continue numbering
26
37
  2. **Add viewpoints** — expand existing sections with Tier 2 (`@normal` + `@low`) scenarios
27
- 3. **Replace all** — overwrite with fresh Tier 1 (`@critical` + `@high`) generation
38
+ 3. **Replace all** — overwrite with fresh Tier 1 (`@high`) generation
39
+
40
+ **If Tier 1 + 2 exist** (`@high` + `@normal` + `@low` scenarios):
41
+ 1. **Deep coverage (Tier 3)** — add advanced scenarios: extra BVA, cross-field validation, negative inputs, race conditions **(Recommended)**
42
+ 2. **Add new sections** — append new sections with Tier 2 scenarios, continue numbering
43
+ 3. **Replace all** — overwrite with fresh Tier 1 (`@high`) generation
44
+
45
+ **If Tier 1 + 2 + 3 exist** (full coverage already):
46
+ 1. **Add new sections** — append new sections if screen has changed
47
+ 2. **Replace all** — overwrite with fresh Tier 1 (`@high`) generation
28
48
 
29
49
  For append: read highest `VP-<CAT>-<NNN>`, continue from next number. Never modify existing scenarios.
30
50
 
@@ -102,8 +122,15 @@ Apply `sungen-test-design-techniques` to spec-extracted conditions:
102
122
  Use `sungen-viewpoint` skill for per-pattern checklists across 4 viewpoints: UI/UX, Data & Validate, Logic, Security.
103
123
 
104
124
  **Tier-aware gap filling:**
105
- - **Tier 1 (first run)**: only add `@critical` and `@high` items from the checklists — core security checks (VP-SEC), required field validation (VP-VAL), key state transitions (VP-LOGIC). Skip `@normal`/`@low` items like hover states, empty states, tooltips.
125
+ - **Tier 1 (first run)**: only add `@high` items from the checklists — core security checks (VP-SEC), required field validation (VP-VAL), key state transitions (VP-LOGIC). Skip `@normal`/`@low` items like hover states, empty states, tooltips.
106
126
  - **Tier 2 (expand run)**: add `@normal` + `@low` scenarios — UI presence, optional validation, edge cases, cosmetic checks, keyboard nav, hover effects.
127
+ - **Tier 3 (deep run)**: do NOT use viewpoint checklists. Instead, re-apply test design techniques with deeper analysis:
128
+ - **BVA**: expand from 4-point to 6-point (`min-1`, `min`, `min+1`, `max-1`, `max`, `max+1`) + typical mid-range value
129
+ - **Decision Table**: enumerate combinations previously capped — cover remaining outcome rows
130
+ - **Cross-field**: identify field dependencies from spec, generate one scenario per dependency
131
+ - **Negative inputs**: add security-oriented inputs (SQL `' OR 1=1`, XSS `<script>`, special chars `<>&"'`, max+100 length)
132
+ - **Race conditions**: double-click submit, browser back after POST, concurrent edit by two users
133
+ - **Full (all-at-once)**: apply Tier 1 + 2 + 3 rules in a single generation pass. Use all techniques at maximum depth.
107
134
 
108
135
  **Validation rule**: capture actual error messages from live page or spec.md. Use `User see {{error_var}}` — never assert just "is visible".
109
136
 
@@ -111,24 +138,29 @@ Use `sungen-viewpoint` skill for per-pattern checklists across 4 viewpoints: UI/
111
138
 
112
139
  Every scenario **MUST** have exactly one priority tag. Add it before the scenario line (after `@extend:` if present).
113
140
 
141
+ **Rule: look at what the WHEN step does and what THEN asserts — that determines the tag.**
142
+
114
143
  | Tag | When to use |
115
144
  |---|---|
116
- | `@critical` | System unusable if fails — login/logout, authentication redirect, main create/submit/delete, permission denied |
117
- | `@high` | Major feature broken required field validation, core business rules, data displays correctly, key navigation |
118
- | `@normal` | Degraded experience UI layout/element presence, secondary flows, optional field validation, search/filter |
119
- | `@low` | Minor/cosmetic — tooltips, hover states, empty states, default sort, placeholder text |
145
+ | `@high` | WHEN performs: login/logout, submit happy path, auth-check, primary CRUD action. THEN asserts: success state, redirect, or security gate |
146
+ | `@normal` | WHEN provides invalid/empty input, OR scenario tests search/filter/notification/secondary flow. THEN asserts: error message, secondary feature result |
147
+ | `@low` | THEN only asserts: element visible, text label, placeholder, tooltip, icon, layout. WHEN (if any) is navigation only |
120
148
 
121
149
  ### Auto-assign heuristics
122
150
 
123
- | Viewpoint + Pattern | Default priority |
151
+ | Scenario type (by WHEN + THEN structure) | Tag |
124
152
  |---|---|
125
- | VP-SEC-* (all security scenarios) | `@critical` |
126
- | VP-LOGIC-* with create/submit/delete/login | `@critical` |
127
- | VP-LOGIC-* other state changes | `@high` |
128
- | VP-VAL-* required field / submit empty | `@high` |
129
- | VP-VAL-* format, boundary, optional fields | `@normal` |
130
- | VP-UI-* form fields present, table columns | `@normal` |
131
- | VP-UI-* hover, tooltip, empty state, placeholder | `@low` |
153
+ | Auth: login / logout / protected page redirect | `@high` |
154
+ | CRUD happy path: create / update / delete → success message or redirect | `@high` |
155
+ | Security: unauthenticated access redirected to login | `@high` |
156
+ | Primary business flow step completes successfully | `@high` |
157
+ | Validation: invalid / empty input → error message shown | `@normal` |
158
+ | Boundary / format validation (email, length, range) | `@normal` |
159
+ | Secondary features: search, filter, sort, pagination, notification | `@normal` |
160
+ | State transition not on primary success path (cancel, undo, back) | `@normal` |
161
+ | Element / component presence check (is it visible?) | `@low` |
162
+ | Text content: label, placeholder, tooltip, warning message text | `@low` |
163
+ | Visual / cosmetic: alignment, icon, empty state, hover style | `@low` |
132
164
 
133
165
  **`@steps:` scenarios** do NOT get a priority tag (they are setup blocks, not test cases).
134
166
 
@@ -188,11 +220,11 @@ Serial (default) is best for: CRUD flows, sequential user journeys, UI checks on
188
220
  @cleanup:forms
189
221
  Feature: kudos Screen
190
222
 
191
- @critical
223
+ @high
192
224
  Scenario: Send kudos
193
225
  ...
194
226
 
195
- @critical @no-auth
227
+ @high @no-auth
196
228
  Scenario: Unauthenticated user is redirected
197
229
  ...
198
230
  ```
@@ -218,9 +250,9 @@ Feature: <Screen> Screen
218
250
  When User click [Create] button
219
251
  Then User see [Form] dialog
220
252
 
221
- # --- Section: Create User Form (Tier 1: @critical + @high) ---
253
+ # --- Section: Create User Form (Tier 1: @high) ---
222
254
 
223
- @critical @extend:open_form
255
+ @high @extend:open_form
224
256
  Scenario: VP-LOGIC-001 Submit form with valid data creates record
225
257
  Given User is on [Form] dialog
226
258
  When User fill [Name] field with {{valid_name}}
@@ -233,7 +265,7 @@ Feature: <Screen> Screen
233
265
  When User click [Submit] button
234
266
  Then User see [Name error] message with {{name_required_error}}
235
267
 
236
- # --- Section: User Table (Tier 1: @critical + @high) ---
268
+ # --- Section: User Table (Tier 1: @high) ---
237
269
 
238
270
  @high
239
271
  Scenario: VP-VAL-010 Table displays correct data
@@ -241,7 +273,7 @@ Feature: <Screen> Screen
241
273
  | Name | Email |
242
274
  | {{name_1}} | {{email_1}} |
243
275
 
244
- @critical
276
+ @high
245
277
  Scenario: VP-SEC-010 Unauthorized user cannot access page
246
278
  Given User is not logged in
247
279
  When User navigate to [Screen] page
@@ -250,6 +282,8 @@ Feature: <Screen> Screen
250
282
 
251
283
  **Tier 2 (expand run)** adds `@normal` + `@low` scenarios like UI field presence, hover states, tooltips, empty states.
252
284
 
285
+ **Tier 3 (deep run)** adds advanced `@high` + `@normal` scenarios: extra BVA boundaries, cross-field validation, negative/destructive inputs, concurrent/race conditions.
286
+
253
287
  ### When to use DataTable vs Row Scope
254
288
 
255
289
  | Pattern | Use when |
@@ -317,14 +351,14 @@ Feature: Award Submission Flow
317
351
  Background:
318
352
  Given User is on [Login] page
319
353
 
320
- @critical
354
+ @high
321
355
  Scenario: User login successfully
322
356
  When User fill [Login:Email] field with {{login.email}}
323
357
  And User fill [Login:Password] field with {{login.password}}
324
358
  And User click [Login:Submit] button
325
359
  Then User see [Dashboard] page
326
360
 
327
- @critical
361
+ @high
328
362
  Scenario: User navigates to awards and submits
329
363
  Given User is on [Awards] page
330
364
  When User fill [Awards:Nominee] field with {{submission.nominee}}
@@ -38,12 +38,12 @@ Score: `(dimensions_covered / 6) * 40`. Validate technique application with `sun
38
38
  | All applicable VP present (UI/VAL/LOGIC/SEC) | 10 |
39
39
  | Correct classification | 8 |
40
40
  | `VP-<CAT>-<NNN>` naming + section grouping | 4 |
41
- | Priority tag present and correct (`@critical`/`@high`/`@normal`/`@low`) | 4 |
41
+ | Priority tag present and correct (`@high`/`@normal`/`@low`) | 4 |
42
42
  | Assertion quality (see rules below) | 4 |
43
43
 
44
44
  **Classification**: UI = static/always-same appearance. VAL = input validation/errors. LOGIC = behavior/state changes (includes persisted state without When). SEC = auth/permissions.
45
45
 
46
- **Tier-aware scoring**: If the feature file only contains `@critical` + `@high` scenarios (Tier 1), do NOT penalize for missing VP-UI viewpoint — UI scenarios are intentionally deferred to Tier 2. Score "All applicable VP present" based on Tier 1-relevant viewpoints only (VAL, LOGIC, SEC). Note in the review output: *"VP-UI deferred to Tier 2 — run create-test with 'Add viewpoints' to expand."*
46
+ **Tier-aware scoring**: If the feature file only contains `@high` scenarios (Tier 1), do NOT penalize for missing VP-UI viewpoint — UI scenarios are intentionally deferred to Tier 2. Score "All applicable VP present" based on Tier 1-relevant viewpoints only (VAL, LOGIC, SEC). Note in the review output: *"VP-UI deferred to Tier 2 — run create-test with 'Add viewpoints' to expand."*
47
47
 
48
48
  ---
49
49
 
@@ -80,7 +80,7 @@ Do NOT mark `@manual` when data is visible in snapshot or documented in spec —
80
80
  2. **Misclassified VP** — UI tests behavior? Move to LOGIC. Logic tests appearance? Move to UI
81
81
  3. **Dynamic content** — exact match on counters/timestamps? Use `contains` instead
82
82
  4. **Duplicate across sections** — SEC scenario identical to UI? Remove duplicate
83
- 5. **Missing/wrong priority tag** — every non-`@steps` scenario needs exactly one of `@critical`/`@high`/`@normal`/`@low`. SEC→`@critical`, VAL required→`@high`, UI layout→`@normal`, hover/tooltip→`@low`
83
+ 5. **Missing/wrong priority tag** — every non-`@steps` scenario needs exactly one of `@high`/`@normal`/`@low`. SEC/CRUD happy path/auth→`@high`, validation/secondary features→`@normal`, presence/cosmetic→`@low`
84
84
  6. **Always-enabled elements** — `is enabled` on never-disabled element? Remove
85
85
  7. **Test-data completeness** — every `{{var}}` must exist in test-data.yaml
86
86
  8. **Missing BVA boundaries** — spec defines min/max range but scenarios only test midpoint? Add `min-1`, `min`, `max`, `max+1`
@@ -53,7 +53,7 @@ Scenario: VP-VAL-003 Above maximum is rejected # value = 101
53
53
  | Mode | Values | Use when |
54
54
  |---|---|---|
55
55
  | **Compact (default)** | `min-1`, `min`, `max`, `max+1` | Most fields |
56
- | **Full 6-point** | `min-1`, `min`, `min+1`, `max-1`, `max`, `max+1` | Critical fields with `@critical`/`@high` priority |
56
+ | **Full 6-point** | `min-1`, `min`, `min+1`, `max-1`, `max`, `max+1` | Critical fields with `@high` priority |
57
57
 
58
58
  **How to apply** (example: "quantity must be 1-10"):
59
59
  - `min-1` = 0 -> invalid
@@ -82,7 +82,7 @@ Scenario: VP-VAL-003 Above maximum is rejected # value = 101
82
82
  - `@normal` Form invalid + no permission → disabled
83
83
  - `@normal` Form valid + no permission → disabled
84
84
  - `@normal` Has permission + form invalid → disabled
85
- - `@critical` Form valid + has permission → succeeds
85
+ - `@high` Form valid + has permission → succeeds
86
86
 
87
87
  ---
88
88
 
@@ -266,7 +266,7 @@ For VP-SEC scenarios testing **unauthorized access** (no login, wrong role, dire
266
266
  - Do NOT use `@manual` for these — they are automatable.
267
267
 
268
268
  ```gherkin
269
- @critical @no-auth
269
+ @high @no-auth
270
270
  Scenario: VP-SEC-001 Unauthenticated user cannot access admin page
271
271
  Given User is on [Admin] page
272
272
  Then User see [Login] page