@sun-asterisk/sungen 2.6.5 → 2.6.7
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/dist/cli/index.js +1 -1
- package/dist/dashboard/templates/index.html +2 -2
- package/dist/dashboard/types.d.ts +1 -1
- package/dist/dashboard/types.d.ts.map +1 -1
- package/dist/exporters/csv-exporter.js +1 -1
- package/dist/exporters/csv-exporter.js.map +1 -1
- package/dist/exporters/feature-parser.d.ts +5 -2
- package/dist/exporters/feature-parser.d.ts.map +1 -1
- package/dist/exporters/feature-parser.js +11 -9
- package/dist/exporters/feature-parser.js.map +1 -1
- package/dist/exporters/json-exporter.js +1 -1
- package/dist/exporters/json-exporter.js.map +1 -1
- package/dist/exporters/spec-parser.js +1 -1
- package/dist/exporters/spec-parser.js.map +1 -1
- package/dist/orchestrator/templates/ai-instructions/claude-cmd-create-test.md +23 -7
- package/dist/orchestrator/templates/ai-instructions/claude-cmd-run-test.md +2 -2
- package/dist/orchestrator/templates/ai-instructions/claude-skill-delivery.md +1 -1
- package/dist/orchestrator/templates/ai-instructions/claude-skill-gherkin-syntax.md +5 -6
- package/dist/orchestrator/templates/ai-instructions/claude-skill-selector-fix.md +7 -7
- package/dist/orchestrator/templates/ai-instructions/claude-skill-tc-generation.md +60 -26
- package/dist/orchestrator/templates/ai-instructions/claude-skill-tc-review.md +3 -3
- package/dist/orchestrator/templates/ai-instructions/claude-skill-test-design-techniques.md +2 -2
- package/dist/orchestrator/templates/ai-instructions/claude-skill-viewpoint.md +1 -1
- package/dist/orchestrator/templates/ai-instructions/copilot-cmd-create-test.md +22 -6
- package/dist/orchestrator/templates/ai-instructions/copilot-cmd-run-test.md +2 -2
- package/dist/orchestrator/templates/ai-instructions/github-skill-sungen-delivery.md +1 -1
- package/dist/orchestrator/templates/ai-instructions/github-skill-sungen-gherkin-syntax.md +5 -6
- package/dist/orchestrator/templates/ai-instructions/github-skill-sungen-selector-fix.md +7 -7
- package/dist/orchestrator/templates/ai-instructions/github-skill-sungen-tc-generation.md +60 -26
- package/dist/orchestrator/templates/ai-instructions/github-skill-sungen-tc-review.md +3 -3
- package/dist/orchestrator/templates/ai-instructions/github-skill-sungen-test-design-techniques.md +2 -2
- package/dist/orchestrator/templates/ai-instructions/github-skill-sungen-viewpoint.md +1 -1
- package/package.json +1 -1
- package/src/cli/index.ts +1 -1
- package/src/dashboard/templates/index.html +2 -2
- package/src/dashboard/types.ts +1 -1
- package/src/exporters/csv-exporter.ts +1 -1
- package/src/exporters/feature-parser.ts +11 -8
- package/src/exporters/json-exporter.ts +1 -1
- package/src/exporters/spec-parser.ts +1 -1
- package/src/generators/test-generator/adapters/adapter-interface.ts +1 -1
- package/src/orchestrator/templates/ai-instructions/claude-cmd-create-test.md +23 -7
- package/src/orchestrator/templates/ai-instructions/claude-cmd-run-test.md +2 -2
- package/src/orchestrator/templates/ai-instructions/claude-skill-delivery.md +1 -1
- package/src/orchestrator/templates/ai-instructions/claude-skill-gherkin-syntax.md +5 -6
- package/src/orchestrator/templates/ai-instructions/claude-skill-selector-fix.md +7 -7
- package/src/orchestrator/templates/ai-instructions/claude-skill-tc-generation.md +60 -26
- package/src/orchestrator/templates/ai-instructions/claude-skill-tc-review.md +3 -3
- package/src/orchestrator/templates/ai-instructions/claude-skill-test-design-techniques.md +2 -2
- package/src/orchestrator/templates/ai-instructions/claude-skill-viewpoint.md +1 -1
- package/src/orchestrator/templates/ai-instructions/copilot-cmd-create-test.md +22 -6
- package/src/orchestrator/templates/ai-instructions/copilot-cmd-run-test.md +2 -2
- package/src/orchestrator/templates/ai-instructions/github-skill-sungen-delivery.md +1 -1
- package/src/orchestrator/templates/ai-instructions/github-skill-sungen-gherkin-syntax.md +5 -6
- package/src/orchestrator/templates/ai-instructions/github-skill-sungen-selector-fix.md +7 -7
- package/src/orchestrator/templates/ai-instructions/github-skill-sungen-tc-generation.md +60 -26
- package/src/orchestrator/templates/ai-instructions/github-skill-sungen-tc-review.md +3 -3
- package/src/orchestrator/templates/ai-instructions/github-skill-sungen-test-design-techniques.md +2 -2
- package/src/orchestrator/templates/ai-instructions/github-skill-sungen-viewpoint.md +1 -1
|
@@ -12,19 +12,39 @@ Generate **focused test cases per screen section** using a **tier-based approach
|
|
|
12
12
|
|
|
13
13
|
| Tier | Priority | What to generate | When |
|
|
14
14
|
|---|---|---|---|
|
|
15
|
-
| **Tier 1** (default) | `@
|
|
15
|
+
| **Tier 1** (default) | `@high` | Happy paths, required validation, core business rules, security basics | First run of `create-test` |
|
|
16
16
|
| **Tier 2** (expand) | `@normal` + `@low` | UI presence, optional validation, edge cases, cosmetic checks | User runs `create-test` again with "Add viewpoints" mode |
|
|
17
|
+
| **Tier 3** (deep) | `@high` + `@normal` | Extra BVA combinations, cross-field validation, negative/destructive inputs, concurrent/race conditions, complex state transitions | Recommended after Tier 2 completes |
|
|
18
|
+
| **Full** (all-at-once) | All | Tier 1 + 2 + 3 combined in one run | Option at first run, **not recommended** — large output |
|
|
17
19
|
|
|
18
|
-
**Round 1 (Tier 1)** targets **~10-15 scenarios per section** — enough to cover critical flows and catch real bugs. This is the default behavior.
|
|
20
|
+
**Round 1 (Tier 1)** targets **~10-15 scenarios per section** — enough to cover critical flows and catch real bugs. This is the default behavior. A **Full** option is available but not recommended — it generates all tiers at once, producing very large output (~40-60 scenarios/section).
|
|
19
21
|
|
|
20
|
-
**Round 2 (Tier 2)** expands
|
|
22
|
+
**Round 2 (Tier 2)** expands coverage when the user explicitly chooses "Add viewpoints" or "Add new sections" update mode. Only then generate `@normal` + `@low` scenarios to fill coverage gaps.
|
|
23
|
+
|
|
24
|
+
**Round 3 (Tier 3)** deepens coverage with advanced test design techniques after Tier 2 is complete. This tier focuses on scenarios that Tier 1+2 didn't cover:
|
|
25
|
+
- **Extra BVA combinations**: additional boundary points not covered in Tier 1 (e.g., `min+1`, `max-1`, typical values between boundaries)
|
|
26
|
+
- **Cross-field validation**: field A value affects field B behavior (dependent fields, conditional required, cascading dropdowns)
|
|
27
|
+
- **Negative/destructive inputs**: SQL injection, XSS payloads, special characters, excessively long strings, unicode edge cases
|
|
28
|
+
- **Concurrent/race conditions**: double submit, back button after submit, multiple tabs, session expiry mid-action
|
|
29
|
+
- **Complex state transitions**: multi-step state changes, undo/redo, conflicting state updates, transition from every non-obvious state
|
|
21
30
|
|
|
22
31
|
## Update Mode
|
|
23
32
|
|
|
24
|
-
When `.feature` already has scenarios, summarize and ask:
|
|
33
|
+
When `.feature` already has scenarios, summarize existing coverage and ask:
|
|
34
|
+
|
|
35
|
+
**If only Tier 1 exists** (only `@high` scenarios):
|
|
25
36
|
1. **Add new sections** — append new sections with Tier 2 (`@normal` + `@low`) scenarios, continue numbering
|
|
26
37
|
2. **Add viewpoints** — expand existing sections with Tier 2 (`@normal` + `@low`) scenarios
|
|
27
|
-
3. **Replace all** — overwrite with fresh Tier 1 (`@
|
|
38
|
+
3. **Replace all** — overwrite with fresh Tier 1 (`@high`) generation
|
|
39
|
+
|
|
40
|
+
**If Tier 1 + 2 exist** (`@high` + `@normal` + `@low` scenarios):
|
|
41
|
+
1. **Deep coverage (Tier 3)** — add advanced scenarios: extra BVA, cross-field validation, negative inputs, race conditions **(Recommended)**
|
|
42
|
+
2. **Add new sections** — append new sections with Tier 2 scenarios, continue numbering
|
|
43
|
+
3. **Replace all** — overwrite with fresh Tier 1 (`@high`) generation
|
|
44
|
+
|
|
45
|
+
**If Tier 1 + 2 + 3 exist** (full coverage already):
|
|
46
|
+
1. **Add new sections** — append new sections if screen has changed
|
|
47
|
+
2. **Replace all** — overwrite with fresh Tier 1 (`@high`) generation
|
|
28
48
|
|
|
29
49
|
For append: read highest `VP-<CAT>-<NNN>`, continue from next number. Never modify existing scenarios.
|
|
30
50
|
|
|
@@ -102,8 +122,15 @@ Apply `sungen-test-design-techniques` to spec-extracted conditions:
|
|
|
102
122
|
Use `sungen-viewpoint` skill for per-pattern checklists across 4 viewpoints: UI/UX, Data & Validate, Logic, Security.
|
|
103
123
|
|
|
104
124
|
**Tier-aware gap filling:**
|
|
105
|
-
- **Tier 1 (first run)**: only add `@
|
|
125
|
+
- **Tier 1 (first run)**: only add `@high` items from the checklists — core security checks (VP-SEC), required field validation (VP-VAL), key state transitions (VP-LOGIC). Skip `@normal`/`@low` items like hover states, empty states, tooltips.
|
|
106
126
|
- **Tier 2 (expand run)**: add `@normal` + `@low` scenarios — UI presence, optional validation, edge cases, cosmetic checks, keyboard nav, hover effects.
|
|
127
|
+
- **Tier 3 (deep run)**: do NOT use viewpoint checklists. Instead, re-apply test design techniques with deeper analysis:
|
|
128
|
+
- **BVA**: expand from 4-point to 6-point (`min-1`, `min`, `min+1`, `max-1`, `max`, `max+1`) + typical mid-range value
|
|
129
|
+
- **Decision Table**: enumerate combinations previously capped — cover remaining outcome rows
|
|
130
|
+
- **Cross-field**: identify field dependencies from spec, generate one scenario per dependency
|
|
131
|
+
- **Negative inputs**: add security-oriented inputs (SQL `' OR 1=1`, XSS `<script>`, special chars `<>&"'`, max+100 length)
|
|
132
|
+
- **Race conditions**: double-click submit, browser back after POST, concurrent edit by two users
|
|
133
|
+
- **Full (all-at-once)**: apply Tier 1 + 2 + 3 rules in a single generation pass. Use all techniques at maximum depth.
|
|
107
134
|
|
|
108
135
|
**Validation rule**: capture actual error messages from live page or spec.md. Use `User see {{error_var}}` — never assert just "is visible".
|
|
109
136
|
|
|
@@ -111,24 +138,29 @@ Use `sungen-viewpoint` skill for per-pattern checklists across 4 viewpoints: UI/
|
|
|
111
138
|
|
|
112
139
|
Every scenario **MUST** have exactly one priority tag. Add it before the scenario line (after `@extend:` if present).
|
|
113
140
|
|
|
141
|
+
**Rule: look at what the WHEN step does and what THEN asserts — that determines the tag.**
|
|
142
|
+
|
|
114
143
|
| Tag | When to use |
|
|
115
144
|
|---|---|
|
|
116
|
-
| `@
|
|
117
|
-
| `@
|
|
118
|
-
| `@
|
|
119
|
-
| `@low` | Minor/cosmetic — tooltips, hover states, empty states, default sort, placeholder text |
|
|
145
|
+
| `@high` | WHEN performs: login/logout, submit happy path, auth-check, primary CRUD action. THEN asserts: success state, redirect, or security gate |
|
|
146
|
+
| `@normal` | WHEN provides invalid/empty input, OR scenario tests search/filter/notification/secondary flow. THEN asserts: error message, secondary feature result |
|
|
147
|
+
| `@low` | THEN only asserts: element visible, text label, placeholder, tooltip, icon, layout. WHEN (if any) is navigation only |
|
|
120
148
|
|
|
121
149
|
### Auto-assign heuristics
|
|
122
150
|
|
|
123
|
-
|
|
|
151
|
+
| Scenario type (by WHEN + THEN structure) | Tag |
|
|
124
152
|
|---|---|
|
|
125
|
-
|
|
|
126
|
-
|
|
|
127
|
-
|
|
|
128
|
-
|
|
|
129
|
-
|
|
|
130
|
-
|
|
|
131
|
-
|
|
|
153
|
+
| Auth: login / logout / protected page redirect | `@high` |
|
|
154
|
+
| CRUD happy path: create / update / delete → success message or redirect | `@high` |
|
|
155
|
+
| Security: unauthenticated access → redirected to login | `@high` |
|
|
156
|
+
| Primary business flow step completes successfully | `@high` |
|
|
157
|
+
| Validation: invalid / empty input → error message shown | `@normal` |
|
|
158
|
+
| Boundary / format validation (email, length, range) | `@normal` |
|
|
159
|
+
| Secondary features: search, filter, sort, pagination, notification | `@normal` |
|
|
160
|
+
| State transition not on primary success path (cancel, undo, back) | `@normal` |
|
|
161
|
+
| Element / component presence check (is it visible?) | `@low` |
|
|
162
|
+
| Text content: label, placeholder, tooltip, warning message text | `@low` |
|
|
163
|
+
| Visual / cosmetic: alignment, icon, empty state, hover style | `@low` |
|
|
132
164
|
|
|
133
165
|
**`@steps:` scenarios** do NOT get a priority tag (they are setup blocks, not test cases).
|
|
134
166
|
|
|
@@ -188,11 +220,11 @@ Serial (default) is best for: CRUD flows, sequential user journeys, UI checks on
|
|
|
188
220
|
@cleanup:forms
|
|
189
221
|
Feature: kudos Screen
|
|
190
222
|
|
|
191
|
-
@
|
|
223
|
+
@high
|
|
192
224
|
Scenario: Send kudos
|
|
193
225
|
...
|
|
194
226
|
|
|
195
|
-
@
|
|
227
|
+
@high @no-auth
|
|
196
228
|
Scenario: Unauthenticated user is redirected
|
|
197
229
|
...
|
|
198
230
|
```
|
|
@@ -218,9 +250,9 @@ Feature: <Screen> Screen
|
|
|
218
250
|
When User click [Create] button
|
|
219
251
|
Then User see [Form] dialog
|
|
220
252
|
|
|
221
|
-
# --- Section: Create User Form (Tier 1: @
|
|
253
|
+
# --- Section: Create User Form (Tier 1: @high) ---
|
|
222
254
|
|
|
223
|
-
@
|
|
255
|
+
@high @extend:open_form
|
|
224
256
|
Scenario: VP-LOGIC-001 Submit form with valid data creates record
|
|
225
257
|
Given User is on [Form] dialog
|
|
226
258
|
When User fill [Name] field with {{valid_name}}
|
|
@@ -233,7 +265,7 @@ Feature: <Screen> Screen
|
|
|
233
265
|
When User click [Submit] button
|
|
234
266
|
Then User see [Name error] message with {{name_required_error}}
|
|
235
267
|
|
|
236
|
-
# --- Section: User Table (Tier 1: @
|
|
268
|
+
# --- Section: User Table (Tier 1: @high) ---
|
|
237
269
|
|
|
238
270
|
@high
|
|
239
271
|
Scenario: VP-VAL-010 Table displays correct data
|
|
@@ -241,7 +273,7 @@ Feature: <Screen> Screen
|
|
|
241
273
|
| Name | Email |
|
|
242
274
|
| {{name_1}} | {{email_1}} |
|
|
243
275
|
|
|
244
|
-
@
|
|
276
|
+
@high
|
|
245
277
|
Scenario: VP-SEC-010 Unauthorized user cannot access page
|
|
246
278
|
Given User is not logged in
|
|
247
279
|
When User navigate to [Screen] page
|
|
@@ -250,6 +282,8 @@ Feature: <Screen> Screen
|
|
|
250
282
|
|
|
251
283
|
**Tier 2 (expand run)** adds `@normal` + `@low` scenarios like UI field presence, hover states, tooltips, empty states.
|
|
252
284
|
|
|
285
|
+
**Tier 3 (deep run)** adds advanced `@high` + `@normal` scenarios: extra BVA boundaries, cross-field validation, negative/destructive inputs, concurrent/race conditions.
|
|
286
|
+
|
|
253
287
|
### When to use DataTable vs Row Scope
|
|
254
288
|
|
|
255
289
|
| Pattern | Use when |
|
|
@@ -317,14 +351,14 @@ Feature: Award Submission Flow
|
|
|
317
351
|
Background:
|
|
318
352
|
Given User is on [Login] page
|
|
319
353
|
|
|
320
|
-
@
|
|
354
|
+
@high
|
|
321
355
|
Scenario: User login successfully
|
|
322
356
|
When User fill [Login:Email] field with {{login.email}}
|
|
323
357
|
And User fill [Login:Password] field with {{login.password}}
|
|
324
358
|
And User click [Login:Submit] button
|
|
325
359
|
Then User see [Dashboard] page
|
|
326
360
|
|
|
327
|
-
@
|
|
361
|
+
@high
|
|
328
362
|
Scenario: User navigates to awards and submits
|
|
329
363
|
Given User is on [Awards] page
|
|
330
364
|
When User fill [Awards:Nominee] field with {{submission.nominee}}
|
|
@@ -38,12 +38,12 @@ Score: `(dimensions_covered / 6) * 40`. Validate technique application with `sun
|
|
|
38
38
|
| All applicable VP present (UI/VAL/LOGIC/SEC) | 10 |
|
|
39
39
|
| Correct classification | 8 |
|
|
40
40
|
| `VP-<CAT>-<NNN>` naming + section grouping | 4 |
|
|
41
|
-
| Priority tag present and correct (`@
|
|
41
|
+
| Priority tag present and correct (`@high`/`@normal`/`@low`) | 4 |
|
|
42
42
|
| Assertion quality (see rules below) | 4 |
|
|
43
43
|
|
|
44
44
|
**Classification**: UI = static/always-same appearance. VAL = input validation/errors. LOGIC = behavior/state changes (includes persisted state without When). SEC = auth/permissions.
|
|
45
45
|
|
|
46
|
-
**Tier-aware scoring**: If the feature file only contains `@
|
|
46
|
+
**Tier-aware scoring**: If the feature file only contains `@high` scenarios (Tier 1), do NOT penalize for missing VP-UI viewpoint — UI scenarios are intentionally deferred to Tier 2. Score "All applicable VP present" based on Tier 1-relevant viewpoints only (VAL, LOGIC, SEC). Note in the review output: *"VP-UI deferred to Tier 2 — run create-test with 'Add viewpoints' to expand."*
|
|
47
47
|
|
|
48
48
|
---
|
|
49
49
|
|
|
@@ -80,7 +80,7 @@ Do NOT mark `@manual` when data is visible in snapshot or documented in spec —
|
|
|
80
80
|
2. **Misclassified VP** — UI tests behavior? Move to LOGIC. Logic tests appearance? Move to UI
|
|
81
81
|
3. **Dynamic content** — exact match on counters/timestamps? Use `contains` instead
|
|
82
82
|
4. **Duplicate across sections** — SEC scenario identical to UI? Remove duplicate
|
|
83
|
-
5. **Missing/wrong priority tag** — every non-`@steps` scenario needs exactly one of `@
|
|
83
|
+
5. **Missing/wrong priority tag** — every non-`@steps` scenario needs exactly one of `@high`/`@normal`/`@low`. SEC/CRUD happy path/auth→`@high`, validation/secondary features→`@normal`, presence/cosmetic→`@low`
|
|
84
84
|
6. **Always-enabled elements** — `is enabled` on never-disabled element? Remove
|
|
85
85
|
7. **Test-data completeness** — every `{{var}}` must exist in test-data.yaml
|
|
86
86
|
8. **Missing BVA boundaries** — spec defines min/max range but scenarios only test midpoint? Add `min-1`, `min`, `max`, `max+1`
|
package/src/orchestrator/templates/ai-instructions/github-skill-sungen-test-design-techniques.md
CHANGED
|
@@ -53,7 +53,7 @@ Scenario: VP-VAL-003 Above maximum is rejected # value = 101
|
|
|
53
53
|
| Mode | Values | Use when |
|
|
54
54
|
|---|---|---|
|
|
55
55
|
| **Compact (default)** | `min-1`, `min`, `max`, `max+1` | Most fields |
|
|
56
|
-
| **Full 6-point** | `min-1`, `min`, `min+1`, `max-1`, `max`, `max+1` | Critical fields with `@
|
|
56
|
+
| **Full 6-point** | `min-1`, `min`, `min+1`, `max-1`, `max`, `max+1` | Critical fields with `@high` priority |
|
|
57
57
|
|
|
58
58
|
**How to apply** (example: "quantity must be 1-10"):
|
|
59
59
|
- `min-1` = 0 -> invalid
|
|
@@ -82,7 +82,7 @@ Scenario: VP-VAL-003 Above maximum is rejected # value = 101
|
|
|
82
82
|
- `@normal` Form invalid + no permission → disabled
|
|
83
83
|
- `@normal` Form valid + no permission → disabled
|
|
84
84
|
- `@normal` Has permission + form invalid → disabled
|
|
85
|
-
- `@
|
|
85
|
+
- `@high` Form valid + has permission → succeeds
|
|
86
86
|
|
|
87
87
|
---
|
|
88
88
|
|
|
@@ -266,7 +266,7 @@ For VP-SEC scenarios testing **unauthorized access** (no login, wrong role, dire
|
|
|
266
266
|
- Do NOT use `@manual` for these — they are automatable.
|
|
267
267
|
|
|
268
268
|
```gherkin
|
|
269
|
-
@
|
|
269
|
+
@high @no-auth
|
|
270
270
|
Scenario: VP-SEC-001 Unauthenticated user cannot access admin page
|
|
271
271
|
Given User is on [Admin] page
|
|
272
272
|
Then User see [Login] page
|