@sun-asterisk/sungen 2.6.5 → 2.6.6
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/dist/cli/index.js +1 -1
- package/dist/dashboard/templates/index.html +2 -2
- package/dist/dashboard/types.d.ts +1 -1
- package/dist/dashboard/types.d.ts.map +1 -1
- package/dist/exporters/csv-exporter.js +1 -1
- package/dist/exporters/csv-exporter.js.map +1 -1
- package/dist/exporters/feature-parser.d.ts +5 -2
- package/dist/exporters/feature-parser.d.ts.map +1 -1
- package/dist/exporters/feature-parser.js +11 -9
- package/dist/exporters/feature-parser.js.map +1 -1
- package/dist/exporters/json-exporter.js +1 -1
- package/dist/exporters/json-exporter.js.map +1 -1
- package/dist/exporters/spec-parser.js +1 -1
- package/dist/exporters/spec-parser.js.map +1 -1
- package/dist/orchestrator/templates/ai-instructions/claude-cmd-run-test.md +2 -2
- package/dist/orchestrator/templates/ai-instructions/claude-skill-delivery.md +1 -1
- package/dist/orchestrator/templates/ai-instructions/claude-skill-gherkin-syntax.md +5 -6
- package/dist/orchestrator/templates/ai-instructions/claude-skill-selector-fix.md +7 -7
- package/dist/orchestrator/templates/ai-instructions/claude-skill-tc-generation.md +28 -23
- package/dist/orchestrator/templates/ai-instructions/claude-skill-tc-review.md +3 -3
- package/dist/orchestrator/templates/ai-instructions/claude-skill-test-design-techniques.md +2 -2
- package/dist/orchestrator/templates/ai-instructions/claude-skill-viewpoint.md +1 -1
- package/dist/orchestrator/templates/ai-instructions/copilot-cmd-run-test.md +2 -2
- package/dist/orchestrator/templates/ai-instructions/github-skill-sungen-delivery.md +1 -1
- package/dist/orchestrator/templates/ai-instructions/github-skill-sungen-gherkin-syntax.md +5 -6
- package/dist/orchestrator/templates/ai-instructions/github-skill-sungen-selector-fix.md +7 -7
- package/dist/orchestrator/templates/ai-instructions/github-skill-sungen-tc-generation.md +28 -23
- package/dist/orchestrator/templates/ai-instructions/github-skill-sungen-tc-review.md +3 -3
- package/dist/orchestrator/templates/ai-instructions/github-skill-sungen-test-design-techniques.md +2 -2
- package/dist/orchestrator/templates/ai-instructions/github-skill-sungen-viewpoint.md +1 -1
- package/package.json +1 -1
- package/src/cli/index.ts +1 -1
- package/src/dashboard/templates/index.html +2 -2
- package/src/dashboard/types.ts +1 -1
- package/src/exporters/csv-exporter.ts +1 -1
- package/src/exporters/feature-parser.ts +11 -8
- package/src/exporters/json-exporter.ts +1 -1
- package/src/exporters/spec-parser.ts +1 -1
- package/src/generators/test-generator/adapters/adapter-interface.ts +1 -1
- package/src/orchestrator/templates/ai-instructions/claude-cmd-run-test.md +2 -2
- package/src/orchestrator/templates/ai-instructions/claude-skill-delivery.md +1 -1
- package/src/orchestrator/templates/ai-instructions/claude-skill-gherkin-syntax.md +5 -6
- package/src/orchestrator/templates/ai-instructions/claude-skill-selector-fix.md +7 -7
- package/src/orchestrator/templates/ai-instructions/claude-skill-tc-generation.md +28 -23
- package/src/orchestrator/templates/ai-instructions/claude-skill-tc-review.md +3 -3
- package/src/orchestrator/templates/ai-instructions/claude-skill-test-design-techniques.md +2 -2
- package/src/orchestrator/templates/ai-instructions/claude-skill-viewpoint.md +1 -1
- package/src/orchestrator/templates/ai-instructions/copilot-cmd-run-test.md +2 -2
- package/src/orchestrator/templates/ai-instructions/github-skill-sungen-delivery.md +1 -1
- package/src/orchestrator/templates/ai-instructions/github-skill-sungen-gherkin-syntax.md +5 -6
- package/src/orchestrator/templates/ai-instructions/github-skill-sungen-selector-fix.md +7 -7
- package/src/orchestrator/templates/ai-instructions/github-skill-sungen-tc-generation.md +28 -23
- package/src/orchestrator/templates/ai-instructions/github-skill-sungen-tc-review.md +3 -3
- package/src/orchestrator/templates/ai-instructions/github-skill-sungen-test-design-techniques.md +2 -2
- package/src/orchestrator/templates/ai-instructions/github-skill-sungen-viewpoint.md +1 -1
|
@@ -179,7 +179,7 @@ Skip if `specs/.auth/<role>.json` already exists and a probe navigation reaches
|
|
|
179
179
|
|
|
180
180
|
## Phase 1: Smoke Check (catch fundamentals)
|
|
181
181
|
|
|
182
|
-
Run **up to 5 scenarios** — pick the first `@
|
|
182
|
+
Run **up to 5 scenarios** — pick the first `@high` scenarios in the feature file.
|
|
183
183
|
|
|
184
184
|
```bash
|
|
185
185
|
npx playwright test --grep "VP-.*-001|VP-.*-002|VP-.*-003|VP-.*-004|VP-.*-005" --reporter=line
|
|
@@ -195,15 +195,15 @@ npx playwright test --grep "VP-.*-001|VP-.*-002|VP-.*-003|VP-.*-004|VP-.*-005" -
|
|
|
195
195
|
|
|
196
196
|
---
|
|
197
197
|
|
|
198
|
-
## Phase 2: Priority Wave (@
|
|
198
|
+
## Phase 2: Priority Wave (@high)
|
|
199
199
|
|
|
200
|
-
Run all `@
|
|
200
|
+
Run all `@high` scenarios:
|
|
201
201
|
|
|
202
202
|
```bash
|
|
203
|
-
npx playwright test --grep "@
|
|
203
|
+
npx playwright test --grep "@high" --reporter=line
|
|
204
204
|
```
|
|
205
205
|
|
|
206
|
-
If your Playwright config doesn't support tag grep, use scenario name grep from the feature file — collect VP IDs of `@
|
|
206
|
+
If your Playwright config doesn't support tag grep, use scenario name grep from the feature file — collect VP IDs of `@high` scenarios.
|
|
207
207
|
|
|
208
208
|
**Fix only failures from this wave.** Most shared selectors (buttons, headings, navigation) get fixed here because critical/high scenarios exercise them.
|
|
209
209
|
|
|
@@ -353,8 +353,8 @@ user detail:
|
|
|
353
353
|
|---|---|---|---|
|
|
354
354
|
| 0. Pre-gen | Playwright MCP snapshot → write selectors.yaml | 1 snapshot | Ask user — skip or retry navigation |
|
|
355
355
|
| 0.5. Auth | Manual login in MCP browser → `browser_storage_state` → `specs/.auth/<role>.json` | 1 login | Ask user — retry login or fall back to `sungen makeauth` |
|
|
356
|
-
| 1. Smoke | First 5 @
|
|
357
|
-
| 2. Priority | All @
|
|
356
|
+
| 1. Smoke | First 5 @high | 2 | Ask user — fundamentals broken |
|
|
357
|
+
| 2. Priority | All @high | 2 | Report failures, continue to Phase 3 |
|
|
358
358
|
| 3. Full | All tests | 1 | Report @low/@normal failures, continue |
|
|
359
359
|
| 4. Regression | All tests | 0 | Report final results |
|
|
360
360
|
|
|
@@ -12,7 +12,7 @@ Generate **focused test cases per screen section** using a **tier-based approach
|
|
|
12
12
|
|
|
13
13
|
| Tier | Priority | What to generate | When |
|
|
14
14
|
|---|---|---|---|
|
|
15
|
-
| **Tier 1** (default) | `@
|
|
15
|
+
| **Tier 1** (default) | `@high` | Happy paths, required validation, core business rules, security basics | First run of `create-test` |
|
|
16
16
|
| **Tier 2** (expand) | `@normal` + `@low` | UI presence, optional validation, edge cases, cosmetic checks | User runs `create-test` again with "Add viewpoints" mode |
|
|
17
17
|
|
|
18
18
|
**Round 1 (Tier 1)** targets **~10-15 scenarios per section** — enough to cover critical flows and catch real bugs. This is the default behavior.
|
|
@@ -24,7 +24,7 @@ Generate **focused test cases per screen section** using a **tier-based approach
|
|
|
24
24
|
When `.feature` already has scenarios, summarize and ask:
|
|
25
25
|
1. **Add new sections** — append new sections with Tier 2 (`@normal` + `@low`) scenarios, continue numbering
|
|
26
26
|
2. **Add viewpoints** — expand existing sections with Tier 2 (`@normal` + `@low`) scenarios
|
|
27
|
-
3. **Replace all** — overwrite with fresh Tier 1 (`@
|
|
27
|
+
3. **Replace all** — overwrite with fresh Tier 1 (`@high`) generation
|
|
28
28
|
|
|
29
29
|
For append: read highest `VP-<CAT>-<NNN>`, continue from next number. Never modify existing scenarios.
|
|
30
30
|
|
|
@@ -102,7 +102,7 @@ Apply `sungen-test-design-techniques` to spec-extracted conditions:
|
|
|
102
102
|
Use `sungen-viewpoint` skill for per-pattern checklists across 4 viewpoints: UI/UX, Data & Validate, Logic, Security.
|
|
103
103
|
|
|
104
104
|
**Tier-aware gap filling:**
|
|
105
|
-
- **Tier 1 (first run)**: only add `@
|
|
105
|
+
- **Tier 1 (first run)**: only add `@high` items from the checklists — core security checks (VP-SEC), required field validation (VP-VAL), key state transitions (VP-LOGIC). Skip `@normal`/`@low` items like hover states, empty states, tooltips.
|
|
106
106
|
- **Tier 2 (expand run)**: add `@normal` + `@low` scenarios — UI presence, optional validation, edge cases, cosmetic checks, keyboard nav, hover effects.
|
|
107
107
|
|
|
108
108
|
**Validation rule**: capture actual error messages from live page or spec.md. Use `User see {{error_var}}` — never assert just "is visible".
|
|
@@ -111,24 +111,29 @@ Use `sungen-viewpoint` skill for per-pattern checklists across 4 viewpoints: UI/
|
|
|
111
111
|
|
|
112
112
|
Every scenario **MUST** have exactly one priority tag. Add it before the scenario line (after `@extend:` if present).
|
|
113
113
|
|
|
114
|
+
**Rule: look at what the WHEN step does and what THEN asserts — that determines the tag.**
|
|
115
|
+
|
|
114
116
|
| Tag | When to use |
|
|
115
117
|
|---|---|
|
|
116
|
-
| `@
|
|
117
|
-
| `@
|
|
118
|
-
| `@
|
|
119
|
-
| `@low` | Minor/cosmetic — tooltips, hover states, empty states, default sort, placeholder text |
|
|
118
|
+
| `@high` | WHEN performs: login/logout, submit happy path, auth-check, primary CRUD action. THEN asserts: success state, redirect, or security gate |
|
|
119
|
+
| `@normal` | WHEN provides invalid/empty input, OR scenario tests search/filter/notification/secondary flow. THEN asserts: error message, secondary feature result |
|
|
120
|
+
| `@low` | THEN only asserts: element visible, text label, placeholder, tooltip, icon, layout. WHEN (if any) is navigation only |
|
|
120
121
|
|
|
121
122
|
### Auto-assign heuristics
|
|
122
123
|
|
|
123
|
-
|
|
|
124
|
+
| Scenario type (by WHEN + THEN structure) | Tag |
|
|
124
125
|
|---|---|
|
|
125
|
-
|
|
|
126
|
-
|
|
|
127
|
-
|
|
|
128
|
-
|
|
|
129
|
-
|
|
|
130
|
-
|
|
|
131
|
-
|
|
|
126
|
+
| Auth: login / logout / protected page redirect | `@high` |
|
|
127
|
+
| CRUD happy path: create / update / delete → success message or redirect | `@high` |
|
|
128
|
+
| Security: unauthenticated access → redirected to login | `@high` |
|
|
129
|
+
| Primary business flow step completes successfully | `@high` |
|
|
130
|
+
| Validation: invalid / empty input → error message shown | `@normal` |
|
|
131
|
+
| Boundary / format validation (email, length, range) | `@normal` |
|
|
132
|
+
| Secondary features: search, filter, sort, pagination, notification | `@normal` |
|
|
133
|
+
| State transition not on primary success path (cancel, undo, back) | `@normal` |
|
|
134
|
+
| Element / component presence check (is it visible?) | `@low` |
|
|
135
|
+
| Text content: label, placeholder, tooltip, warning message text | `@low` |
|
|
136
|
+
| Visual / cosmetic: alignment, icon, empty state, hover style | `@low` |
|
|
132
137
|
|
|
133
138
|
**`@steps:` scenarios** do NOT get a priority tag (they are setup blocks, not test cases).
|
|
134
139
|
|
|
@@ -188,11 +193,11 @@ Serial (default) is best for: CRUD flows, sequential user journeys, UI checks on
|
|
|
188
193
|
@cleanup:forms
|
|
189
194
|
Feature: kudos Screen
|
|
190
195
|
|
|
191
|
-
@
|
|
196
|
+
@high
|
|
192
197
|
Scenario: Send kudos
|
|
193
198
|
...
|
|
194
199
|
|
|
195
|
-
@
|
|
200
|
+
@high @no-auth
|
|
196
201
|
Scenario: Unauthenticated user is redirected
|
|
197
202
|
...
|
|
198
203
|
```
|
|
@@ -218,9 +223,9 @@ Feature: <Screen> Screen
|
|
|
218
223
|
When User click [Create] button
|
|
219
224
|
Then User see [Form] dialog
|
|
220
225
|
|
|
221
|
-
# --- Section: Create User Form (Tier 1: @
|
|
226
|
+
# --- Section: Create User Form (Tier 1: @high) ---
|
|
222
227
|
|
|
223
|
-
@
|
|
228
|
+
@high @extend:open_form
|
|
224
229
|
Scenario: VP-LOGIC-001 Submit form with valid data creates record
|
|
225
230
|
Given User is on [Form] dialog
|
|
226
231
|
When User fill [Name] field with {{valid_name}}
|
|
@@ -233,7 +238,7 @@ Feature: <Screen> Screen
|
|
|
233
238
|
When User click [Submit] button
|
|
234
239
|
Then User see [Name error] message with {{name_required_error}}
|
|
235
240
|
|
|
236
|
-
# --- Section: User Table (Tier 1: @
|
|
241
|
+
# --- Section: User Table (Tier 1: @high) ---
|
|
237
242
|
|
|
238
243
|
@high
|
|
239
244
|
Scenario: VP-VAL-010 Table displays correct data
|
|
@@ -241,7 +246,7 @@ Feature: <Screen> Screen
|
|
|
241
246
|
| Name | Email |
|
|
242
247
|
| {{name_1}} | {{email_1}} |
|
|
243
248
|
|
|
244
|
-
@
|
|
249
|
+
@high
|
|
245
250
|
Scenario: VP-SEC-010 Unauthorized user cannot access page
|
|
246
251
|
Given User is not logged in
|
|
247
252
|
When User navigate to [Screen] page
|
|
@@ -317,14 +322,14 @@ Feature: Award Submission Flow
|
|
|
317
322
|
Background:
|
|
318
323
|
Given User is on [Login] page
|
|
319
324
|
|
|
320
|
-
@
|
|
325
|
+
@high
|
|
321
326
|
Scenario: User login successfully
|
|
322
327
|
When User fill [Login:Email] field with {{login.email}}
|
|
323
328
|
And User fill [Login:Password] field with {{login.password}}
|
|
324
329
|
And User click [Login:Submit] button
|
|
325
330
|
Then User see [Dashboard] page
|
|
326
331
|
|
|
327
|
-
@
|
|
332
|
+
@high
|
|
328
333
|
Scenario: User navigates to awards and submits
|
|
329
334
|
Given User is on [Awards] page
|
|
330
335
|
When User fill [Awards:Nominee] field with {{submission.nominee}}
|
|
@@ -38,12 +38,12 @@ Score: `(dimensions_covered / 6) * 40`. Validate technique application with `sun
|
|
|
38
38
|
| All applicable VP present (UI/VAL/LOGIC/SEC) | 10 |
|
|
39
39
|
| Correct classification | 8 |
|
|
40
40
|
| `VP-<CAT>-<NNN>` naming + section grouping | 4 |
|
|
41
|
-
| Priority tag present and correct (`@
|
|
41
|
+
| Priority tag present and correct (`@high`/`@normal`/`@low`) | 4 |
|
|
42
42
|
| Assertion quality (see rules below) | 4 |
|
|
43
43
|
|
|
44
44
|
**Classification**: UI = static/always-same appearance. VAL = input validation/errors. LOGIC = behavior/state changes (includes persisted state without When). SEC = auth/permissions.
|
|
45
45
|
|
|
46
|
-
**Tier-aware scoring**: If the feature file only contains `@
|
|
46
|
+
**Tier-aware scoring**: If the feature file only contains `@high` scenarios (Tier 1), do NOT penalize for missing VP-UI viewpoint — UI scenarios are intentionally deferred to Tier 2. Score "All applicable VP present" based on Tier 1-relevant viewpoints only (VAL, LOGIC, SEC). Note in the review output: *"VP-UI deferred to Tier 2 — run create-test with 'Add viewpoints' to expand."*
|
|
47
47
|
|
|
48
48
|
---
|
|
49
49
|
|
|
@@ -80,7 +80,7 @@ Do NOT mark `@manual` when data is visible in snapshot or documented in spec —
|
|
|
80
80
|
2. **Misclassified VP** — UI tests behavior? Move to LOGIC. Logic tests appearance? Move to UI
|
|
81
81
|
3. **Dynamic content** — exact match on counters/timestamps? Use `contains` instead
|
|
82
82
|
4. **Duplicate across sections** — SEC scenario identical to UI? Remove duplicate
|
|
83
|
-
5. **Missing/wrong priority tag** — every non-`@steps` scenario needs exactly one of `@
|
|
83
|
+
5. **Missing/wrong priority tag** — every non-`@steps` scenario needs exactly one of `@high`/`@normal`/`@low`. SEC/CRUD happy path/auth→`@high`, validation/secondary features→`@normal`, presence/cosmetic→`@low`
|
|
84
84
|
6. **Always-enabled elements** — `is enabled` on never-disabled element? Remove
|
|
85
85
|
7. **Test-data completeness** — every `{{var}}` must exist in test-data.yaml
|
|
86
86
|
8. **Missing BVA boundaries** — spec defines min/max range but scenarios only test midpoint? Add `min-1`, `min`, `max`, `max+1`
|
package/dist/orchestrator/templates/ai-instructions/github-skill-sungen-test-design-techniques.md
CHANGED
|
@@ -53,7 +53,7 @@ Scenario: VP-VAL-003 Above maximum is rejected # value = 101
|
|
|
53
53
|
| Mode | Values | Use when |
|
|
54
54
|
|---|---|---|
|
|
55
55
|
| **Compact (default)** | `min-1`, `min`, `max`, `max+1` | Most fields |
|
|
56
|
-
| **Full 6-point** | `min-1`, `min`, `min+1`, `max-1`, `max`, `max+1` | Critical fields with `@
|
|
56
|
+
| **Full 6-point** | `min-1`, `min`, `min+1`, `max-1`, `max`, `max+1` | Critical fields with `@high` priority |
|
|
57
57
|
|
|
58
58
|
**How to apply** (example: "quantity must be 1-10"):
|
|
59
59
|
- `min-1` = 0 -> invalid
|
|
@@ -82,7 +82,7 @@ Scenario: VP-VAL-003 Above maximum is rejected # value = 101
|
|
|
82
82
|
- `@normal` Form invalid + no permission → disabled
|
|
83
83
|
- `@normal` Form valid + no permission → disabled
|
|
84
84
|
- `@normal` Has permission + form invalid → disabled
|
|
85
|
-
- `@
|
|
85
|
+
- `@high` Form valid + has permission → succeeds
|
|
86
86
|
|
|
87
87
|
---
|
|
88
88
|
|
|
@@ -266,7 +266,7 @@ For VP-SEC scenarios testing **unauthorized access** (no login, wrong role, dire
|
|
|
266
266
|
- Do NOT use `@manual` for these — they are automatable.
|
|
267
267
|
|
|
268
268
|
```gherkin
|
|
269
|
-
@
|
|
269
|
+
@high @no-auth
|
|
270
270
|
Scenario: VP-SEC-001 Unauthenticated user cannot access admin page
|
|
271
271
|
Given User is on [Admin] page
|
|
272
272
|
Then User see [Login] page
|
package/package.json
CHANGED