@sun-asterisk/sungen 2.6.5 → 2.6.7
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/dist/cli/index.js +1 -1
- package/dist/dashboard/templates/index.html +2 -2
- package/dist/dashboard/types.d.ts +1 -1
- package/dist/dashboard/types.d.ts.map +1 -1
- package/dist/exporters/csv-exporter.js +1 -1
- package/dist/exporters/csv-exporter.js.map +1 -1
- package/dist/exporters/feature-parser.d.ts +5 -2
- package/dist/exporters/feature-parser.d.ts.map +1 -1
- package/dist/exporters/feature-parser.js +11 -9
- package/dist/exporters/feature-parser.js.map +1 -1
- package/dist/exporters/json-exporter.js +1 -1
- package/dist/exporters/json-exporter.js.map +1 -1
- package/dist/exporters/spec-parser.js +1 -1
- package/dist/exporters/spec-parser.js.map +1 -1
- package/dist/orchestrator/templates/ai-instructions/claude-cmd-create-test.md +23 -7
- package/dist/orchestrator/templates/ai-instructions/claude-cmd-run-test.md +2 -2
- package/dist/orchestrator/templates/ai-instructions/claude-skill-delivery.md +1 -1
- package/dist/orchestrator/templates/ai-instructions/claude-skill-gherkin-syntax.md +5 -6
- package/dist/orchestrator/templates/ai-instructions/claude-skill-selector-fix.md +7 -7
- package/dist/orchestrator/templates/ai-instructions/claude-skill-tc-generation.md +60 -26
- package/dist/orchestrator/templates/ai-instructions/claude-skill-tc-review.md +3 -3
- package/dist/orchestrator/templates/ai-instructions/claude-skill-test-design-techniques.md +2 -2
- package/dist/orchestrator/templates/ai-instructions/claude-skill-viewpoint.md +1 -1
- package/dist/orchestrator/templates/ai-instructions/copilot-cmd-create-test.md +22 -6
- package/dist/orchestrator/templates/ai-instructions/copilot-cmd-run-test.md +2 -2
- package/dist/orchestrator/templates/ai-instructions/github-skill-sungen-delivery.md +1 -1
- package/dist/orchestrator/templates/ai-instructions/github-skill-sungen-gherkin-syntax.md +5 -6
- package/dist/orchestrator/templates/ai-instructions/github-skill-sungen-selector-fix.md +7 -7
- package/dist/orchestrator/templates/ai-instructions/github-skill-sungen-tc-generation.md +60 -26
- package/dist/orchestrator/templates/ai-instructions/github-skill-sungen-tc-review.md +3 -3
- package/dist/orchestrator/templates/ai-instructions/github-skill-sungen-test-design-techniques.md +2 -2
- package/dist/orchestrator/templates/ai-instructions/github-skill-sungen-viewpoint.md +1 -1
- package/package.json +1 -1
- package/src/cli/index.ts +1 -1
- package/src/dashboard/templates/index.html +2 -2
- package/src/dashboard/types.ts +1 -1
- package/src/exporters/csv-exporter.ts +1 -1
- package/src/exporters/feature-parser.ts +11 -8
- package/src/exporters/json-exporter.ts +1 -1
- package/src/exporters/spec-parser.ts +1 -1
- package/src/generators/test-generator/adapters/adapter-interface.ts +1 -1
- package/src/orchestrator/templates/ai-instructions/claude-cmd-create-test.md +23 -7
- package/src/orchestrator/templates/ai-instructions/claude-cmd-run-test.md +2 -2
- package/src/orchestrator/templates/ai-instructions/claude-skill-delivery.md +1 -1
- package/src/orchestrator/templates/ai-instructions/claude-skill-gherkin-syntax.md +5 -6
- package/src/orchestrator/templates/ai-instructions/claude-skill-selector-fix.md +7 -7
- package/src/orchestrator/templates/ai-instructions/claude-skill-tc-generation.md +60 -26
- package/src/orchestrator/templates/ai-instructions/claude-skill-tc-review.md +3 -3
- package/src/orchestrator/templates/ai-instructions/claude-skill-test-design-techniques.md +2 -2
- package/src/orchestrator/templates/ai-instructions/claude-skill-viewpoint.md +1 -1
- package/src/orchestrator/templates/ai-instructions/copilot-cmd-create-test.md +22 -6
- package/src/orchestrator/templates/ai-instructions/copilot-cmd-run-test.md +2 -2
- package/src/orchestrator/templates/ai-instructions/github-skill-sungen-delivery.md +1 -1
- package/src/orchestrator/templates/ai-instructions/github-skill-sungen-gherkin-syntax.md +5 -6
- package/src/orchestrator/templates/ai-instructions/github-skill-sungen-selector-fix.md +7 -7
- package/src/orchestrator/templates/ai-instructions/github-skill-sungen-tc-generation.md +60 -26
- package/src/orchestrator/templates/ai-instructions/github-skill-sungen-tc-review.md +3 -3
- package/src/orchestrator/templates/ai-instructions/github-skill-sungen-test-design-techniques.md +2 -2
- package/src/orchestrator/templates/ai-instructions/github-skill-sungen-viewpoint.md +1 -1
|
@@ -266,7 +266,7 @@ For VP-SEC scenarios testing **unauthorized access** (no login, wrong role, dire
|
|
|
266
266
|
- Do NOT use `@manual` for these — they are automatable.
|
|
267
267
|
|
|
268
268
|
```gherkin
|
|
269
|
-
@
|
|
269
|
+
@high @no-auth
|
|
270
270
|
Scenario: VP-SEC-001 Unauthenticated user cannot access admin page
|
|
271
271
|
Given User is on [Admin] page
|
|
272
272
|
Then User see [Login] page
|
|
@@ -22,7 +22,11 @@ You are a **Senior QA Engineer**. You structure test cases by viewpoint categori
|
|
|
22
22
|
|
|
23
23
|
1. **Flow**: Verify `qa/flows/${input:name}/` exists. If not → `/sungen-add-flow` first.
|
|
24
24
|
**Screen**: Verify `qa/screens/${input:name}/` exists. If not → `/sungen-add-screen` first.
|
|
25
|
-
2. Check if `.feature` already has scenarios.
|
|
25
|
+
2. Check if `.feature` already has scenarios.
|
|
26
|
+
- If yes → summarize existing coverage and ask update mode (options depend on which tiers already exist — see `sungen-tc-generation` skill for details).
|
|
27
|
+
- If no → fresh creation. Ask generation scope:
|
|
28
|
+
- **1) Tier 1 — Critical & High priority** — ~10-15 scenarios/section covering happy paths, core validation, security basics **(Recommended)**
|
|
29
|
+
- **2) Full coverage — All tiers at once** — generates Tier 1 + 2 + 3 in one run. Large output (~40-60 scenarios/section), best for experienced users who want complete coverage immediately
|
|
26
30
|
3. **Read requirements & resolve visual source** — check `<base>/${input:name}/requirements/`:
|
|
27
31
|
- If `spec.md` exists → read it as PRIMARY source (sections, fields, validation rules, business rules, states).
|
|
28
32
|
- If `test-viewpoint.md` exists → read it. If it only contains HTML comments (scaffold template), ask:
|
|
@@ -44,11 +48,23 @@ You are a **Senior QA Engineer**. You structure test cases by viewpoint categori
|
|
|
44
48
|
|
|
45
49
|
4. Follow the `sungen-tc-generation` skill for section identification, viewpoint generation, and output format. **For flows**, use the "Flow Test Generation" section in the skill. When requirements exist, use the "Requirements-Driven Generation" strategy. Present sections as a numbered list and let user pick.
|
|
46
50
|
5. Generate or update `.feature` + `test-data.yaml` following `sungen-gherkin-syntax` and `sungen-tc-generation` skills. **For flows**: use `[Screen:Element]` namespace format, namespace test-data by phase, add `@flow` tag.
|
|
47
|
-
6. Show summary and offer next steps:
|
|
51
|
+
6. Show summary and offer next steps based on which tier was just generated:
|
|
48
52
|
|
|
49
|
-
|
|
50
|
-
- **`/sungen-
|
|
51
|
-
- **`/sungen-
|
|
52
|
-
-
|
|
53
|
+
**After Tier 1 generation:**
|
|
54
|
+
- **`/sungen-review ${input:name}`** — Review syntax, coverage, viewpoint quality (Recommended)
|
|
55
|
+
- **`/sungen-run-test ${input:name}`** — Skip review, generate selectors and run tests now
|
|
56
|
+
- **`/sungen-create-test ${input:name}`** — Expand coverage: add @normal + @low scenarios (Tier 2)
|
|
57
|
+
- **Done for now** — I'll come back later
|
|
58
|
+
|
|
59
|
+
**After Tier 2 generation:**
|
|
60
|
+
- **`/sungen-create-test ${input:name}`** — Deep coverage: add BVA combos, cross-field validation, negative inputs, race conditions (Tier 3) (Recommended)
|
|
61
|
+
- **`/sungen-review ${input:name}`** — Review syntax, coverage, viewpoint quality
|
|
62
|
+
- **`/sungen-run-test ${input:name}`** — Generate selectors and run tests now
|
|
63
|
+
- **Done for now** — I'll come back later
|
|
64
|
+
|
|
65
|
+
**After Tier 3 or Full generation:**
|
|
66
|
+
- **`/sungen-review ${input:name}`** — Review syntax, coverage, viewpoint quality (Recommended)
|
|
67
|
+
- **`/sungen-run-test ${input:name}`** — Generate selectors and run tests now
|
|
68
|
+
- **Done for now** — I'll come back later
|
|
53
69
|
|
|
54
70
|
**No selectors.yaml** — selectors are generated during `/sungen-run-test`.
|
|
@@ -56,8 +56,8 @@ Parse **name** from `$ARGUMENTS`. If missing, ask the user.
|
|
|
56
56
|
|
|
57
57
|
## Run & Fix (phased — per `sungen-selector-fix` skill)
|
|
58
58
|
|
|
59
|
-
5. **Phase 1 — Smoke Check**: Run first 5 `@
|
|
60
|
-
6. **Phase 2 — Priority Wave**: Run all `@
|
|
59
|
+
5. **Phase 1 — Smoke Check**: Run first 5 `@high` scenarios only. If failures → diagnose, fix fundamentals (page selector, auth, base @steps), re-run. Max 2 attempts. If still broken → ask user.
|
|
60
|
+
6. **Phase 2 — Priority Wave**: Run all `@high` scenarios. Fix only failures from this wave. Max 2 attempts. Shared selectors fixed here cascade to later phases.
|
|
61
61
|
7. **Phase 3 — Full Run**: Run all tests. Fix only **new** failures (elements unique to `@normal`/`@low`). Max 1 attempt. Don't loop on low-priority failures.
|
|
62
62
|
8. **Phase 4 — Regression**: One final full run. Report results. No more fix loops.
|
|
63
63
|
|
|
@@ -68,7 +68,7 @@ The CLI reads the **per-target result file first** (co-located with `.spec.ts`),
|
|
|
68
68
|
| Test Data | `{{vars}}` from scenario resolved via test-data.yaml → `key: value; key2: value2` |
|
|
69
69
|
| Steps | `.spec.ts` code comments for interactions (numbered) |
|
|
70
70
|
| Expected results | `.spec.ts` `expect(...)` comments (numbered) |
|
|
71
|
-
| Priority | Tag: `@
|
|
71
|
+
| Priority | Tag: `@high`/`@normal`/`@low` (default: Normal) |
|
|
72
72
|
| Testcase type | `@manual` → Manual, else Auto. Not compiled → "Not compiled" |
|
|
73
73
|
| Test Result | results.json status: passed→Passed, failed/timedOut→Failed, skipped→N/A, else Pending |
|
|
74
74
|
| Executed Date | results.json startTime formatted as `dd/mm/yyyy` |
|
|
@@ -192,18 +192,17 @@ Any tag not listed above passes through to Playwright `{ tag: [...] }`. Feature-
|
|
|
192
192
|
|---|---|
|
|
193
193
|
| `@smoke` | Quick sanity check — run after every deploy |
|
|
194
194
|
| `@regression` | Full test suite — run nightly or before release |
|
|
195
|
-
| `@
|
|
196
|
-
| `@
|
|
197
|
-
| `@
|
|
198
|
-
| `@low` | Priority: minor/cosmetic (tooltips, hover states, empty states) |
|
|
195
|
+
| `@high` | Priority: must pass — login, auth, core CRUD, primary business flow |
|
|
196
|
+
| `@normal` | Priority: important — validation rules, secondary features, search/filter |
|
|
197
|
+
| `@low` | Priority: minor/cosmetic — tooltips, hover states, element presence |
|
|
199
198
|
| `@auto` | Standard scenario, ready for automation |
|
|
200
199
|
| Any custom | e.g., `@sprint-42`, `@team-payment` — any tag works |
|
|
201
200
|
|
|
202
201
|
**Run filtered:**
|
|
203
202
|
```bash
|
|
204
203
|
npx playwright test --grep "@smoke" # only smoke tests
|
|
205
|
-
npx playwright test --grep "@
|
|
206
|
-
npx playwright test --grep "@smoke|@
|
|
204
|
+
npx playwright test --grep "@high" # only high priority
|
|
205
|
+
npx playwright test --grep "@smoke|@high" # smoke OR high
|
|
207
206
|
```
|
|
208
207
|
|
|
209
208
|
### Serial vs Parallel (test execution mode)
|
|
@@ -179,7 +179,7 @@ Skip if `specs/.auth/<role>.json` already exists and a probe navigation reaches
|
|
|
179
179
|
|
|
180
180
|
## Phase 1: Smoke Check (catch fundamentals)
|
|
181
181
|
|
|
182
|
-
Run **up to 5 scenarios** — pick the first `@
|
|
182
|
+
Run **up to 5 scenarios** — pick the first `@high` scenarios in the feature file.
|
|
183
183
|
|
|
184
184
|
```bash
|
|
185
185
|
npx playwright test --grep "VP-.*-001|VP-.*-002|VP-.*-003|VP-.*-004|VP-.*-005" --reporter=line
|
|
@@ -195,15 +195,15 @@ npx playwright test --grep "VP-.*-001|VP-.*-002|VP-.*-003|VP-.*-004|VP-.*-005" -
|
|
|
195
195
|
|
|
196
196
|
---
|
|
197
197
|
|
|
198
|
-
## Phase 2: Priority Wave (@
|
|
198
|
+
## Phase 2: Priority Wave (@high)
|
|
199
199
|
|
|
200
|
-
Run all `@
|
|
200
|
+
Run all `@high` scenarios:
|
|
201
201
|
|
|
202
202
|
```bash
|
|
203
|
-
npx playwright test --grep "@
|
|
203
|
+
npx playwright test --grep "@high" --reporter=line
|
|
204
204
|
```
|
|
205
205
|
|
|
206
|
-
If your Playwright config doesn't support tag grep, use scenario name grep from the feature file — collect VP IDs of `@
|
|
206
|
+
If your Playwright config doesn't support tag grep, use scenario name grep from the feature file — collect VP IDs of `@high` scenarios.
|
|
207
207
|
|
|
208
208
|
**Fix only failures from this wave.** Most shared selectors (buttons, headings, navigation) get fixed here because critical/high scenarios exercise them.
|
|
209
209
|
|
|
@@ -353,8 +353,8 @@ user detail:
|
|
|
353
353
|
|---|---|---|---|
|
|
354
354
|
| 0. Pre-gen | Playwright MCP snapshot → write selectors.yaml | 1 snapshot | Ask user — skip or retry navigation |
|
|
355
355
|
| 0.5. Auth | Manual login in MCP browser → `browser_storage_state` → `specs/.auth/<role>.json` | 1 login | Ask user — retry login or fall back to `sungen makeauth` |
|
|
356
|
-
| 1. Smoke | First 5 @
|
|
357
|
-
| 2. Priority | All @
|
|
356
|
+
| 1. Smoke | First 5 @high | 2 | Ask user — fundamentals broken |
|
|
357
|
+
| 2. Priority | All @high | 2 | Report failures, continue to Phase 3 |
|
|
358
358
|
| 3. Full | All tests | 1 | Report @low/@normal failures, continue |
|
|
359
359
|
| 4. Regression | All tests | 0 | Report final results |
|
|
360
360
|
|
|
@@ -12,19 +12,39 @@ Generate **focused test cases per screen section** using a **tier-based approach
|
|
|
12
12
|
|
|
13
13
|
| Tier | Priority | What to generate | When |
|
|
14
14
|
|---|---|---|---|
|
|
15
|
-
| **Tier 1** (default) | `@
|
|
15
|
+
| **Tier 1** (default) | `@high` | Happy paths, required validation, core business rules, security basics | First run of `create-test` |
|
|
16
16
|
| **Tier 2** (expand) | `@normal` + `@low` | UI presence, optional validation, edge cases, cosmetic checks | User runs `create-test` again with "Add viewpoints" mode |
|
|
17
|
+
| **Tier 3** (deep) | `@high` + `@normal` | Extra BVA combinations, cross-field validation, negative/destructive inputs, concurrent/race conditions, complex state transitions | Recommended after Tier 2 completes |
|
|
18
|
+
| **Full** (all-at-once) | All | Tier 1 + 2 + 3 combined in one run | Option at first run, **not recommended** — large output |
|
|
17
19
|
|
|
18
|
-
**Round 1 (Tier 1)** targets **~10-15 scenarios per section** — enough to cover critical flows and catch real bugs. This is the default behavior.
|
|
20
|
+
**Round 1 (Tier 1)** targets **~10-15 scenarios per section** — enough to cover critical flows and catch real bugs. This is the default behavior. A **Full** option is available but not recommended — it generates all tiers at once, producing very large output (~40-60 scenarios/section).
|
|
19
21
|
|
|
20
|
-
**Round 2 (Tier 2)** expands
|
|
22
|
+
**Round 2 (Tier 2)** expands coverage when the user explicitly chooses "Add viewpoints" or "Add new sections" update mode. Only then generate `@normal` + `@low` scenarios to fill coverage gaps.
|
|
23
|
+
|
|
24
|
+
**Round 3 (Tier 3)** deepens coverage with advanced test design techniques after Tier 2 is complete. This tier focuses on scenarios that Tier 1+2 didn't cover:
|
|
25
|
+
- **Extra BVA combinations**: additional boundary points not covered in Tier 1 (e.g., `min+1`, `max-1`, typical values between boundaries)
|
|
26
|
+
- **Cross-field validation**: field A value affects field B behavior (dependent fields, conditional required, cascading dropdowns)
|
|
27
|
+
- **Negative/destructive inputs**: SQL injection, XSS payloads, special characters, excessively long strings, unicode edge cases
|
|
28
|
+
- **Concurrent/race conditions**: double submit, back button after submit, multiple tabs, session expiry mid-action
|
|
29
|
+
- **Complex state transitions**: multi-step state changes, undo/redo, conflicting state updates, transition from every non-obvious state
|
|
21
30
|
|
|
22
31
|
## Update Mode
|
|
23
32
|
|
|
24
|
-
When `.feature` already has scenarios, summarize and ask:
|
|
33
|
+
When `.feature` already has scenarios, summarize existing coverage and ask:
|
|
34
|
+
|
|
35
|
+
**If only Tier 1 exists** (only `@high` scenarios):
|
|
25
36
|
1. **Add new sections** — append new sections with Tier 2 (`@normal` + `@low`) scenarios, continue numbering
|
|
26
37
|
2. **Add viewpoints** — expand existing sections with Tier 2 (`@normal` + `@low`) scenarios
|
|
27
|
-
3. **Replace all** — overwrite with fresh Tier 1 (`@
|
|
38
|
+
3. **Replace all** — overwrite with fresh Tier 1 (`@high`) generation
|
|
39
|
+
|
|
40
|
+
**If Tier 1 + 2 exist** (`@high` + `@normal` + `@low` scenarios):
|
|
41
|
+
1. **Deep coverage (Tier 3)** — add advanced scenarios: extra BVA, cross-field validation, negative inputs, race conditions **(Recommended)**
|
|
42
|
+
2. **Add new sections** — append new sections with Tier 2 scenarios, continue numbering
|
|
43
|
+
3. **Replace all** — overwrite with fresh Tier 1 (`@high`) generation
|
|
44
|
+
|
|
45
|
+
**If Tier 1 + 2 + 3 exist** (full coverage already):
|
|
46
|
+
1. **Add new sections** — append new sections if screen has changed
|
|
47
|
+
2. **Replace all** — overwrite with fresh Tier 1 (`@high`) generation
|
|
28
48
|
|
|
29
49
|
For append: read highest `VP-<CAT>-<NNN>`, continue from next number. Never modify existing scenarios.
|
|
30
50
|
|
|
@@ -102,8 +122,15 @@ Apply `sungen-test-design-techniques` to spec-extracted conditions:
|
|
|
102
122
|
Use `sungen-viewpoint` skill for per-pattern checklists across 4 viewpoints: UI/UX, Data & Validate, Logic, Security.
|
|
103
123
|
|
|
104
124
|
**Tier-aware gap filling:**
|
|
105
|
-
- **Tier 1 (first run)**: only add `@
|
|
125
|
+
- **Tier 1 (first run)**: only add `@high` items from the checklists — core security checks (VP-SEC), required field validation (VP-VAL), key state transitions (VP-LOGIC). Skip `@normal`/`@low` items like hover states, empty states, tooltips.
|
|
106
126
|
- **Tier 2 (expand run)**: add `@normal` + `@low` scenarios — UI presence, optional validation, edge cases, cosmetic checks, keyboard nav, hover effects.
|
|
127
|
+
- **Tier 3 (deep run)**: do NOT use viewpoint checklists. Instead, re-apply test design techniques with deeper analysis:
|
|
128
|
+
- **BVA**: expand from 4-point to 6-point (`min-1`, `min`, `min+1`, `max-1`, `max`, `max+1`) + typical mid-range value
|
|
129
|
+
- **Decision Table**: enumerate combinations previously capped — cover remaining outcome rows
|
|
130
|
+
- **Cross-field**: identify field dependencies from spec, generate one scenario per dependency
|
|
131
|
+
- **Negative inputs**: add security-oriented inputs (SQL `' OR 1=1`, XSS `<script>`, special chars `<>&"'`, max+100 length)
|
|
132
|
+
- **Race conditions**: double-click submit, browser back after POST, concurrent edit by two users
|
|
133
|
+
- **Full (all-at-once)**: apply Tier 1 + 2 + 3 rules in a single generation pass. Use all techniques at maximum depth.
|
|
107
134
|
|
|
108
135
|
**Validation rule**: capture actual error messages from live page or spec.md. Use `User see {{error_var}}` — never assert just "is visible".
|
|
109
136
|
|
|
@@ -111,24 +138,29 @@ Use `sungen-viewpoint` skill for per-pattern checklists across 4 viewpoints: UI/
|
|
|
111
138
|
|
|
112
139
|
Every scenario **MUST** have exactly one priority tag. Add it before the scenario line (after `@extend:` if present).
|
|
113
140
|
|
|
141
|
+
**Rule: look at what the WHEN step does and what THEN asserts — that determines the tag.**
|
|
142
|
+
|
|
114
143
|
| Tag | When to use |
|
|
115
144
|
|---|---|
|
|
116
|
-
| `@
|
|
117
|
-
| `@
|
|
118
|
-
| `@
|
|
119
|
-
| `@low` | Minor/cosmetic — tooltips, hover states, empty states, default sort, placeholder text |
|
|
145
|
+
| `@high` | WHEN performs: login/logout, submit happy path, auth-check, primary CRUD action. THEN asserts: success state, redirect, or security gate |
|
|
146
|
+
| `@normal` | WHEN provides invalid/empty input, OR scenario tests search/filter/notification/secondary flow. THEN asserts: error message, secondary feature result |
|
|
147
|
+
| `@low` | THEN only asserts: element visible, text label, placeholder, tooltip, icon, layout. WHEN (if any) is navigation only |
|
|
120
148
|
|
|
121
149
|
### Auto-assign heuristics
|
|
122
150
|
|
|
123
|
-
|
|
|
151
|
+
| Scenario type (by WHEN + THEN structure) | Tag |
|
|
124
152
|
|---|---|
|
|
125
|
-
|
|
|
126
|
-
|
|
|
127
|
-
|
|
|
128
|
-
|
|
|
129
|
-
|
|
|
130
|
-
|
|
|
131
|
-
|
|
|
153
|
+
| Auth: login / logout / protected page redirect | `@high` |
|
|
154
|
+
| CRUD happy path: create / update / delete → success message or redirect | `@high` |
|
|
155
|
+
| Security: unauthenticated access → redirected to login | `@high` |
|
|
156
|
+
| Primary business flow step completes successfully | `@high` |
|
|
157
|
+
| Validation: invalid / empty input → error message shown | `@normal` |
|
|
158
|
+
| Boundary / format validation (email, length, range) | `@normal` |
|
|
159
|
+
| Secondary features: search, filter, sort, pagination, notification | `@normal` |
|
|
160
|
+
| State transition not on primary success path (cancel, undo, back) | `@normal` |
|
|
161
|
+
| Element / component presence check (is it visible?) | `@low` |
|
|
162
|
+
| Text content: label, placeholder, tooltip, warning message text | `@low` |
|
|
163
|
+
| Visual / cosmetic: alignment, icon, empty state, hover style | `@low` |
|
|
132
164
|
|
|
133
165
|
**`@steps:` scenarios** do NOT get a priority tag (they are setup blocks, not test cases).
|
|
134
166
|
|
|
@@ -188,11 +220,11 @@ Serial (default) is best for: CRUD flows, sequential user journeys, UI checks on
|
|
|
188
220
|
@cleanup:forms
|
|
189
221
|
Feature: kudos Screen
|
|
190
222
|
|
|
191
|
-
@
|
|
223
|
+
@high
|
|
192
224
|
Scenario: Send kudos
|
|
193
225
|
...
|
|
194
226
|
|
|
195
|
-
@
|
|
227
|
+
@high @no-auth
|
|
196
228
|
Scenario: Unauthenticated user is redirected
|
|
197
229
|
...
|
|
198
230
|
```
|
|
@@ -218,9 +250,9 @@ Feature: <Screen> Screen
|
|
|
218
250
|
When User click [Create] button
|
|
219
251
|
Then User see [Form] dialog
|
|
220
252
|
|
|
221
|
-
# --- Section: Create User Form (Tier 1: @
|
|
253
|
+
# --- Section: Create User Form (Tier 1: @high) ---
|
|
222
254
|
|
|
223
|
-
@
|
|
255
|
+
@high @extend:open_form
|
|
224
256
|
Scenario: VP-LOGIC-001 Submit form with valid data creates record
|
|
225
257
|
Given User is on [Form] dialog
|
|
226
258
|
When User fill [Name] field with {{valid_name}}
|
|
@@ -233,7 +265,7 @@ Feature: <Screen> Screen
|
|
|
233
265
|
When User click [Submit] button
|
|
234
266
|
Then User see [Name error] message with {{name_required_error}}
|
|
235
267
|
|
|
236
|
-
# --- Section: User Table (Tier 1: @
|
|
268
|
+
# --- Section: User Table (Tier 1: @high) ---
|
|
237
269
|
|
|
238
270
|
@high
|
|
239
271
|
Scenario: VP-VAL-010 Table displays correct data
|
|
@@ -241,7 +273,7 @@ Feature: <Screen> Screen
|
|
|
241
273
|
| Name | Email |
|
|
242
274
|
| {{name_1}} | {{email_1}} |
|
|
243
275
|
|
|
244
|
-
@
|
|
276
|
+
@high
|
|
245
277
|
Scenario: VP-SEC-010 Unauthorized user cannot access page
|
|
246
278
|
Given User is not logged in
|
|
247
279
|
When User navigate to [Screen] page
|
|
@@ -250,6 +282,8 @@ Feature: <Screen> Screen
|
|
|
250
282
|
|
|
251
283
|
**Tier 2 (expand run)** adds `@normal` + `@low` scenarios like UI field presence, hover states, tooltips, empty states.
|
|
252
284
|
|
|
285
|
+
**Tier 3 (deep run)** adds advanced `@high` + `@normal` scenarios: extra BVA boundaries, cross-field validation, negative/destructive inputs, concurrent/race conditions.
|
|
286
|
+
|
|
253
287
|
### When to use DataTable vs Row Scope
|
|
254
288
|
|
|
255
289
|
| Pattern | Use when |
|
|
@@ -317,14 +351,14 @@ Feature: Award Submission Flow
|
|
|
317
351
|
Background:
|
|
318
352
|
Given User is on [Login] page
|
|
319
353
|
|
|
320
|
-
@
|
|
354
|
+
@high
|
|
321
355
|
Scenario: User login successfully
|
|
322
356
|
When User fill [Login:Email] field with {{login.email}}
|
|
323
357
|
And User fill [Login:Password] field with {{login.password}}
|
|
324
358
|
And User click [Login:Submit] button
|
|
325
359
|
Then User see [Dashboard] page
|
|
326
360
|
|
|
327
|
-
@
|
|
361
|
+
@high
|
|
328
362
|
Scenario: User navigates to awards and submits
|
|
329
363
|
Given User is on [Awards] page
|
|
330
364
|
When User fill [Awards:Nominee] field with {{submission.nominee}}
|
|
@@ -38,12 +38,12 @@ Score: `(dimensions_covered / 6) * 40`. Validate technique application with `sun
|
|
|
38
38
|
| All applicable VP present (UI/VAL/LOGIC/SEC) | 10 |
|
|
39
39
|
| Correct classification | 8 |
|
|
40
40
|
| `VP-<CAT>-<NNN>` naming + section grouping | 4 |
|
|
41
|
-
| Priority tag present and correct (`@
|
|
41
|
+
| Priority tag present and correct (`@high`/`@normal`/`@low`) | 4 |
|
|
42
42
|
| Assertion quality (see rules below) | 4 |
|
|
43
43
|
|
|
44
44
|
**Classification**: UI = static/always-same appearance. VAL = input validation/errors. LOGIC = behavior/state changes (includes persisted state without When). SEC = auth/permissions.
|
|
45
45
|
|
|
46
|
-
**Tier-aware scoring**: If the feature file only contains `@
|
|
46
|
+
**Tier-aware scoring**: If the feature file only contains `@high` scenarios (Tier 1), do NOT penalize for missing VP-UI viewpoint — UI scenarios are intentionally deferred to Tier 2. Score "All applicable VP present" based on Tier 1-relevant viewpoints only (VAL, LOGIC, SEC). Note in the review output: *"VP-UI deferred to Tier 2 — run create-test with 'Add viewpoints' to expand."*
|
|
47
47
|
|
|
48
48
|
---
|
|
49
49
|
|
|
@@ -80,7 +80,7 @@ Do NOT mark `@manual` when data is visible in snapshot or documented in spec —
|
|
|
80
80
|
2. **Misclassified VP** — UI tests behavior? Move to LOGIC. Logic tests appearance? Move to UI
|
|
81
81
|
3. **Dynamic content** — exact match on counters/timestamps? Use `contains` instead
|
|
82
82
|
4. **Duplicate across sections** — SEC scenario identical to UI? Remove duplicate
|
|
83
|
-
5. **Missing/wrong priority tag** — every non-`@steps` scenario needs exactly one of `@
|
|
83
|
+
5. **Missing/wrong priority tag** — every non-`@steps` scenario needs exactly one of `@high`/`@normal`/`@low`. SEC/CRUD happy path/auth→`@high`, validation/secondary features→`@normal`, presence/cosmetic→`@low`
|
|
84
84
|
6. **Always-enabled elements** — `is enabled` on never-disabled element? Remove
|
|
85
85
|
7. **Test-data completeness** — every `{{var}}` must exist in test-data.yaml
|
|
86
86
|
8. **Missing BVA boundaries** — spec defines min/max range but scenarios only test midpoint? Add `min-1`, `min`, `max`, `max+1`
|
package/dist/orchestrator/templates/ai-instructions/github-skill-sungen-test-design-techniques.md
CHANGED
|
@@ -53,7 +53,7 @@ Scenario: VP-VAL-003 Above maximum is rejected # value = 101
|
|
|
53
53
|
| Mode | Values | Use when |
|
|
54
54
|
|---|---|---|
|
|
55
55
|
| **Compact (default)** | `min-1`, `min`, `max`, `max+1` | Most fields |
|
|
56
|
-
| **Full 6-point** | `min-1`, `min`, `min+1`, `max-1`, `max`, `max+1` | Critical fields with `@
|
|
56
|
+
| **Full 6-point** | `min-1`, `min`, `min+1`, `max-1`, `max`, `max+1` | Critical fields with `@high` priority |
|
|
57
57
|
|
|
58
58
|
**How to apply** (example: "quantity must be 1-10"):
|
|
59
59
|
- `min-1` = 0 -> invalid
|
|
@@ -82,7 +82,7 @@ Scenario: VP-VAL-003 Above maximum is rejected # value = 101
|
|
|
82
82
|
- `@normal` Form invalid + no permission → disabled
|
|
83
83
|
- `@normal` Form valid + no permission → disabled
|
|
84
84
|
- `@normal` Has permission + form invalid → disabled
|
|
85
|
-
- `@
|
|
85
|
+
- `@high` Form valid + has permission → succeeds
|
|
86
86
|
|
|
87
87
|
---
|
|
88
88
|
|
|
@@ -266,7 +266,7 @@ For VP-SEC scenarios testing **unauthorized access** (no login, wrong role, dire
|
|
|
266
266
|
- Do NOT use `@manual` for these — they are automatable.
|
|
267
267
|
|
|
268
268
|
```gherkin
|
|
269
|
-
@
|
|
269
|
+
@high @no-auth
|
|
270
270
|
Scenario: VP-SEC-001 Unauthenticated user cannot access admin page
|
|
271
271
|
Given User is on [Admin] page
|
|
272
272
|
Then User see [Login] page
|
package/package.json
CHANGED