@sun-asterisk/sungen 2.6.15 → 2.7.0-beta.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/dist/cli/index.js +3 -1
- package/dist/cli/index.js.map +1 -1
- package/dist/exporters/feature-parser.d.ts +9 -2
- package/dist/exporters/feature-parser.d.ts.map +1 -1
- package/dist/exporters/feature-parser.js +12 -4
- package/dist/exporters/feature-parser.js.map +1 -1
- package/dist/orchestrator/ai-rules-updater.d.ts.map +1 -1
- package/dist/orchestrator/ai-rules-updater.js +10 -0
- package/dist/orchestrator/ai-rules-updater.js.map +1 -1
- package/dist/orchestrator/project-initializer.d.ts +5 -0
- package/dist/orchestrator/project-initializer.d.ts.map +1 -1
- package/dist/orchestrator/project-initializer.js +16 -0
- package/dist/orchestrator/project-initializer.js.map +1 -1
- package/dist/orchestrator/templates/ai-instructions/claude-cmd-create-test.md +9 -1
- package/dist/orchestrator/templates/ai-instructions/claude-cmd-review.md +13 -12
- package/dist/orchestrator/templates/ai-instructions/claude-cmd-run-test.md +4 -2
- package/dist/orchestrator/templates/ai-instructions/claude-config.md +1 -1
- package/dist/orchestrator/templates/ai-instructions/claude-skill-delivery.md +1 -1
- package/dist/orchestrator/templates/ai-instructions/claude-skill-gherkin-syntax.md +14 -0
- package/dist/orchestrator/templates/ai-instructions/claude-skill-selector-fix.md +57 -11
- package/dist/orchestrator/templates/ai-instructions/claude-skill-selector-keys.md +41 -31
- package/dist/orchestrator/templates/ai-instructions/claude-skill-tc-generation.md +386 -326
- package/dist/orchestrator/templates/ai-instructions/claude-skill-tc-review.md +268 -90
- package/dist/orchestrator/templates/ai-instructions/claude-skill-test-design-techniques.md +23 -49
- package/dist/orchestrator/templates/ai-instructions/claude-skill-viewpoint-group-a-data-entry.md +203 -0
- package/dist/orchestrator/templates/ai-instructions/claude-skill-viewpoint-group-b-data-ops.md +179 -0
- package/dist/orchestrator/templates/ai-instructions/claude-skill-viewpoint-group-c-data-explore.md +233 -0
- package/dist/orchestrator/templates/ai-instructions/claude-skill-viewpoint-group-d-display.md +226 -0
- package/dist/orchestrator/templates/ai-instructions/claude-skill-viewpoint-group-e-identity.md +177 -0
- package/dist/orchestrator/templates/ai-instructions/claude-skill-viewpoint.md +69 -240
- package/dist/orchestrator/templates/ai-instructions/copilot-cmd-create-test.md +9 -1
- package/dist/orchestrator/templates/ai-instructions/copilot-cmd-review.md +13 -12
- package/dist/orchestrator/templates/ai-instructions/copilot-cmd-run-test.md +4 -2
- package/dist/orchestrator/templates/ai-instructions/copilot-config.md +1 -1
- package/dist/orchestrator/templates/ai-instructions/github-skill-sungen-delivery.md +1 -1
- package/dist/orchestrator/templates/ai-instructions/github-skill-sungen-gherkin-syntax.md +15 -21
- package/dist/orchestrator/templates/ai-instructions/github-skill-sungen-selector-fix.md +61 -15
- package/dist/orchestrator/templates/ai-instructions/github-skill-sungen-selector-keys.md +41 -31
- package/dist/orchestrator/templates/ai-instructions/github-skill-sungen-tc-generation.md +371 -324
- package/dist/orchestrator/templates/ai-instructions/github-skill-sungen-tc-review.md +262 -102
- package/dist/orchestrator/templates/ai-instructions/github-skill-sungen-test-design-techniques.md +23 -49
- package/dist/orchestrator/templates/ai-instructions/github-skill-sungen-viewpoint-group-a-data-entry.md +203 -0
- package/dist/orchestrator/templates/ai-instructions/github-skill-sungen-viewpoint-group-b-data-ops.md +179 -0
- package/dist/orchestrator/templates/ai-instructions/github-skill-sungen-viewpoint-group-c-data-explore.md +233 -0
- package/dist/orchestrator/templates/ai-instructions/github-skill-sungen-viewpoint-group-d-display.md +226 -0
- package/dist/orchestrator/templates/ai-instructions/github-skill-sungen-viewpoint-group-e-identity.md +177 -0
- package/dist/orchestrator/templates/ai-instructions/github-skill-sungen-viewpoint.md +69 -240
- package/dist/orchestrator/templates/qa-context.md +90 -0
- package/dist/orchestrator/templates/readme.md +16 -13
- package/package.json +9 -1
- package/src/cli/index.ts +4 -1
- package/src/exporters/feature-parser.ts +12 -4
- package/src/orchestrator/ai-rules-updater.ts +10 -0
- package/src/orchestrator/project-initializer.ts +20 -0
- package/src/orchestrator/templates/ai-instructions/claude-cmd-create-test.md +9 -1
- package/src/orchestrator/templates/ai-instructions/claude-cmd-review.md +13 -12
- package/src/orchestrator/templates/ai-instructions/claude-cmd-run-test.md +4 -2
- package/src/orchestrator/templates/ai-instructions/claude-config.md +1 -1
- package/src/orchestrator/templates/ai-instructions/claude-skill-delivery.md +1 -1
- package/src/orchestrator/templates/ai-instructions/claude-skill-gherkin-syntax.md +14 -0
- package/src/orchestrator/templates/ai-instructions/claude-skill-selector-fix.md +57 -11
- package/src/orchestrator/templates/ai-instructions/claude-skill-selector-keys.md +41 -31
- package/src/orchestrator/templates/ai-instructions/claude-skill-tc-generation.md +386 -326
- package/src/orchestrator/templates/ai-instructions/claude-skill-tc-review.md +268 -90
- package/src/orchestrator/templates/ai-instructions/claude-skill-test-design-techniques.md +23 -49
- package/src/orchestrator/templates/ai-instructions/claude-skill-viewpoint-group-a-data-entry.md +203 -0
- package/src/orchestrator/templates/ai-instructions/claude-skill-viewpoint-group-b-data-ops.md +179 -0
- package/src/orchestrator/templates/ai-instructions/claude-skill-viewpoint-group-c-data-explore.md +233 -0
- package/src/orchestrator/templates/ai-instructions/claude-skill-viewpoint-group-d-display.md +226 -0
- package/src/orchestrator/templates/ai-instructions/claude-skill-viewpoint-group-e-identity.md +177 -0
- package/src/orchestrator/templates/ai-instructions/claude-skill-viewpoint.md +69 -240
- package/src/orchestrator/templates/ai-instructions/copilot-cmd-create-test.md +9 -1
- package/src/orchestrator/templates/ai-instructions/copilot-cmd-review.md +13 -12
- package/src/orchestrator/templates/ai-instructions/copilot-cmd-run-test.md +4 -2
- package/src/orchestrator/templates/ai-instructions/copilot-config.md +1 -1
- package/src/orchestrator/templates/ai-instructions/github-skill-sungen-delivery.md +1 -1
- package/src/orchestrator/templates/ai-instructions/github-skill-sungen-gherkin-syntax.md +15 -21
- package/src/orchestrator/templates/ai-instructions/github-skill-sungen-selector-fix.md +61 -15
- package/src/orchestrator/templates/ai-instructions/github-skill-sungen-selector-keys.md +41 -31
- package/src/orchestrator/templates/ai-instructions/github-skill-sungen-tc-generation.md +371 -324
- package/src/orchestrator/templates/ai-instructions/github-skill-sungen-tc-review.md +262 -102
- package/src/orchestrator/templates/ai-instructions/github-skill-sungen-test-design-techniques.md +23 -49
- package/src/orchestrator/templates/ai-instructions/github-skill-sungen-viewpoint-group-a-data-entry.md +203 -0
- package/src/orchestrator/templates/ai-instructions/github-skill-sungen-viewpoint-group-b-data-ops.md +179 -0
- package/src/orchestrator/templates/ai-instructions/github-skill-sungen-viewpoint-group-c-data-explore.md +233 -0
- package/src/orchestrator/templates/ai-instructions/github-skill-sungen-viewpoint-group-d-display.md +226 -0
- package/src/orchestrator/templates/ai-instructions/github-skill-sungen-viewpoint-group-e-identity.md +177 -0
- package/src/orchestrator/templates/ai-instructions/github-skill-sungen-viewpoint.md +69 -240
- package/src/orchestrator/templates/qa-context.md +90 -0
- package/src/orchestrator/templates/readme.md +16 -13
|
@@ -1,417 +1,470 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: sungen-tc-generation
|
|
3
|
-
description: '
|
|
3
|
+
description: 'Use when create-test needs to translate spec.md/Figma/UI into .feature + test-data.yaml. Invoke after requirements are read, before any selectors.yaml work.'
|
|
4
4
|
user-invocable: false
|
|
5
5
|
---
|
|
6
6
|
|
|
7
|
-
##
|
|
7
|
+
## ⚠️ Gotchas — read before generating
|
|
8
8
|
|
|
9
|
-
|
|
9
|
+
- `spec_figma.md` exists → read file only, **NEVER** call `mcp__figma__*`
|
|
10
|
+
→ PAT auth flow already done by `sungen-figma-source`; re-calling fails or duplicates work.
|
|
10
11
|
|
|
11
|
-
|
|
12
|
+
- `selectors.yaml` → do **NOT** generate — handled by `run-test`
|
|
13
|
+
→ Selectors need live DOM inspection via Playwright MCP, only `run-test` triggers it.
|
|
12
14
|
|
|
13
|
-
|
|
14
|
-
|
|
15
|
-
| **Tier 1** (default) | `@high` | Happy paths, required validation, core business rules, security basics | First run of `create-test` |
|
|
16
|
-
| **Tier 2** (expand) | `@normal` + `@low` | UI presence, optional validation, edge cases, cosmetic checks | User runs `create-test` again with "Add viewpoints" mode |
|
|
17
|
-
| **Tier 3** (deep) | `@high` + `@normal` + `@low` | Extra BVA combinations, cross-field validation, negative/destructive inputs, concurrent/race conditions, complex state transitions | Recommended after Tier 2 completes |
|
|
18
|
-
| **Full** (all-at-once) | All | Tier 1 + 2 + 3 combined in one run | Option at first run, **not recommended** — large output |
|
|
15
|
+
- `@steps:` scenarios → **NO** priority tag (setup blocks, not test cases)
|
|
16
|
+
→ Priority tags filter test runs; setup blocks must always run via `@extend:`, never be filtered out.
|
|
19
17
|
|
|
20
|
-
|
|
18
|
+
- Hardcode exact error messages in test-data.yaml — never leave `{{error}}` vague
|
|
19
|
+
→ Assertions are string-equal; vague placeholders produce false-positive passes.
|
|
21
20
|
|
|
22
|
-
|
|
21
|
+
- `@parallel` required when mixing `@auth:X` + `@no-auth` in the same feature
|
|
22
|
+
→ Playwright shares browser context per worker; auth state leaks across scenarios without isolation.
|
|
23
23
|
|
|
24
|
-
|
|
24
|
+
- XSS ≠ SQL injection — generate as **2 separate scenarios**, never merge
|
|
25
|
+
→ Different attack vectors (client render vs DB query) and different observable failures (alert popup vs DB error).
|
|
25
26
|
|
|
26
|
-
|
|
27
|
+
- **SQL injection has 2 layers — generate both when field reaches a backend query:**
|
|
28
|
+
- Layer 1 → field UI: special chars blocked by input field → `@high` automated scenario
|
|
29
|
+
- Layer 2 → API server: field accepts alphanumeric (e.g. LIKE/search field) → send payload like `1 OR 1=1` directly to API (curl/Postman, bypass UI) → verify server uses parameterized query, not string concat → `@high @manual`
|
|
30
|
+
→ Skipping Layer 2 = missing real attack vector even when field validation is correct.
|
|
31
|
+
→ Trigger: any free-text field whose value reaches a LIKE, partial-match, or dynamic WHERE clause.
|
|
27
32
|
|
|
28
|
-
|
|
29
|
-
|
|
30
|
-
|
|
31
|
-
|
|
33
|
+
- **Concurrent / race conditions — generate `@manual` when spec or viewpoint mentions multi-tab, multi-user, or simultaneous actions:**
|
|
34
|
+
→ Examples: two browser tabs same search with different conditions must not contaminate each other; double-click submit must not create duplicates.
|
|
35
|
+
→ `@normal @manual` by default — unless data integrity at risk → `@high @manual`.
|
|
36
|
+
→ Do NOT skip because "hard to automate" — document as `@manual` so it appears in the test plan.
|
|
32
37
|
|
|
33
|
-
**
|
|
34
|
-
|
|
35
|
-
|
|
38
|
+
- Spec mentions outcomes on a **different user-facing surface** (another URL, mobile app, user portal, widget)
|
|
39
|
+
→ "Single screen focus" means don't test other screens' UI patterns — it does NOT exempt you from documenting business outcomes on other surfaces.
|
|
40
|
+
→ **Always generate `@manual` scenarios for the output surface** — at minimum one `@high @manual` per cross-surface business rule.
|
|
41
|
+
→ If the output surface is reachable with a different auth role in the same test run, use `@auth:role` + `@extend` setup instead of `@manual`.
|
|
42
|
+
→ Skipping these entirely = zero test can catch "admin creates record correctly but user never sees it" — the most silent failure class.
|
|
43
|
+
→ Add `Display surfaces` + `Cross-surface rules` to the Coverage Map (see Step 2).
|
|
36
44
|
|
|
37
|
-
**
|
|
38
|
-
|
|
39
|
-
|
|
45
|
+
- Spec uses **inclusive boundary operators** (`<=`, `>=`, `≤`, `≥`, "at least N", "no more than N", "up to and including")
|
|
46
|
+
→ The boundary point itself **must** be a test case — not just "inside" and "outside" regions.
|
|
47
|
+
→ Generate: `boundary - 1` (must fail/not-display), `boundary` (must pass/display), `boundary + 1` (must pass or fail depending on direction).
|
|
48
|
+
→ Example: `end_at >= now` → test `end_at = now` (inclusive: display), `end_at = now - 1s` (exclusive: no display).
|
|
49
|
+
→ Off-by-one at inclusive boundaries is the most common display-logic bug.
|
|
40
50
|
|
|
41
|
-
**
|
|
42
|
-
|
|
51
|
+
- **AND condition in business rule — test each branch failing independently:**
|
|
52
|
+
When spec defines "A AND B AND C must all be true", generate:
|
|
53
|
+
(1) A fails while B/C pass, (2) B fails while A/C pass, (3) C fails while A/B pass.
|
|
54
|
+
OR condition: generate 1 scenario per branch where that branch alone triggers the outcome.
|
|
55
|
+
→ Happy-path only = missing the most common multi-condition implementation bug.
|
|
43
56
|
|
|
44
|
-
|
|
57
|
+
---
|
|
45
58
|
|
|
46
|
-
|
|
59
|
+
## Tier System
|
|
47
60
|
|
|
48
|
-
|
|
61
|
+
| Tier | Priority | What to generate | When |
|
|
62
|
+
|---|---|---|---|
|
|
63
|
+
| **Tier 1** (default) | `@high` | Happy paths, required validation, core business rules, security basics | First run of `create-test` |
|
|
64
|
+
| **Tier 2** (expand) | `@normal` + `@low` | UI presence, optional validation, edge cases, cosmetic checks | User runs `create-test` with "Add viewpoints" mode |
|
|
49
65
|
|
|
50
|
-
|
|
51
|
-
-
|
|
52
|
-
-
|
|
53
|
-
-
|
|
66
|
+
**Tier 1 stops when:**
|
|
67
|
+
- Every core user task has a happy-path scenario
|
|
68
|
+
- Every validation rule has at least one negative case (required, format, boundary)
|
|
69
|
+
- Every business rule has at least one scenario
|
|
70
|
+
- Security basics covered (auth redirect, permission blocked, injection)
|
|
54
71
|
|
|
55
|
-
|
|
72
|
+
**Tier 2** only when the user explicitly chooses "Add viewpoints" or "Add new sections".
|
|
56
73
|
|
|
57
|
-
|
|
74
|
+
**Fast-path** — display-only screens (no form, no state machine, ≤2 sections):
|
|
75
|
+
skip Coverage Map (Step 2) entirely. Go straight to Output Format with Tier 1 VP-UI presence checks
|
|
76
|
+
and any obvious VP-SEC (auth redirect). Spec drives nothing here — viewpoint covers it.
|
|
58
77
|
|
|
59
|
-
|
|
78
|
+
## Update Mode
|
|
60
79
|
|
|
61
|
-
|
|
80
|
+
When `.feature` already has scenarios, summarize and ask:
|
|
81
|
+
1. **Add new sections** — append Tier 1 scenarios, continue numbering
|
|
82
|
+
2. **Add viewpoints** — expand existing sections with Tier 2 scenarios
|
|
83
|
+
3. **Replace all** — overwrite with fresh Tier 1 generation
|
|
62
84
|
|
|
63
|
-
|
|
64
|
-
|---|---|---|
|
|
65
|
-
| Fields row, `Required=yes` | At least 1 required-error TC per required field | `@high` |
|
|
66
|
-
| Validation Rules row | At least 1 exact-message TC per validation rule | `@high` |
|
|
67
|
-
| Business Rule bullet | At least 1 behavioral TC per business rule | `@high` |
|
|
68
|
-
| Security-sensitive actions (auth, permissions) | At least 1 security check (VP-SEC) | `@high` |
|
|
69
|
-
| Entity with lifecycle states | At least 1 key state transition (VP-LOGIC) | `@high` |
|
|
70
|
-
|
|
71
|
-
Other spec sections (Section header, Default, Accessibility, Notes) are deferred to Tier 2.
|
|
72
|
-
|
|
73
|
-
### Mapping Contract (Tier 2+ — MANDATORY)
|
|
74
|
-
|
|
75
|
-
> **Full contract applies from Tier 2 onwards.** Walk `spec.md` top-to-bottom. For every section listed below, you **MUST** produce the indicated TCs. If a section is missing or empty, note it explicitly — do **NOT** silently skip.
|
|
76
|
-
|
|
77
|
-
#### Table 1 — `spec.md` → TC Mapping Contract
|
|
78
|
-
|
|
79
|
-
| Spec section | TC requirement | Tier | Tag | Viewpoint |
|
|
80
|
-
|---|---|---|---|---|
|
|
81
|
-
| Section header | 1 presence TC | T2 | `@low` | VP-UI |
|
|
82
|
-
| Fields row, `Required=yes` | 1 per-field required-error TC (individual, not bulk) | T2 | `@normal` | VP-VAL |
|
|
83
|
-
| Fields `Constraints` cell (non-empty, non-`—`) | 4 BVA for numeric range; 1 EP for format | T2 | `@normal` | VP-VAL |
|
|
84
|
-
| Fields `Constraints` cell (numeric) | 2 extra BVA (`min+1`, `max-1`) + mid-range → total 7-point with T2 | T3 | `@normal` | VP-VAL |
|
|
85
|
-
| Fields `Default` cell (non-empty, non-`—`) | 1 default-state TC | T2 | `@low` | VP-UI |
|
|
86
|
-
| Fields with dependencies (conditional required, cascading) | 1 cross-field TC per dependency | T3 | `@normal` | VP-VAL |
|
|
87
|
-
| Fields accepting free text input | 1 negative input TC (XSS/SQL injection/special chars) | T3 | `@high` | VP-SEC |
|
|
88
|
-
| Actions row (primary: submit, create, delete) | 1 edge/alternate behavior TC | T2 | `@normal` | VP-LOGIC |
|
|
89
|
-
| Actions row (secondary: cancel, reset, export) | 1 behavior TC | T2 | `@normal` | VP-LOGIC |
|
|
90
|
-
| Actions with submit/save | 1 race condition TC (double-submit, back after POST) | T3 | `@normal` | VP-LOGIC |
|
|
91
|
-
| Validation Rules row | 1 exact-message TC | T2 | `@normal` | VP-VAL |
|
|
92
|
-
| States row (Default/Loading/Error/Success) | 1 visual state TC | T2 | `@normal` | VP-UI / VP-LOGIC |
|
|
93
|
-
| States row (other states) | 1 visual TC | T2 | `@low` | VP-UI |
|
|
94
|
-
| Business Rule bullet | 1 edge-case behavioral TC; `@manual` if unautomatable | T2 | `@normal` | VP-LOGIC |
|
|
95
|
-
| Accessibility tab order | 1 tab-walking TC | T2 | `@low` | VP-UI |
|
|
96
|
-
| Notes bullet | classify per content (page title→VP-UI, env→`@manual`, edge→relevant section) | T2 | `@low` | varies |
|
|
97
|
-
|
|
98
|
-
#### Table 2 — `test-viewpoint.md` → TC Mapping Contract
|
|
99
|
-
|
|
100
|
-
| Section / format | TC requirement |
|
|
101
|
-
|---|---|
|
|
102
|
-
| `## Edge Cases` (bulleted) | classify per spec Notes |
|
|
103
|
-
| `## Known Issues` (bulleted) | `@manual` TCs with bug ID comment |
|
|
104
|
-
| `## Design Decisions` | behavioral TCs |
|
|
105
|
-
| `## UI Patterns Identified` | confirm `sungen-viewpoint` pattern checklists |
|
|
106
|
-
| `## Priority Viewpoints` | adjust Tier-2 emphasis |
|
|
107
|
-
| `## Element Overview` (table) | 1 presence/state TC per row (T2, `@low`) |
|
|
108
|
-
| `## State Transitions` (table) | 1 TC per valid transition + key blocked |
|
|
109
|
-
| Free-form notes | re-read with heading as context |
|
|
110
|
-
|
|
111
|
-
### Figma supplement (`spec_figma.md`)
|
|
112
|
-
|
|
113
|
-
When `requirements/spec_figma.md` is present alongside `spec.md`, treat it as a **secondary input** with these rules:
|
|
114
|
-
|
|
115
|
-
- **Never override `spec.md`**: `spec.md` is authoritative for all business rules, field constraints, and behavior. `spec_figma.md` only supplements with visual/text data that `spec.md` may lack.
|
|
116
|
-
- **`## Text Inventory` → literal strings**: use text label values from this section verbatim in `test-data.yaml` (button labels, input placeholders, error messages shown in Figma). Do not paraphrase or invent alternatives.
|
|
117
|
-
- **`## Interaction States` → state coverage checkpoints**: use the listed variants (e.g., empty, loading, error, success) as a checklist for state-coverage scenarios. Only generate scenarios for states that are either (a) confirmed in both `spec.md` and `spec_figma.md`, or (b) explicitly documented in one source without contradiction from the other.
|
|
118
|
-
- **Flag disagreements**: if a field name, label, or behavior in `spec_figma.md` contradicts `spec.md`, insert an HTML comment at the top of the `.feature` file:
|
|
119
|
-
```gherkin
|
|
120
|
-
<!-- FIGMA-SPEC CONFLICT: <brief description> — using spec.md value -->
|
|
121
|
-
```
|
|
122
|
-
Then proceed using the `spec.md` value.
|
|
123
|
-
|
|
124
|
-
## Screen Input Sources
|
|
125
|
-
|
|
126
|
-
**Auto-detect** — the parent command (`create-test`) resolves visual sources before invoking this skill. By the time generation starts, the available sources are already determined:
|
|
127
|
-
- `spec.md` — primary, always read if present
|
|
128
|
-
- `spec_figma.md` — Figma supplement, read if present (PAT flow already completed)
|
|
129
|
-
- `ui/*.png` — visual context, read if present
|
|
130
|
-
- `test-viewpoint.md` — edge cases and known issues, read if present
|
|
85
|
+
For append: read highest `VP-<CAT>-<NNN>`, continue from next number. Never modify existing scenarios.
|
|
131
86
|
|
|
132
|
-
**
|
|
87
|
+
> **IDs are immutable & append-only.** The dashboard tracks each test case by its ID (namespaced at export as `<SCREEN>-<CAT>-<NNN>`, e.g. `LOGIN-SEC-001`). When updating a `.feature`:
|
|
88
|
+
> - **Never renumber** existing scenarios — even when earlier ones were deleted. Renumbering changes the dashboard key and loses that test's pass/fail history.
|
|
89
|
+
> - **New scenarios** take the next unused number per `VP-<CAT>` group (continue from the highest; do NOT fill gaps left by deletions).
|
|
90
|
+
> - **Deleting** a scenario retires its number permanently — never reuse it.
|
|
91
|
+
> - Editing a scenario's title text or steps is fine — the ID stays the same.
|
|
133
92
|
|
|
134
|
-
|
|
93
|
+
---
|
|
135
94
|
|
|
136
|
-
|
|
95
|
+
## Input Sources
|
|
137
96
|
|
|
138
|
-
|
|
139
|
-
-
|
|
140
|
-
-
|
|
97
|
+
Auto-detected by `create-test` before invoking this skill:
|
|
98
|
+
- `spec.md` — primary, always read if present
|
|
99
|
+
- `spec_figma.md` — Figma supplement (PAT flow already done — just read the file)
|
|
100
|
+
- `ui/*.png` — visual context
|
|
101
|
+
- `test-viewpoint.md` — test conditions checklist; scan entirely regardless of format.
|
|
102
|
+
Format varies across projects: hierarchical table (Level 1–4 columns), flat checklist,
|
|
103
|
+
prose, or mixed. Regardless of format:
|
|
104
|
+
1. Scan entire file top to bottom.
|
|
105
|
+
2. Each row / bullet / item = 1 viewpoint → add to `Viewpoint items` in Coverage Map.
|
|
106
|
+
3. Do NOT pre-classify into buckets before scanning — classify only when
|
|
107
|
+
writing the scenario.
|
|
141
108
|
|
|
142
|
-
|
|
109
|
+
**Single screen focus**: one URL = one screen. Modals on same page = part of this screen.
|
|
110
|
+
This means: do not test other screens' UI layout or navigation. It does NOT mean skip documenting business outcomes that your screen's actions cause on other surfaces. Those cross-surface outcomes must appear in the Coverage Map and be covered by at least `@manual` scenarios.
|
|
143
111
|
|
|
144
|
-
|
|
112
|
+
**Capture real data** from live page or Figma: option labels, error messages, counter keywords. Hardcode in `test-data.yaml` — stale data that fails fast beats `@manual` that never runs.
|
|
145
113
|
|
|
146
|
-
|
|
114
|
+
---
|
|
147
115
|
|
|
148
|
-
|
|
116
|
+
## Generation Workflow
|
|
149
117
|
|
|
150
|
-
|
|
151
|
-
- **Validation rules**: field constraints, error messages, required/optional
|
|
152
|
-
- **Business rules**: eligibility, calculation logic, permission-based behavior
|
|
153
|
-
- **State lifecycle**: allowed transitions, blocked transitions
|
|
154
|
-
- **Edge cases**: boundary values, empty states, concurrent conditions
|
|
118
|
+
### Step 1 — Orient: patterns + risk
|
|
155
119
|
|
|
156
|
-
|
|
120
|
+
Identify core user tasks. Categorize page sections by UI pattern group using `sungen-viewpoint`:
|
|
121
|
+
data entry (A), data manipulation (B), data exploration (C), display/feedback (D), identity (E).
|
|
157
122
|
|
|
158
|
-
|
|
123
|
+
Proceed directly to Step 2. Pause to ask the user only when section priority is genuinely ambiguous from spec (e.g. multiple top-level forms without business context).
|
|
159
124
|
|
|
160
|
-
|
|
125
|
+
### Step 2 — Build Coverage Map
|
|
161
126
|
|
|
162
|
-
|
|
163
|
-
|---|---|
|
|
164
|
-
| EP | Valid/invalid ranges, categories → **one** scenario per class, not per value |
|
|
165
|
-
| BVA | Numeric range, string length → `min-1`, `min`, `max`, `max+1` (compact 4-point default) |
|
|
166
|
-
| Decision Table | 2+ dependent conditions → one scenario per combination (cap at distinct outcomes if >3 conditions) |
|
|
167
|
-
| State Transition | Entity lifecycle → one scenario per valid transition + key invalid transitions |
|
|
127
|
+
Read `spec.md` fully, then extract into a Coverage Map **before writing any scenario**. Nothing is skipped at this stage — risk only affects depth in Step 3.
|
|
168
128
|
|
|
169
|
-
|
|
129
|
+
**Risk tags:** HIGH = complex business rules, cascading fields, multi-step state changes, auth/integration. LOW = display-only, static labels, read-only fields.
|
|
170
130
|
|
|
171
|
-
|
|
131
|
+
```
|
|
132
|
+
User journeys: [J1 – ...], [J2 – ...]
|
|
133
|
+
Validation rules: [V1 – field → "exact error text"], [V2 – ...]
|
|
134
|
+
Business rules: [B1 HIGH – ...], [B2 LOW – ...]
|
|
135
|
+
Secondary behaviors: [SB1 – tiebreaker: when X identical → sort by Y DESC]
|
|
136
|
+
[SB2 – default state when no condition applies: ...]
|
|
137
|
+
→ capture "otherwise", "when tied", "secondary sort", "fallback" rules here
|
|
138
|
+
States: [StateA → StateB], [blocked: StateC → StateA]
|
|
139
|
+
Security: [S1 HIGH – OAuth/SSO flow], [S2 HIGH – free-text inputs → XSS + SQL injection]
|
|
140
|
+
Concurrency risks: [C1 – spec/viewpoint mentions multi-tab, concurrent edit, or simultaneous action]
|
|
141
|
+
→ leave empty if neither spec nor viewpoint mentions concurrency
|
|
142
|
+
Display surfaces: [Surface1 – /url, auth:role], [Surface2 – /url, auth:user] ← list every surface spec mentions as output
|
|
143
|
+
Cross-surface rules: [B1 – action on /admin → outcome on /user (3-condition AND logic)] ← mandatory if Display surfaces exist
|
|
144
|
+
Inclusive bounds: [start_at <= now → boundary: start_at=now must display], [end_at >= now → boundary: end_at=now must display]
|
|
145
|
+
Viewpoint items: [TV-01 – <exact wording from test-viewpoint row 1>]
|
|
146
|
+
[TV-02 – <exact wording from test-viewpoint row 2>]
|
|
147
|
+
→ one entry per row/bullet in test-viewpoint.md, regardless of format
|
|
148
|
+
→ leave empty if no test-viewpoint.md present
|
|
149
|
+
Spec gaps: [G1 – behavior undocumented → verify with dev]
|
|
150
|
+
```
|
|
172
151
|
|
|
173
|
-
**
|
|
174
|
-
|
|
175
|
-
|
|
176
|
-
|
|
177
|
-
|
|
178
|
-
|
|
179
|
-
|
|
180
|
-
|
|
181
|
-
|
|
182
|
-
|
|
152
|
+
**Example — login screen (2 sections, 5 validation rules):**
|
|
153
|
+
```
|
|
154
|
+
User journeys: [J1 – account login with valid credentials]
|
|
155
|
+
Validation rules: [V1 – empty email → "Invalid email address"],
|
|
156
|
+
[V2 – short password → "Password must be at least 6 characters"],
|
|
157
|
+
[V3 – invalid format → "Invalid email address"],
|
|
158
|
+
[V4 – wrong credentials → "Invalid email or password"]
|
|
159
|
+
Business rules: [B1 HIGH – toggle is same-page state, not route change]
|
|
160
|
+
States: [Default → Account form], [Account form → Google login],
|
|
161
|
+
[Account form → Loading → Success/Error]
|
|
162
|
+
Security: [S1 HIGH – free-text email/password → XSS + SQL injection]
|
|
163
|
+
Display surfaces: [Login page – /login, no-auth] ← same surface, no cross-surface rules needed
|
|
164
|
+
Cross-surface rules: (none)
|
|
165
|
+
Inclusive bounds: (none)
|
|
166
|
+
Spec gaps: [G1 – auth error on 401 unconfirmed on live]
|
|
167
|
+
|
|
168
|
+
Tier 1 output: VP-LOGIC-001 (toggle reveals form), VP-LOGIC-002 (restore Google state),
|
|
169
|
+
VP-LOGIC-003 (login success → redirect), VP-VAL-001 (empty fields),
|
|
170
|
+
VP-VAL-002 (invalid format), VP-VAL-003 (short password),
|
|
171
|
+
VP-VAL-004 (wrong credentials), VP-SEC-001 (XSS), VP-SEC-002 (SQL injection)
|
|
172
|
+
```
|
|
183
173
|
|
|
184
|
-
**
|
|
174
|
+
**Example — admin notice management screen (cross-surface + inclusive bounds):**
|
|
175
|
+
```
|
|
176
|
+
User journeys: [J1 – create notice], [J2 – edit notice], [J3 – toggle active], [J4 – delete notice]
|
|
177
|
+
Validation rules: [V1 – title empty → "必須項目です"], [V2 – end_at <= start_at → "終了日時は開始日時より後に…"],
|
|
178
|
+
[V3 – title > 255 chars → "255文字以内で入力してください"]
|
|
179
|
+
Business rules: [B1 CRITICAL – notice displays on user-facing portal only when: is_active=ON AND start_at<=now AND end_at>=now],
|
|
180
|
+
[B2 HIGH – is_active=ON by default on create; not on form],
|
|
181
|
+
[B3 HIGH – toggle not blocked by date range]
|
|
182
|
+
States: [Scheduled → Live → Expired], [any → OFF via toggle]
|
|
183
|
+
Security: [S1 HIGH – free-text title/content → XSS + SQL injection], [S2 HIGH – admin-only, non-admin blocked]
|
|
184
|
+
Display surfaces: [Admin panel – /admin/notices, auth:admin], [User portal – /user-facing-url, auth:user] ← TWO surfaces
|
|
185
|
+
Cross-surface rules: [B1 – admin creates/toggles → user portal shows/hides banner; 8-combination decision table]
|
|
186
|
+
Inclusive bounds: [start_at <= now → boundary: start_at=now must display], [end_at >= now → boundary: end_at=now must display]
|
|
187
|
+
Spec gaps: [G1 – exact content max length unconfirmed]
|
|
188
|
+
|
|
189
|
+
Tier 1 output (admin surface): VP-LOGIC-001 to VP-LOGIC-013, VP-VAL-001 to VP-VAL-007, VP-SEC-001 to VP-SEC-003
|
|
190
|
+
Cross-surface output (user portal): VP-LOGIC-CS-001 @manual (M-01: all 3 ON → display),
|
|
191
|
+
VP-LOGIC-CS-002 @manual (M-05: is_active=OFF → no display),
|
|
192
|
+
VP-LOGIC-CS-003 @manual (M-03: start_at future → no display),
|
|
193
|
+
VP-LOGIC-CS-004 @manual (M-02: end_at past → no display),
|
|
194
|
+
VP-LOGIC-CS-005 @manual (boundary: start_at=now → display),
|
|
195
|
+
VP-LOGIC-CS-006 @manual (boundary: end_at=now → display)
|
|
196
|
+
```
|
|
185
197
|
|
|
186
|
-
|
|
198
|
+
**❌ Bad Coverage Map — silently misses validation rules AND cross-surface outcomes:**
|
|
199
|
+
```
|
|
200
|
+
User journeys: [J1 – create notice], [J2 – edit notice]
|
|
201
|
+
Business rules: [B1 – display logic: is_active + date range]
|
|
202
|
+
Security: [S1 – admin only]
|
|
203
|
+
```
|
|
204
|
+
→ Output misses VP-VAL-* entirely (validation rules not listed with exact messages).
|
|
205
|
+
→ Output misses user portal scenarios entirely (no `Display surfaces` or `Cross-surface rules` field).
|
|
206
|
+
→ Output misses boundary tests for `start_at <= now`, `end_at >= now` (no `Inclusive bounds` field).
|
|
207
|
+
→ Root cause: Coverage Map only captured admin CRUD journeys; spec section on "display flow" and "user-facing surface" was not scanned into the map.
|
|
187
208
|
|
|
188
|
-
|
|
209
|
+
**✅ Good** — see admin notice example above: `Display surfaces` lists every URL spec mentions as output, `Cross-surface rules` maps each admin action to its user-facing outcome, `Inclusive bounds` flags every `<=`/`>=` for BVA. Every item maps to a VP-ID in `Tier 1 output`.
|
|
189
210
|
|
|
190
|
-
|
|
211
|
+
#### Tier 1 guard — minimum before writing scenarios
|
|
191
212
|
|
|
192
|
-
|
|
|
213
|
+
| Spec section | Minimum requirement | Tag |
|
|
193
214
|
|---|---|---|
|
|
194
|
-
|
|
|
195
|
-
|
|
|
196
|
-
|
|
|
197
|
-
|
|
|
198
|
-
|
|
|
199
|
-
|
|
|
200
|
-
|
|
|
201
|
-
|
|
|
202
|
-
|
|
|
203
|
-
| `@
|
|
204
|
-
|
|
|
205
|
-
|
|
206
|
-
|
|
215
|
+
| Required field | 1 required-error TC per field | `@high` |
|
|
216
|
+
| Validation rule | 1 exact-message TC per rule | `@high` |
|
|
217
|
+
| Business rule | 1 behavioral TC per rule | `@high` |
|
|
218
|
+
| **Secondary behavior / tiebreaker** | **1 TC per tiebreaker or fallback rule in `Secondary behaviors`** | **`@high`** |
|
|
219
|
+
| Auth / OAuth / permissions | 1 VP-SEC TC | `@high` |
|
|
220
|
+
| Free-text input | 1 XSS TC **and** 1 SQL injection TC (separate) | `@high` |
|
|
221
|
+
| **Free-text LIKE / partial-match field** | **1 field-level SQL TC + 1 API-level SQL `@manual` TC** | **`@high`** |
|
|
222
|
+
| Lifecycle states | 1 key state transition TC | `@high` |
|
|
223
|
+
| Public page | 1 accessible-without-auth TC | `@high` |
|
|
224
|
+
| **Cross-surface business rule** | **1 `@high @manual` TC per affected surface per rule** — even if not automated today | **`@high`** |
|
|
225
|
+
| **Inclusive boundary condition** (`<=`, `>=`) | **4 TCs per range: `min-1` (fail), `min` (pass), `max` (pass), `max+1` (fail)** — never skip the exact boundary point | **`@high`** |
|
|
226
|
+
| **Concurrency risk (from `Concurrency risks`)** | **1 `@manual` TC per risk — `@normal` unless data integrity at risk → `@high`** | **`@normal`** |
|
|
227
|
+
| States row (Default / Loading / Error / Success) | 1 visual state TC per named state — separate from lifecycle transition TC | `@normal` |
|
|
228
|
+
| Actions secondary (cancel, reset, export) | 1 behavior TC per secondary action | `@normal` |
|
|
207
229
|
|
|
208
|
-
|
|
230
|
+
> **Completeness check** — after generating all scenarios, scan Coverage Map line by line.
|
|
231
|
+
> Every spec item must map to at least one scenario. Every `Viewpoint items` entry must be mapped to a VP-ID or marked `[covered: VP-ID]`. Do NOT silently skip either source.
|
|
232
|
+
> Specifically verify: `Display surfaces` each have ≥1 scenario; `Inclusive bounds` each have a boundary-point TC.
|
|
209
233
|
|
|
210
|
-
|
|
234
|
+
#### Tier 2 strategy
|
|
211
235
|
|
|
212
|
-
|
|
213
|
-
|
|
214
|
-
| 0 When steps + Then asserts only `User see [T] type` (presence) | `@low` (always) |
|
|
215
|
-
| 0 When steps + Then asserts `User see [T] type with {{value}}` (content) | `@low` (label/text) |
|
|
216
|
-
| When steps but no state change (e.g., navigation only) | `@low` |
|
|
217
|
-
| When provides invalid input + Then asserts validation error | `@normal` |
|
|
218
|
-
| When triggers business logic + Then asserts state change | `@high` or `@normal` per heuristics |
|
|
236
|
+
Walk `spec.md` top-to-bottom and apply `sungen-viewpoint` Tier 2 checklist for each identified pattern. Depth by risk: high-risk section → all items; low-risk → presence checks only. Skip items already covered by Tier 1.
|
|
237
|
+
For HIGH-tagged sections: use the same depth as Step 3 (full BVA + Decision Table) when generating new Tier 2 VP-VAL items not already covered by Step 3. For LOW-tagged sections: EP valid + EP invalid only — skip BVA and Decision Table. Do NOT re-generate scenarios already created in Step 3.
|
|
219
238
|
|
|
220
|
-
**
|
|
239
|
+
**When `test-viewpoint.md` is present — mandatory line-by-line scan:**
|
|
240
|
+
Every item in `test-viewpoint.md` must appear in `Viewpoint items` of the Coverage Map (done in Step 2). In Tier 2, walk `Viewpoint items` line by line:
|
|
241
|
+
- Item already covered by Tier 1 → mark `[covered: VP-ID]`, skip.
|
|
242
|
+
- Item automatable → generate `@normal` scenario.
|
|
243
|
+
- Item requires DB setup / network manipulation → `@normal @manual`.
|
|
244
|
+
- Item marked as known bug → `@manual` with comment `# Known issue: <ID>`.
|
|
221
245
|
|
|
222
|
-
|
|
246
|
+
> **Viewpoint completeness check** — after Tier 2, every `Viewpoint items`
|
|
247
|
+
> entry must be mapped to a VP-ID or marked `[covered: VP-ID]`.
|
|
248
|
+
> If any item has neither → generate missing scenario before finishing.
|
|
249
|
+
> Do NOT silently skip viewpoint items because they seem redundant or
|
|
250
|
+
> hard to automate.
|
|
223
251
|
|
|
224
|
-
|
|
225
|
-
Given User is on [Screen] page
|
|
226
|
-
And User wait for [Page Title] heading is visible
|
|
227
|
-
```
|
|
252
|
+
### Step 3 — Apply test design techniques
|
|
228
253
|
|
|
229
|
-
|
|
254
|
+
Delegate to `sungen-test-design-techniques` for technique selection and depth.
|
|
255
|
+
Risk level from Coverage Map drives depth: HIGH section → full BVA / Decision Table; LOW section → EP valid+invalid only.
|
|
230
256
|
|
|
231
|
-
|
|
257
|
+
**BVA constraint:** Use compact 4-point mode only (min-1, min, max, max+1). Do not apply Full 6-point mode.
|
|
232
258
|
|
|
233
|
-
|
|
259
|
+
### Step 4 — Supplement with viewpoint checklists
|
|
234
260
|
|
|
235
|
-
|
|
236
|
-
|---|---|---|
|
|
237
|
-
| Modal / Dialog / Drawer | `@cleanup:overlay` | Dismiss leftover overlays between tests |
|
|
238
|
-
| Form & Inputs / Search / Filter | `@cleanup:forms` | Clear form fields, reset selects |
|
|
239
|
-
| Long scrollable content | `@cleanup:scroll` | Scroll to top for consistent assertions |
|
|
240
|
-
| Auth tokens / session data in tests | `@cleanup:storage` | Clear sessionStorage |
|
|
241
|
-
| CI/CD or debug-heavy screens | `@screenshot:on-failure` | Auto-capture screenshot on test failure |
|
|
261
|
+
Use `sungen-viewpoint` for defect-prone patterns spec never documents. Skip items already covered by spec-driven scenarios.
|
|
242
262
|
|
|
243
|
-
**
|
|
263
|
+
> **XSS and SQL injection** are generated at Tier 1 guard (Step 2). Skip them when applying `sungen-viewpoint` Shared Checks — do not generate again.
|
|
244
264
|
|
|
245
|
-
|
|
265
|
+
#### 4a. Shared Checks — generate ONCE per screen
|
|
246
266
|
|
|
247
|
-
|
|
267
|
+
Apply the **Shared Checks table in `sungen-viewpoint`** (single source). Generate each at most once per screen, not per pattern. Since XSS and SQL injection are already covered at the Tier 1 guard (Step 2, above), at this step generate only the remaining ones that fit the screen: **Loading State** (any async fetch), **Empty State** (any list/table/card that can return 0 records), **URL Manipulation** (URL params affecting displayed data).
|
|
248
268
|
|
|
249
|
-
|
|
250
|
-
- Reset a dropdown filter to default value (not just clear)
|
|
251
|
-
- Navigate away from a sub-tab back to the main tab
|
|
252
|
-
- Close a specific sidebar panel
|
|
269
|
+
#### 4b. Pattern Tier 1 / Tier 2
|
|
253
270
|
|
|
254
|
-
|
|
255
|
-
|
|
256
|
-
|
|
257
|
-
When User select [Status Filter] dropdown with {{default_status}}
|
|
258
|
-
```
|
|
271
|
+
**Gap check rule — Step 4b fills gaps from Step 3, not a new generation source:**
|
|
272
|
+
- **VP-VAL BVA/EP items**: skip if Step 3 already generated scenarios for that field. Generate only if Step 3 missed the field entirely.
|
|
273
|
+
- **VP-LOGIC, VP-UI, VP-SEC items**: generate normally — Step 3 does not cover these.
|
|
259
274
|
|
|
260
|
-
|
|
275
|
+
- **Tier 1**: all items from `#### Tier 1 — @high` → VP-LOGIC, VP-VAL, VP-SEC
|
|
276
|
+
- **Tier 2**: items from `#### Tier 2 — @normal + @low` → VP-UI
|
|
261
277
|
|
|
262
|
-
|
|
278
|
+
Risk weighting: high-risk section → all Tier 1 items. Low-risk (e.g. read-only table) → only clearly applicable items.
|
|
263
279
|
|
|
264
|
-
|
|
280
|
+
#### 4c. Cross-pattern interactions — where spec-invisible bugs live
|
|
265
281
|
|
|
266
|
-
|
|
282
|
+
For every pair of patterns present on the screen, walk the **⚡ Cross-pattern interactions** list in each `sungen-viewpoint` group file (single source). Never-miss pairs:
|
|
283
|
+
- **Create / Update / Delete × Data Table** — new record missing or count not updated; row shows stale data; last-record delete → blank page.
|
|
284
|
+
- **Search / Filter × Pagination** — results don't reset to page 1.
|
|
285
|
+
- **Form × Modal/Dialog** — modal stays open on success / data wiped on failure.
|
|
286
|
+
- **Import × Data Table** — table not refreshed / count wrong after import.
|
|
287
|
+
- **Sort × Data Table** — all cells in a row must reorder together (data must not shift to wrong columns).
|
|
288
|
+
- **Filter (≥2 active)** — generate 1 scenario with all filters active simultaneously to verify AND logic.
|
|
267
289
|
|
|
268
|
-
|
|
269
|
-
2. **Validation-heavy features** (recommended) — each scenario fills forms with different invalid data and needs a clean form. Serial shared page keeps previous test's input.
|
|
290
|
+
Cross-pattern: `@high` by default.
|
|
270
291
|
|
|
271
|
-
|
|
292
|
+
**Assert values, not existence.** Never bare "is visible". Capture exact error messages into test-data.yaml.
|
|
272
293
|
|
|
273
|
-
|
|
274
|
-
@parallel @auth:user
|
|
275
|
-
@cleanup:forms
|
|
276
|
-
Feature: kudos Screen
|
|
294
|
+
---
|
|
277
295
|
|
|
278
|
-
|
|
279
|
-
Scenario: Send kudos
|
|
280
|
-
...
|
|
296
|
+
## Priority Tags
|
|
281
297
|
|
|
282
|
-
|
|
283
|
-
Scenario: Unauthenticated user is redirected
|
|
284
|
-
...
|
|
285
|
-
```
|
|
298
|
+
Every scenario **MUST** have exactly one priority tag. Assign by user impact per the **"Assign priority by user impact"** table in `sungen-gherkin-syntax`; override only when context differs.
|
|
286
299
|
|
|
287
|
-
|
|
300
|
+
Quick fallback: auth / CRUD / security / required-field / unique-constraint → `@high`; format validation / standard-range boundary / search / filter / sort → `@normal`; presence / label / placeholder / cosmetic → `@low`. (Inclusive-operator or critical boundary → `@high`.)
|
|
288
301
|
|
|
289
|
-
|
|
302
|
+
---
|
|
290
303
|
|
|
291
|
-
|
|
304
|
+
## SPA Wait-For Steps
|
|
292
305
|
|
|
293
306
|
```gherkin
|
|
294
|
-
|
|
295
|
-
|
|
296
|
-
|
|
297
|
-
Feature: <Screen> Screen
|
|
307
|
+
Given User is on [Screen] page
|
|
308
|
+
And User wait for [Page Title] heading is visible
|
|
309
|
+
```
|
|
298
310
|
|
|
299
|
-
|
|
300
|
-
Given User is on [Screen] page
|
|
311
|
+
---
|
|
301
312
|
|
|
302
|
-
|
|
303
|
-
@steps:open_form
|
|
304
|
-
Scenario: Open form
|
|
305
|
-
When User click [Create] button
|
|
306
|
-
Then User see [Form] dialog
|
|
313
|
+
## Cleanup & Hooks
|
|
307
314
|
|
|
308
|
-
|
|
315
|
+
Add cleanup tags per the `sungen-gherkin-syntax` Cleanup table. Key rules:
|
|
316
|
+
- **Always `@cleanup:overlay`** if ANY section opens a dialog; **always `@cleanup:forms`** if the screen has inline search, filter, or editable forms.
|
|
317
|
+
- **`@parallel`** is required when mixing auth groups (`@auth:X` + `@no-auth`); recommended for validation-heavy features needing a clean form state per scenario.
|
|
318
|
+
- **`@afterEach`** hook only when `@cleanup:*` tags aren't enough (feature-specific reset logic).
|
|
309
319
|
|
|
310
|
-
|
|
311
|
-
Scenario: VP-LOGIC-001 Submit form with valid data creates record
|
|
312
|
-
Given User is on [Form] dialog
|
|
313
|
-
When User fill [Name] field with {{valid_name}}
|
|
314
|
-
And User click [Submit] button
|
|
315
|
-
Then User see {{success_message}} message
|
|
320
|
+
---
|
|
316
321
|
|
|
317
|
-
|
|
318
|
-
Scenario: VP-VAL-001 Submit with all empty fields shows errors
|
|
319
|
-
Given User is on [Form] dialog
|
|
320
|
-
When User click [Submit] button
|
|
321
|
-
Then User see [Name error] message with {{name_required_error}}
|
|
322
|
+
## Output Format
|
|
322
323
|
|
|
323
|
-
|
|
324
|
+
**Files:** `qa/screens/<screen>/features/<screen>.feature` + `qa/screens/<screen>/test-data/<screen>.yaml`
|
|
324
325
|
|
|
325
|
-
|
|
326
|
-
|
|
327
|
-
Then User see [Users] table match data:
|
|
328
|
-
| Name | Email |
|
|
329
|
-
| {{name_1}} | {{email_1}} |
|
|
326
|
+
Use step patterns and element types from `sungen-gherkin-syntax`.
|
|
327
|
+
**Naming**: `VP-<CATEGORY>-<NNN>`. Scenario name must use the **same element type** as the steps.
|
|
330
328
|
|
|
331
|
-
|
|
332
|
-
Scenario: VP-SEC-010 Unauthorized user cannot access page
|
|
333
|
-
Given User is not logged in
|
|
334
|
-
When User navigate to [Screen] page
|
|
335
|
-
Then User is on [Login] page
|
|
336
|
-
```
|
|
329
|
+
**Test data** — grouped by section, loaded at runtime:
|
|
337
330
|
|
|
338
|
-
|
|
331
|
+
```yaml
|
|
332
|
+
# login.yaml (base)
|
|
333
|
+
valid_email: admin@dev.example.com
|
|
334
|
+
valid_password: DevPass123
|
|
339
335
|
|
|
340
|
-
|
|
336
|
+
# login.staging.yaml (override)
|
|
337
|
+
valid_email: admin@staging.example.com
|
|
338
|
+
valid_password: StagingPass456
|
|
339
|
+
```
|
|
341
340
|
|
|
342
|
-
|
|
341
|
+
**DataTable vs Row Scope:**
|
|
343
342
|
|
|
344
343
|
| Pattern | Use when |
|
|
345
344
|
|---|---|
|
|
346
345
|
| `table match data:` + DataTable | Verifying **multiple rows** exist with expected values |
|
|
347
|
-
| `row in [Table] table with {{v}}` + `column with {{v}}` |
|
|
348
|
-
|
|
349
|
-
**Naming**: `VP-<CATEGORY>-<NNN>` prefix. Scenario name must use the **same element type** as the steps — e.g., if the step uses `dialog`, write "dialog opens" not "modal opens".
|
|
346
|
+
| `row in [Table] table with {{v}}` + `column with {{v}}` | Single row detail or action on a row |
|
|
350
347
|
|
|
351
|
-
|
|
352
|
-
|
|
353
|
-
**Environment-specific data**: create `<screen>.<env>.yaml` alongside the base file with only the keys that change. Users run `SUNGEN_ENV=staging npx playwright test` to merge overrides.
|
|
348
|
+
---
|
|
354
349
|
|
|
355
|
-
|
|
350
|
+
## Worked Example — spec → Coverage Map → output
|
|
356
351
|
|
|
357
|
-
|
|
352
|
+
A minimal end-to-end pass. Use this to calibrate output: every Coverage Map line below maps to exactly one scenario, and every `{{var}}` exists in the YAML.
|
|
358
353
|
|
|
359
|
-
|
|
354
|
+
**① Input — `spec.md` (excerpt):**
|
|
355
|
+
```
|
|
356
|
+
## Coupon Form
|
|
357
|
+
- Code: required, free text, max 20 chars, must be unique.
|
|
358
|
+
- Discount %: required, integer 1–100 inclusive.
|
|
359
|
+
- Submit creates a coupon; it appears at the top of the Coupon List.
|
|
360
|
+
- Only authenticated admins can create coupons.
|
|
361
|
+
```
|
|
360
362
|
|
|
361
|
-
|
|
363
|
+
**② Coverage Map (Step 2 output):**
|
|
364
|
+
```
|
|
365
|
+
User journeys: [J1 – admin creates coupon → appears top of list]
|
|
366
|
+
Validation rules: [V1 – code empty → "Code is required"],
|
|
367
|
+
[V2 – code > 20 chars → "Code must be 20 characters or less"],
|
|
368
|
+
[V3 – duplicate code → "Code already exists"],
|
|
369
|
+
[V4 – discount empty → "Discount is required"]
|
|
370
|
+
Business rules: [B1 HIGH – new coupon sorts to top of list]
|
|
371
|
+
States: (none — no lifecycle)
|
|
372
|
+
Security: [S1 HIGH – free-text code → XSS + SQL injection], [S2 HIGH – admin-only]
|
|
373
|
+
Display surfaces: [Coupon admin – /admin/coupons, auth:admin] ← single surface
|
|
374
|
+
Cross-surface rules: (none)
|
|
375
|
+
Inclusive bounds: [discount 1–100 inclusive → BVA: 0, 1, 100, 101]
|
|
376
|
+
Spec gaps: (none)
|
|
377
|
+
|
|
378
|
+
Tier 1 output: VP-LOGIC-001 (create → top of list), VP-VAL-001 (code empty),
|
|
379
|
+
VP-VAL-002 (code > 20), VP-VAL-003 (duplicate code), VP-VAL-004 (discount empty),
|
|
380
|
+
VP-VAL-005 (discount=0 reject), VP-VAL-006 (discount=1 accept),
|
|
381
|
+
VP-VAL-007 (discount=100 accept), VP-VAL-008 (discount=101 reject),
|
|
382
|
+
VP-SEC-001 (XSS), VP-SEC-002 (SQL injection), VP-SEC-003 (non-admin blocked)
|
|
383
|
+
```
|
|
384
|
+
> Note how `Inclusive bounds` line forces 4 BVA scenarios (0/1/100/101), not just "valid + invalid".
|
|
362
385
|
|
|
363
|
-
|
|
364
|
-
|
|
365
|
-
|
|
366
|
-
|
|
367
|
-
|
|
386
|
+
**③ Output — `coupons.feature` (excerpt, 4 of 12 scenarios):**
|
|
387
|
+
```gherkin
|
|
388
|
+
@parallel @auth:admin
|
|
389
|
+
@cleanup:overlay
|
|
390
|
+
@cleanup:forms
|
|
391
|
+
Feature: Coupons Screen
|
|
368
392
|
|
|
369
|
-
|
|
393
|
+
Background:
|
|
394
|
+
Given User is on [Coupons] page
|
|
395
|
+
|
|
396
|
+
@steps:open_create_form
|
|
397
|
+
Scenario: Open create form
|
|
398
|
+
When User click [Create Coupon] button
|
|
399
|
+
Then User see [Create Coupon] dialog
|
|
400
|
+
|
|
401
|
+
# --- Section: Create Coupon (Tier 1) ---
|
|
402
|
+
|
|
403
|
+
@high @extend:open_create_form
|
|
404
|
+
Scenario: VP-LOGIC-001 Submit valid coupon adds it to top of list
|
|
405
|
+
Given User is on [Create Coupon] dialog
|
|
406
|
+
When User fill [Code] field with {{valid_coupon.code}}
|
|
407
|
+
And User fill [Discount] field with {{valid_coupon.discount}}
|
|
408
|
+
And User click [Save] button
|
|
409
|
+
Then User see [Create Coupon] dialog is hidden
|
|
410
|
+
And User see [Coupon List] row in [Coupon List] table with {{valid_coupon.code}}
|
|
411
|
+
|
|
412
|
+
@high @extend:open_create_form
|
|
413
|
+
Scenario: VP-VAL-005 Discount of 0 is rejected (boundary min-1)
|
|
414
|
+
Given User is on [Create Coupon] dialog
|
|
415
|
+
When User fill [Code] field with {{valid_coupon.code}}
|
|
416
|
+
And User fill [Discount] field with {{discount_zero}}
|
|
417
|
+
And User click [Save] button
|
|
418
|
+
Then User see [Discount Error] message with {{error.discount_range}}
|
|
419
|
+
|
|
420
|
+
@high @extend:open_create_form
|
|
421
|
+
Scenario: VP-VAL-006 Discount of 1 is accepted (boundary min)
|
|
422
|
+
Given User is on [Create Coupon] dialog
|
|
423
|
+
When User fill [Code] field with {{valid_coupon.code}}
|
|
424
|
+
And User fill [Discount] field with {{discount_min}}
|
|
425
|
+
And User click [Save] button
|
|
426
|
+
Then User see [Create Coupon] dialog is hidden
|
|
370
427
|
|
|
371
|
-
|
|
372
|
-
|
|
373
|
-
|
|
374
|
-
|
|
375
|
-
| Tier 1 focus | Happy path + required validation per section | Happy path through entire flow + auth + key error recovery |
|
|
376
|
-
| Background | Navigate to screen | Navigate to starting page |
|
|
428
|
+
@high @no-auth
|
|
429
|
+
Scenario: VP-SEC-003 Unauthenticated user is redirected to login
|
|
430
|
+
Then User see [Login] page
|
|
431
|
+
```
|
|
377
432
|
|
|
378
|
-
|
|
433
|
+
**④ Output — `coupons.yaml` (excerpt):**
|
|
434
|
+
```yaml
|
|
435
|
+
valid_coupon:
|
|
436
|
+
code: "SAVE-{{$uuid}}"
|
|
437
|
+
discount: "20"
|
|
438
|
+
discount_zero: "0" # boundary min-1 (reject)
|
|
439
|
+
discount_min: "1" # boundary min (accept)
|
|
440
|
+
discount_max: "100" # boundary max (accept)
|
|
441
|
+
discount_over: "101" # boundary max+1 (reject)
|
|
442
|
+
error:
|
|
443
|
+
discount_range: "Discount must be between 1 and 100"
|
|
444
|
+
```
|
|
379
445
|
|
|
380
|
-
|
|
381
|
-
|---|---|---|
|
|
382
|
-
| **Happy path** | T1 | Complete flow end-to-end with valid data |
|
|
383
|
-
| **Auth persistence** | T1 | Auth state maintained across screen transitions |
|
|
384
|
-
| **Error recovery** | T1 | Invalid input mid-flow → fix → continue |
|
|
385
|
-
| **Incomplete flow** | T2 | User abandons at each phase → state cleanup |
|
|
386
|
-
| **Cross-screen data** | T2 | Data entered on screen A visible on screen B |
|
|
387
|
-
| **Screen transition edges** | T2 | Back button, refresh, deep link mid-flow |
|
|
388
|
-
| **UI state across screens** | T2 | Breadcrumb, progress indicator, nav state |
|
|
389
|
-
| **Negative inputs mid-flow** | T3 | XSS/SQL in cross-screen fields, special chars |
|
|
390
|
-
| **Concurrent flow** | T3 | Two tabs same flow, session expiry mid-flow |
|
|
391
|
-
| **Cross-screen validation** | T3 | Field dependency across screens, cascading errors |
|
|
392
|
-
|
|
393
|
-
### Flow Lightweight Guard (Tier 1)
|
|
394
|
-
|
|
395
|
-
> After generating Tier 1 flow scenarios, verify these checks — if any are missing, add them:
|
|
396
|
-
|
|
397
|
-
| Flow requirement | Minimum TC | Tag |
|
|
398
|
-
|---|---|---|
|
|
399
|
-
| Complete flow end-to-end with valid data | At least 1 happy path scenario | `@high` |
|
|
400
|
-
| Auth state across screen transitions | At least 1 auth persistence check (if flow crosses auth) | `@high` |
|
|
401
|
-
| Invalid input mid-flow → fix → continue | At least 1 error recovery scenario | `@high` |
|
|
446
|
+
---
|
|
402
447
|
|
|
403
|
-
|
|
448
|
+
## Flow Test Generation
|
|
404
449
|
|
|
405
|
-
|
|
450
|
+
> **Auto-detect**: if path is `qa/flows/<name>/` → use this section. Skip Steps 1–4 above.
|
|
406
451
|
|
|
407
|
-
|
|
408
|
-
|
|
409
|
-
|
|
410
|
-
|
|
452
|
+
| Aspect | Screen | Flow |
|
|
453
|
+
|---|---|---|
|
|
454
|
+
| Section focus | UI patterns per section | Journey phases across screens |
|
|
455
|
+
| Selector format | `[Element]` | `[Screen:Element]` (namespaced) |
|
|
456
|
+
| Test data keys | `{{variable}}` | `{{phase.variable}}` |
|
|
457
|
+
| Feature tag | `@auto` / `@smoke` etc. | `@flow` (required) |
|
|
458
|
+
| Viewpoints | VP-UI/VAL/LOGIC/SEC per section | VP-LOGIC (transitions), VP-SEC (auth persistence), VP-VAL (cross-screen data) |
|
|
411
459
|
|
|
412
|
-
|
|
460
|
+
**Scenarios to generate:**
|
|
413
461
|
|
|
414
|
-
|
|
462
|
+
| Category | What to test |
|
|
463
|
+
|---|---|
|
|
464
|
+
| Happy path | Complete flow end-to-end with valid data |
|
|
465
|
+
| Auth persistence | Auth state maintained across screen transitions |
|
|
466
|
+
| Error recovery | Invalid input mid-flow → fix → continue |
|
|
467
|
+
| Cross-screen data | Data entered on screen A visible on screen B |
|
|
415
468
|
|
|
416
469
|
```gherkin
|
|
417
470
|
@flow @auth:user
|
|
@@ -420,36 +473,30 @@ Feature: Award Submission Flow
|
|
|
420
473
|
Background:
|
|
421
474
|
Given User is on [Login] page
|
|
422
475
|
|
|
423
|
-
# --- Section: Login Phase (Tier 1: @high) ---
|
|
424
|
-
|
|
425
476
|
@high
|
|
426
|
-
Scenario: User
|
|
477
|
+
Scenario: User logs in successfully
|
|
427
478
|
When User fill [Login:Email] field with {{login.email}}
|
|
428
479
|
And User fill [Login:Password] field with {{login.password}}
|
|
429
480
|
And User click [Login:Submit] button
|
|
430
481
|
Then User see [Dashboard] page
|
|
431
482
|
|
|
432
|
-
# --- Section: Submission Phase (Tier 1: @high) ---
|
|
433
|
-
|
|
434
483
|
@high
|
|
435
|
-
Scenario: User
|
|
436
|
-
|
|
484
|
+
Scenario: User submits nomination
|
|
485
|
+
When User click [Dashboard:Awards] link
|
|
486
|
+
Then User see [Awards] page
|
|
437
487
|
When User fill [Awards:Nominee] field with {{submission.nominee}}
|
|
438
|
-
And User fill [Awards:Reason] field with {{submission.reason}}
|
|
439
488
|
And User click [Awards:Submit] button
|
|
440
|
-
Then User see
|
|
489
|
+
Then User see {{success_message}} message
|
|
441
490
|
```
|
|
442
491
|
|
|
443
492
|
```yaml
|
|
444
|
-
# test-data
|
|
493
|
+
# flows/<name>/test-data/<name>.yaml
|
|
445
494
|
login:
|
|
446
|
-
email: "
|
|
447
|
-
password: "
|
|
448
|
-
|
|
495
|
+
email: "admin@example.com"
|
|
496
|
+
password: "secret123"
|
|
449
497
|
submission:
|
|
450
498
|
nominee: "John Doe"
|
|
451
|
-
|
|
452
|
-
success_message: "Award submitted successfully"
|
|
499
|
+
success_message: "Award submitted successfully"
|
|
453
500
|
```
|
|
454
501
|
|
|
455
|
-
**Do NOT generate**: `selectors.yaml` (created during run-test), Playwright code (sungen compiles).
|
|
502
|
+
**Do NOT generate**: `selectors.yaml` (created during `run-test`), Playwright code (sungen compiles).
|