@sun-asterisk/sungen 2.6.6 → 2.6.8

package/src/orchestrator/templates/ai-instructions/github-skill-sungen-tc-generation.md

@@ -14,17 +14,34 @@ Generate **focused test cases per screen section** using a **tier-based approach
 |---|---|---|---|
 | **Tier 1** (default) | `@high` | Happy paths, required validation, core business rules, security basics | First run of `create-test` |
 | **Tier 2** (expand) | `@normal` + `@low` | UI presence, optional validation, edge cases, cosmetic checks | User runs `create-test` again with "Add viewpoints" mode |
+| **Tier 3** (deep) | `@high` + `@normal` + `@low` | Extra BVA combinations, cross-field validation, negative/destructive inputs, concurrent/race conditions, complex state transitions | Recommended after Tier 2 completes |
+| **Full** (all-at-once) | All | Tier 1 + 2 + 3 combined in one run | Option at first run, **not recommended** — large output |
 
-**Round 1 (Tier 1)** targets **~10-15 scenarios per section** — enough to cover critical flows and catch real bugs. This is the default behavior.
+**Round 1 (Tier 1)** targets **~10-15 scenarios per section** — enough to cover critical flows and catch real bugs. This is the default behavior. A **Full** option is available but not recommended — it generates all tiers at once, producing very large output (~40-60 scenarios/section).
 
-**Round 2 (Tier 2)** expands to full coverage when the user explicitly chooses "Add viewpoints" or "Add new sections" update mode. Only then generate `@normal` + `@low` scenarios to fill coverage gaps.
+**Round 2 (Tier 2)** expands coverage when the user explicitly chooses "Add viewpoints" or "Add new sections" update mode. Only then generate `@normal` + `@low` scenarios to fill coverage gaps.
 
 ## Update Mode
 
-When `.feature` already has scenarios, summarize and ask:
+When `.feature` already has scenarios, detect which tiers exist by scanning section comments, then ask the appropriate update mode.
+
+**Tier detection** — scan for `# --- Section:` comments:
+- Only `(Tier 1: @high)` comments → **Tier 1 only**
+- Has `(Tier 2: @normal + @low)` comments → **Tier 1 + 2**
+- Has `(Tier 3: deep)` comments → **Tier 1 + 2 + 3**
+
+**If Tier 1 only:**
 1. **Add new sections** — append new sections with Tier 2 (`@normal` + `@low`) scenarios, continue numbering
 2. **Add viewpoints** — expand existing sections with Tier 2 (`@normal` + `@low`) scenarios
-3. **Replace all** — overwrite with fresh Tier 1 (`@high`) generation
+
+**If Tier 1 + 2:**
+1. **Deep coverage (Tier 3)** — add advanced scenarios: extra BVA, cross-field validation, negative inputs, race conditions **(Recommended)**
+2. **Add new sections** — append new sections with Tier 2 scenarios, continue numbering
+
+**If Tier 1 + 2 + 3** (full coverage):
+1. **Add new sections** — append new sections if screen has changed
+
+> **Replace all** (available in all modes) — overwrite with fresh Tier 1 (`@high`) generation. Use when spec has changed significantly.
 
 For append: read highest `VP-<CAT>-<NNN>`, continue from next number. Never modify existing scenarios.
 
@@ -39,6 +56,58 @@ Requirements improve every viewpoint: exact error messages for VAL, business rul
 
 If also exploring live page: verify spec vs actual, flag mismatches, capture exact text.
 
+### Tier 1 — Lightweight Guard
+
+> Tier 1 uses the quick 3-step strategy (Step 1–3) without full section walking. After generation, verify these checks against `spec.md` — if any are missing, add them before finalizing:
+
+| Spec section | Minimum TC requirement | Tag |
+|---|---|---|
+| Fields row, `Required=yes` | At least 1 required-error TC per required field | `@high` |
+| Validation Rules row | At least 1 exact-message TC per validation rule | `@high` |
+| Business Rule bullet | At least 1 behavioral TC per business rule | `@high` |
+| Security-sensitive actions (auth, permissions) | At least 1 security check (VP-SEC) | `@high` |
+| Entity with lifecycle states | At least 1 key state transition (VP-LOGIC) | `@high` |
+
+Other spec sections (Section header, Default, Accessibility, Notes) are deferred to Tier 2.
+
+### Mapping Contract (Tier 2+ — MANDATORY)
+
+> **Full contract applies from Tier 2 onwards.** Walk `spec.md` top-to-bottom. For every section listed below, you **MUST** produce the indicated TCs. If a section is missing or empty, note it explicitly — do **NOT** silently skip.
+
+#### Table 1 — `spec.md` → TC Mapping Contract
+
+| Spec section | TC requirement | Tier | Tag | Viewpoint |
+|---|---|---|---|---|
+| Section header | 1 presence TC | T2 | `@low` | VP-UI |
+| Fields row, `Required=yes` | 1 per-field required-error TC (individual, not bulk) | T2 | `@normal` | VP-VAL |
+| Fields `Constraints` cell (non-empty, non-`—`) | 4 BVA for numeric range; 1 EP for format | T2 | `@normal` | VP-VAL |
+| Fields `Constraints` cell (numeric) | 2 extra BVA (`min+1`, `max-1`) + mid-range → total 7-point with T2 | T3 | `@normal` | VP-VAL |
+| Fields `Default` cell (non-empty, non-`—`) | 1 default-state TC | T2 | `@low` | VP-UI |
+| Fields with dependencies (conditional required, cascading) | 1 cross-field TC per dependency | T3 | `@normal` | VP-VAL |
+| Fields accepting free text input | 1 negative input TC (XSS/SQL injection/special chars) | T3 | `@high` | VP-SEC |
+| Actions row (primary: submit, create, delete) | 1 edge/alternate behavior TC | T2 | `@normal` | VP-LOGIC |
+| Actions row (secondary: cancel, reset, export) | 1 behavior TC | T2 | `@normal` | VP-LOGIC |
+| Actions with submit/save | 1 race condition TC (double-submit, back after POST) | T3 | `@normal` | VP-LOGIC |
+| Validation Rules row | 1 exact-message TC | T2 | `@normal` | VP-VAL |
+| States row (Default/Loading/Error/Success) | 1 visual state TC | T2 | `@normal` | VP-UI / VP-LOGIC |
+| States row (other states) | 1 visual TC | T2 | `@low` | VP-UI |
+| Business Rule bullet | 1 edge-case behavioral TC; `@manual` if unautomatable | T2 | `@normal` | VP-LOGIC |
+| Accessibility tab order | 1 tab-walking TC | T2 | `@low` | VP-UI |
+| Notes bullet | classify per content (page title→VP-UI, env→`@manual`, edge→relevant section) | T2 | `@low` | varies |
+
+#### Table 2 — `test-viewpoint.md` → TC Mapping Contract
+
+| Section / format | TC requirement |
+|---|---|
+| `## Edge Cases` (bulleted) | classify per spec Notes |
+| `## Known Issues` (bulleted) | `@manual` TCs with bug ID comment |
+| `## Design Decisions` | behavioral TCs |
+| `## UI Patterns Identified` | confirm `sungen-viewpoint` pattern checklists |
+| `## Priority Viewpoints` | adjust Tier-2 emphasis |
+| `## Element Overview` (table) | 1 presence/state TC per row (T2, `@low`) |
+| `## State Transitions` (table) | 1 TC per valid transition + key blocked |
+| Free-form notes | re-read with heading as context |
+
 ### Figma supplement (`spec_figma.md`)
 
 When `requirements/spec_figma.md` is present alongside `spec.md`, treat it as a **secondary input** with these rules:
@@ -102,8 +171,15 @@ Apply `sungen-test-design-techniques` to spec-extracted conditions:
 Use `sungen-viewpoint` skill for per-pattern checklists across 4 viewpoints: UI/UX, Data & Validate, Logic, Security.
 
 **Tier-aware gap filling:**
-- **Tier 1 (first run)**: only add `@high` items from the checklists — core security checks (VP-SEC), required field validation (VP-VAL), key state transitions (VP-LOGIC). Skip `@normal`/`@low` items like hover states, empty states, tooltips.
+- **Tier 1 (first run)**: only add `@high` items from the checklists. Skip `@normal`/`@low` items. After generation, apply the **Lightweight Guard** to verify coverage.
 - **Tier 2 (expand run)**: add `@normal` + `@low` scenarios — UI presence, optional validation, edge cases, cosmetic checks, keyboard nav, hover effects.
+- **Tier 3 (deep run)**: do NOT use viewpoint checklists. Instead, re-apply test design techniques with deeper analysis:
+  - **BVA**: expand from 4-point to 6-point (`min-1`, `min`, `min+1`, `max-1`, `max`, `max+1`) + typical mid-range value
+  - **Decision Table**: enumerate combinations previously capped — cover remaining outcome rows
+  - **Cross-field**: identify field dependencies from spec, generate one scenario per dependency
+  - **Negative inputs**: add security-oriented inputs (SQL `' OR 1=1`, XSS `<script>`, special chars `<>&"'`, max+100 length)
+  - **Race conditions**: double-click submit, browser back after POST, concurrent edit by two users
+- **Full (all-at-once)**: apply Tier 1 + 2 + 3 rules in a single pass. Use all viewpoint checklists and all test design technique depths. Mark sections with all three tier comments.
 
 **Validation rule**: capture actual error messages from live page or spec.md. Use `User see {{error_var}}` — never assert just "is visible".
 
@@ -113,29 +189,35 @@ Every scenario **MUST** have exactly one priority tag. Add it before the scenari
 
 **Rule: look at what the WHEN step does and what THEN asserts — that determines the tag.**
 
-| Tag | When to use |
-|---|---|
-| `@high` | WHEN performs: login/logout, submit happy path, auth-check, primary CRUD action. THEN asserts: success state, redirect, or security gate |
-| `@normal` | WHEN provides invalid/empty input, OR scenario tests search/filter/notification/secondary flow. THEN asserts: error message, secondary feature result |
-| `@low` | THEN only asserts: element visible, text label, placeholder, tooltip, icon, layout. WHEN (if any) is navigation only |
+| Tag | Scenario type | Examples |
+|---|---|---|
+| `@high` | Auth actions | login, logout, protected page redirect |
+| `@high` | CRUD happy path | create / update / delete → success message or redirect |
+| `@high` | Security gate | unauthenticated access → redirected to login |
+| `@high` | Primary business flow | core flow step completes successfully |
+| `@normal` | Input validation | invalid / empty input → error message shown |
+| `@normal` | Boundary / format | email format, length limit, numeric range |
+| `@normal` | Secondary features | search, filter, sort, pagination, notification |
+| `@normal` | Non-primary state transition | cancel, undo, back |
+| `@low` | Element presence | component / section is visible |
+| `@low` | Text content | label, placeholder, tooltip, warning text |
+| `@low` | Visual / cosmetic | alignment, icon, empty state, hover style |
+
+**`@steps:` scenarios** do NOT get a priority tag (they are setup blocks, not test cases).
+
+### Priority assignment — disambiguation tie-breaker
 
-### Auto-assign heuristics
+When in doubt between `@low` and `@normal`, apply this rule:
 
-| Scenario type (by WHEN + THEN structure) | Tag |
+| Pattern | Tag |
 |---|---|
-| Auth: login / logout / protected page redirect | `@high` |
-| CRUD happy path: create / update / delete → success message or redirect | `@high` |
-| Security: unauthenticated access → redirected to login | `@high` |
-| Primary business flow step completes successfully | `@high` |
-| Validation: invalid / empty input → error message shown | `@normal` |
-| Boundary / format validation (email, length, range) | `@normal` |
-| Secondary features: search, filter, sort, pagination, notification | `@normal` |
-| State transition not on primary success path (cancel, undo, back) | `@normal` |
-| Element / component presence check (is it visible?) | `@low` |
-| Text content: label, placeholder, tooltip, warning message text | `@low` |
-| Visual / cosmetic: alignment, icon, empty state, hover style | `@low` |
+| 0 When steps + Then asserts only `User see [T] type` (presence) | `@low` (always) |
+| 0 When steps + Then asserts `User see [T] type with {{value}}` (content) | `@low` (label/text) |
+| When steps but no state change (e.g., navigation only) | `@low` |
+| When provides invalid input + Then asserts validation error | `@normal` |
+| When triggers business logic + Then asserts state change | `@high` or `@normal` per heuristics |
 
-**`@steps:` scenarios** do NOT get a priority tag (they are setup blocks, not test cases).
+**Example:** `Then User see [Email] field` (no When) → `@low`. `When User fill ... And User click ... Then User see [Error]` → `@normal`.
 
 ## SPA Wait-For Steps
 
@@ -253,7 +335,9 @@ Feature: <Screen> Screen
     Then User is on [Login] page
 ```
 
-**Tier 2 (expand run)** adds `@normal` + `@low` scenarios like UI field presence, hover states, tooltips, empty states.
+**Tier 2 (expand run)** appends sections with `# --- Section: X (Tier 2: @normal + @low) ---` comments. Adds UI field presence, hover states, tooltips, empty states.
+
+**Tier 3 (deep run)** appends sections with `# --- Section: X (Tier 3: deep) ---` comments. Adds extra BVA boundaries, cross-field validation, negative/destructive inputs, concurrent/race conditions.
 
 ### When to use DataTable vs Row Scope
 
@@ -266,21 +350,7 @@ Feature: <Screen> Screen
 
 **Test data** — `qa/screens/<screen>/test-data/<screen>.yaml`, grouped by section. Data is loaded **at runtime** — keys become runtime lookups, not hardcoded strings. The same compiled test works across environments.
 
-**Environment-specific data**: For values that differ per environment (credentials, URLs, test users), create `<screen>.<env>.yaml` alongside the base file. Users run `SUNGEN_ENV=staging npx playwright test` to merge overrides. Structure env YAML with the same keys, only including values that change:
-
-```yaml
-# login.yaml (base)
-valid_email: admin@dev.example.com
-valid_password: DevPass123
-
-# login.staging.yaml (override for staging)
-valid_email: admin@staging.example.com
-valid_password: StagingPass456
-```
-
-**Do NOT generate**: `selectors.yaml` (created during run-test), Playwright code (sungen compiles).
-
----
+**Environment-specific data**: create `<screen>.<env>.yaml` alongside the base file with only the keys that change. Users run `SUNGEN_ENV=staging npx playwright test` to merge overrides.
 
 ## Flow Test Generation
 
@@ -303,15 +373,41 @@ When generating tests for a **flow** (`qa/flows/<name>/`), adapt the strategy:
 | Tier 1 focus | Happy path + required validation per section | Happy path through entire flow + auth + key error recovery |
 | Background | Navigate to screen | Navigate to starting page |
 
-### Flow-specific scenarios to generate
+### Flow-specific scenarios by tier
 
-| Category | Examples |
-|---|---|
-| **Happy path** | Complete flow end-to-end with valid data |
-| **Auth persistence** | Auth state maintained across screen transitions |
-| **Error recovery** | Invalid input mid-flow → fix → continue |
-| **Incomplete flow** | User abandons at each phase → state cleanup |
-| **Cross-screen data** | Data entered on screen A visible on screen B |
+| Category | Tier | Examples |
+|---|---|---|
+| **Happy path** | T1 | Complete flow end-to-end with valid data |
+| **Auth persistence** | T1 | Auth state maintained across screen transitions |
+| **Error recovery** | T1 | Invalid input mid-flow → fix → continue |
+| **Incomplete flow** | T2 | User abandons at each phase → state cleanup |
+| **Cross-screen data** | T2 | Data entered on screen A visible on screen B |
+| **Screen transition edges** | T2 | Back button, refresh, deep link mid-flow |
+| **UI state across screens** | T2 | Breadcrumb, progress indicator, nav state |
+| **Negative inputs mid-flow** | T3 | XSS/SQL in cross-screen fields, special chars |
+| **Concurrent flow** | T3 | Two tabs same flow, session expiry mid-flow |
+| **Cross-screen validation** | T3 | Field dependency across screens, cascading errors |
+
+### Flow Lightweight Guard (Tier 1)
+
+> After generating Tier 1 flow scenarios, verify these checks — if any are missing, add them:
+
+| Flow requirement | Minimum TC | Tag |
+|---|---|---|
+| Complete flow end-to-end with valid data | At least 1 happy path scenario | `@high` |
+| Auth state across screen transitions | At least 1 auth persistence check (if flow crosses auth) | `@high` |
+| Invalid input mid-flow → fix → continue | At least 1 error recovery scenario | `@high` |
+
+### Flow Section Comments
+
+Flow features use the same `# --- Section:` comment format, with phase names instead of UI sections:
+
+```gherkin
+# --- Section: Login Phase (Tier 1: @high) ---
+# --- Section: Submission Phase (Tier 2: @normal + @low) ---
+```
+
+Update Mode tier detection works identically — scan for tier markers in these comments.
 
 ### Output Format (Flow)
 
@@ -322,6 +418,8 @@ Feature: Award Submission Flow
   Background:
     Given User is on [Login] page
 
+  # --- Section: Login Phase (Tier 1: @high) ---
+
   @high
   Scenario: User login successfully
     When User fill [Login:Email] field with {{login.email}}
@@ -329,6 +427,8 @@ Feature: Award Submission Flow
     And User click [Login:Submit] button
     Then User see [Dashboard] page
 
+  # --- Section: Submission Phase (Tier 1: @high) ---
+
   @high
   Scenario: User navigates to awards and submits
     Given User is on [Awards] page
@@ -349,3 +449,5 @@ submission:
   reason: "Outstanding contribution to the team"
   success_message: "Award submitted successfully"
 ```
+
+**Do NOT generate**: `selectors.yaml` (created during run-test), Playwright code (sungen compiles).

package/src/orchestrator/templates/ai-instructions/github-skill-sungen-test-design-techniques.md

@@ -68,6 +68,45 @@ Scenario: VP-VAL-003 Above maximum is rejected           # value = 101
 - `@high VP-VAL-012 Maximum boundary (10) is accepted`
 - `@high VP-VAL-013 Above maximum (11) is rejected`
 
+### Expected outcome contract (MANDATORY)
+
+When generating BVA scenarios, the **Then** assertion **MUST** match the scenario name's outcome verb:
+
+| Name verb | Expected `Then` assertion |
+|---|---|
+| "is accepted" / "passes validation" | NO validation error visible. Form proceeds OR field is in normal state. Use `User see [Field Error] message is hidden` if uncertain. |
+| "is rejected" / "fails validation" / "shows error" | Specific validation error visible with exact message text from spec. |
+
+**Anti-pattern (do NOT generate):**
+
+```gherkin
+# BAD: name says "passes validation" but asserts auth error
+Scenario: VP-VAL-005 Email at maximum boundary (254 chars) passes validation
+  When User fill [Email] field with {{email_max_254}}
+  And User fill [Password] field with {{valid_password}}
+  And User click [Login] button
+  Then User see [Auth Error] message with {{auth_error_message}}
+```
+
+**Correct pattern:**
+
+```gherkin
+# GOOD: name and assertion aligned
+@normal
+Scenario: VP-VAL-005 Email at maximum boundary (254 chars) passes validation
+  When User fill [Email] field with {{email_max_254}}
+  And User fill [Password] field with {{valid_password}}
+  And User click [Login] button
+  Then User see [Email Error] message is hidden
+
+@normal
+Scenario: VP-VAL-006 Email at maximum+1 (255 chars) fails validation
+  When User fill [Email] field with {{email_over_max_255}}
+  And User fill [Password] field with {{valid_password}}
+  And User click [Login] button
+  Then User see [Email Error] message with {{email_too_long_error}}
+```
+
 ---
 
 ## 3. Decision Table