@sun-asterisk/sungen 2.2.2 → 2.3.0

package/src/orchestrator/templates/ai-instructions/github-skill-sungen-tc-generation.md

@@ -33,6 +33,46 @@ For options 1 and 2:
 For option 3:
 - Overwrite both `.feature` and `test-data.yaml` completely
 
+### Requirements-Driven Generation
+
+When `qa/screens/<screen>/requirements/` exists, use it as the **primary source** for test case generation. This produces higher quality tests because requirements contain exact validation messages, field constraints, business rules, and states.
+
+#### Reading Requirements
+
+1. **`spec.md`** (primary) — structured screen specification:
+   - **Sections** → map directly to test case sections
+   - **Fields table** → generate boundary value tests (min/max constraints), required field tests, format tests
+   - **Validation Rules table** → generate exact assertion tests using the error messages as `{{test_data}}` values
+   - **Actions table** → generate interaction tests (click, submit, navigate)
+   - **States table** → generate state transition tests (loading, error, success, disabled)
+   - **Business Rules** → generate logic tests (limits, permissions, conditional behavior)
+   - **Accessibility** → generate tab-order and aria tests
+
+2. **`ui/`** (supplementary) — screenshots, mockups, design images:
+   - Read images to understand layout, element positions, visual states
+   - Cross-reference with spec.md — identify elements visible in UI but missing from spec
+   - Use for UI/UX viewpoint tests (element visibility, placement, responsive)
+
+3. **`notes.md`** (supplementary) — free-form edge cases, decisions, known issues:
+   - Extract edge cases → add to test coverage
+   - Flag known issues → add TODO scenarios
+
+#### How Requirements Improve Each Viewpoint
+
+| Viewpoint | Without Requirements | With Requirements |
+|-----------|---------------------|-------------------|
+| **UI/UX** | "see [Field] is visible" (generic) | "see [Field] field" + verify label, placeholder, default from spec |
+| **Validation** | "submit empty → see error" (vague) | "submit empty email → see {{email_required_error}}" with exact message from spec |
+| **Logic** | Based on observed page behavior | Business rules drive specific tests (lockout after N attempts, session expiry) |
+| **Security** | Generic injection tests | Role-based tests from auth requirements, permission-specific scenarios |
+
+#### Merging Requirements + Live Page
+
+If the user also explores the live page:
+- **Verify** spec.md against actual page — flag mismatches (e.g., field in spec but not on page)
+- **Supplement** — discover elements on page not in spec (e.g., footer links, tooltips)
+- **Exact text** — capture actual placeholder text, button labels, error messages from live page and update test-data accordingly
+
 ### Page Exploration & Auth
 
 Use Playwright MCP to explore the live page. If the page requires authentication:
@@ -55,26 +95,7 @@ Use Playwright MCP to explore the live page. If the page requires authentication
 
 #### Detail screens with dynamic IDs
 
-For screens like `/admin/users/:id` or `/products/:slug`:
-1. Navigate to the **list page** first via MCP browser to find a real record ID
-2. Use that ID in the page selector value
-3. Use `User is on [X] page` — sungen resolves the path from the selector
-
-```yaml
-# selectors.yaml — full path with real ID (generated during make-test)
-user detail:
-  type: 'page'
-  value: '/admin/users/de42d800-0f5a-490e-9dcf-344fedbd34a5'
-```
-
-```gherkin
-Scenario: VP-UI-001 User detail displays name
-  Given User is on [User Detail] page
-  And User wait for [User Name] heading is visible
-  Then User see [User Name] heading with {{user_name}}
-```
-
-Note: the selector uses a hardcoded ID from the live page. If the record is deleted, update the ID in `selectors.yaml`.
+For screens like `/admin/users/:id` or `/products/:slug`, write Gherkin normally (`User is on [User Detail] page`). The real ID will be resolved during `/sungen-make-test` when selectors are generated from the live page. See `sungen-selector-fix` skill for details.
 
 ### Section-Focused Approach
 
@@ -113,72 +134,90 @@ For each selected section, generate **20+ scenarios per applicable viewpoint**.
 
 Present a test plan summary and wait for user confirmation before generating files.
 
+### Assertion Quality Rules
+
+**CRITICAL** — these rules prevent shallow, low-value test cases:
+
+1. **NEVER use `is visible`** — `User see [T] type` already asserts visibility. Writing `is visible` is redundant noise. Only use `is hidden` to assert something is NOT shown.
+2. **Group related assertions** — one scenario can have 3-7 `Then/And` steps. Don't waste a whole scenario on one element. Group elements that verify the same concern.
+3. **Assert content, not just existence** — verify values, text, states, not just "it's there":
+   - `User see [Title] heading with {{page_title}}` — verify text
+   - `User see [Email] field with {{default_email}}` — verify default value
+   - `User see [Submit] button is disabled` — verify state
+   - `User see [Error] message with {{error_text}}` — verify exact error
+4. **Every assertion must have test value** — if you write `User see [T] type`, ask: what EXACTLY should the user see? Add `with {{value}}` or `is state` whenever the expected content/state is known.
+
+**Bad** (shallow — just checks existence):
+```gherkin
+Scenario: VP-UI-001 Email field is visible
+  Given User is on [Login] page
+  Then User see [Email] field is visible
+
+Scenario: VP-UI-002 Password field is visible
+  Given User is on [Login] page
+  Then User see [Password] field is visible
+```
+
+**Good** (rich — verifies content, states, groups related checks):
+```gherkin
+Scenario: VP-UI-001 Login form displays all fields with correct defaults
+  Given User is on [Login] page
+  Then User see [Login] heading with {{login_title}}
+  And User see [Email] field
+  And User see [Password] field
+  And User see [Remember me] checkbox is unchecked
+  And User see [Submit] button is enabled
+  And User see [Forgot password] link
+```
+
 ### Viewpoint Categories
 
 | VP Category | Description |
 |---|---|
-| **UI/UX** | Default appearance, element visibility, default states |
-| **Validation** | Input validation, error messages, edge cases. **Must explore live page via MCP to capture actual error messages before writing tests. Use `User see {{error_var}}` pattern — never assert just "is visible".** |
+| **UI/UX** | Default appearance, default values/text, default states. Group related elements per scenario. |
+| **Validation** | Input validation, error messages, edge cases. **Assert exact error messages via `User see [T] with {{error_var}}`** — store texts in test-data.yaml. |
 | **Logic** | Business logic, interactions, state changes |
 | **Security** | Auth guards, injection, permission checks |
 
 ### Coverage Checklist per Section Pattern
 
-Apply the relevant checklist based on the section pattern:
-
 #### Form sections
-- **UI/UX**: All fields visible, labels correct, default values, placeholder text, required indicators
-- **Validation**: Empty submit, each required field empty, invalid format (email, phone, URL), min/max length, special chars, unicode, XSS, SQL injection, whitespace-only, boundary values
-- **Validation**: Empty submit, each required field empty, invalid format (email, phone, URL), min/max length, special chars, unicode, XSS, SQL injection, whitespace-only, boundary values — **assert exact error messages via `User see {{error_var}}`**, store texts in test-data.yaml
-- **Logic**: Successful submit, submit with all optional fields, edit existing record, cancel resets form, field dependencies (show/hide), auto-complete, date range validation
-- **Security**: CSRF protection, unauthorized submit, role-based field visibility, input sanitization
+- **UI/UX**: All fields + labels + default values in 1-2 grouped scenarios. Default states (button enabled/disabled, checkbox unchecked). Placeholder text via `with {{placeholder}}`.
+- **Validation**: Empty submit → verify all error messages. Each required field empty individually → verify specific error. Invalid formats → verify specific error. Boundary values (min/max length). Special chars, unicode, XSS, SQL injection.
+- **Logic**: Successful submit → verify success state/redirect. Edit existing → verify pre-filled values. Cancel → verify form resets. Field dependencies (show/hide).
+- **Security**: Unauthorized submit, role-based field visibility, input sanitization
 
 #### Data Table sections
-- **UI/UX**: All columns visible, column headers, row data displays correctly, empty state, loading state, action buttons visible
-- **Validation**: N/A (read-only) or inline edit validation
-- **Logic**: Sort ascending/descending per sortable column, filter by each column, search within table, row action menu (edit/delete/view), bulk select, bulk action, row click navigation, column resize
-- **Security**: Cannot access other users' data, action permissions per role, data not exposed in DOM
+- **UI/UX**: All column headers in one scenario (`User see [Col] column in [Table] table`). Row data displays with correct values. Empty state message. Action buttons per row.
+- **Logic**: Sort ascending/descending. Filter by column. Search. Row actions (edit/delete/view). Bulk select + action.
+- **Security**: Action permissions per role
 
 #### Search & Filter sections
-- **UI/UX**: Search field visible, filter buttons visible, clear button state
-- **Validation**: Empty search, special chars, SQL injection, XSS, very long text, unicode/emoji, whitespace, partial match, case insensitive
-- **Logic**: Apply single filter, combine multiple filters, clear single filter, clear all filters, search + filter combined, filter persists across pagination, no results state
-- **Security**: Injection via search, injection via filter params
+- **UI/UX**: Search field + filter buttons + clear button states in one scenario
+- **Validation**: Empty search, special chars, injection, long text, unicode
+- **Logic**: Apply/clear single filter, combine filters, no results state with message
+- **Security**: Injection via search/filter params
 
 #### Pagination sections
-- **UI/UX**: Page indicator visible, previous disabled on page 1, next enabled
-- **Logic**: Next page, previous page, navigate to last page, boundary (first/last), page indicator updates, data refreshes on navigation
-- **Security**: Filters persist across pages, search persists, cannot access page beyond max
+- **UI/UX**: Page indicator + button states (previous disabled on page 1, next enabled)
+- **Logic**: Navigate pages, boundary behavior, indicator updates
 
 #### Modal / Dialog sections
-- **UI/UX**: Modal opens, close button visible, overlay visible, form fields inside modal
-- **Validation**: Submit empty form in modal, field validation inside modal
-- **Logic**: Open modal, close with X, close with Escape, close with overlay click, submit success closes modal, submit error keeps modal open
-- **Security**: Cannot open unauthorized modals, CSRF on modal submit
-
-#### Card Grid / List sections
-- **UI/UX**: Cards display all expected fields (title, image, description, metadata), empty state, loading state
-- **Logic**: Click card navigates to detail, load more / infinite scroll, card action buttons (like, share, delete), responsive layout
-- **Security**: User-generated content sanitized, cannot access other users' cards
-
-#### Carousel / Slider sections
-- **UI/UX**: Arrows visible, indicators visible, current slide highlighted
-- **Logic**: Next slide, previous slide, wrap at end, auto-play, indicator click navigates, swipe gesture
-- **Security**: N/A
+- **UI/UX**: Modal content + all fields + close button in one scenario
+- **Validation**: Field validation inside modal with exact errors
+- **Logic**: Open/close methods (X, Escape, overlay), submit success/error behavior
 
 #### Tabs / Accordion sections
-- **UI/UX**: All tabs visible, active tab highlighted, default tab selected
-- **Logic**: Switch tabs, content updates per tab, deep link to tab, accordion expand/collapse, only one accordion open at a time
-- **Security**: Tab content respects permissions
+- **UI/UX**: All tab labels + active tab highlighted + default tab content
+- **Logic**: Switch tabs → verify content changes, accordion expand/collapse
 
 ### General Coverage Dimensions (aim for 20+ total per viewpoint)
 
-**Happy paths (3-5):** Standard flow, all optional fields, minimum required
-**Edge cases (5-8):** Empty, max length, special chars, unicode/CJK/emoji, whitespace, leading/trailing spaces, overflow
-**Boundary values (3-5):** At min/max limit, one above/below, zero/null
-**Negative cases (3-5):** Invalid format, missing required, wrong type, injection attempts
-**State transitions (2-4):** Before/after action, undo, cancel mid-flow
-**Combinatorial (2-4):** Multiple invalid fields, valid+invalid combos, different roles
+**Happy paths (3-5):** Standard flow with full assertion of result state
+**Edge cases (5-8):** Empty, max length, special chars, unicode, whitespace, boundary values
+**Negative cases (3-5):** Invalid format, missing required, wrong type, exact error messages
+**State transitions (2-4):** Before/after action, verify both old and new state
+**Combinatorial (2-4):** Multiple invalid fields, valid+invalid combos
 
 ### SPA Wait-For Steps
 
@@ -209,24 +248,37 @@ Feature: <Screen> Screen
   # Section: Create User Form
   # ============================================================
 
-  # --- UI/UX (20+) ---
+  # --- UI/UX ---
 
-  Scenario: VP-UI-001 Name field is visible
-    ...
+  Scenario: VP-UI-001 Form displays all fields with correct defaults
+    Given User is on [Create User] page
+    Then User see [Create User] heading with {{form_title}}
+    And User see [Name] field
+    And User see [Email] field
+    And User see [Role] dropdown with {{default_role}}
+    And User see [Submit] button is disabled
+    And User see [Cancel] button is enabled
 
-  # --- Validation (20+) ---
+  # --- Validation ---
 
-  Scenario: VP-VAL-001 Submit with empty name
-    ...
+  Scenario: VP-VAL-001 Submit with all empty fields shows errors
+    Given User is on [Create User] page
+    When User click [Submit] button
+    Then User see [Name error] message with {{name_required_error}}
+    And User see [Email error] message with {{email_required_error}}
 
   # ============================================================
   # Section: User Table
   # ============================================================
 
-  # --- UI/UX (20+) ---
+  # --- UI/UX ---
 
-  Scenario: VP-UI-025 Table displays all columns
-    ...
+  Scenario: VP-UI-010 Table displays all columns
+    Given User is on [Users] page
+    Then User see [Name] column in [Users] table
+    And User see [Email] column in [Users] table
+    And User see [Status] column in [Users] table
+    And User see [Actions] column in [Users] table
 ```
 
 **Naming convention:** `VP-<CATEGORY>-<NNN>` prefix in Scenario name.

package/src/orchestrator/templates/readme.md

@@ -15,7 +15,11 @@ sungen generate → compiles Gherkin + selectors + data → Playwright .spec.ts
 ├── qa/screens/<name>/
 │   ├── features/         # .feature files (Gherkin)
 │   ├── selectors/        # Element locator YAML mappings
-│   └── test-data/        # Test data YAML values
+│   ├── test-data/        # Test data YAML values
+│   └── requirements/     # Screen specs, UI designs, notes
+│       ├── spec.md       # Structured screen specification
+│       ├── ui/           # Screenshots, mockups, design images
+│       └── notes.md      # Edge cases, decisions (optional)
 ├── specs/
 │   └── generated/        # Auto-generated Playwright tests
 ├── .claude/
@@ -53,7 +57,7 @@ Use AI commands (Claude Code or GitHub Copilot) to drive the workflow:
 |---|---|
 | `/sungen:add-screen` | `/sungen-add-screen` |
 
-Scaffolds `qa/screens/<name>/` with empty feature, selectors, and test-data files, then asks if you want to create test cases.
+Scaffolds `qa/screens/<name>/` with empty feature, selectors, test-data, and requirements files. Fill `requirements/spec.md` with screen specs before creating test cases for higher quality output.
 
 ### Step 2: Create test cases
 
@@ -62,10 +66,11 @@ Scaffolds `qa/screens/<name>/` with empty feature, selectors, and test-data file
 | `/sungen:make-tc login` | `/sungen-make-tc login` |
 
 AI acts as a **Senior QA Engineer**:
-1. Explores the live page via Playwright MCP (asks user to log in if auth needed)
-2. Identifies screen sections → asks user which to focus on
-3. Generates **20+ scenarios per viewpoint** (UI/UX, Validation, Logic, Security) for each section
-4. Confirms test plan before generating `.feature` + `test-data.yaml`
+1. Reads `requirements/spec.md` for screen specs (fields, validation, business rules, states)
+2. Optionally explores the live page via Playwright MCP to verify and supplement
+3. Identifies screen sections → asks user which to focus on
+4. Generates **20+ scenarios per viewpoint** (UI/UX, Validation, Logic, Security) for each section
+5. Confirms test plan before generating `.feature` + `test-data.yaml`
 
 ### Step 3: Compile & run tests
 
@@ -136,8 +141,8 @@ User <action> [<Target>] <type> with {{<Value>}}
 | Assert text | `User see [Title] heading with {{title}}` |
 | Assert state | `User see [Submit] button is disabled` |
 | Wait for | `User wait for [Modal] dialog is visible` |
-| Table row | `User see [Users] table has row with {{name}}` |
-| Table column | `User see [Users] table has [Email] column` |
+| Table row | `User see [Users] table row with {{name}}` |
+| Table column | `User see [Email] column in [Users] table` |
 
 States: `hidden` `visible` `disabled` `enabled` `checked` `unchecked` `focused` `empty` `loading` `selected`