@sun-asterisk/sungen 2.2.2 → 2.3.0

package/dist/orchestrator/templates/ai-instructions/claude-skill-tc-generation.md

@@ -33,6 +33,46 @@ For options 1 and 2:
 For option 3:
 - Overwrite both `.feature` and `test-data.yaml` completely
 
+### Requirements-Driven Generation
+
+When `qa/screens/<screen>/requirements/` exists, use it as the **primary source** for test case generation. This produces higher quality tests because requirements contain exact validation messages, field constraints, business rules, and states.
+
+#### Reading Requirements
+
+1. **`spec.md`** (primary) — structured screen specification:
+   - **Sections** → map directly to test case sections
+   - **Fields table** → generate boundary value tests (min/max constraints), required field tests, format tests
+   - **Validation Rules table** → generate exact assertion tests using the error messages as `{{test_data}}` values
+   - **Actions table** → generate interaction tests (click, submit, navigate)
+   - **States table** → generate state transition tests (loading, error, success, disabled)
+   - **Business Rules** → generate logic tests (limits, permissions, conditional behavior)
+   - **Accessibility** → generate tab-order and aria tests
+
+2. **`ui/`** (supplementary) — screenshots, mockups, design images:
+   - Read images to understand layout, element positions, visual states
+   - Cross-reference with spec.md — identify elements visible in UI but missing from spec
+   - Use for UI/UX viewpoint tests (element visibility, placement, responsive)
+
+3. **`notes.md`** (supplementary) — free-form edge cases, decisions, known issues:
+   - Extract edge cases → add to test coverage
+   - Flag known issues → add TODO scenarios
+
+#### How Requirements Improve Each Viewpoint
+
+| Viewpoint | Without Requirements | With Requirements |
+|-----------|---------------------|-------------------|
+| **UI/UX** | "see [Field] is visible" (generic) | "see [Field] field" + verify label, placeholder, default from spec |
+| **Validation** | "submit empty → see error" (vague) | "submit empty email → see {{email_required_error}}" with exact message from spec |
+| **Logic** | Based on observed page behavior | Business rules drive specific tests (lockout after N attempts, session expiry) |
+| **Security** | Generic injection tests | Role-based tests from auth requirements, permission-specific scenarios |
+
+#### Merging Requirements + Live Page
+
+If the user also explores the live page:
+- **Verify** spec.md against actual page — flag mismatches (e.g., field in spec but not on page)
+- **Supplement** — discover elements on page not in spec (e.g., footer links, tooltips)
+- **Exact text** — capture actual placeholder text, button labels, error messages from live page and update test-data accordingly
+
 ### Page Exploration & Auth
 
 Use Playwright MCP to explore the live page. If the page requires authentication:
@@ -55,26 +95,7 @@ Use Playwright MCP to explore the live page. If the page requires authentication
 
 #### Detail screens with dynamic IDs
 
-For screens like `/admin/users/:id` or `/products/:slug`:
-1. Navigate to the **list page** first via MCP browser to find a real record ID
-2. Use that ID in the page selector value
-3. Use `User is on [X] page` — sungen resolves the path from the selector
-
-```yaml
-# selectors.yaml — full path with real ID (generated during make-test)
-user detail:
-  type: 'page'
-  value: '/admin/users/de42d800-0f5a-490e-9dcf-344fedbd34a5'
-```
-
-```gherkin
-Scenario: VP-UI-001 User detail displays name
-  Given User is on [User Detail] page
-  And User wait for [User Name] heading is visible
-  Then User see [User Name] heading with {{user_name}}
-```
-
-Note: the selector uses a hardcoded ID from the live page. If the record is deleted, update the ID in `selectors.yaml`.
+For screens like `/admin/users/:id` or `/products/:slug`, write Gherkin normally (`User is on [User Detail] page`). The real ID will be resolved during `/sungen:make-test` when selectors are generated from the live page. See `sungen-selector-fix` skill for details.
 
 ### Section-Focused Approach
 
@@ -113,71 +134,90 @@ For each selected section, generate **20+ scenarios per applicable viewpoint**.
 
 Present a test plan summary and wait for user confirmation before generating files.
 
+### Assertion Quality Rules
+
+**CRITICAL** — these rules prevent shallow, low-value test cases:
+
+1. **NEVER use `is visible`** — `User see [T] type` already asserts visibility. Writing `is visible` is redundant noise. Only use `is hidden` to assert something is NOT shown.
+2. **Group related assertions** — one scenario can have 3-7 `Then/And` steps. Don't waste a whole scenario on one element. Group elements that verify the same concern.
+3. **Assert content, not just existence** — verify values, text, states, not just "it's there":
+   - `User see [Title] heading with {{page_title}}` — verify text
+   - `User see [Email] field with {{default_email}}` — verify default value
+   - `User see [Submit] button is disabled` — verify state
+   - `User see [Error] message with {{error_text}}` — verify exact error
+4. **Every assertion must have test value** — if you write `User see [T] type`, ask: what EXACTLY should the user see? Add `with {{value}}` or `is state` whenever the expected content/state is known.
+
+**Bad** (shallow — just checks existence):
+```gherkin
+Scenario: VP-UI-001 Email field is visible
+  Given User is on [Login] page
+  Then User see [Email] field is visible
+
+Scenario: VP-UI-002 Password field is visible
+  Given User is on [Login] page
+  Then User see [Password] field is visible
+```
+
+**Good** (rich — verifies content, states, groups related checks):
+```gherkin
+Scenario: VP-UI-001 Login form displays all fields with correct defaults
+  Given User is on [Login] page
+  Then User see [Login] heading with {{login_title}}
+  And User see [Email] field
+  And User see [Password] field
+  And User see [Remember me] checkbox is unchecked
+  And User see [Submit] button is enabled
+  And User see [Forgot password] link
+```
+
 ### Viewpoint Categories
 
 | VP Category | Description |
 |---|---|
-| **UI/UX** | Default appearance, element visibility, default states |
-| **Validation** | Input validation, error messages, edge cases. **Must explore live page via MCP to capture actual error messages before writing tests. Use `User see {{error_var}}` pattern — never assert just "is visible".** |
+| **UI/UX** | Default appearance, default values/text, default states. Group related elements per scenario. |
+| **Validation** | Input validation, error messages, edge cases. **Assert exact error messages via `User see [T] with {{error_var}}`** — store texts in test-data.yaml. |
 | **Logic** | Business logic, interactions, state changes |
 | **Security** | Auth guards, injection, permission checks |
 
 ### Coverage Checklist per Section Pattern
 
-Apply the relevant checklist based on the section pattern:
-
 #### Form sections
-- **UI/UX**: All fields visible, labels correct, default values, placeholder text, required indicators
-- **Validation**: Empty submit, each required field empty, invalid format (email, phone, URL), min/max length, special chars, unicode, XSS, SQL injection, whitespace-only, boundary values — **assert exact error messages via `User see {{error_var}}`**, store texts in test-data.yaml
-- **Logic**: Successful submit, submit with all optional fields, edit existing record, cancel resets form, field dependencies (show/hide), auto-complete, date range validation
-- **Security**: CSRF protection, unauthorized submit, role-based field visibility, input sanitization
+- **UI/UX**: All fields + labels + default values in 1-2 grouped scenarios. Default states (button enabled/disabled, checkbox unchecked). Placeholder text via `with {{placeholder}}`.
+- **Validation**: Empty submit → verify all error messages. Each required field empty individually → verify specific error. Invalid formats → verify specific error. Boundary values (min/max length). Special chars, unicode, XSS, SQL injection.
+- **Logic**: Successful submit → verify success state/redirect. Edit existing → verify pre-filled values. Cancel → verify form resets. Field dependencies (show/hide).
+- **Security**: Unauthorized submit, role-based field visibility, input sanitization
 
 #### Data Table sections
-- **UI/UX**: All columns visible, column headers, row data displays correctly, empty state, loading state, action buttons visible
-- **Validation**: N/A (read-only) or inline edit validation
-- **Logic**: Sort ascending/descending per sortable column, filter by each column, search within table, row action menu (edit/delete/view), bulk select, bulk action, row click navigation, column resize
-- **Security**: Cannot access other users' data, action permissions per role, data not exposed in DOM
+- **UI/UX**: All column headers in one scenario. Row data displays with correct values. Empty state message. Action buttons per row.
+- **Logic**: Sort ascending/descending. Filter by column. Search. Row actions (edit/delete/view). Bulk select + action.
+- **Security**: Action permissions per role
 
 #### Search & Filter sections
-- **UI/UX**: Search field visible, filter buttons visible, clear button state
-- **Validation**: Empty search, special chars, SQL injection, XSS, very long text, unicode/emoji, whitespace, partial match, case insensitive
-- **Logic**: Apply single filter, combine multiple filters, clear single filter, clear all filters, search + filter combined, filter persists across pagination, no results state
-- **Security**: Injection via search, injection via filter params
+- **UI/UX**: Search field + filter buttons + clear button states in one scenario
+- **Validation**: Empty search, special chars, injection, long text, unicode
+- **Logic**: Apply/clear single filter, combine filters, no results state with message
+- **Security**: Injection via search/filter params
 
 #### Pagination sections
-- **UI/UX**: Page indicator visible, previous disabled on page 1, next enabled
-- **Logic**: Next page, previous page, navigate to last page, boundary (first/last), page indicator updates, data refreshes on navigation
-- **Security**: Filters persist across pages, search persists, cannot access page beyond max
+- **UI/UX**: Page indicator + button states (previous disabled on page 1, next enabled)
+- **Logic**: Navigate pages, boundary behavior, indicator updates
 
 #### Modal / Dialog sections
-- **UI/UX**: Modal opens, close button visible, overlay visible, form fields inside modal
-- **Validation**: Submit empty form in modal, field validation inside modal
-- **Logic**: Open modal, close with X, close with Escape, close with overlay click, submit success closes modal, submit error keeps modal open
-- **Security**: Cannot open unauthorized modals, CSRF on modal submit
-
-#### Card Grid / List sections
-- **UI/UX**: Cards display all expected fields (title, image, description, metadata), empty state, loading state
-- **Logic**: Click card navigates to detail, load more / infinite scroll, card action buttons (like, share, delete), responsive layout
-- **Security**: User-generated content sanitized, cannot access other users' cards
-
-#### Carousel / Slider sections
-- **UI/UX**: Arrows visible, indicators visible, current slide highlighted
-- **Logic**: Next slide, previous slide, wrap at end, auto-play, indicator click navigates, swipe gesture
-- **Security**: N/A
+- **UI/UX**: Modal content + all fields + close button in one scenario
+- **Validation**: Field validation inside modal with exact errors
+- **Logic**: Open/close methods (X, Escape, overlay), submit success/error behavior
 
 #### Tabs / Accordion sections
-- **UI/UX**: All tabs visible, active tab highlighted, default tab selected
-- **Logic**: Switch tabs, content updates per tab, deep link to tab, accordion expand/collapse, only one accordion open at a time
-- **Security**: Tab content respects permissions
+- **UI/UX**: All tab labels + active tab highlighted + default tab content
+- **Logic**: Switch tabs → verify content changes, accordion expand/collapse
 
 ### General Coverage Dimensions (aim for 20+ total per viewpoint)
 
-**Happy paths (3-5):** Standard flow, all optional fields, minimum required
-**Edge cases (5-8):** Empty, max length, special chars, unicode/CJK/emoji, whitespace, leading/trailing spaces, overflow
-**Boundary values (3-5):** At min/max limit, one above/below, zero/null
-**Negative cases (3-5):** Invalid format, missing required, wrong type, injection attempts
-**State transitions (2-4):** Before/after action, undo, cancel mid-flow
-**Combinatorial (2-4):** Multiple invalid fields, valid+invalid combos, different roles
+**Happy paths (3-5):** Standard flow with full assertion of result state
+**Edge cases (5-8):** Empty, max length, special chars, unicode, whitespace, boundary values
+**Negative cases (3-5):** Invalid format, missing required, wrong type, exact error messages
+**State transitions (2-4):** Before/after action, verify both old and new state
+**Combinatorial (2-4):** Multiple invalid fields, valid+invalid combos
 
 ### SPA Wait-For Steps
 
@@ -208,24 +248,37 @@ Feature: <Screen> Screen
   # Section: Create User Form
   # ============================================================
 
-  # --- UI/UX (20+) ---
+  # --- UI/UX ---
 
-  Scenario: VP-UI-001 Name field is visible
-    ...
+  Scenario: VP-UI-001 Form displays all fields with correct defaults
+    Given User is on [Create User] page
+    Then User see [Create User] heading with {{form_title}}
+    And User see [Name] field
+    And User see [Email] field
+    And User see [Role] dropdown with {{default_role}}
+    And User see [Submit] button is disabled
+    And User see [Cancel] button is enabled
 
-  # --- Validation (20+) ---
+  # --- Validation ---
 
-  Scenario: VP-VAL-001 Submit with empty name
-    ...
+  Scenario: VP-VAL-001 Submit with all empty fields shows errors
+    Given User is on [Create User] page
+    When User click [Submit] button
+    Then User see [Name error] message with {{name_required_error}}
+    And User see [Email error] message with {{email_required_error}}
 
   # ============================================================
   # Section: User Table
   # ============================================================
 
-  # --- UI/UX (20+) ---
+  # --- UI/UX ---
 
-  Scenario: VP-UI-025 Table displays all columns
-    ...
+  Scenario: VP-UI-010 Table displays all columns
+    Given User is on [Users] page
+    Then User see [Name] column in [Users] table
+    And User see [Email] column in [Users] table
+    And User see [Status] column in [Users] table
+    And User see [Actions] column in [Users] table
 ```
 
 **Naming convention:** `VP-<CATEGORY>-<NNN>` prefix in Scenario name.

package/dist/orchestrator/templates/ai-instructions/copilot-cmd-add-screen.md

@@ -24,15 +24,23 @@ Run with #tool:terminal:
 sungen add --screen ${input:screen} --path ${input:path}
 ```
 
-### 2. Create test cases
+### 2. Fill requirements (recommended)
+
+Ask the user: "Would you like to fill in `requirements/spec.md` now? This helps generate higher quality test cases."
+
+- If yes → open `qa/screens/${input:screen}/requirements/spec.md` and help the user fill sections, fields, validation rules, business rules, and states.
+- If they have UI designs (screenshots, Figma exports, mockups) → suggest copying them to `requirements/ui/`.
+- If no → proceed to step 3.
+
+### 3. Create test cases
 
 Ask the user: "Would you like to create test cases now?"
 
-If yes, tell the user to run `/sungen-make-tc ${input:screen}`.
+If yes → **tell the user to run `/sungen-make-tc ${input:screen}` as a separate command.** Do NOT attempt to generate test cases yourself in this session — the `make-tc` command auto-loads the `sungen-gherkin-syntax` and `sungen-tc-generation` skills which contain the full Gherkin syntax rules, pattern shapes, viewpoint checklists, and output format.
 
-### 3. Confirm
+### 4. Confirm
 
-Tell the user what was created and next steps:
-- If the page requires authentication, the user will be asked to log in via the MCP browser during `/sungen-make-tc`
+If the user declined test case creation, tell them next steps:
+- Fill `requirements/spec.md` with screen specs (if not done)
 - Run `/sungen-make-tc` to create test cases
 - Run `/sungen-make-test` to generate selectors, compile, and run tests

package/dist/orchestrator/templates/ai-instructions/copilot-cmd-make-tc.md

@@ -20,9 +20,17 @@ You are a **Senior QA Engineer**. You structure test cases by viewpoint categori
 
 1. Verify `qa/screens/${input:screen}/` exists. If not → run `/sungen-add-screen` first.
 2. Check if `.feature` already has scenarios. If yes → summarize existing coverage and ask: **1) Add new sections**, **2) Add viewpoints to existing sections**, or **3) Replace all**. See `sungen-tc-generation` skill for update mode details.
-3. Ask the user: "How should I explore the page? **1) Live page** (via Playwright MCP) or **2) Static designs** (screenshots, Figma)?". If live page, explore via #tool:playwright (see [copilot-instructions.md](../../copilot-instructions.md) for MCP rules). If auth needed, ask user to log in manually.
-4. Identify screen sections → ask user which to focus on (per `sungen-tc-generation` skill). Present sections as a numbered list and let user pick.
-5. Generate or update `.feature` + `test-data.yaml` following `sungen-gherkin-syntax` and `sungen-tc-generation` skills.
-6. Show summary → next: `/sungen-make-test ${input:screen}`
+3. **Read requirements** — check `qa/screens/${input:screen}/requirements/`:
+   - If `spec.md` exists → read it as PRIMARY source (sections, fields, validation rules, business rules, states).
+   - If `ui/` has images → read them for visual context (layout, element positions, states).
+   - If `notes.md` exists → read for edge cases and additional context.
+   - Summarize what you found in requirements and present to the user.
+4. **Explore page** (supplements requirements, or is primary source if no requirements):
+   - Ask: "How should I explore the page? **1) Live page** (via Playwright MCP) or **2) Static designs** (screenshots, Figma) or **3) Skip** (if requirements are sufficient)?"
+   - If live page, explore via #tool:playwright (see [copilot-instructions.md](../../copilot-instructions.md) for MCP rules). If auth needed, ask user to log in manually.
+   - If exploring, verify and supplement requirements — flag any discrepancies found.
+5. Identify screen sections → ask user which to focus on (per `sungen-tc-generation` skill). When requirements exist, use the "Requirements-Driven Generation" strategy. Present sections as a numbered list and let user pick.
+6. Generate or update `.feature` + `test-data.yaml` following `sungen-gherkin-syntax` and `sungen-tc-generation` skills.
+7. Show summary → next: `/sungen-make-test ${input:screen}`
 
 **No selectors.yaml** — selectors are generated during `/sungen-make-test`.

package/dist/orchestrator/templates/ai-instructions/copilot-cmd-make-test.md

@@ -19,9 +19,14 @@ You are a **Senior Developer** specialized in Playwright test debugging. You gen
 ## Steps
 
 1. Verify `qa/screens/${input:screen}/` has `.feature` + `test-data.yaml`. If not → run `/sungen-make-tc` first.
-2. Explore live page via #tool:playwright to generate `selectors.yaml` (per `sungen-selector-fix` skill).
-3. Compile with #tool:terminal: `sungen generate --screen ${input:screen}`
-4. Run with #tool:terminal: `npx playwright test specs/generated/${input:screen}/*.spec.ts`
-5. If fail → fix selectors/test-data per `sungen-selector-fix` + `sungen-error-mapping` skills, retry (max 5).
-6. After 5 fails → ask user about direct `.spec.ts` fix.
-7. Show: pass/fail, attempt count, files changed.
+2. **Generate selectors** — explore live page via #tool:playwright, build `selectors.yaml` (per `sungen-selector-fix` skill).
+3. **Proactive validation** — verify EVERY selector against the live page using `browser_snapshot` + `browser_evaluate` BEFORE running any test. Fix mismatches immediately. See `sungen-selector-fix` skill "Proactive Selector Validation" section. Target: 80%+ issues fixed before first run.
+4. **Compile** with #tool:terminal: `sungen generate --screen ${input:screen}`
+5. **Batched test run** — run tests in batches of 20 via `--grep`:
+   `npx playwright test specs/generated/${input:screen}/*.spec.ts --grep "VP-UI-001|...|VP-UI-020" --reporter=line`
+   - If failures in batch → group by root cause, fix, recompile, re-run only failing tests
+   - If batch passes → move to next 20 tests
+   - Max 5 fix attempts per batch
+6. **Final confirmation** — run ALL tests once to catch regressions.
+7. After 5 fix attempts still failing → ask user about direct `.spec.ts` fix.
+8. Show: pass/fail, attempt count, files changed.

package/dist/orchestrator/templates/ai-instructions/github-skill-sungen-error-mapping.md

@@ -6,16 +6,29 @@ user-invocable: false
 
 ## Playwright Errors → Fix
 
-| Error | Fix in `selectors.yaml` |
+| Error | Fix |
 |---|---|
-| strict mode violation | add `nth`, `exact: true`, or specific `name` |
-| Element is not an input | change `type` or `value` |
-| Timeout waiting for selector | fix `type`/`value`/`name` to match element |
-| toBeVisible Timeout | verify accessible name or use `testid` |
-| toHaveText mismatch | fix in `test-data.yaml` |
-| Navigation failed | fix page `value` path |
-| not a select | set `variant: 'custom'` |
-| Frame not found | fix `frame` value |
+| strict mode violation | add `nth`, `exact: true`, or specific `name` in selectors.yaml |
+| Element is not an input | change `type` or `value` in selectors.yaml |
+| Timeout waiting for selector | fix `type`/`value`/`name` to match element in selectors.yaml |
+| toBeVisible Timeout | verify accessible name or use `testid` in selectors.yaml |
+| toHaveText mismatch | fix expected text in `test-data.yaml` — OR if element is an input (field/textarea/search/dropdown), change Gherkin type so compiler uses `toHaveValue` instead |
+| toHaveValue mismatch | fix expected value in `test-data.yaml` — this is for input elements (field, textarea, search, dropdown, slider, date-picker) |
+| toContainText mismatch | fix expected partial text in `test-data.yaml` |
+| Navigation failed | fix page `value` path in selectors.yaml |
+| not a select | set `variant: 'custom'` in selectors.yaml |
+| Frame not found | fix `frame` value in selectors.yaml |
+| dialog was dismissed | alert step is AFTER the trigger — move `click [OK] alert` BEFORE the button click that opens the dialog |
+| page.once('dialog') never fired | no browser dialog appeared — check if the action actually triggers a `window.alert/confirm/prompt`, not a CSS modal |
+
+### Assertion type mismatch (toHaveText vs toHaveValue)
+
+Sungen picks the assertion based on element type:
+- **Input types** (`field`, `textarea`, `search`, `dropdown`, `slider`, `date-picker`) → `toHaveValue()`
+- **Text types** (everything else: `message`, `header`, `label`, `row`, etc.) → `toHaveText()`
+- **Partial match** (uses `contains` keyword) → `toContainText()`
+
+If you see `toHaveText` failing on an input element, the Gherkin step has the wrong target type. Fix: change the target type in the `.feature` file (e.g., `see [Email] message with {{v}}` → `see [Email] field with {{v}}`).
 
 ## Auth Errors → Fix