@sun-asterisk/sungen 2.3.0 → 2.4.0

package/dist/orchestrator/templates/ai-instructions/claude-skill-error-mapping.md

@@ -1,69 +1,145 @@
 ---
 name: sungen-error-mapping
-description: 'Playwright and Sungen error to fix mapping. Auto-loaded when running or debugging tests.'
+description: 'Error diagnosis and fix strategy — systematic debugging flow, error patterns, fix priorities. Auto-loaded during run-test.'
 user-invocable: false
 ---
 
-## Playwright Errors → Fix
+## Diagnosis Flow (before fixing anything)
 
-| Error | Fix |
-|---|---|
-| strict mode violation | add `nth`, `exact: true`, or specific `name` in selectors.yaml |
-| Element is not an input | change `type` or `value` in selectors.yaml |
-| Timeout waiting for selector | fix `type`/`value`/`name` to match element in selectors.yaml |
-| toBeVisible Timeout | verify accessible name or use `testid` in selectors.yaml |
-| toHaveText mismatch | fix expected text in `test-data.yaml` — OR if element is an input (field/textarea/search/dropdown), change Gherkin type so compiler uses `toHaveValue` instead |
-| toHaveValue mismatch | fix expected value in `test-data.yaml` — this is for input elements (field, textarea, search, dropdown, slider, date-picker) |
-| toContainText mismatch | fix expected partial text in `test-data.yaml` |
-| Navigation failed | fix page `value` path in selectors.yaml |
-| not a select | set `variant: 'custom'` in selectors.yaml |
-| Frame not found | fix `frame` value in selectors.yaml |
-| dialog was dismissed | alert step is AFTER the trigger — move `click [OK] alert` BEFORE the button click that opens the dialog |
-| page.once('dialog') never fired | no browser dialog appeared — check if the action actually triggers a `window.alert/confirm/prompt`, not a CSS modal |
-
-### Assertion type mismatch (toHaveText vs toHaveValue)
-
-Sungen picks the assertion based on element type:
-- **Input types** (`field`, `textarea`, `search`, `dropdown`, `slider`, `date-picker`) → `toHaveValue()`
-- **Text types** (everything else: `message`, `header`, `label`, `row`, etc.) → `toHaveText()`
-- **Partial match** (uses `contains` keyword) → `toContainText()`
-
-If you see `toHaveText` failing on an input element, the Gherkin step has the wrong target type. Fix: change the target type in the `.feature` file (e.g., `see [Email] message with {{v}}` → `see [Email] field with {{v}}`).
-
-## Auth Errors → Fix
+**Step 1: Read the error** — extract: error type, element name, expected vs actual, file:line.
 
-| Symptom | Fix |
-|---|---|
-| Tests redirect to login page | `specs/.auth/<role>.json` missing or expired. Tell user: `sungen makeauth <role> --url <baseURL>` |
-| `storageState` file not found | Run `sungen makeauth <role> --url <baseURL>` to create auth state |
-| Most tests timeout on first step | Auth expired — re-run `sungen makeauth <role> --url <baseURL>` |
-| Page shows home/login instead of target | SPA routing + expired auth. Re-run `sungen makeauth`, add `wait for` step after navigation |
+**Step 2: Read error context** — check `test-results/` for the snapshot at failure time. This shows the exact page state when the test failed — more reliable than re-navigating with MCP.
+
+**Step 3: Compare** — match the failing selector against the snapshot. Ask:
+- Does the element exist on the page?
+- Does the accessible name match exactly?
+- Are there multiple matches (strict mode)?
+- Is the element inside an iframe or dialog?
+- Is the page in the expected state (correct URL, loaded)?
+
+Then choose the fix from the patterns below.
+
+---
+
+## Fix Priority (try in order)
+
+1. **Auth issue** — page redirected to login? Fix auth first, everything else is noise
+2. **Element not found** — wrong name/type/value in selectors.yaml. Re-snapshot, copy exact name
+3. **Multiple matches** — add `nth`, `exact: true`, or `scope` to narrow down
+4. **Wrong assertion** — `toHaveText` vs `toHaveValue` mismatch, wrong expected data
+5. **Timing** — SPA not loaded, async content. Add `wait for` step in .feature
+
+---
+
+## Playwright Errors
+
+### Selector errors → fix in `selectors.yaml`
+
+| Error | Diagnosis | Fix |
+|---|---|---|
+| strict mode violation | Multiple elements match | Add `nth: 0`, `exact: true`, or more specific `name` |
+| Timeout / not found | Element doesn't exist or name wrong | Re-snapshot → copy exact accessible name. Check iframe/dialog scope |
+| Element is not an input | Wrong element type targeted | Change `type` or `value` to match actual element |
+| not a select | Custom dropdown, not native `<select>` | Set `variant: 'custom'` |
+| Frame not found | iframe selector wrong or doesn't exist | Fix `frame` value, verify iframe in snapshot |
+
+### Assertion errors → fix in `test-data.yaml` or `.feature`
+
+| Error | Diagnosis | Fix |
+|---|---|---|
+| toHaveText mismatch | Expected text differs from actual | Fix value in test-data. If element is input type → change Gherkin type to `field`/`textarea` (triggers `toHaveValue` instead) |
+| toHaveValue mismatch | Expected value differs from actual | Fix value in test-data |
+| toContainText mismatch | Partial text not found | Fix expected partial text in test-data |
+| toBeVisible timeout | Element exists but hidden, or name wrong | Check: is element conditionally visible? Wrong name? Inside dialog? |
+| toHaveCount mismatch | Row count differs | Fix expected count in test-data. Verify: is table loaded? Filtered? |
 
-**Never run `sungen makeauth` yourself.** Always tell the user to run it manually.
+### Assertion type rule
 
-## Performance Errors → Fix (specs/base.ts)
+Sungen picks assertion based on element type:
+- **Input** (`field`, `textarea`, `search`, `dropdown`, `slider`) → `toHaveValue()`
+- **Text** (everything else: `message`, `heading`, `label`, `row`) → `toHaveText()`
+- **Partial** (`contains` keyword) → `toContainText()`
 
-All generated `.spec.ts` files import from `specs/base.ts`. This shared file handles context caching, navigation skip, and overlay cleanup.
+If `toHaveText` fails on an input → the Gherkin step has wrong target type. Fix: change type in `.feature`.
 
-| Symptom | Fix in `specs/base.ts` |
+---
+
+## Table-Specific Errors
+
+| Error | Diagnosis | Fix |
+|---|---|---|
+| `tableRow is not defined` | Column assertion without preceding row scope step | Add `User see [Ref] row in [Table] table with {{v}}` before `User see [Col] column with {{v}}` |
+| `toHaveText` on cell fails (with columns) | Wrong column index in `columns` config | Re-count columns in snapshot (0-indexed). Fix `index` in selectors.yaml |
+| `toBeVisible` on cell fails (no columns) | `filter({ hasText })` didn't match | Check exact cell text in snapshot. Fix value in test-data |
+| Row filter matches 0 rows | Filter text doesn't match any row content | Re-snapshot → find actual row text. Fix filter value in test-data |
+| Row filter matches multiple rows | Filter text is too generic (matches multiple rows) | Use more specific filter text (unique identifier like email, ID) |
+| Table not found | Wrong table name or table not rendered | Re-snapshot → copy exact table accessible name |
+
+---
+
+## Auth Errors
+
+| Symptom | Fix |
 |---|---|
-| Server returns 429 (rate limited) | Context caching broken — check `contextCache` Map reuses sessions per `storageState` |
-| Tests slow even with `--workers=1` | Navigation skip not working — check `goto` patch compares `currentPath === url` |
-| Previous test's modal blocks next test | Overlay cleanup insufficient — improve Escape/click dismiss logic |
-| All tests fail on first navigation | `page.url()` is `about:blank` — check try/catch in goto patch |
-| Multiple session invalidation errors | Too many contexts created — ensure `contextCache.get(cacheKey)` returns existing context |
+| Redirect to login page | Auth expired. Tell user: `sungen makeauth <role> --url <baseURL>` |
+| `storageState` file not found | Run `sungen makeauth <role> --url <baseURL>` |
+| Most tests timeout on first step | Auth expired — re-run makeauth |
+| Page shows home instead of target | SPA + expired auth. Re-run makeauth + add `wait for` step |
 
-**Rules:**
-- `base.ts` changes affect ALL tests — edit carefully
-- Small project-specific tweaks are OK (e.g., custom overlay dismiss, timeout tuning, extra wait logic)
-- Do NOT move individual selector/data fixes into `base.ts` — those belong in `selectors.yaml` / `test-data.yaml`
-- If unsure whether a fix belongs in `base.ts` or selectors, ask the user
+**Never run `sungen makeauth` yourself.** Tell the user.
 
-## Sungen Errors → Fix
+---
+
+## Sungen Compile Errors
 
 | Error | Fix |
 |---|---|
-| Unknown step pattern | rewrite `.feature` step to match supported pattern |
-| Missing selector | add key to `selectors.yaml` |
-| Missing variable | add key to `test-data.yaml` |
-| Invalid selector type | use: role/testid/placeholder/label/text/locator/page/upload/frame |
+| Unknown step pattern | Rewrite step to match `sungen-gherkin-syntax` patterns |
+| Missing selector | Add key to `selectors.yaml` |
+| Missing variable | Add key to `test-data.yaml` |
+| Invalid selector type | Use: role/testid/placeholder/label/text/locator/page/upload/frame |
+
+---
+
+## Performance & Infrastructure Errors → Fix in `specs/base.ts`
+
+All generated `.spec.ts` import from `specs/base.ts` — shared context caching, navigation, overlay cleanup. AI **can and should** tune `base.ts` to match the project.
+
+| Symptom | Root cause | Fix |
+|---|---|---|
+| Server 429 (rate limited) | Too many browser contexts | Fix `contextCache` to reuse sessions per `storageState` |
+| Tests slow with `--workers=1` | Redundant navigation | Fix goto patch: skip if `currentPath === url` |
+| Previous test's modal blocks next | Overlay not cleaned up | Add/improve Escape + backdrop click in cleanup hook |
+| All tests fail on first navigation | `page.url()` is `about:blank` | Add try/catch in goto patch |
+| Flaky timeouts on SPA pages | Default timeout too short for app | Increase `actionTimeout` / `navigationTimeout` |
+| Tests pass individually but fail in batch | Shared state leaking between tests | Isolate context per test or reset state in `beforeEach` |
+
+### What AI CAN fix in `base.ts`
+
+- Timeout tuning (increase for slow APIs, decrease for fast apps)
+- Custom overlay/modal dismiss logic (project has unique close patterns)
+- Navigation wait strategy (`networkidle` vs `domcontentloaded` vs custom)
+- Context caching strategy (per-role, per-session)
+- Cookie/localStorage cleanup between tests
+
+### What AI should NOT move into `base.ts`
+
+- Individual selector fixes → belong in `selectors.yaml`
+- Test data values → belong in `test-data.yaml`
+- Single-test workarounds → belong in the `.spec.ts` directly
+
+---
+
+## Smart Fix Strategy
+
+### When multiple tests fail on the same element
+Fix the **root cause once** — don't fix each test individually. Group by selector key, fix selectors.yaml, recompile, re-run batch.
+
+### When you're unsure about the fix
+**Re-snapshot** — `browser_navigate` to the failing page, take `browser_snapshot`, compare the live state against what the test expects. This is faster than guessing.
+
+### When the fix might break other tests
+After fixing selectors.yaml, run ALL tests (not just the failing one) to catch regressions from renamed/changed selectors.
+
+### When nothing works after 3 attempts
+Ask the user. The issue might be: dynamic content, race condition, environment-specific state, or a real application bug.

package/dist/orchestrator/templates/ai-instructions/claude-skill-gherkin-syntax.md

@@ -33,7 +33,6 @@ AND    →  inherits from preceding keyword
 User is on [T] page                              # navigate to page
 User is on [T] page with {{v}}                   # navigate with data (e.g., detail page with ID)
 User is on [T] dialog                            # enter dialog scope
-User is logged in | is not logged in              # auth state
 ```
 
 ### Form
@@ -89,8 +88,8 @@ User see [message text] alert                    # assert dialog message
 ### Keyboard
 
 ```
-User press [Escape] key                          # global key press
-User press [Enter] on [T] field                  # key on element
+User press Escape key                            # global key press
+User press Enter on [T] field                    # key on element
 ```
 
 ### Wait
@@ -155,29 +154,16 @@ User see [T] page                                # URL assertion
 ### Table
 
 ```
-User see [Col] column in [Table] table           # column exists (parent scoping)
-User see [Table] table row with {{f}}            # row exists
-User see [Table] table has no row with {{f}}     # row not exists
-User see [Table] table with {{count}} rows       # row count
-User see [Table] table is empty                  # empty table
-User see [Table] table row with {{f}} has [Col] with {{v}}  # cell by filter
-User see [Table] table row 1 [Col] cell with {{v}}          # cell by index
-User click [Act] in [Table] table row with {{f}}             # action in row
+User see [Col] column in [Table] table                          # column exists
+User see [Ref] row in [Table] table with {{v}}                  # row exists (enters row scope)
+User see [Ref] row in [Table] table with {{v}} is hidden        # row hidden
+User see [Table] table with {{count}}                           # row count
+User see [Table] table is empty                                 # empty table
+User see [Col] column with {{v}}                                # cell value (row scoped)
+User click [Act] button in [Table] table with {{v}}             # action in row
 ```
 
-### Parent Scoping (disambiguation)
-
-```
-User click [Submit] button in [User Info] form           # button inside specific form
-User fill [Email] field in [Registration] form with {{v}} # field inside specific form
-User see [Total] text in [Summary] section with {{v}}    # text inside specific section
-User click [Delete] button in [Active Users] table       # button inside specific table
-```
-
-- **Optional** — only use when page has 2+ similar UI blocks
-- **Valid parent types**: `table`, `list`, `section`, `dialog`, `form`
-- **Max 2 levels**: `[Target] in [Parent]`. **NEVER** nest 3 levels: `[A] in [B] in [C]`
-- Parent resolves from selectors YAML first, falls back to auto-infer `getByRole(parentType, { name })`
+Row scope: `see [Ref] row in [Table] table with {{v}}` defines `tableRow`. Subsequent `see [Col] column with {{v}}` checks the cell in that row. With `columns` config → exact `nth(index)`, without → `filter({ hasText })`.
 
 ### States
 
@@ -196,24 +182,7 @@ User click [Delete] button in [Active Users] table       # button inside specifi
 
 ### Auto-infer (no YAML entry needed)
 
-| Gherkin | Playwright locator |
-|---|---|
-| `[Submit] button` | `getByRole('button', { name: 'Submit' })` |
-| `[Home] link` | `getByRole('link', { name: 'Home' })` |
-| `[Welcome] heading` / `header` | `getByRole('heading', { name: 'Welcome' })` |
-| `[Email] field` | `getByPlaceholder('Email')` |
-| `[Success] text` / `message` / `label` | `getByText('Success')` |
-| `[Terms] checkbox` | `getByRole('checkbox', { name: 'Terms' })` |
-| `[Male] radio` | `getByRole('radio', { name: 'Male' })` |
-| `[Global] search` | `getByRole('searchbox', { name: 'Global' })` |
-| `[Vietnam] option` | `getByRole('option', { name: 'Vietnam' })` |
-| `[Price] slider` | `getByRole('slider', { name: 'Price' })` |
-| `[Notification] toggle` | `getByRole('switch', { name: 'Notification' })` |
-| `[Profile] tab` | `getByRole('tab', { name: 'Profile' })` |
-| `[Orders] table` | `getByRole('table', { name: 'Orders' })` |
-| `[Products] list` | `getByRole('list', { name: 'Products' })` |
-| `[Name] column` | `getByRole('columnheader', { name: 'Name' })` |
-| `[Confirm] dialog` / `modal` / `drawer` | `getByRole('dialog', { name: 'Confirm' })` |
+Most elements auto-infer from `[Label] type` → `getByRole(type, { name: 'Label' })`. Only add YAML when the accessible name differs, needs `nth`, or needs `testid`. Full auto-infer table → see `sungen-selector-keys` skill.
 
 ## YAML Keys
 
@@ -265,7 +234,7 @@ Options: `nth` `exact` `scope` `match` `variant` `frame` `contenteditable` `colu
 |---|---|---|
 | Wrong keyword | `Given User click [T] button` | `When User click [T] button` |
 | Wrong action for type | `When User click [T] checkbox` | `When User check [T] checkbox` |
-| press wrong target | `When User press [Submit] button` | `When User press [Enter] key` |
+| press wrong target | `When User press [Submit] button` | `When User press Enter key` |
 | uncheck radio | `When User uncheck [Male] radio` | `When User check [Female] radio` |
 | Hardcode data | `with {{admin@mail.com}}` | `with {{invalid_email}}` |
 | Missing `is` for state | `with {{text}} hidden` | `with {{text}} is hidden` |