@sun-asterisk/sungen 3.2.2-beta.9 → 3.2.3-beta.3

package/src/orchestrator/templates/ai-instructions/github-skill-sungen-delivery.md

@@ -1,141 +0,0 @@
----
-name: sungen-delivery
-description: 'Export Gherkin scenarios + Playwright results → CSV test case deliverable. Auto-loaded by delivery command.'
-user-invocable: false
----
-
-## Purpose
-
-Export test cases from Sungen screens and flows to a standardized CSV file (format BM-2-901-13) for QA delivery.
-
-**This skill delegates all heavy work to the `sungen delivery` CLI.** The CLI is the single source of truth for parsing logic — do NOT re-parse files in AI. Your role is only to:
-
-1. Invoke the CLI
-2. Show its output verbatim
-3. Help the user react to pre-flight failures
-
----
-
-## Architecture
-
-```
-User → /sungen:delivery [name...]
-       │
-       ▼
-  sungen delivery CLI  (deterministic — no AI tokens)
-  ├─ Scope detection (no-arg = all screens + flows)
-  ├─ Auto-detect: qa/flows/<name>/ → flow, qa/screens/<name>/ → screen
-  ├─ Pre-flight source checks per target
-  ├─ Parse .feature (metadata)
-  ├─ Parse .spec.ts (resolved Playwright code — source of truth)
-  ├─ Parse test-data.yaml (resolve {{vars}})
-  ├─ Parse test-results/results.json (match test titles)
-  ├─ Merge sources + generate CSV rows
-  └─ Write qa/deliverables/<name>-testcases.csv
-```
-
-Source modules: `src/exporters/*.ts`
-
----
-
-## Required sources (CLI pre-flight checks these)
-
-| # | Source | Screen path | Flow path | Created by |
-|---|--------|-------------|-----------|------------|
-| 1 | Feature file | `qa/screens/<name>/features/<name>.feature` | `qa/flows/<name>/features/<name>.feature` | `add-screen`/`add-flow` + `create-test` |
-| 2 | Test data | `qa/screens/<name>/test-data/<name>.yaml` | `qa/flows/<name>/test-data/<name>.yaml` | `create-test` |
-| 3 | Selectors | `qa/screens/<name>/selectors/<name>.yaml` | `qa/flows/<name>/selectors/<name>.yaml` | `run-test` |
-| 4 | Compiled spec | `specs/generated/<name>/<name>.spec.ts` | `specs/generated/flows/<name>/<name>.spec.ts` | `sungen generate` |
-| 5 | Test results | `specs/generated/<name>/<name>-test-result.json` or `test-results/results.json` | `specs/generated/flows/<name>/<name>-test-result.json` or global fallback | `run-test` |
-
-**Sources 1-4 are blocking** — CLI aborts if any is missing.
-**Source 5 is optional** — CSV is still generated but Test Result/Date/Executor/Env columns are empty (all tests show as Pending).
-
-The CLI reads the **per-target result file first** (co-located with `.spec.ts`), then falls back to the global `test-results/results.json`. Per-target is preferred because the global file gets OVERWRITTEN each time Playwright runs, losing results from earlier targets.
-
----
-
-## Column mapping (handled by CLI)
-
-| CSV Column | Source |
-|------------|--------|
-| TC ID | Generated, namespaced per screen/flow: `<SCREEN_UPPER>-<CAT>-<NNN>` (e.g. `VP-SEC-001` on screen `login` → `LOGIN-SEC-001`). The namespace makes it globally unique — the stable key the dashboard tracks each test case by. |
-| Category 1 | Scenario name with VP prefix stripped |
-| Category 2 | VP group: `VP-SEC`→Accessing, `VP-UI`→GUI, `VP-VAL`/`VP-LOGIC`→Function |
-| Category 3 | Feature name (first line of `.feature`) |
-| Category 4 | Screen name |
-| Pre-condition | Auth tag → "Logged in as X" / "Not authenticated" + Given steps (natural language) |
-| Test Data | `{{vars}}` from scenario resolved via test-data.yaml → `key: value; key2: value2` |
-| Steps | `.spec.ts` code comments for interactions (numbered) |
-| Expected results | `.spec.ts` `expect(...)` comments (numbered) |
-| Priority | Tag: `@high`/`@normal`/`@low` (default: Normal) |
-| Testcase type | `@manual` → Manual, else Auto. Not compiled → "Not compiled" |
-| Test Result | results.json status: passed→Passed, failed/timedOut→Failed, skipped→N/A, else Pending |
-| Executed Date | results.json startTime formatted as `dd/mm/yyyy` |
-| Test Executor | `git config user.name` |
-| Test Environment | `playwright.config.ts` baseURL + project name |
-| Note | Error message + trace path (for failed tests) |
-
----
-
-## XLSX sheets — Auto / Manual split
-
-The `.xlsx` is split into two sheets so QA manages the sets separately:
-- **`Auto`** — automatable test cases (`Auto` + `Not compiled`).
-- **`Manual`** — `@manual` test cases (always present, header-only when there are none).
-
-Multi-locale (no `SUNGEN_ENV`): one **`<LOCALE> Auto`** sheet per locale + a single shared **`Manual`** sheet (manual TCs are locale-invariant). The **CSV stays one file with every row** — the `Testcase type` column distinguishes Auto vs Manual.
-
----
-
-## API delivery — extra worksheet
-
-For **api-kind units** (`qa/api/<area>/`), the `.xlsx` gains a third worksheet **`API detail`** (appended after Auto/Manual). The main BM-2-901-13 Testcases layout is unchanged. The CSV is unchanged (16-column, no extra sheet).
-
-### Required sources (API detail sheet only)
-
-| Source | Path | Created by |
-|--------|------|------------|
-| Endpoint catalog | `qa/api/<area>/api/apis.yaml` | `sungen add --api` or `sungen api import` |
-| Scenario annotations | `qa/api/<area>/features/<feature>.feature` | `create-test` |
-
-### API detail column mapping
-
-| Column | Source |
-|--------|--------|
-| Endpoint | `path` from `apis.yaml` catalog entry |
-| Method | `method` from catalog entry (uppercased) |
-| Auth / Datasource | catalog `datasource` + any `@auth:<role>` tag from scenarios calling this endpoint |
-| Request shape | catalog `body` + `params` fields composed as `body: {…}; params: [a, b]` |
-| Expected-status matrix | `@cases:<dataset>` label for data-driven scenarios; catalog `expect.status` as fallback |
-| Flow steps | Ordered `@api:<name>` call chain from multi-call scenarios (e.g. `register → count_users`) |
-| Concurrency invariant | `@concurrent:<N>` + `@query:<oracle>` from concurrent scenarios (e.g. `ok_count=2; @query user_count`) |
-
-**Sources are catalog + annotations only** — Field Metadata (FM) is not required for this sheet.
-
----
-
-## Excluded from CSV
-
-- `@steps:<name>` **base** scenarios — these are setup-only, inlined into `@extend:...` scenarios at compile time
-- Default scaffold `Sample scenario for <screen>` — not a real test
-
----
-
-## CLI command reference
-
-```bash
-# Export all screens and flows
-sungen delivery
-
-# Export specific targets (auto-detects screen vs flow)
-sungen delivery kudos awards nomination-flow
-
-# Skip pre-flight (CI only)
-sungen delivery --skip-preflight
-
-# Skip targets with blocking misses
-sungen delivery --continue-on-missing
-```
-
-Output: `qa/deliverables/<name>-testcases.csv` (UTF-8 with BOM)

package/src/orchestrator/templates/ai-instructions/github-skill-sungen-error-mapping.md

@@ -1,176 +0,0 @@
----
-name: sungen-error-mapping
-description: 'Error diagnosis and fix strategy — systematic debugging flow, error patterns, fix priorities. Auto-loaded during run-test.'
-user-invocable: false
----
-
-## Diagnosis Flow (before fixing anything)
-
-**Step 1: Read the error** — extract: error type, element name, expected vs actual, file:line.
-
-**Step 2: Read error context** — check `test-results/` for the snapshot at failure time. This shows the exact page state when the test failed — more reliable than re-navigating with MCP.
-
-**Step 3: Compare** — match the failing selector against the snapshot. Ask:
-- Does the element exist on the page?
-- Does the accessible name match exactly?
-- Are there multiple matches (strict mode)?
-- Is the element inside an iframe or dialog?
-- Is the page in the expected state (correct URL, loaded)?
-
-Then choose the fix from the patterns below.
-
----
-
-## ⛔ Source of truth — classify EVERY failure before you "fix" it
-
-`.feature` + `test-data.yaml` + `spec.md` are the **oracle**. The **live page is NOT** — it may be the thing that's broken. A failing test is not automatically a test to "make pass". Classify first:
-
-- **Selector-resolution failure** (element not found / wrong locator / strict-mode / wrong element type) → the test looked in the wrong place. **Fix the locator in `selectors.yaml`** (re-snapshot, copy the exact accessible name). Legit auto-fix.
-- **Assertion-value failure** (element FOUND, but observed value ≠ expected) → STOP and ask: *is the TEST wrong, or is the APP wrong?*
-  - Expected value/rule is wrong **relative to `spec.md`** (typo, stale test-data) → fix `test-data.yaml`/`.feature` so it matches the **spec** — never the live page.
-  - App behaviour contradicts `spec.md` (spec says X, app shows Y) → **CANDIDATE BUG**. **Report it** (let the test FAIL / surface to the QA in the run summary). **NEVER** change the expected value, loosen the rule, weaken the assertion (`toHaveText`→`toContainText` to dodge a mismatch), edit `.feature`, or edit the generated `.spec.ts` to make it pass.
-
-> **Cardinal sin (do NOT do this):** a `password > 8 chars` rule fails on a 6-char input → "fix" it to `>= 6` so the test passes. The logic is now meaningless. A failing assertion is a **finding**, not a chore.
-
-**Auto-fix loop scope:** the run-test auto-fix loop engages ONLY on **selector-resolution** failures. On an assertion-value failure where the app contradicts the spec → **HALT and report**, do not loop it into passing.
-
-**Never hand-edit the generated `.spec.ts`** (e.g. inserting `page.evaluate`/`fetch` to bypass a broken control). `sungen script-check` regenerates the spec from `.feature` and flags any edit as DRIFT — regenerate, don't patch.
-
----
-
-## Fix Priority (try in order)
-
-1. **Auth issue** — page redirected to login? Fix auth first, everything else is noise
-2. **Element not found** — wrong name/type/value in selectors.yaml. Re-snapshot, copy exact name
-3. **Multiple matches** — add `nth`, `exact: true`, or `scope` to narrow down
-4. **Wrong assertion** — `toHaveText` vs `toHaveValue` mismatch, wrong expected data
-5. **Timing** — SPA not loaded, async content. Add `wait for` step in .feature
-
----
-
-## Playwright Errors
-
-### Selector errors → fix in `selectors.yaml`
-
-| Error | Diagnosis | Fix |
-|---|---|---|
-| strict mode violation | Multiple elements match | Add `nth: 0`, `exact: true`, or more specific `name` |
-| Timeout / not found | Element doesn't exist or name wrong | Re-snapshot → copy exact accessible name. Check iframe/dialog scope |
-| Element is not an input | Wrong element type targeted | Change `type` or `value` to match actual element |
-| not a select | Custom dropdown, not native `<select>` | Set `variant: 'custom'` |
-| Frame not found | iframe selector wrong or doesn't exist | Fix `frame` value, verify iframe in snapshot |
-
-### Assertion errors → apply the Source-of-truth gate above FIRST
-
-> The "Fix" column below applies **only when the expected value was wrong relative to `spec.md`** (a test defect). If the app's value contradicts the spec, the row is a **candidate bug → report it, do not edit the expected to match live**. Never weaken `toHaveText`→`toContainText` just to pass.
-
-| Error | Diagnosis | Fix (only if the TEST was wrong per spec) |
-|---|---|---|
-| toHaveText mismatch | Expected text differs from actual | If the test's expected was wrong per spec → fix value in test-data. If element is input type → change Gherkin type to `field`/`textarea` (triggers `toHaveValue`). If the app value contradicts spec → **report as bug**. |
-| toHaveValue mismatch | Expected value differs from actual | Fix value in test-data |
-| toContainText mismatch | Partial text not found | Fix expected partial text in test-data |
-| toBeVisible timeout | Element exists but hidden, or name wrong | Check: is element conditionally visible? Wrong name? Inside dialog? |
-| toHaveCount mismatch | Row count differs | Fix expected count in test-data. Verify: is table loaded? Filtered? |
-
-### Assertion type rule
-
-Sungen picks assertion based on element type:
-- **Input** (`field`, `textarea`, `search`, `dropdown`, `slider`) → `toHaveValue()`
-- **Text** (everything else: `message`, `heading`, `label`, `row`) → `toHaveText()`
-- **Partial** (`contains` keyword) → `toContainText()`
-
-If `toHaveText` fails on an input → the Gherkin step has wrong target type. Fix: change type in `.feature`.
-
----
-
-## Table-Specific Errors
-
-| Error | Diagnosis | Fix |
-|---|---|---|
-| `tableRow is not defined` | Column assertion without preceding row scope step | Add `User see [Ref] row in [Table] table with {{v}}` before `User see [Col] column with {{v}}` |
-| `toHaveText` on cell fails (with columns) | Wrong column index in `columns` config | Re-count columns in snapshot (0-indexed). Fix `index` in selectors.yaml |
-| `toBeVisible` on cell fails (no columns) | `filter({ hasText })` didn't match | Check exact cell text in snapshot. Fix value in test-data |
-| Row filter matches 0 rows | Filter text doesn't match any row content | Re-snapshot → find actual row text. Fix filter value in test-data |
-| Row filter matches multiple rows | Filter text is too generic (matches multiple rows) | Use more specific filter text (unique identifier like email, ID) |
-| Table not found | Wrong table name or table not rendered | Re-snapshot → copy exact table accessible name |
-
----
-
-## Auth Errors
-
-| Symptom | Fix |
-|---|---|
-| Redirect to login page | Auth expired. Ask user to log in manually via MCP browser |
-| `storageState` file not found | Ask user to log in manually via MCP browser, then save storage state |
-| Most tests timeout on first step | Auth expired — ask user to re-authenticate via MCP browser |
-| Page shows home instead of target | SPA + expired auth. Re-authenticate + add `wait for` step |
-
-**Never use `sungen makeauth`.** Always let the user log in manually via the MCP browser.
-
----
-
-## Sungen Compile Errors
-
-| Error | Fix |
-|---|---|
-| Unknown step pattern | Rewrite step to match `sungen-gherkin-syntax` patterns |
-| Missing selector | Add key to `selectors.yaml` |
-| Missing variable | Add key to `test-data.yaml` |
-| Invalid selector type | Use: role/testid/placeholder/label/text/locator/page/upload/frame |
-
----
-
-## Flow-Specific Errors
-
-| Error | Diagnosis | Fix |
-|---|---|---|
-| Navigation timeout between screens | Cross-screen transition takes too long or URL mismatch | Add explicit `wait for page` step between screen transitions in `.feature`. Verify target URL path |
-| Selector `"screen:element"` not found | Namespace key missing or wrong format | Ensure colon-namespaced key in `selectors.yaml` is **quoted**: `"login:submit":`. Check screen prefix matches `[Screen:Element]` ref in Gherkin |
-| Test data `screen.key` undefined | Phase namespace mismatch | Verify `test-data.yaml` uses dot-namespaced keys: `login.email`, `submission.nominee`. Keys must match `{{screen.key}}` refs in `.feature` |
-| State lost between screens | Auth/session expired during multi-screen flow | Ensure all screens in the flow share the same `@auth:role` tag. Check if the app invalidates sessions on navigation |
-| Duplicate selector key across screens | Two screens use same element name without namespace | Always use `[Screen:Element]` format in flow `.feature`. Selectors must use `"screen:element":` quoted keys |
-
----
-
-## Performance & Infrastructure Errors → Fix in `specs/base.ts`
-
-All generated `.spec.ts` import from `specs/base.ts` — shared context caching, navigation, overlay cleanup. AI **can and should** tune `base.ts` to match the project.
-
-| Symptom | Root cause | Fix |
-|---|---|---|
-| Server 429 (rate limited) | Too many browser contexts | Fix `contextCache` to reuse sessions per `storageState` |
-| Tests slow with `--workers=1` | Redundant navigation | Fix goto patch: skip if `currentPath === url` |
-| Previous test's modal blocks next | Overlay not cleaned up | Add/improve Escape + backdrop click in cleanup hook |
-| All tests fail on first navigation | `page.url()` is `about:blank` | Add try/catch in goto patch |
-| Flaky timeouts on SPA pages | Default timeout too short for app | Increase `actionTimeout` / `navigationTimeout` |
-| Tests pass individually but fail in batch | Shared state leaking between tests | Isolate context per test or reset state in `beforeEach` |
-
-### What AI CAN fix in `base.ts`
-
-- Timeout tuning (increase for slow APIs, decrease for fast apps)
-- Custom overlay/modal dismiss logic (project has unique close patterns)
-- Navigation wait strategy (`networkidle` vs `domcontentloaded` vs custom)
-- Context caching strategy (per-role, per-session)
-- Cookie/localStorage cleanup between tests
-
-### What AI should NOT move into `base.ts`
-
-- Individual selector fixes → belong in `selectors.yaml`
-- Test data values → belong in `test-data.yaml`
-- Single-test workarounds → belong in the `.spec.ts` directly
-
----
-
-## Smart Fix Strategy
-
-### When multiple tests fail on the same element
-Fix the **root cause once** — don't fix each test individually. Group by selector key, fix selectors.yaml, recompile, re-run batch.
-
-### When you're unsure about the fix
-**Re-snapshot** — `browser_navigate` to the failing page, take `browser_snapshot`, compare the live state against what the test expects. This is faster than guessing.
-
-### When the fix might break other tests
-After fixing selectors.yaml, run ALL tests (not just the failing one) to catch regressions from renamed/changed selectors.
-
-### When nothing works after 3 attempts
-Ask the user. The issue might be: dynamic content, race condition, environment-specific state, or a real application bug.