@sun-asterisk/sungen 2.7.0-beta.0 → 2.7.0-beta.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (28) hide show
  1. package/dist/orchestrator/project-initializer.d.ts +5 -0
  2. package/dist/orchestrator/project-initializer.d.ts.map +1 -1
  3. package/dist/orchestrator/project-initializer.js +16 -0
  4. package/dist/orchestrator/project-initializer.js.map +1 -1
  5. package/dist/orchestrator/templates/ai-instructions/claude-cmd-create-test.md +9 -1
  6. package/dist/orchestrator/templates/ai-instructions/claude-cmd-run-test.md +4 -2
  7. package/dist/orchestrator/templates/ai-instructions/claude-skill-selector-fix.md +57 -11
  8. package/dist/orchestrator/templates/ai-instructions/claude-skill-selector-keys.md +41 -31
  9. package/dist/orchestrator/templates/ai-instructions/claude-skill-tc-generation.md +13 -0
  10. package/dist/orchestrator/templates/ai-instructions/copilot-cmd-create-test.md +9 -1
  11. package/dist/orchestrator/templates/ai-instructions/copilot-cmd-run-test.md +4 -2
  12. package/dist/orchestrator/templates/ai-instructions/github-skill-sungen-selector-fix.md +61 -15
  13. package/dist/orchestrator/templates/ai-instructions/github-skill-sungen-selector-keys.md +41 -31
  14. package/dist/orchestrator/templates/qa-context.md +90 -0
  15. package/dist/orchestrator/templates/readme.md +16 -13
  16. package/package.json +1 -1
  17. package/src/orchestrator/project-initializer.ts +20 -0
  18. package/src/orchestrator/templates/ai-instructions/claude-cmd-create-test.md +9 -1
  19. package/src/orchestrator/templates/ai-instructions/claude-cmd-run-test.md +4 -2
  20. package/src/orchestrator/templates/ai-instructions/claude-skill-selector-fix.md +57 -11
  21. package/src/orchestrator/templates/ai-instructions/claude-skill-selector-keys.md +41 -31
  22. package/src/orchestrator/templates/ai-instructions/claude-skill-tc-generation.md +13 -0
  23. package/src/orchestrator/templates/ai-instructions/copilot-cmd-create-test.md +9 -1
  24. package/src/orchestrator/templates/ai-instructions/copilot-cmd-run-test.md +4 -2
  25. package/src/orchestrator/templates/ai-instructions/github-skill-sungen-selector-fix.md +61 -15
  26. package/src/orchestrator/templates/ai-instructions/github-skill-sungen-selector-keys.md +41 -31
  27. package/src/orchestrator/templates/qa-context.md +90 -0
  28. package/src/orchestrator/templates/readme.md +16 -13
@@ -166,37 +166,47 @@ Resolver searches in this order:
166
166
 
167
167
  If no YAML key exists, the resolver infers from the Gherkin element type:
168
168
 
169
- | Gherkin | Inferred locator |
170
- |---|---|
171
- | `[X] button` | `getByRole('button', { name: 'X' })` |
172
- | `[X] link` | `getByRole('link', { name: 'X' })` |
173
- | `[X] heading` / `header` | `getByRole('heading', { name: 'X' })` |
174
- | `[X] checkbox` | `getByRole('checkbox', { name: 'X' })` |
175
- | `[X] radio` | `getByRole('radio', { name: 'X' })` |
176
- | `[X] field` | `getByPlaceholder('X')` |
177
- | `[X] text` / `message` / `label` | `getByText('X')` |
178
- | `[X] logo/image/icon` | `getByRole('img', { name: 'X' })` |
179
- | `[X] search` | `getByRole('searchbox', { name: 'X' })` |
180
- | `[X] option` | `getByRole('option', { name: 'X' })` |
181
- | `[X] slider` | `getByRole('slider', { name: 'X' })` |
182
- | `[X] toggle` | `getByRole('switch', { name: 'X' })` |
183
- | `[X] tab` | `getByRole('tab', { name: 'X' })` |
184
- | `[X] table` | `getByRole('table', { name: 'X' })` |
185
- | `[X] list` | `getByRole('list', { name: 'X' })` |
186
- | `[X] column` | `getByRole('columnheader', { name: 'X' })` |
187
- | `[X] dialog` / `modal` / `drawer` | `getByRole('dialog', { name: 'X' })` |
188
- | `[X] dropdown` / `select` | `getByRole('combobox', { name: 'X' })` |
189
- | `[X] menuitem` | `getByRole('menuitem', { name: 'X' })` |
190
- | `[X] progressbar` | `getByRole('progressbar', { name: 'X' })` |
191
- | `[X] section` | `getByRole('region', { name: 'X' })` |
192
- | `[X] card` | `getByRole('article', { name: 'X' })` |
193
- | `[X] item` | `getByRole('listitem', { name: 'X' })` |
194
- | `[X] cell` | `getByRole('cell', { name: 'X' })` |
195
- | `[X] spinner` | `getByRole('status', { name: 'X' })` |
196
- | `[X] breadcrumb` | `getByRole('navigation', { name: 'X' })` |
197
- | `[X] badge` / `tooltip` / `tag` | `getByText('X')` |
198
-
199
- **Only add a YAML entry when** the auto-inferred locator won't work (wrong name, need testid, need nth, etc.).
169
+ > ⚠️ **Auto-infer pitfall the #1 cause of selector failures in production.**
170
+ >
171
+ > `[X] button` auto-infers as `getByRole('button', { name: 'X' })`. This **only works** when the button's accessible name in the DOM is **exactly `X`** — same language, same text, same casing.
172
+ >
173
+ > The Gherkin `[Reference]` is your human label for the element, **not** the DOM name. If the app is in Vietnamese (or any language where the Gherkin label differs from DOM text), auto-infer will produce `No element found` at runtime. **Write an explicit YAML entry** with the real DOM name instead.
174
+ >
175
+ > **Decision rule**: auto-infer is safe ONLY when you have confirmed in the snapshot that the DOM element's accessible name / placeholder text is literally `X`. When in doubt → write YAML.
176
+
177
+ | Gherkin | Inferred locator | Safe when… |
178
+ |---|---|---|
179
+ | `[X] button` | `getByRole('button', { name: 'X' })` | Button's accessible name = X |
180
+ | `[X] link` | `getByRole('link', { name: 'X' })` | Link text = X |
181
+ | `[X] heading` / `header` | `getByRole('heading', { name: 'X' })` | Heading text = X |
182
+ | `[X] checkbox` | `getByRole('checkbox', { name: 'X' })` | Checkbox label = X |
183
+ | `[X] radio` | `getByRole('radio', { name: 'X' })` | Radio label = X |
184
+ | `[X] field` | `getByPlaceholder('X')` | Placeholder text = X AND field has a placeholder |
185
+ | `[X] text` / `message` / `label` | `getByText('X')` | Visible text = X (partial match) |
186
+ | `[X] logo/image/icon` | `getByRole('img', { name: 'X' })` | Image alt = X |
187
+ | `[X] search` | `getByRole('searchbox', { name: 'X' })` | Searchbox label = X |
188
+ | `[X] option` | `getByRole('option', { name: 'X' })` | Option text = X |
189
+ | `[X] slider` | `getByRole('slider', { name: 'X' })` | Slider label = X |
190
+ | `[X] toggle` | `getByRole('switch', { name: 'X' })` | Toggle label = X |
191
+ | `[X] tab` | `getByRole('tab', { name: 'X' })` | Tab text = X |
192
+ | `[X] table` | `getByRole('table', { name: 'X' })` | Table aria-label = X |
193
+ | `[X] list` | `getByRole('list', { name: 'X' })` | List aria-label = X |
194
+ | `[X] column` | `getByRole('columnheader', { name: 'X' })` | Column header text = X |
195
+ | `[X] dialog` / `modal` / `drawer` | `getByRole('dialog', { name: 'X' })` | Dialog aria-label/heading = X |
196
+ | `[X] dropdown` / `select` | `getByRole('combobox', { name: 'X' })` | Combobox label = X |
197
+ | `[X] menuitem` | `getByRole('menuitem', { name: 'X' })` | Menu item text = X |
198
+ | `[X] progressbar` | `getByRole('progressbar', { name: 'X' })` | Progressbar label = X |
199
+ | `[X] section` | `getByRole('region', { name: 'X' })` | Section aria-label = X |
200
+ | `[X] card` | `getByRole('article', { name: 'X' })` | Card aria-label = X |
201
+ | `[X] item` | `getByRole('listitem', { name: 'X' })` | List item text = X |
202
+ | `[X] cell` | `getByRole('cell', { name: 'X' })` | Cell text = X |
203
+ | `[X] spinner` | `getByRole('status', { name: 'X' })` | Spinner aria-label = X |
204
+ | `[X] breadcrumb` | `getByRole('navigation', { name: 'X' })` | Navigation aria-label = X |
205
+ | `[X] badge` / `tooltip` / `tag` | `getByText('X')` | Visible text = X |
206
+
207
+ **Special note on `[X] field`**: `getByPlaceholder('X')` only works when (1) the field has a placeholder attribute AND (2) the placeholder text equals X. For fields without placeholders (floating labels, aria-label), write explicit YAML: `type: label, value: "Actual label text"`.
208
+
209
+ **Only add a YAML entry when** auto-infer cannot work: DOM name differs from Gherkin label, need `testid`, need `nth`, need `exact: true`, or the field type requires explicit config.
200
210
 
201
211
  ### Types requiring YAML entry (no auto-infer)
202
212
 
@@ -0,0 +1,90 @@
1
+ # Project Context
2
+
3
+ > Read by the AI before generating test cases for any screen in this project.
4
+ > Fill in what applies — leave sections empty if not relevant.
5
+ > **The more specific you are, the more accurate the generated test cases.**
6
+
7
+ ---
8
+
9
+ ## Project Overview
10
+
11
+ **Application:**
12
+ <!-- One sentence: what does this app do? -->
13
+ <!-- Example: B2B award nomination platform for enterprise HR teams. -->
14
+
15
+ **Target users:**
16
+ <!-- Who uses this app and in what context? -->
17
+ <!-- Example: HR managers submit nominations; employees view results. -->
18
+
19
+ **Domain notes:**
20
+ <!-- Key terminology, conventions, or constraints the AI should know. -->
21
+ <!-- Example: "Nomination = an award record. Once submitted, status cannot revert to Draft." -->
22
+ <!-- Example: "All monetary values are in JPY. No decimal places." -->
23
+
24
+ ---
25
+
26
+ ## Auth Roles
27
+
28
+ > The AI maps these directly to `@auth:X` tags and generates permission-boundary test scenarios.
29
+ > Leave the table empty (or delete it) if the app has no auth system.
30
+
31
+ | Role | Can do | Cannot do |
32
+ |------|--------|-----------|
33
+ | | | |
34
+
35
+ <!--
36
+ Example:
37
+ | Role | Can do | Cannot do |
38
+ |---------|---------------------------------------------|--------------------------------------|
39
+ | admin | All CRUD, manage users, configure settings | Nothing blocked |
40
+ | manager | Create/edit records, view reports | Delete records, manage users |
41
+ | staff | View and submit own records only | Edit others' records, view reports |
42
+ -->
43
+
44
+ ---
45
+
46
+ ## Testing Strategy
47
+
48
+ **Focus areas** — what to cover thoroughly:
49
+ <!-- List from: functional, security, ui, accessibility, performance -->
50
+ <!-- Example: functional, security -->
51
+
52
+ **Mandatory coverage:**
53
+ <!-- Rules that override the AI's default tier decisions for every screen. -->
54
+ <!-- Example: "Every screen with admin-only actions MUST have a non-admin blocked-access scenario." -->
55
+ <!-- Example: "All free-text inputs MUST have XSS + SQL injection scenarios regardless of screen risk level." -->
56
+
57
+ **Deprioritize / skip:**
58
+ <!-- What to move to @low or skip entirely for this project. -->
59
+ <!-- Example: "Skip VP-UI cosmetic checks (label/placeholder presence) — handled separately by design review." -->
60
+ <!-- Example: "Skip accessibility scenarios — separate audit planned." -->
61
+
62
+ ---
63
+
64
+ ## Global Business Rules
65
+
66
+ > Rules that apply across multiple screens.
67
+ > The AI adds these to the Coverage Map for every screen as `[G]`-tagged Business rules.
68
+ > Screen-specific rules belong in `requirements/spec.md`, not here.
69
+
70
+ <!-- - Soft-delete only: records are never hard-deleted, only marked inactive -->
71
+ <!-- - All timestamps stored in UTC, displayed in UTC+7 -->
72
+ <!-- - Pagination default: 20 items per page; max 100 -->
73
+ <!-- - File uploads: PNG/JPG/PDF only, max 5 MB -->
74
+ <!-- - After any write operation, the list view must refresh automatically -->
75
+
76
+ ---
77
+
78
+ ## Error Message Patterns
79
+
80
+ > If your app follows consistent validation error formats, list them here.
81
+ > The AI uses these to fill `test-data.yaml` error keys when `spec.md` doesn't specify exact text.
82
+ > Leave empty to let the AI infer from spec.md.
83
+
84
+ - Required field: `<!-- "This field is required" -->`
85
+ - Max length: `<!-- "Must be X characters or less" -->`
86
+ - Min length: `<!-- "Must be at least X characters" -->`
87
+ - Invalid format: `<!-- "Invalid format" -->`
88
+ - Unique constraint: `<!-- "Already exists" -->`
89
+ - Not found: `<!-- "Not found" -->`
90
+ - Unauthorized: `<!-- "You do not have permission to perform this action" -->`
@@ -12,14 +12,16 @@ sungen generate → compiles Gherkin + selectors + data → Playwright .spec.ts
12
12
  ## Directory Structure
13
13
 
14
14
  ```
15
- ├── qa/screens/<name>/
16
- │ ├── features/ # .feature files (Gherkin)
17
- │ ├── selectors/ # Element locator YAML mappings
18
- │ ├── test-data/ # Test data YAML values
19
- └── requirements/ # Screen specs, UI designs, notes
20
- ├── spec.md # Structured screen specification
21
- ├── ui/ # Screenshots, mockups, design images
22
- └── test-viewpoint.md # Edge cases, decisions (optional)
15
+ ├── qa/
16
+ │ ├── context.md # Project-wide context: roles, testing strategy, global rules (fill once)
17
+ │ ├── screens/<name>/
18
+ ├── features/ # .feature files (Gherkin)
19
+ │ ├── selectors/ # Element locator YAML mappings
20
+ ├── test-data/ # Test data YAML values
21
+ │ └── requirements/ # Screen specs, UI designs, notes
22
+ ├── spec.md # Structured screen specification
23
+ │ │ ├── ui/ # Screenshots, mockups, design images
24
+ │ │ └── test-viewpoint.md # Edge cases, decisions (optional)
23
25
  ├── specs/
24
26
  │ └── generated/ # Auto-generated Playwright tests
25
27
  ├── .claude/
@@ -66,11 +68,12 @@ Scaffolds `qa/screens/<name>/` with empty feature, selectors, test-data, and req
66
68
  | `/sungen:create-test login` | `/sungen-create-test login` |
67
69
 
68
70
  AI acts as a **Senior QA Engineer**:
69
- 1. Reads `requirements/spec.md` for screen specs (fields, validation, business rules, states)
70
- 2. Optionally explores the live page via Playwright MCP to verify and supplement
71
- 3. Identifies screen sections asks user which to focus on
72
- 4. Generates **20+ scenarios per viewpoint** (UI/UX, Validation, Logic, Security) for each section
73
- 5. Confirms test plan before generating `.feature` + `test-data.yaml`
71
+ 1. Reads `qa/context.md` for project-wide context (roles, testing strategy, global rules)
72
+ 2. Reads `requirements/spec.md` for screen specs (fields, validation, business rules, states)
73
+ 3. Optionally explores the live page via Playwright MCP to verify and supplement
74
+ 4. Identifies screen sections asks user which to focus on
75
+ 5. Generates **20+ scenarios per viewpoint** (UI/UX, Validation, Logic, Security) for each section
76
+ 6. Confirms test plan before generating `.feature` + `test-data.yaml`
74
77
 
75
78
  ### Step 3: Compile & run tests
76
79
 
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@sun-asterisk/sungen",
3
- "version": "2.7.0-beta.0",
3
+ "version": "2.7.0-beta.1",
4
4
  "description": "Deterministic E2E Test Compiler - Gherkin + Selectors → Playwright tests",
5
5
  "main": "dist/index.js",
6
6
  "types": "dist/index.d.ts",
@@ -39,6 +39,9 @@ export class ProjectInitializer {
39
39
  // Create directories
40
40
  this.createDirectories();
41
41
 
42
+ // Create qa/context.md for QA lead to fill project-wide context
43
+ this.createContext();
44
+
42
45
  // Ensure package.json and install Playwright
43
46
  await this.setupDependencies();
44
47
 
@@ -363,6 +366,23 @@ export class ProjectInitializer {
363
366
 
364
367
  }
365
368
 
369
+ /**
370
+ * Create qa/context.md for the QA lead to fill project-wide context
371
+ * (roles, testing strategy, global rules, error patterns).
372
+ */
373
+ private createContext(): void {
374
+ const contextPath = path.join(this.cwd, 'qa', 'context.md');
375
+
376
+ if (fs.existsSync(contextPath)) {
377
+ this.skippedItems.push('qa/context.md');
378
+ return;
379
+ }
380
+
381
+ const content = this.readTemplate('qa-context.md');
382
+ fs.writeFileSync(contextPath, content, 'utf-8');
383
+ this.createdItems.push('qa/context.md');
384
+ }
385
+
366
386
  /**
367
387
  * Create specs/base.ts for shared browser context
368
388
  */
@@ -32,7 +32,15 @@ Parse **name** from `$ARGUMENTS`. If missing, ask the user.
32
32
  - If no → fresh creation. Use `AskUserQuestion` to ask generation scope:
33
33
  - **Tier 1 — Critical & High priority** — ~10-15 scenarios/section covering happy paths, core validation, security basics **(Recommended)**
34
34
  - **Full coverage — All tiers at once** — generates Tier 1 + 2 + 3 in one run. Large output (~40-60 scenarios/section), best for experienced users who want complete coverage immediately
35
- 3. **Read requirements & resolve visual source** — check `qa/<screens|flows>/<name>/requirements/`:
35
+ 3. **Read project context + screen requirements**
36
+
37
+ **Project context** — check `qa/context.md` (project root, not screen-specific):
38
+ - If exists → read it. Extract: roles, testing strategy directives, global business rules, error patterns.
39
+ - Summarize what you found in one line (e.g. `"Roles: admin/staff/user | Strategy: focus security, skip VP-UI T1 | 2 global rules"`).
40
+ - These are carried into the Coverage Map when invoking `sungen-tc-generation`.
41
+ - If absent → continue without it, no action needed.
42
+
43
+ **Screen requirements** — check `qa/<screens|flows>/<name>/requirements/`:
36
44
  - If `spec.md` exists → read it as PRIMARY source (sections, fields, validation rules, business rules, states).
37
45
  - If `test-viewpoint.md` exists → read it. If it only contains HTML comments (scaffold template), use `AskUserQuestion` to ask:
38
46
  - **Fill test-viewpoint.md first** — I'll help you identify edge cases, known issues, and design decisions for this screen before generating tests
@@ -41,11 +41,13 @@ Skip this pre-flight when `--env` matches the base locale (no overlay needed in
41
41
  Phase 0 — Selector Generation decision tree
42
42
 
43
43
  Live page reachable? (URL provided and loads without error)
44
- YES → existing flow: browser_navigate → one browser_snapshot generate selectors.yaml (verified entries)
44
+ YES → existing flow: browser_navigate → wait for page to fully load (no spinner/skeleton/empty table)
45
+ one browser_snapshot → cross-verify every [Reference] label vs snapshot name →
46
+ generate selectors.yaml (verified entries; explicit YAML for any label≠DOM-name mismatch)
45
47
  NO → spec_figma.md exists in requirements/?
46
48
  YES → provisional flow (sungen-figma-source + sungen-selector-fix skills):
47
49
  1. Read filtered Figma node data from spec_figma.md (## Components + ## Text Inventory)
48
- 2. Apply selector heuristics from sungen-figma-source skill (testid > role+name > placeholder > label > locator > text)
50
+ 2. Apply selector priority from sungen-selector-fix § Step 3 (testid > role+name > label > placeholder > text > locator CSS last)
49
51
  3. Write selectors.yaml — every provisional entry gets this comment on the line above:
50
52
  # @needs-live-verify source=figma node_id=<id>
51
53
  4. Compile: Screen: sungen generate --screen <name>. Flow: sungen generate --flow <name> — must succeed
@@ -55,13 +55,23 @@ When running Phase 0 for a **flow** (`qa/flows/<name>/`), check existing screen
55
55
  - Read `baseURL` from `playwright.config.ts`.
56
56
  - `browser_navigate` to the page URL.
57
57
  - If redirected to login → run **Phase 0.5: Auth Persistence** first (see below), then re-navigate to the target page.
58
- 5. **Snapshot**: take **ONE** `browser_snapshot`. All Phase 0 selectors come from this single snapshot.
58
+ 5. **Snapshot**: Wait for the page to fully load before snapshotting.
59
+ - Check if the page is still loading (spinner visible, skeleton placeholders, empty table with 0 rows). If so, use `browser_wait_for` to wait until content is rendered.
60
+ - Then take **ONE** `browser_snapshot`. All Phase 0 selectors come from this single snapshot.
59
61
  6. **Generate YAML entries**:
60
62
  - Keys: follow `sungen-selector-keys` (lowercase, Unicode preserved, `--type` / `--N` suffixes).
61
- - Selector priority: follow the table in **Diagnosis & Fix § Step 3** (`testid` > `role`+name > `placeholder` > `label` > `locator` > `text`).
63
+ - Selector priority: follow the table in **Diagnosis & Fix § Step 3** (`testid` > `role`+name > `label` > `placeholder` > `text` > `locator` CSS last resort).
62
64
  - Copy names **character-for-character** from the snapshot. Never infer from the Gherkin label.
63
65
  - If an element is auto-inferable per `sungen-selector-keys` § Auto-Infer, **omit it** from YAML — keep the file minimal.
64
66
  - **i18n sites**: if the site supports multiple languages, use `{{variable}}` in `name`/`value` fields instead of hardcoded text. Add corresponding `lbl_*` keys to `test-data.yaml` + locale overlay files (see `sungen-selector-keys` § i18n).
67
+ - **Selector quality rule**: the Playwright MCP accessibility tree snapshot gives you roles and accessible names directly — use them. Do NOT write XPath or class-based CSS selectors. Only write `type: locator` when no role/text/label/placeholder/testid is available, and restrict the CSS to `#id` or `[data-*]` / `[aria-*]` attribute selectors.
68
+ 6b. **Cross-verify Gherkin labels vs snapshot** (prevents the #1 production failure):
69
+ - For **every** `[Reference]` in the `.feature` that will rely on auto-infer (not written to YAML), check the snapshot:
70
+ - `[X] button` — is there a button with accessible name **exactly** `X`?
71
+ - `[X] field` — does an input have placeholder **exactly** `X`? Does it even have a placeholder?
72
+ - `[X] heading` / `text` / `message` — is that text literally visible in the snapshot?
73
+ - If any mismatch → write an explicit YAML entry using the real DOM name. Do not leave a mismatch to be caught at runtime.
74
+ - **Typical mismatch cases**: Gherkin uses English label (`[Submit]`) but app displays Vietnamese (`"Gửi"`); placeholder is descriptive (`"Nhập email của bạn"`) not a bare field name (`"Email"`); button text includes an icon glyph before/after the word.
65
75
  7. **Substring ambiguity check**: for each `role` + `name` selector, check if any other element in the snapshot has a name that **contains** this name as a substring (e.g., `"Đăng ký"` vs `"Đăng ký bằng Google"`). If yes → add `exact: true` to prevent strict mode violation at runtime.
66
76
  8. **Merge, don't overwrite**: preserve the page selector and any user-authored entries in `selectors.yaml`. Only add missing keys.
67
77
  9. **Show summary + confirm**: list the keys that will be added, ask the user to approve, then write the file.
@@ -69,9 +79,13 @@ When running Phase 0 for a **flow** (`qa/flows/<name>/`), check existing screen
69
79
 
70
80
  ### Common Phase 0 pitfalls
71
81
 
72
- - Writing keys inferred from the Gherkin label instead of the snapshot name → Phase 1 will fail with "no element found".
82
+ - Writing keys inferred from the Gherkin label instead of the snapshot name → Phase 1 will fail with `No element found`.
73
83
  - Skipping Phase 0.5 when an auth redirect happened → snapshot captures the login page, all selectors wrong.
84
+ - Taking snapshot while page is still loading (spinner visible, table empty) → selectors for dynamic content will be missing or wrong.
85
+ - Skipping step 6b for "simple" elements like buttons → silent mismatch between Gherkin label and DOM name fails at runtime.
74
86
  - Using `browser_evaluate` alone to scrape cookies → misses httpOnly session cookies. Always use `browser_storage_state` (or the `browser_run_code` fallback).
87
+ - Writing XPath or class-based CSS selectors → breaks on DOM/style refactoring. Use role/testid/text/label/placeholder from the accessibility tree.
88
+ - Falling back to `locator: 'div.some-class > span'` when the element IS visible in the accessibility snapshot with a role + name → the snapshot gives you `getByRole` for free; use it.
75
89
  - Overwriting user-authored selectors → always merge.
76
90
 
77
91
  ---
@@ -210,12 +224,24 @@ Selector priority (use first applicable):
210
224
 
211
225
  | Priority | type | When |
212
226
  |---|---|---|
213
- | 1 | `testid` | `data-testid` exists |
214
- | 2 | `role` + exact name | Interactive elements |
215
- | 3 | `placeholder` | Input with placeholder |
216
- | 4 | `label` | Form field with `<label>` |
217
- | 5 | `locator` (CSS) | No accessible name |
218
- | 6 | `text` | Static text only |
227
+ | 1 | `testid` | `data-testid` or any stable test attribute exists |
228
+ | 2 | `role` + exact name | Interactive elements with an accessible name |
229
+ | 3 | `label` | Form field with a visible `<label>` |
230
+ | 4 | `placeholder` | Input/textarea with a placeholder attribute |
231
+ | 5 | `text` | Static visible text content |
232
+ | 6 | `locator` (CSS) | Last resort — `#id` or `[attr=value]` **only** (see restrictions below) |
233
+
234
+ > ⚠️ **Playwright best practice** ([source](https://playwright.dev/docs/best-practices#use-locators)): user-facing locators (`role`, `label`, `text`, `placeholder`, `testid`) are resilient to refactoring and far less likely to break. CSS class selectors and XPath break whenever a developer renames a class or restructures the DOM — even without changing the UI.
235
+ >
236
+ > **Never write these in `selectors.yaml`**:
237
+ > - XPath: `xpath=//div[@class='...']` or `//button[contains(@class,'btn')]`
238
+ > - Class-based CSS: `div.btn-primary`, `.modal-footer > .submit-btn`
239
+ > - Deep structural CSS: `div:nth-child(3) > ul > li > button`
240
+ >
241
+ > **Acceptable CSS (last resort only)**:
242
+ > - Stable `id`: `#submit-button` (only if the id is truly stable and not dynamic)
243
+ > - Data attributes: `[data-id="123"]`, `[aria-controls="menu"]`
244
+ > - Input type: `input[type="file"]` (when no testid/label exists)
219
245
 
220
246
  **Exact name rule**: copy name character-for-character from snapshot. Never infer from Gherkin label.
221
247
 
@@ -229,9 +255,9 @@ Common fixes:
229
255
  - Name mismatch → copy exact name from snapshot
230
256
  - Multiple matches → add `nth` or `exact: true`
231
257
  - Substring ambiguity (e.g., `"Submit"` matches `"Submit"` and `"Submit & Continue"`) → add `exact: true`
232
- - No accessible name → use `testid` or `locator` (CSS)
258
+ - No accessible name → use `testid`; only fall back to `locator` CSS as last resort
233
259
  - Element in iframe → add `frame` field
234
- - Dynamic content → use `testid` or structural `role` + `nth`
260
+ - Dynamic content → use `testid` or `role` + `nth`
235
261
 
236
262
  ### Step 4: Recompile After Fix
237
263
 
@@ -248,6 +274,26 @@ Then re-run only the current phase's failing tests, not all tests.
248
274
 
249
275
  ---
250
276
 
277
+ ## Common Failure Patterns
278
+
279
+ Quick reference for the most frequent production failures:
280
+
281
+ | Symptom | Root cause | Fix |
282
+ |---------|-----------|-----|
283
+ | `No element found` on button/link/heading | Gherkin `[Reference]` label ≠ DOM accessible name (different language or text) | Write explicit YAML: `type: role, value: button, name: "<exact DOM name>"` |
284
+ | `No element found` on `[X] field` | Field has no placeholder, or placeholder ≠ X | Write explicit YAML: `type: label, value: "Actual label"` or `type: placeholder, value: "Actual placeholder"` |
285
+ | `No element found` on `[X] text` / `message` | Visible text differs from Gherkin label, or text is dynamic | Write explicit YAML or use `{{variable}}` for dynamic content |
286
+ | `strict mode violation` | Multiple elements match the same name/text | Add `exact: true` to YAML entry, or add `nth` |
287
+ | `toBeVisible` timeout on dynamic content | Snapshot was taken while page was still loading | Wait for spinner/skeleton to clear before snapshotting; add `browser_wait_for` |
288
+ | All tests fail with page navigate error | Page selector URL wrong or baseURL mismatch | Re-check `playwright.config.ts` `baseURL` and page selector `value` path |
289
+ | Auth redirect on every test | `specs/.auth/<role>.json` missing or expired | Run Phase 0.5 to capture fresh session |
290
+ | Table row assertions fail | `columns` config has wrong indices | Count column headers left-to-right (0-indexed) from snapshot |
291
+ | Wrong text assertions on locale page | Hardcoded Vietnamese/English text in YAML `name`/`value` | Use `{{lbl_*}}` variables with locale overlay files |
292
+ | Element inside iframe not found | `frame` field missing in YAML entry | Add `frame: "iframe[src*='...']"` to the selector entry |
293
+ | Selector breaks after UI redesign with no functional change | CSS class or XPath used — brittle to style refactoring | Rewrite with `role`/`testid`/`label`/`text` from accessibility snapshot |
294
+
295
+ ---
296
+
251
297
  ## Table Selectors
252
298
 
253
299
  For table patterns, add table selectors with `columns` config:
@@ -166,37 +166,47 @@ Resolver searches in this order:
166
166
 
167
167
  If no YAML key exists, the resolver infers from the Gherkin element type:
168
168
 
169
- | Gherkin | Inferred locator |
170
- |---|---|
171
- | `[X] button` | `getByRole('button', { name: 'X' })` |
172
- | `[X] link` | `getByRole('link', { name: 'X' })` |
173
- | `[X] heading` / `header` | `getByRole('heading', { name: 'X' })` |
174
- | `[X] checkbox` | `getByRole('checkbox', { name: 'X' })` |
175
- | `[X] radio` | `getByRole('radio', { name: 'X' })` |
176
- | `[X] field` | `getByPlaceholder('X')` |
177
- | `[X] text` / `message` / `label` | `getByText('X')` |
178
- | `[X] logo/image/icon` | `getByRole('img', { name: 'X' })` |
179
- | `[X] search` | `getByRole('searchbox', { name: 'X' })` |
180
- | `[X] option` | `getByRole('option', { name: 'X' })` |
181
- | `[X] slider` | `getByRole('slider', { name: 'X' })` |
182
- | `[X] toggle` | `getByRole('switch', { name: 'X' })` |
183
- | `[X] tab` | `getByRole('tab', { name: 'X' })` |
184
- | `[X] table` | `getByRole('table', { name: 'X' })` |
185
- | `[X] list` | `getByRole('list', { name: 'X' })` |
186
- | `[X] column` | `getByRole('columnheader', { name: 'X' })` |
187
- | `[X] dialog` / `modal` / `drawer` | `getByRole('dialog', { name: 'X' })` |
188
- | `[X] dropdown` / `select` | `getByRole('combobox', { name: 'X' })` |
189
- | `[X] menuitem` | `getByRole('menuitem', { name: 'X' })` |
190
- | `[X] progressbar` | `getByRole('progressbar', { name: 'X' })` |
191
- | `[X] section` | `getByRole('region', { name: 'X' })` |
192
- | `[X] card` | `getByRole('article', { name: 'X' })` |
193
- | `[X] item` | `getByRole('listitem', { name: 'X' })` |
194
- | `[X] cell` | `getByRole('cell', { name: 'X' })` |
195
- | `[X] spinner` | `getByRole('status', { name: 'X' })` |
196
- | `[X] breadcrumb` | `getByRole('navigation', { name: 'X' })` |
197
- | `[X] badge` / `tooltip` / `tag` | `getByText('X')` |
198
-
199
- **Only add a YAML entry when** the auto-inferred locator won't work (wrong name, need testid, need nth, etc.).
169
+ > ⚠️ **Auto-infer pitfall the #1 cause of selector failures in production.**
170
+ >
171
+ > `[X] button` auto-infers as `getByRole('button', { name: 'X' })`. This **only works** when the button's accessible name in the DOM is **exactly `X`** — same language, same text, same casing.
172
+ >
173
+ > The Gherkin `[Reference]` is your human label for the element, **not** the DOM name. If the app is in Vietnamese (or any language where the Gherkin label differs from DOM text), auto-infer will produce `No element found` at runtime. **Write an explicit YAML entry** with the real DOM name instead.
174
+ >
175
+ > **Decision rule**: auto-infer is safe ONLY when you have confirmed in the snapshot that the DOM element's accessible name / placeholder text is literally `X`. When in doubt → write YAML.
176
+
177
+ | Gherkin | Inferred locator | Safe when… |
178
+ |---|---|---|
179
+ | `[X] button` | `getByRole('button', { name: 'X' })` | Button's accessible name = X |
180
+ | `[X] link` | `getByRole('link', { name: 'X' })` | Link text = X |
181
+ | `[X] heading` / `header` | `getByRole('heading', { name: 'X' })` | Heading text = X |
182
+ | `[X] checkbox` | `getByRole('checkbox', { name: 'X' })` | Checkbox label = X |
183
+ | `[X] radio` | `getByRole('radio', { name: 'X' })` | Radio label = X |
184
+ | `[X] field` | `getByPlaceholder('X')` | Placeholder text = X AND field has a placeholder |
185
+ | `[X] text` / `message` / `label` | `getByText('X')` | Visible text = X (partial match) |
186
+ | `[X] logo/image/icon` | `getByRole('img', { name: 'X' })` | Image alt = X |
187
+ | `[X] search` | `getByRole('searchbox', { name: 'X' })` | Searchbox label = X |
188
+ | `[X] option` | `getByRole('option', { name: 'X' })` | Option text = X |
189
+ | `[X] slider` | `getByRole('slider', { name: 'X' })` | Slider label = X |
190
+ | `[X] toggle` | `getByRole('switch', { name: 'X' })` | Toggle label = X |
191
+ | `[X] tab` | `getByRole('tab', { name: 'X' })` | Tab text = X |
192
+ | `[X] table` | `getByRole('table', { name: 'X' })` | Table aria-label = X |
193
+ | `[X] list` | `getByRole('list', { name: 'X' })` | List aria-label = X |
194
+ | `[X] column` | `getByRole('columnheader', { name: 'X' })` | Column header text = X |
195
+ | `[X] dialog` / `modal` / `drawer` | `getByRole('dialog', { name: 'X' })` | Dialog aria-label/heading = X |
196
+ | `[X] dropdown` / `select` | `getByRole('combobox', { name: 'X' })` | Combobox label = X |
197
+ | `[X] menuitem` | `getByRole('menuitem', { name: 'X' })` | Menu item text = X |
198
+ | `[X] progressbar` | `getByRole('progressbar', { name: 'X' })` | Progressbar label = X |
199
+ | `[X] section` | `getByRole('region', { name: 'X' })` | Section aria-label = X |
200
+ | `[X] card` | `getByRole('article', { name: 'X' })` | Card aria-label = X |
201
+ | `[X] item` | `getByRole('listitem', { name: 'X' })` | List item text = X |
202
+ | `[X] cell` | `getByRole('cell', { name: 'X' })` | Cell text = X |
203
+ | `[X] spinner` | `getByRole('status', { name: 'X' })` | Spinner aria-label = X |
204
+ | `[X] breadcrumb` | `getByRole('navigation', { name: 'X' })` | Navigation aria-label = X |
205
+ | `[X] badge` / `tooltip` / `tag` | `getByText('X')` | Visible text = X |
206
+
207
+ **Special note on `[X] field`**: `getByPlaceholder('X')` only works when (1) the field has a placeholder attribute AND (2) the placeholder text equals X. For fields without placeholders (floating labels, aria-label), write explicit YAML: `type: label, value: "Actual label text"`.
208
+
209
+ **Only add a YAML entry when** auto-infer cannot work: DOM name differs from Gherkin label, need `testid`, need `nth`, need `exact: true`, or the field type requires explicit config.
200
210
 
201
211
  ### Types requiring YAML entry (no auto-infer)
202
212
 
@@ -105,6 +105,14 @@ Auto-detected by `create-test` before invoking this skill:
105
105
  2. Each row / bullet / item = 1 viewpoint → add to `Viewpoint items` in Coverage Map.
106
106
  3. Do NOT pre-classify into buckets before scanning — classify only when
107
107
  writing the scenario.
108
+ - `qa/context.md` — project-wide context set by the QA lead. Read ONCE before building the Coverage Map; apply to every screen. Extraction rules:
109
+ - **Roles** → for each role in the table: add to the `@auth:X` tag pool; generate a VP-SEC blocked-access scenario for every role boundary relevant to this screen.
110
+ - **Testing strategy → Focus areas** → if `security` listed: VP-SEC is mandatory Tier 1 for every free-text input regardless of spec risk level; if `ui` not listed: all VP-UI scenarios move to Tier 2 minimum.
111
+ - **Testing strategy → Mandatory coverage** → each line is a hard override applied to this screen regardless of spec risk; document in `Context constraints` of the Coverage Map.
112
+ - **Testing strategy → Deprioritize/skip** → record in `Context constraints`; suppress those VP categories from Tier 1 generation.
113
+ - **Global business rules** → add each to the `Business rules` section tagged `[G]` (e.g. `[G1 – soft-delete only]`); treat as `HIGH` risk unless stated otherwise.
114
+ - **Error patterns** → use as fallback only when `spec.md` does not give exact error text; never override spec-specified messages.
115
+ - If `qa/context.md` is absent: proceed without it — no impact on the generation flow.
108
116
 
109
117
  **Single screen focus**: one URL = one screen. Modals on same page = part of this screen.
110
118
  This means: do not test other screens' UI layout or navigation. It does NOT mean skip documenting business outcomes that your screen's actions cause on other surfaces. Those cross-surface outcomes must appear in the Coverage Map and be covered by at least `@manual` scenarios.
@@ -129,6 +137,11 @@ Read `spec.md` fully, then extract into a Coverage Map **before writing any scen
129
137
  **Risk tags:** HIGH = complex business rules, cascading fields, multi-step state changes, auth/integration. LOW = display-only, static labels, read-only fields.
130
138
 
131
139
  ```
140
+ Context constraints: [populated from qa/context.md before writing any scenario]
141
+ roles: [list roles, e.g. admin / manager / staff]
142
+ strategy: [active overrides, e.g. "VP-SEC mandatory T1", "VP-UI → T2 only"]
143
+ global rules: [G1 – ...] → also appear in Business rules below tagged [G]
144
+ → leave empty if qa/context.md is absent or has no entries applicable to this screen
132
145
  User journeys: [J1 – ...], [J2 – ...]
133
146
  Validation rules: [V1 – field → "exact error text"], [V2 – ...]
134
147
  Business rules: [B1 HIGH – ...], [B2 LOW – ...]
@@ -27,7 +27,15 @@ You are a **Senior QA Engineer**. You structure test cases by viewpoint categori
27
27
  - If no → fresh creation. Ask generation scope:
28
28
  - **1) Tier 1 — Critical & High priority** — ~10-15 scenarios/section covering happy paths, core validation, security basics **(Recommended)**
29
29
  - **2) Full coverage — All tiers at once** — generates Tier 1 + 2 + 3 in one run. Large output (~40-60 scenarios/section), best for experienced users who want complete coverage immediately
30
- 3. **Read requirements & resolve visual source** — check `<base>/${input:name}/requirements/`:
30
+ 3. **Read project context + screen requirements**
31
+
32
+ **Project context** — check `qa/context.md` (project root, not screen-specific):
33
+ - If exists → read it. Extract: roles, testing strategy directives, global business rules, error patterns.
34
+ - Summarize what you found in one line (e.g. `"Roles: admin/staff/user | Strategy: focus security, skip VP-UI T1 | 2 global rules"`).
35
+ - These are carried into the Coverage Map when invoking the `sungen-tc-generation` skill.
36
+ - If absent → continue without it, no action needed.
37
+
38
+ **Screen requirements** — check `<base>/${input:name}/requirements/`:
31
39
  - If `spec.md` exists → read it as PRIMARY source (sections, fields, validation rules, business rules, states).
32
40
  - If `test-viewpoint.md` exists → read it. If it only contains HTML comments (scaffold template), ask:
33
41
  - **1) Fill test-viewpoint.md first** — identify edge cases, known issues, and design decisions before generating tests
@@ -41,11 +41,13 @@ Skip when `--env` matches the base locale.
41
41
  Phase 0 — Selector Generation decision tree
42
42
 
43
43
  Live page reachable? (URL provided and loads without error)
44
- YES → existing flow: browser_navigate → one browser_snapshot generate selectors.yaml (verified entries)
44
+ YES → existing flow: browser_navigate → wait for page to fully load (no spinner/skeleton/empty table)
45
+ one browser_snapshot → cross-verify every [Reference] label vs snapshot name →
46
+ generate selectors.yaml (verified entries; explicit YAML for any label≠DOM-name mismatch)
45
47
  NO → spec_figma.md exists in requirements/?
46
48
  YES → provisional flow (sungen-figma-source + sungen-selector-fix skills):
47
49
  1. Read filtered Figma node data from spec_figma.md (## Components + ## Text Inventory)
48
- 2. Apply selector heuristics from sungen-figma-source skill (testid > role+name > placeholder > label > locator > text)
50
+ 2. Apply selector priority from sungen-selector-fix § Step 3 (testid > role+name > label > placeholder > text > locator CSS last)
49
51
  3. Write selectors.yaml — every provisional entry gets this comment on the line above:
50
52
  # @needs-live-verify source=figma node_id=<id>
51
53
  4. Compile: Screen: sungen generate --screen <name>. Flow: sungen generate --flow <name> — must succeed