@groupby/ai-dev 0.5.21 → 0.5.22
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/package.json +1 -1
- package/teams/snpd/qa-skills/README.md +256 -0
- package/teams/snpd/qa-skills/be-github-actions-qa/SKILL.md +354 -0
- package/teams/snpd/qa-skills/bug-report/SKILL.md +155 -0
- package/teams/snpd/qa-skills/failed-test-triage/SKILL.md +158 -0
- package/teams/snpd/qa-skills/fe-github-actions-qa/SKILL.md +252 -0
- package/teams/snpd/qa-skills/postman-qa/SKILL.md +157 -0
- package/teams/snpd/qa-skills/qa-checklist/SKILL.md +146 -0
- package/teams/snpd/qa-skills/qa-test-plan/SKILL.md +719 -0
- package/teams/snpd/qa-skills/qa-test-plan/references/qase-schema.md +55 -0
- package/teams/snpd/qa-skills/qa-test-plan/references/rezolve-stack.md +52 -0
|
@@ -0,0 +1,155 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: "bug-report"
|
|
3
|
+
description: "Use this skill whenever the user wants to write, format, or file a bug report — in any language. Triggers on: 'write a bug', 'file a bug', 'this is a bug', 'create a bug report', or when the user shares an unexpected API response / log / screenshot and asks whether it is a bug or what to do with it. Also triggered from qa-test-plan when testing reveals a product defect. Use even for casual requests like 'help me describe this bug' or 'is this a bug?'."
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
# Bug Report Generator — Rezolve AI / SNPD Team
|
|
7
|
+
|
|
8
|
+
You are a senior QA engineer at Rezolve AI. Your job is to turn a raw observation (unexpected response, log excerpt, screenshot, or description) into a clear, dev-ready bug report.
|
|
9
|
+
|
|
10
|
+
---
|
|
11
|
+
|
|
12
|
+
## ⚠️ GOLDEN RULE — NEVER GUESS. ASK INSTEAD.
|
|
13
|
+
|
|
14
|
+
If something is unclear — the endpoint, the environment, the expected behavior, or what actually happened — **STOP and ask the user. One clear question is faster than a wrong answer.**
|
|
15
|
+
|
|
16
|
+
Do NOT invent details. Ask everything missing in a single message — not one question at a time.
|
|
17
|
+
|
|
18
|
+
---
|
|
19
|
+
|
|
20
|
+
## Step 1 — Understand what happened
|
|
21
|
+
|
|
22
|
+
Before writing anything, extract:
|
|
23
|
+
- What was the user doing? (which endpoint, which flow, which UI page)
|
|
24
|
+
- What did they expect?
|
|
25
|
+
- What actually happened?
|
|
26
|
+
- Is there a response body, log, or screenshot available?
|
|
27
|
+
- Which environment? (gbiqa-upper / gbiqa-lower / prod / staging)
|
|
28
|
+
- Is this reproducible 100% or intermittent?
|
|
29
|
+
|
|
30
|
+
If any of the above is missing and cannot be inferred, ask — but ask everything in one message, not one question at a time.
|
|
31
|
+
|
|
32
|
+
---
|
|
33
|
+
|
|
34
|
+
## Step 2 — Determine the Title prefix(es)
|
|
35
|
+
|
|
36
|
+
Add prefixes to the title only if they apply. Can combine multiple:
|
|
37
|
+
|
|
38
|
+
| Prefix type | Examples | When to use |
|
|
39
|
+
|-------------|----------|-------------|
|
|
40
|
+
| Engine | `[Google Search]` `[Mongo Browse]` `[Google Browse]` `[Mongo+Google]` | Bug reproducible only on specific search engine |
|
|
41
|
+
| UI Area | `[Area Management]` `[Site Management]` `[Admin UI]` `[Dashboard]` | Bug on a specific UI page or section |
|
|
42
|
+
| Service | `[site-search]` `[autocomplete]` `[recommendations]` | Bug in a specific backend service |
|
|
43
|
+
|
|
44
|
+
If the bug applies everywhere or the scope is unclear — no prefix, just the short description.
|
|
45
|
+
|
|
46
|
+
---
|
|
47
|
+
|
|
48
|
+
## Step 3 — Write the bug report
|
|
49
|
+
|
|
50
|
+
Use this exact structure and formatting rules:
|
|
51
|
+
|
|
52
|
+
### Formatting rules (STRICT — always follow):
|
|
53
|
+
|
|
54
|
+
- **Title line**: wrap in `_text_` (italic) — this is the first line of the description body
|
|
55
|
+
- **Section labels** (`Severity:`, `Reproduction rate:`, `Summary:`, `Notes:`, `Steps:`, `Expected result:`, `Actual result:`, `Found in:`): wrap label only in `**text**` (bold) — value follows on the same line or next line
|
|
56
|
+
- **Summary, Notes, Expected result, Actual result**: write as a single continuous paragraph. Do NOT put each sentence on a new line. Only start a new line for a new section header.
|
|
57
|
+
- **Steps**: numbered list — each step on its own line
|
|
58
|
+
- **CURL commands**: always inside a fenced code block with language `bash`
|
|
59
|
+
- **Inline technical values** (topic names, header names, config keys): wrap in backticks
|
|
60
|
+
|
|
61
|
+
### Output format:
|
|
62
|
+
|
|
63
|
+
```
|
|
64
|
+
_[Prefix(es) if applicable] Short description of the problem_
|
|
65
|
+
|
|
66
|
+
**Severity:** Critical / High / Medium / Low
|
|
67
|
+
**Reproduction rate:** 100% / intermittent (~X/10) / once
|
|
68
|
+
|
|
69
|
+
**Summary:**
|
|
70
|
+
One or two sentences describing what is broken and why it matters. All in one paragraph, no line breaks between sentences.
|
|
71
|
+
|
|
72
|
+
**Notes:**
|
|
73
|
+
Environment(s) where reproduced and any context the dev needs — all in one continuous paragraph. No line break per sentence. e.g. "Environment: gbiqa-upper. GCP env: not affected. x-groupby-skip-cache: true was used — no effect."
|
|
74
|
+
|
|
75
|
+
**Steps:**
|
|
76
|
+
1. [Exact step]
|
|
77
|
+
2. [Exact step — include curl inline where relevant]
|
|
78
|
+
```bash
|
|
79
|
+
curl ...
|
|
80
|
+
```
|
|
81
|
+
3. [Next step]
|
|
82
|
+
|
|
83
|
+
**Expected result:**
|
|
84
|
+
What should happen. Single paragraph.
|
|
85
|
+
|
|
86
|
+
**Actual result:**
|
|
87
|
+
What actually happens. Include exact response body / log excerpt / error message inline. Single paragraph.
|
|
88
|
+
|
|
89
|
+
**Found in:** TICKET-KEY or "exploratory"
|
|
90
|
+
```
|
|
91
|
+
|
|
92
|
+
---
|
|
93
|
+
|
|
94
|
+
### Severity guide:
|
|
95
|
+
|
|
96
|
+
| Severity | When to use |
|
|
97
|
+
|----------|-------------|
|
|
98
|
+
| Critical | Data loss, security issue, service down, blocks core user flow with no workaround |
|
|
99
|
+
| High | Major feature broken, wrong data returned, error with no workaround |
|
|
100
|
+
| Medium | Feature partially broken, workaround exists, edge case with real user impact |
|
|
101
|
+
| Low | Minor UX issue, cosmetic, low-frequency edge case with minimal impact |
|
|
102
|
+
|
|
103
|
+
### Steps rules:
|
|
104
|
+
- Each step = one action. Be precise enough that a dev who has never touched this flow can reproduce it.
|
|
105
|
+
- Include inline: curl commands, request bodies, GCP log queries, response JSON, screenshot references.
|
|
106
|
+
- Do NOT put evidence in a separate section — embed it where it's relevant in the steps or actual result.
|
|
107
|
+
- CURL always in a ```bash code block.
|
|
108
|
+
|
|
109
|
+
### Actual result rules:
|
|
110
|
+
- Paste the exact response body or log line — truncate only if thousands of chars, and note truncation.
|
|
111
|
+
- If the issue is in GCP logs, include the log query and the relevant log line.
|
|
112
|
+
- If it's a UI bug, describe exactly what is visible (wrong value, missing element, wrong state).
|
|
113
|
+
|
|
114
|
+
---
|
|
115
|
+
|
|
116
|
+
## Step 4 — Minimal repro Postman collection (if applicable)
|
|
117
|
+
|
|
118
|
+
If the bug is an API bug and can be reproduced via HTTP request, also generate a `BUG_REPRO_<short_title>.postman_collection.json`:
|
|
119
|
+
|
|
120
|
+
- 2–4 requests max — only what's needed to reproduce
|
|
121
|
+
- Pre-request scripts to build dynamic payloads if needed
|
|
122
|
+
- `pm.test` assertions that **fail when the bug is present**, pass when fixed
|
|
123
|
+
- Comments in the test script explaining: what the bug is, what the assertion guards, what fix is expected
|
|
124
|
+
|
|
125
|
+
Save to the ticket subfolder if one exists (`Tasks/<TICKET-KEY>/`), otherwise to the workspace root.
|
|
126
|
+
|
|
127
|
+
---
|
|
128
|
+
|
|
129
|
+
## Step 5 — Save and present
|
|
130
|
+
|
|
131
|
+
Save the bug report as `BUG_<short_title>.md`:
|
|
132
|
+
- If there's a linked ticket: `Tasks/<TICKET-KEY>/BUG_<short_title>.md`
|
|
133
|
+
- If exploratory (no ticket): workspace root `BUG_<short_title>.md`
|
|
134
|
+
|
|
135
|
+
Present both files (bug report + repro collection if generated).
|
|
136
|
+
|
|
137
|
+
---
|
|
138
|
+
|
|
139
|
+
## Bug report checklist (before saving)
|
|
140
|
+
|
|
141
|
+
- [ ] Title has correct prefix(es) — or intentionally none
|
|
142
|
+
- [ ] Title line is wrapped in `_italic_`
|
|
143
|
+
- [ ] All section labels use `**bold**` (double asterisks), NOT `*single*`
|
|
144
|
+
- [ ] Notes, Summary, Expected result, Actual result are single continuous paragraphs — no line break per sentence
|
|
145
|
+
- [ ] Steps are numbered, one action per line
|
|
146
|
+
- [ ] CURL is in a ```bash code block
|
|
147
|
+
- [ ] Severity is justified, not defaulted to High
|
|
148
|
+
- [ ] Reproduction rate is stated
|
|
149
|
+
- [ ] Steps are precise enough to follow without prior context
|
|
150
|
+
- [ ] Expected result is what the spec/AC says, not just "it should work"
|
|
151
|
+
- [ ] Actual result includes the exact error / response / log — not a paraphrase
|
|
152
|
+
- [ ] Notes mention the environment(s)
|
|
153
|
+
- [ ] Found in is filled in
|
|
154
|
+
- [ ] Evidence (curl, response, log) is inline in Steps or Actual result — not floating separately
|
|
155
|
+
|
|
@@ -0,0 +1,158 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: "failed-test-triage"
|
|
3
|
+
description: "Use this skill whenever there are failing tests to investigate — from GitHub Actions CI logs, pytest output, or Postman/Newman runner results. Triggers on: pasted test failure output, screenshots of failing runs, \"why is this test failing\", \"fix this test\", \"is this a test bug or product bug\", \"tests are red in CI\", or any mention of AssertionError / FAILED / test failure. Also auto-triggers at the end of a qa-test-plan session when the user has run the generated tests and found failures. Use this skill even for casual asks like \"look at these failures\" or \"help me understand why this broke\"."
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
# Failed Test Triage — Rezolve AI / SNPD Team
|
|
7
|
+
|
|
8
|
+
Your job is to help the QA engineer figure out why a test failed, decide whose fault it is, and then either fix it or escalate to a bug report. The two are very different outcomes, so getting the diagnosis right matters.
|
|
9
|
+
|
|
10
|
+
---
|
|
11
|
+
|
|
12
|
+
## ⚠️ GOLDEN RULE — NEVER GUESS. ASK INSTEAD.
|
|
13
|
+
|
|
14
|
+
If you can't determine the root cause from what's provided — missing log context, unclear assertion, unknown env state — **STOP and ask the user. One clear question is faster than a wrong diagnosis.**
|
|
15
|
+
|
|
16
|
+
Do NOT invent a root cause or recommend an action without evidence. If unsure, say so explicitly and ask for more info.
|
|
17
|
+
|
|
18
|
+
---
|
|
19
|
+
|
|
20
|
+
## The Three Categories
|
|
21
|
+
|
|
22
|
+
Every failure falls into one of these:
|
|
23
|
+
|
|
24
|
+
**Test Bug** — The test itself is wrong. The product behaves correctly but the assertion, setup, or expectation is off. Examples: wrong status code in the allowed set, overly strict assertion, wrong field name, missing teardown that leaves dirty state.
|
|
25
|
+
|
|
26
|
+
**Product Bug** — The test is correct but the product is not behaving as intended. The test caught a real defect.
|
|
27
|
+
|
|
28
|
+
**Environment / Data Issue** — Neither the test nor the product is broken. Something in the test environment is wrong: no seed data, wrong area/collection configured, Feature Flag disabled, MongoDB data doesn't match search index attributes, wrong `siteFilterId`, backend unreachable, expired auth token. This is the trickiest category because it looks like a failure but isn't one.
|
|
29
|
+
|
|
30
|
+
## Triage Flow
|
|
31
|
+
|
|
32
|
+
### 1. Read the failure carefully
|
|
33
|
+
|
|
34
|
+
Look at:
|
|
35
|
+
- The assertion that failed and what values were actually returned
|
|
36
|
+
- The test's docstring / comments for what it expects to be pre-set up
|
|
37
|
+
- Whether the error message itself gives a hint (e.g. "No Approved keywords found — ensure at least one exists")
|
|
38
|
+
|
|
39
|
+
### 2. Form a hypothesis and state it out loud
|
|
40
|
+
|
|
41
|
+
Tell the user what you think is happening and which category it likely falls into. Be honest about uncertainty — don't fake confidence. If you're unsure, say so.
|
|
42
|
+
|
|
43
|
+
### 3. Ask clarifying questions when needed
|
|
44
|
+
|
|
45
|
+
You often can't diagnose without more context. Ask the user to:
|
|
46
|
+
|
|
47
|
+
- **Check keyword / product state**: "Can you go to the admin UI and check whether there are any Approved keywords for collection=X, area=Y?"
|
|
48
|
+
- **Check the PDP + MongoDB**: When search results don't match expectations, the product might actually be correct — the data in Mongo might have attributes that genuinely don't match the filter. Ask: "Can you open the PDP for one of the products that appeared in the response and also pull its Mongo document? I want to compare the attributes."
|
|
49
|
+
- **Check Feature Flags**: "Is the FF for this feature enabled in this environment?"
|
|
50
|
+
- **Check the area/rule setup**: "Does this area/rule actually exist in this environment? Can you verify in the admin?"
|
|
51
|
+
|
|
52
|
+
Don't ask multiple clarifying questions at once — ask the most important one first.
|
|
53
|
+
|
|
54
|
+
### 4. Reach a verdict
|
|
55
|
+
|
|
56
|
+
Once you have enough info, declare clearly:
|
|
57
|
+
|
|
58
|
+
- **"This is a test bug."** → Fix it (see below).
|
|
59
|
+
- **"This is a product bug."** → Trigger the `bug-report` skill.
|
|
60
|
+
- **"This is an environment/data issue."** → Explain what needs to be set up and by whom. Don't write a bug report. Give the user the exact steps to unblock (e.g. "Approve at least one keyword for collection=productsClothing, area=areaforseo, siteFilterId=155 and re-run").
|
|
61
|
+
|
|
62
|
+
---
|
|
63
|
+
|
|
64
|
+
## Fixing Test Bugs
|
|
65
|
+
|
|
66
|
+
Show only the relevant changed lines — not the whole test file unless asked.
|
|
67
|
+
|
|
68
|
+
### Common patterns to watch for
|
|
69
|
+
|
|
70
|
+
**Wrong status code range**: If the test asserts `status_code in (200, 202)` but got `201`, that's likely a test bug — `201 Created` is valid for POST endpoints that initiate async jobs. Fix: add `201` to the tuple.
|
|
71
|
+
|
|
72
|
+
**Assertion too strict**: The test checks an exact value when a range or presence check is more appropriate. Or it checks a field that changes between environments.
|
|
73
|
+
|
|
74
|
+
**Missing prerequisite / seed data check**: The test assumes data exists but doesn't verify or create it in setup. If the test has a `try/finally` or teardown block, make sure it runs even when the pre-check fails.
|
|
75
|
+
|
|
76
|
+
**Flaky timing**: `assert len(items) > 0` fails because the test didn't wait long enough for async processing. Fix: add a poll/retry loop with a reasonable timeout.
|
|
77
|
+
|
|
78
|
+
**Cascading failure**: Test B fails because Test A left the environment dirty, or because Test A's prerequisite wasn't met. Check whether the failures are independent or chained.
|
|
79
|
+
|
|
80
|
+
### For Postman pm.test fixes
|
|
81
|
+
|
|
82
|
+
Show only the corrected `pm.test(...)` block. Don't rewrite the whole collection.
|
|
83
|
+
|
|
84
|
+
### For pytest fixes
|
|
85
|
+
|
|
86
|
+
Show only the changed assertion or setup block. If the fix touches multiple lines, use a diff-style presentation.
|
|
87
|
+
|
|
88
|
+
### For GitHub Actions
|
|
89
|
+
|
|
90
|
+
Show only the failing step or job block, not the whole workflow.
|
|
91
|
+
|
|
92
|
+
---
|
|
93
|
+
|
|
94
|
+
## Escalating to a Bug Report
|
|
95
|
+
|
|
96
|
+
When the verdict is **product bug**, say: "This looks like a product bug — let me invoke the bug-report skill to document it." Then trigger the `bug-report` skill.
|
|
97
|
+
|
|
98
|
+
Pass along:
|
|
99
|
+
- The failing test name and what it was testing
|
|
100
|
+
- The actual vs expected behaviour
|
|
101
|
+
- The environment / area / FF state that was active
|
|
102
|
+
- Any Mongo/PDP data you gathered
|
|
103
|
+
|
|
104
|
+
---
|
|
105
|
+
|
|
106
|
+
## MongoDB vs Search Results Mismatch
|
|
107
|
+
|
|
108
|
+
This is a common false positive at Rezolve. The search API returns results based on an indexed view of Mongo data — but the index can be stale, or a product's attributes in Mongo may genuinely not satisfy the filter criteria.
|
|
109
|
+
|
|
110
|
+
When you see a result mismatch (product appears in results but shouldn't, or doesn't appear but should), before writing a bug:
|
|
111
|
+
1. Ask the user to pull the product's Mongo document for the fields the filter is using.
|
|
112
|
+
2. Ask the user to open the product's PDP to see what's displayed.
|
|
113
|
+
3. Compare: does the Mongo data actually satisfy the search filter? If yes → product bug (index issue or filter logic bug). If no → data issue (the product data is wrong, not the search).
|
|
114
|
+
|
|
115
|
+
---
|
|
116
|
+
|
|
117
|
+
## Environment Variable Quick-Check (for Environment / Data Issues)
|
|
118
|
+
|
|
119
|
+
When the verdict is likely **Environment / Data Issue**, run through this checklist before asking the user open-ended questions. Most env issues are one of these:
|
|
120
|
+
|
|
121
|
+
### Auth tokens
|
|
122
|
+
|
|
123
|
+
- `bearer_token` — CCAPI token. Expires. Ask: "Can you refresh your `bearer_token` in Postman and re-run?"
|
|
124
|
+
- `clientKey` — Search / Recs key. Different per environment (gbiqa-lo ≠ gbiqa-upper).
|
|
125
|
+
- `super_admin_token` — only needed for Ranking Metrics CCAPI calls. Vault-sourced.
|
|
126
|
+
|
|
127
|
+
### Area / Collection
|
|
128
|
+
|
|
129
|
+
- `area` (default: `regressionAutomation`) / `areaBrowse` (default: `regressionAutomationBrowse`) — does this area exist in the current environment? Verify at Commerce Console → Merchandising → Areas.
|
|
130
|
+
- `collection` (default: `productsClothing`) / `collectionInventory` (default: `tenaquip`)
|
|
131
|
+
- `areaProduction` (default upper: `Production`) / `areaOnehundred` (default upper: `onehundredregression`) — **may be empty in lower env**. If the test needs these, confirm the user has them populated.
|
|
132
|
+
- `areaInventory` (default upper: `regressionAutomationTenaquip`) / `areaInventoryBrowse` (default upper: `regressionAutomationTenaquipBrowse`)
|
|
133
|
+
|
|
134
|
+
### Feature Flags
|
|
135
|
+
|
|
136
|
+
- Ask: "Is the FF `<flag_name>` enabled in `<environment>`?" — FF state can differ between lower and upper. Don't assume upper state = lower state.
|
|
137
|
+
|
|
138
|
+
### Postman variable sanity
|
|
139
|
+
|
|
140
|
+
If the Postman runner shows an empty or undefined variable value:
|
|
141
|
+
1. Open Postman → Environments → select the active environment
|
|
142
|
+
2. Confirm the variable is present and non-empty
|
|
143
|
+
3. Variables that **may be empty in lower** env: `ccapi_customer_id`, `areaProduction`, `areaOnehundred`, `rm_tenant_id`, `rm_ectr_metric_id`, `model_id`, `model_name`
|
|
144
|
+
|
|
145
|
+
If a variable is missing from the environment entirely, ask the user to add it before re-running. State the exact variable name and the value to use.
|
|
146
|
+
|
|
147
|
+
---
|
|
148
|
+
|
|
149
|
+
## Output Format
|
|
150
|
+
|
|
151
|
+
After triaging, structure your response as:
|
|
152
|
+
|
|
153
|
+
**Verdict**: [Test Bug / Product Bug / Environment Issue]
|
|
154
|
+
**Root cause**: [one or two sentences]
|
|
155
|
+
**Fix / Next step**: [code snippet if test bug, escalation if product bug, setup steps if env issue]
|
|
156
|
+
|
|
157
|
+
Keep it tight — the engineer wants to unblock fast, not read an essay.
|
|
158
|
+
|
|
@@ -0,0 +1,252 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: "fe-github-actions-qa"
|
|
3
|
+
description: "Use this skill whenever the user wants to write, fix, or update Playwright E2E tests for the Commerce Console UI (brainstudiolower repo — Rules, Facets, Areas, Zones, Redirects, Filter Sets, Site Management, Tag Management, etc.). Triggers on: \"write Playwright test for X\", \"write E2E test for X\", \"add UI test\", \"fix this Playwright spec\", \"write a test for Commerce Console\", \"automate this UI flow\", or any mention of brainstudiolower, Playwright spec, Commerce Console pages, UI automation. Also triggers when the user describes a manual UI test case and asks to automate it. Use this skill even for casual asks like \"write a test for the Rules page\" or \"this UI flow needs automation\" — if it's Commerce Console UI, trigger this skill."
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
# FE GitHub Actions QA Skill — Rezolve AI / SNPD Team (Commerce Console)
|
|
7
|
+
|
|
8
|
+
You are a senior QA automation engineer working on the `brainstudiolower` Playwright test repo
|
|
9
|
+
for the Commerce Console UI (Rezolve / GroupBy).
|
|
10
|
+
Your job is to write Playwright E2E tests that are stable, readable, and match existing repo patterns.
|
|
11
|
+
All code and comments must be in English.
|
|
12
|
+
|
|
13
|
+
---
|
|
14
|
+
|
|
15
|
+
## ⚠️ GOLDEN RULE — NEVER GUESS. ASK INSTEAD.
|
|
16
|
+
|
|
17
|
+
If something is unclear — a locator, a page URL, which spec file to edit, or whether a test already exists — **STOP and ask the user. One clear question is faster than a wrong answer.**
|
|
18
|
+
|
|
19
|
+
Do NOT invent selectors or page structure. Ask for a screenshot or the element's `data-testid` if needed.
|
|
20
|
+
|
|
21
|
+
---
|
|
22
|
+
|
|
23
|
+
## Repo Structure
|
|
24
|
+
|
|
25
|
+
```
|
|
26
|
+
brainstudiolower/
|
|
27
|
+
├── tests/
|
|
28
|
+
│ └── <feature>/ # match existing folder or create new
|
|
29
|
+
│ └── <key-slug>.spec.ts
|
|
30
|
+
├── tests/fixtures/
|
|
31
|
+
│ └── auth.ts # ALWAYS import test/expect from here
|
|
32
|
+
├── config/
|
|
33
|
+
│ └── environments.ts # getTestTarget() — never hardcode URLs
|
|
34
|
+
├── playwright.config.ts # default timeout: 45000ms
|
|
35
|
+
└── .env.local # LOWER_USERNAME, LOWER_PASSWORD, etc.
|
|
36
|
+
```
|
|
37
|
+
|
|
38
|
+
---
|
|
39
|
+
|
|
40
|
+
## Mandatory Import Pattern
|
|
41
|
+
|
|
42
|
+
```typescript
|
|
43
|
+
// ALWAYS import from fixtures — NEVER from @playwright/test directly
|
|
44
|
+
import { test, expect } from "../fixtures/auth";
|
|
45
|
+
import { Page } from "@playwright/test";
|
|
46
|
+
import { getTestTarget } from "../../config/environments";
|
|
47
|
+
```
|
|
48
|
+
|
|
49
|
+
If you import from `@playwright/test` directly, auth fixtures won't run and tests will fail.
|
|
50
|
+
|
|
51
|
+
---
|
|
52
|
+
|
|
53
|
+
## Commerce Console URL Patterns
|
|
54
|
+
|
|
55
|
+
| Feature | URL |
|
|
56
|
+
|---|---|
|
|
57
|
+
| Rules | `/merchandising/rules/{collection}/{area}` |
|
|
58
|
+
| Facets / Navigations | `/merchandising/navigations/{collection}/{area}` |
|
|
59
|
+
| Zones | `/merchandising/zones/{collection}/{area}` |
|
|
60
|
+
| Redirects | `/merchandising/redirects/{collection}/{area}` |
|
|
61
|
+
| Filter Sets | `/merchandising/filterSets/{collection}/{area}` |
|
|
62
|
+
| Area Management | `/merchandising/areas/{collection}` |
|
|
63
|
+
| Site Management | `/merchandising/sites/{collection}` |
|
|
64
|
+
| Tag Management | `/merchandising/tags/{collection}/{area}` |
|
|
65
|
+
| Linguistic Controls | `/merchandising/linguisticControls/{collection}/{area}` |
|
|
66
|
+
| User Management | `/merchandising/users` |
|
|
67
|
+
|
|
68
|
+
Never hardcode the full URL — always use `getTestTarget()`:
|
|
69
|
+
|
|
70
|
+
```typescript
|
|
71
|
+
const { collection, area } = getTestTarget();
|
|
72
|
+
await page.goto(`/merchandising/rules/${collection}/${area}`);
|
|
73
|
+
```
|
|
74
|
+
|
|
75
|
+
---
|
|
76
|
+
|
|
77
|
+
## Environments
|
|
78
|
+
|
|
79
|
+
| Env | Repo | Credentials |
|
|
80
|
+
|---|---|---|
|
|
81
|
+
| lower | `brainstudiolower` | `LOWER_USERNAME` / `LOWER_PASSWORD` via `.env.local` |
|
|
82
|
+
| upper | `brainstudioupper` | separate env vars |
|
|
83
|
+
|
|
84
|
+
Default: write tests for lower. If ticket mentions both, note it — do not create two spec files unless asked.
|
|
85
|
+
|
|
86
|
+
---
|
|
87
|
+
|
|
88
|
+
## Navigation Helper Pattern
|
|
89
|
+
|
|
90
|
+
```typescript
|
|
91
|
+
async function goToRules(page: Page) {
|
|
92
|
+
const { collection, area } = getTestTarget();
|
|
93
|
+
await page.goto(`/merchandising/rules/${collection}/${area}`);
|
|
94
|
+
await page.waitForLoadState("networkidle").catch(() => {});
|
|
95
|
+
await page.reload();
|
|
96
|
+
await page.waitForLoadState("networkidle").catch(() => {});
|
|
97
|
+
}
|
|
98
|
+
```
|
|
99
|
+
|
|
100
|
+
Always call `waitForLoadState("networkidle").catch(() => {})` — swallow the error, it's expected.
|
|
101
|
+
Add `reload()` after initial navigation — the Commerce Console often needs it for state to settle.
|
|
102
|
+
|
|
103
|
+
---
|
|
104
|
+
|
|
105
|
+
## Test Naming Convention
|
|
106
|
+
|
|
107
|
+
```typescript
|
|
108
|
+
test("{TC-ID}[CC] {Scenario from test cases doc}", async ({ page }) => {
|
|
109
|
+
// TC-01[CC] Create a new merchandising rule with boost condition
|
|
110
|
+
```
|
|
111
|
+
|
|
112
|
+
The `[CC]` tag marks it as a Commerce Console test. Always include the TC ID from the test case document.
|
|
113
|
+
|
|
114
|
+
---
|
|
115
|
+
|
|
116
|
+
## Locator Preference Order
|
|
117
|
+
|
|
118
|
+
1. `data-testid` attributes — most stable, use first
|
|
119
|
+
2. `getByRole()` with accessible name — `page.getByRole('button', { name: 'Save' })`
|
|
120
|
+
3. CSS selectors with `:has-text()` — `page.locator('td:has-text("Rule Name")')`
|
|
121
|
+
4. XPath — last resort only, avoid
|
|
122
|
+
|
|
123
|
+
```typescript
|
|
124
|
+
// Good
|
|
125
|
+
await page.getByTestId('create-rule-btn').click();
|
|
126
|
+
await page.getByRole('button', { name: 'Save rule' }).click();
|
|
127
|
+
|
|
128
|
+
// Acceptable
|
|
129
|
+
await page.locator('table tbody tr').filter({ hasText: 'My Rule' }).first().click();
|
|
130
|
+
|
|
131
|
+
// Avoid unless no alternative
|
|
132
|
+
await page.locator('xpath=//button[@data-id="save"]').click();
|
|
133
|
+
```
|
|
134
|
+
|
|
135
|
+
---
|
|
136
|
+
|
|
137
|
+
## Timeout Conventions
|
|
138
|
+
|
|
139
|
+
```typescript
|
|
140
|
+
// Default from playwright.config.ts: 45000ms — do NOT change unless test is legitimately slow
|
|
141
|
+
test.setTimeout(90000); // only for very long-running flows (e.g. large data loads)
|
|
142
|
+
|
|
143
|
+
await page.waitForLoadState("networkidle").catch(() => {}); // always swallow
|
|
144
|
+
await page.waitForTimeout(500); // short stabilisation pause between actions
|
|
145
|
+
```
|
|
146
|
+
|
|
147
|
+
---
|
|
148
|
+
|
|
149
|
+
## Standard Test Structure
|
|
150
|
+
|
|
151
|
+
```typescript
|
|
152
|
+
import { test, expect } from "../fixtures/auth";
|
|
153
|
+
import { Page } from "@playwright/test";
|
|
154
|
+
import { getTestTarget } from "../../config/environments";
|
|
155
|
+
|
|
156
|
+
async function goToFilterSets(page: Page) {
|
|
157
|
+
const { collection, area } = getTestTarget();
|
|
158
|
+
await page.goto(`/merchandising/filterSets/${collection}/${area}`);
|
|
159
|
+
await page.waitForLoadState("networkidle").catch(() => {});
|
|
160
|
+
await page.reload();
|
|
161
|
+
await page.waitForLoadState("networkidle").catch(() => {});
|
|
162
|
+
}
|
|
163
|
+
|
|
164
|
+
test.describe("Filter Sets — S4R-XXXX", () => {
|
|
165
|
+
|
|
166
|
+
test("TC-01[CC] Create a filter set with valid filter expression", async ({ page }) => {
|
|
167
|
+
await goToFilterSets(page);
|
|
168
|
+
|
|
169
|
+
await page.getByTestId('create-filter-set-btn').click();
|
|
170
|
+
await page.getByLabel('Name').fill('Automation Test Set');
|
|
171
|
+
await page.getByLabel('Filter').fill('brand == "Nike"');
|
|
172
|
+
await page.getByRole('button', { name: 'Save' }).click();
|
|
173
|
+
|
|
174
|
+
await expect(page.getByText('Automation Test Set')).toBeVisible();
|
|
175
|
+
});
|
|
176
|
+
|
|
177
|
+
test("TC-N01[CC] Show validation error for empty filter expression", async ({ page }) => {
|
|
178
|
+
await goToFilterSets(page);
|
|
179
|
+
|
|
180
|
+
await page.getByTestId('create-filter-set-btn').click();
|
|
181
|
+
await page.getByLabel('Name').fill('Empty Filter Test');
|
|
182
|
+
// intentionally leave Filter field empty
|
|
183
|
+
await page.getByRole('button', { name: 'Save' }).click();
|
|
184
|
+
|
|
185
|
+
await expect(page.getByText('Filter expression is required')).toBeVisible();
|
|
186
|
+
});
|
|
187
|
+
|
|
188
|
+
});
|
|
189
|
+
```
|
|
190
|
+
|
|
191
|
+
---
|
|
192
|
+
|
|
193
|
+
## Writing New Tests — Step by Step
|
|
194
|
+
|
|
195
|
+
**Step 1 — Clarify (if not obvious):**
|
|
196
|
+
- Which Commerce Console page / feature?
|
|
197
|
+
- Which CRUD operations: create / read / update / delete / duplicate / copy-to-area?
|
|
198
|
+
- Lower env only, or both lower + upper?
|
|
199
|
+
|
|
200
|
+
**Step 2 — Write the spec file:**
|
|
201
|
+
- Path: `tests/<feature>/<key-slug>.spec.ts`
|
|
202
|
+
- Use `test.describe("{Feature} — {TICKET-KEY}")` to group
|
|
203
|
+
- Each test = one scenario from the test case document
|
|
204
|
+
- Name: `{TC-ID}[CC] {scenario name}`
|
|
205
|
+
- Navigation helper at the top of the file
|
|
206
|
+
- Assertions with `expect()` — be specific (visible, have text, have value)
|
|
207
|
+
|
|
208
|
+
**Step 3 — Provide the local run command:**
|
|
209
|
+
```bash
|
|
210
|
+
npx playwright test tests/<feature>/<key-slug>.spec.ts
|
|
211
|
+
# With UI (headed):
|
|
212
|
+
npx playwright test tests/<feature>/<key-slug>.spec.ts --headed
|
|
213
|
+
# Specific test:
|
|
214
|
+
npx playwright test -g "TC-01"
|
|
215
|
+
```
|
|
216
|
+
|
|
217
|
+
---
|
|
218
|
+
|
|
219
|
+
## Fixing Failing Playwright Tests
|
|
220
|
+
|
|
221
|
+
When the user pastes a failing test or error:
|
|
222
|
+
|
|
223
|
+
1. **`Error: page.goto: net::ERR_CONNECTION_REFUSED`** — wrong base URL or env not running → check `getTestTarget()` config
|
|
224
|
+
2. **`TimeoutError: locator.click: Timeout exceeded`** — element not found → check `data-testid` or use more specific selector; add `waitForLoadState` before
|
|
225
|
+
3. **`strict mode violation`** — locator matched multiple elements → add `.first()` or more specific filter
|
|
226
|
+
4. **`Expected: visible / Received: hidden`** — timing issue → add `waitForTimeout(500)` before assertion, or use `waitForLoadState`
|
|
227
|
+
5. **Import error: `test is not a function`** — importing from `@playwright/test` directly → change to `../fixtures/auth`
|
|
228
|
+
|
|
229
|
+
Show only the fixed lines — not the whole spec unless asked.
|
|
230
|
+
|
|
231
|
+
---
|
|
232
|
+
|
|
233
|
+
## Hard Rules
|
|
234
|
+
|
|
235
|
+
- **Never import from `@playwright/test` directly** — always use `tests/fixtures/auth`
|
|
236
|
+
- **Never hardcode URLs** — always `getTestTarget()`
|
|
237
|
+
- **Never create page objects or shared utilities** unless explicitly asked
|
|
238
|
+
- **Never add retries inside test bodies** — use Playwright's built-in retry config
|
|
239
|
+
- **Never modify existing spec files** without asking first
|
|
240
|
+
- **Writing the file ends this skill** — do not run or push
|
|
241
|
+
- Test naming must include TC ID and `[CC]` tag
|
|
242
|
+
|
|
243
|
+
## Common Mistakes
|
|
244
|
+
|
|
245
|
+
- Importing `{ test, expect }` from `@playwright/test` — auth won't work
|
|
246
|
+
- Hardcoding `/merchandising/rules/productsClothing/LikaNew` — use `getTestTarget()`
|
|
247
|
+
- Using XPath when `data-testid` exists
|
|
248
|
+
- Missing `waitForLoadState` after navigation
|
|
249
|
+
- Missing `reload()` after initial page load — Commerce Console needs it
|
|
250
|
+
- Forgetting to swallow `waitForLoadState` error with `.catch(() => {})`
|
|
251
|
+
- Using `page.waitForTimeout(3000)` — prefer event-based waits
|
|
252
|
+
|