@keber/qa-framework 1.4.0 → 1.5.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@keber/qa-framework",
3
- "version": "1.4.0",
3
+ "version": "1.5.1",
4
4
  "description": "Reusable spec-driven QA framework for IDE-agent-assisted automated testing. Installable as an npm package. Provides structure, templates, agent instructions, and optional integrations for Playwright and Azure DevOps.",
5
5
  "keywords": [
6
6
  "qa",
package/scripts/init.js CHANGED
@@ -288,7 +288,10 @@ This project uses \`@keber/qa-framework\` v${config.frameworkVersion ?? '1.0.0'}
288
288
  5. Read previous test suites code in \`qa/07-automation/e2e/tests/\` if available before writing new automation to maintain consistency in style and approach
289
289
  6. Project QA config is at \`qa/qa-framework.config.json\`
290
290
  7. Save learnings in \`qa/memory/\` with proper naming and metadata. Always update \`qa/memory/INDEX.md\` after adding or updating any memory file.
291
- 8. **On module/sprint completion** (all TCs passing): (1) update the module status row in \`qa/README.md\`; (2) move the completed sprint checklist from \`AGENT-NEXT-STEPS.md\` to the \`## Sprint History\` section of \`qa/README.md\`; (3) trim \`AGENT-NEXT-STEPS.md\` so it contains only the next sprint — never let it accumulate more than one active sprint.
291
+ 8. **Sprint/module completion criteria:** A sprint or module is complete when every TC in scope has one of these three valid states - and no other: (a) \`test()\` passing: the app meets the spec; (b) \`test.fail()\`: the app violates the spec **and** a file \`qa/06-defects/open/DEF-SIS-NNN_*.md\` exists documenting which business rule is violated and what the app actually does; (c) \`test.skip()\`: the TC is not yet executable, annotated with \`PENDING-CODE\` and a linked issue. A sprint is **not** complete if any TC is failing without \`test.fail()\`, or has \`test.fail()\` without a defect file, or has \`test.skip()\` without a \`PENDING-CODE\` reason.
292
+ 9. **On sprint/module completion:** (1) update the module status row in \`qa/README.md\`; (2) move the completed sprint checklist from \`AGENT-NEXT-STEPS.md\` to the \`## Sprint History\` section of \`qa/README.md\`; (3) trim \`AGENT-NEXT-STEPS.md\` so it contains only the next sprint — never let it accumulate more than one active sprint.
293
+ 10. **Assertion polarity rule (BLOCKING):** A test must assert what the **spec** requires, not what the app currently does. If the app fails to meet the spec, use \`test.fail()\` + correct assertion to document the defect — **never invert or weaken an assertion to make a test go green.** A failing test that exposes a real defect is always more valuable than a passing test that hides one.
294
+ 11. **Character encoding safety (BLOCKING):** In ALL generated content (markdown, scripts, test titles, ADO fields, config files), never use characters at these Unicode code points: em-dash (U+2014), en-dash (U+2013), horizontal ellipsis (U+2026), smart/curly quotes (U+201C, U+201D, U+2018, U+2019), or directional arrows (U+2190-U+21FF). Use ASCII equivalents instead: \` - \` (hyphen with surrounding spaces) for dashes, \`...\` for ellipsis, \`"\` and \`'\` for quotes, \`->\` and \`<-\` for arrows. Exception: Latin Extended characters (U+00C0-U+024F) are always permitted - this covers Spanish vowels with accents (U+00E0-U+00FA), n with tilde (U+00F1/U+00D1), and u/o with umlaut (U+00FC/U+00DC, U+00F6/U+00D6).
292
295
 
293
296
  ## Formatting rules for generated content
294
297
  1. Generate the output in \`UTF-8\` encoding without BOM (\`UTF-8\`, no signature).
@@ -27,11 +27,38 @@ description: >
27
27
  2. `qa/07-automation/playwright.config.ts` — confirm project name maps to submodule tag
28
28
  3. `qa/07-automation/fixtures/auth.ts` — identify available auth fixtures
29
29
  4. TC list (from test plan or Stage 4) — defines which TCs to automate in this session
30
+ 5. `qa/qa-framework.config.json` → screenshotPath, automationRoot
30
31
 
31
32
  ---
32
33
 
33
34
  ## Process
34
35
 
36
+ ### Step 0 — Pre-inspection (MANDATORY before writing any test code)
37
+
38
+ For every new submodule, run a dedicated inspection script before writing tests.
39
+ **Template**: `references/dom-inspection-template.js` — copy, set the 4 constants at the top, run.
40
+
41
+ 1. Copy `references/dom-inspection-template.js` → `qa/07-automation/_inspect-{submodule}.js`
42
+ 2. Set the 4 constants at the top of the script:
43
+ - `MODULE_ROUTE` — the submodule's URL path (e.g. `'/Users'`, `'/Products/list'`)
44
+ - `APP_SHELL_SEL` — a selector that confirms the SPA has loaded (nav, sidebar, app shell)
45
+ - `CREATE_BTN` — regex matching the create/new button label in this app
46
+ - `BASE_URL` — already read from `process.env.QA_BASE_URL`
47
+ 3. Run: `QA_BASE_URL=<env> node _inspect-{submodule}.js`
48
+ 4. Paste key findings as a comment block at the **top of the `.spec.ts` file** before writing any test:
49
+ ```
50
+ // INSPECTION: {date}
51
+ // Grid headers: [...]
52
+ // Create form: inputs[0]=Name(maxLen=100), dd[0]=Category(lazy=true)
53
+ // Validation msg: "El campo Nombre es obligatorio."
54
+ // API: POST /api/{Entity} { pageNumber, pageSize }
55
+ ```
56
+
57
+ The template covers 5 inspection areas automatically: SPA warmup, list view (grid headers + seed row + buttons), create form (all input types, tabs, validation messages), unauth redirect, and API call interception. Adapt the script for tabs or nested forms by adding steps after the create form section.
58
+
59
+ **Never skip this step for submodules inside complex forms (tabs, dialogs, nested entities).
60
+ The cost of one inspection run is far lower than 10+ debugging iterations.**
61
+
35
62
  ### Step 1 — Scan for blockers
36
63
 
37
64
  Before writing a line of code:
@@ -39,13 +66,24 @@ Before writing a line of code:
39
66
  - Confirm `playwright.config.ts` exists; scaffold it if missing using `references/config-checklist.md`
40
67
  - Verify `fixtures/auth.ts` has the roles required by this submodule
41
68
 
69
+ ### Step 1b — Decide POM vs inline locators
70
+
71
+ Create a Page Object in `qa/07-automation/e2e/page-objects/{SubmoduleName}Page.ts` when ANY of these is true:
72
+ - The submodule has a form with 5+ fields (locators will be reused across P0 + P1 suites)
73
+ - The submodule is referenced as a precondition by another submodule (BP, tabs, modals)
74
+ - The submodule requires navigation to 2+ distinct URLs (list + create + edit)
75
+
76
+ Otherwise, inline locators are acceptable for simple catalog submodules (single URL, < 5 fields).
77
+
78
+ POM template: `references/pom-template.md`
79
+
42
80
  ### Step 2 — Scaffold spec file
43
81
 
44
82
  Full spec file template and 7 implementation patterns: `references/patterns.md`
45
83
 
46
84
  Required scaffold elements:
47
85
  - `import { test, expect } from '@playwright/test'`
48
- - `const EXEC_IDX = process.env.EXEC_IDX ?? '0'` (prevents DB conflicts in parallel runs)
86
+ - `const EXEC_IDX = Math.floor(Date.now() / 60_000) % 100_000` (prevents DB conflicts in parallel runs)
49
87
  - `test.describe('{Submodule Name}', () => { ... })`
50
88
  - Use fixture-based auth (never hardcode credentials)
51
89
 
@@ -66,6 +104,20 @@ For each TC:
66
104
  - Locators attached to dynamic data must use `EXEC_IDX` suffix
67
105
  - All test data cleared in `afterAll` — never leave residue
68
106
 
107
+ ### Step 4b — Assertion polarity check (MANDATORY before committing any test)
108
+
109
+ Every assertion must be verified against the spec, not against the app's
110
+ observed output.
111
+
112
+ **Rule**: if the app does X but the spec requires NOT X, use `test.fail()`
113
+ with the correct assertion — do not adapt the assertion to match the app.
114
+
115
+ Red flags that require review before proceeding:
116
+ - `expect(X).toBe(false)` where the spec describes a positive condition
117
+ - Assertions with `|| fallback`, `?.`, or `Math.max(a, b)` to avoid zero/falsy
118
+ - Assertions inside an `if (condition)` that only execute on the happy path
119
+ - A test that starts passing after a refactor without a clear spec justification
120
+
69
121
  ### Step 5 — Module Completion Checklist
70
122
 
71
123
  Before marking the submodule as ✅ Automation Complete:
@@ -84,11 +136,13 @@ Before marking the submodule as ✅ Automation Complete:
84
136
  |---|---|
85
137
  | `EXEC_IDX` | Any TC that creates persistent data |
86
138
  | `auth-fixture` | All authenticated TCs |
87
- | `fast-fail` | P0 TCs — skip P1+ if P0 fails |
139
+ | `fast-fail` | Element state checks where the default timeout would cause silent hangs |
88
140
  | `email-intercept` | Email verification flows |
89
- | `skip-conditional` | Scenarios blocked by PENDING-CODE |
141
+ | `skip-conditional` | TCs blocked by a known app defect (DEF-xxx) |
90
142
  | `beforeAll-setup` | Shared precondition across TC group |
91
143
  | `password-env` | Any credential usage |
144
+ | `afterAll-cleanup` | When a test assigns, creates, or modifies persistent data |
145
+ | `debug-screenshots` | beforeAll warmup verification, catch blocks in setup, and inspection scripts |
92
146
 
93
147
  See `references/patterns.md` for full TypeScript implementations.
94
148
 
@@ -97,5 +151,6 @@ See `references/patterns.md` for full TypeScript implementations.
97
151
  ## Outputs
98
152
 
99
153
  - `qa/07-automation/e2e/{module}/{submodule}.spec.ts`
154
+ - `qa/07-automation/e2e/page-objects/{SubmoduleName}Page.ts` (when POM criteria met)
100
155
  - `qa/07-automation/playwright.config.ts` (updated or confirmed)
101
156
  - `qa/README.md` automation status updated
@@ -0,0 +1,245 @@
1
+ /**
2
+ * DOM / API Inspection Script — {SUBMODULE_DISPLAY_NAME}
3
+ *
4
+ * Purpose: Capture real UI structure before writing automation specs.
5
+ * Run ONCE per submodule. Output drives selector and fixture decisions.
6
+ *
7
+ * Usage:
8
+ * QA_BASE_URL=https://your-qa-env.example.com node _inspect-{submodule}.js
9
+ *
10
+ * Pre-requisites:
11
+ * - Valid storageState in .auth/user-default.json (run global-setup first)
12
+ * - playwright installed: npm install @playwright/test
13
+ *
14
+ * ─── What to do with the output ─────────────────────────────────────────────
15
+ * Paste key findings as a comment block at the top of the generated .spec.ts:
16
+ *
17
+ * // INSPECTION: {YYYY-MM-DD}
18
+ * // Grid headers: ["Col1", "Col2", ...]
19
+ * // Create form: inputs[0]=Name(maxLen=100), dd[0]=Category(lazy=true)
20
+ * // Validation msg: "El campo Nombre es obligatorio."
21
+ * // API: POST /api/{Entity} { pageNumber, pageSize }
22
+ *
23
+ * ── For tabbed/nested forms, add steps after [3] following this pattern: ────
24
+ * await page.getByRole('tab', { name: /tab name/i }).click();
25
+ * await page.waitForTimeout(1_000);
26
+ * // then re-run the inputs/dropdowns/buttons queries inside the panel
27
+ * ─────────────────────────────────────────────────────────────────────────────
28
+ */
29
+
30
+ 'use strict';
31
+
32
+ const { chromium } = require('@playwright/test');
33
+ const path = require('path');
34
+
35
+ // ── Configuration — edit these 4 constants, leave the rest ───────────────────
36
+ const STATE_FILE = path.resolve(__dirname, '.auth/user-default.json');
37
+ const BASE_URL = process.env.QA_BASE_URL || '';
38
+ const MODULE_ROUTE = '/{EntityRoute}'; // e.g. '/Users', '/Orders/list'
39
+ const APP_SHELL_SEL = 'nav, [role="navigation"], .sidebar'; // selector that confirms SPA loaded
40
+ const CREATE_BTN = /new|nuevo|agregar|add/i; // regex matching your app's create button
41
+ // ─────────────────────────────────────────────────────────────────────────────
42
+
43
+ async function run() {
44
+ if (!BASE_URL) {
45
+ console.error('ERROR: QA_BASE_URL is not set. Run with: QA_BASE_URL=https://... node _inspect-{submodule}.js');
46
+ process.exit(1);
47
+ }
48
+
49
+ const browser = await chromium.launch({ headless: true });
50
+ const ctx = await browser.newContext({ storageState: STATE_FILE, baseURL: BASE_URL });
51
+ ctx.setDefaultNavigationTimeout(120_000);
52
+ const page = await ctx.newPage();
53
+
54
+ console.log('BASE_URL:', BASE_URL);
55
+ console.log('MODULE_ROUTE:', MODULE_ROUTE);
56
+
57
+ // ── 0. Intercept API calls — runs passively throughout the entire session ──
58
+ const apiCalls = [];
59
+ page.on('request', req => {
60
+ try {
61
+ if (!req.url().includes(new URL(BASE_URL).hostname)) return;
62
+ } catch { return; }
63
+ const method = req.method();
64
+ if (['GET', 'POST', 'PUT', 'PATCH', 'DELETE'].includes(method)) {
65
+ apiCalls.push({
66
+ method,
67
+ url: req.url().replace(BASE_URL, ''),
68
+ body: req.postData()?.substring(0, 200) ?? null,
69
+ });
70
+ }
71
+ });
72
+
73
+ // ── 1. SPA / WASM warmup ──────────────────────────────────────────────────
74
+ console.log('\n[STEP 1] SPA warmup...');
75
+ await page.goto('/');
76
+ try {
77
+ await page.waitForSelector(APP_SHELL_SEL, { timeout: 120_000 });
78
+ console.log(' App shell visible');
79
+ } catch {
80
+ console.log(' App shell not found with selector:', APP_SHELL_SEL);
81
+ console.log(' Update APP_SHELL_SEL. Body text (200 chars):',
82
+ await page.$eval('body', b => b.innerText.substring(0, 200)).catch(() => 'N/A'));
83
+ }
84
+
85
+ // ── 2. List view ──────────────────────────────────────────────────────────
86
+ console.log('\n[STEP 2] List view:', MODULE_ROUTE);
87
+ await page.goto(MODULE_ROUTE);
88
+ await page.waitForSelector('[role="columnheader"], th, table', { timeout: 30_000 }).catch(() => null);
89
+ await page.waitForTimeout(1_500);
90
+
91
+ console.log(' URL:', page.url());
92
+ console.log(' Title:', await page.title());
93
+
94
+ const headers = await page.$$eval(
95
+ '[role="columnheader"], th',
96
+ ths => ths.map(th => th.innerText.trim()).filter(t => t && t.length < 80)
97
+ ).catch(() => []);
98
+ console.log(' Grid headers:', JSON.stringify(headers));
99
+
100
+ const firstRow = await page.$$eval(
101
+ 'table tbody tr:first-child td',
102
+ tds => tds.map(td => td.innerText.trim()).filter(t => t).slice(0, 8)
103
+ ).catch(() => []);
104
+ console.log(' First row (seed data):', JSON.stringify(firstRow));
105
+
106
+ const allBtns = await page.$$eval(
107
+ 'button',
108
+ btns => [...new Set(btns.map(b => b.innerText.trim()).filter(t => t && t.length < 60))]
109
+ ).catch(() => []);
110
+ console.log(' Buttons on list page:', JSON.stringify(allBtns));
111
+
112
+ // ── 3. Create form ────────────────────────────────────────────────────────
113
+ console.log('\n[STEP 3] Create form...');
114
+ const newBtn = page.getByRole('button', { name: CREATE_BTN });
115
+ const newBtnVisible = await newBtn.isVisible({ timeout: 5_000 }).catch(() => false);
116
+
117
+ if (newBtnVisible) {
118
+ await newBtn.click();
119
+ await page.waitForLoadState('domcontentloaded');
120
+ await page.waitForTimeout(2_000);
121
+ console.log(' Form URL:', page.url());
122
+
123
+ // Text inputs (all types)
124
+ const textInputs = await page.$$eval(
125
+ 'input[type="text"], input[type="email"], input[type="number"], input[type="date"], textarea',
126
+ inps => inps.map((inp, i) => ({
127
+ nth: i,
128
+ type: inp.type,
129
+ placeholder: inp.placeholder || null,
130
+ maxLength: inp.maxLength > 0 ? inp.maxLength : null,
131
+ required: inp.required,
132
+ disabled: inp.disabled,
133
+ name: inp.name || inp.id || null,
134
+ }))
135
+ ).catch(() => []);
136
+ console.log('\n Text inputs:', JSON.stringify(textInputs, null, 2));
137
+
138
+ // Native <select> elements
139
+ const selects = await page.$$eval(
140
+ 'select',
141
+ sels => sels.map((sel, i) => ({
142
+ nth: i,
143
+ name: sel.name || sel.id || null,
144
+ required: sel.required,
145
+ disabled: sel.disabled,
146
+ options: Array.from(sel.options).map(o => o.text).slice(0, 15),
147
+ }))
148
+ ).catch(() => []);
149
+ console.log(' <select> elements:', JSON.stringify(selects, null, 2));
150
+
151
+ // Custom dropdowns — uses ARIA roles (framework-agnostic)
152
+ const customDDs = await page.$$eval(
153
+ '[role="combobox"], [role="listbox"], [aria-haspopup="listbox"]',
154
+ dds => dds.map((dd, i) => ({
155
+ nth: i,
156
+ role: dd.getAttribute('role'),
157
+ text: dd.innerText.trim().substring(0, 80),
158
+ disabled: dd.getAttribute('aria-disabled') === 'true',
159
+ expanded: dd.getAttribute('aria-expanded') === 'true',
160
+ }))
161
+ ).catch(() => []);
162
+ console.log(' Custom dropdowns (ARIA):', JSON.stringify(customDDs, null, 2));
163
+
164
+ // Checkboxes and radio buttons
165
+ const checkboxes = await page.$$eval(
166
+ 'input[type="checkbox"], input[type="radio"]',
167
+ els => els.map((el, i) => ({
168
+ nth: i,
169
+ type: el.type,
170
+ checked: el.checked,
171
+ name: el.name || el.id || null,
172
+ label: el.labels?.[0]?.innerText?.trim() ?? null,
173
+ }))
174
+ ).catch(() => []);
175
+ console.log(' Checkboxes/radios:', JSON.stringify(checkboxes, null, 2));
176
+
177
+ // Tabs (if the form is tabbed)
178
+ const tabs = await page.$$eval(
179
+ '[role="tab"]',
180
+ tabs => tabs.map((t, i) => ({ nth: i, text: t.innerText.trim() }))
181
+ ).catch(() => []);
182
+ if (tabs.length > 0) console.log(' Tabs:', JSON.stringify(tabs));
183
+
184
+ // Validation — submit empty form to capture required-field messages
185
+ const saveBtn = page.getByRole('button', { name: /guardar|save|submit|crear|create/i }).first();
186
+ if (await saveBtn.isVisible({ timeout: 3_000 }).catch(() => false)) {
187
+ await saveBtn.click();
188
+ await page.waitForTimeout(1_500);
189
+ const msgs = await page.$$eval(
190
+ '[class*="validation"], [class*="error"], [class*="invalid"], [aria-live="polite"], [aria-live="assertive"]',
191
+ els => els.map(el => el.innerText.trim()).filter(t => t && t.length < 200)
192
+ ).catch(() => []);
193
+ console.log('\n Validation messages (empty submit):', JSON.stringify(msgs));
194
+ }
195
+
196
+ // ── ADD EXTRA STEPS HERE for tabbed/nested forms ──────────────────────
197
+ // Example — inspect a specific tab:
198
+ // await page.getByRole('tab', { name: /identif/i }).click();
199
+ // await page.waitForTimeout(1_000);
200
+ // const tabDDs = await page.locator('[role="tab"][aria-selected="true"] + ... .rz-dropdown')...
201
+ // ─────────────────────────────────────────────────────────────────────
202
+
203
+ } else {
204
+ console.log(' Create button NOT found. Searched for:', CREATE_BTN.toString());
205
+ console.log(' Buttons visible:', allBtns);
206
+ }
207
+
208
+ // ── 4. Unauth redirect check ──────────────────────────────────────────────
209
+ console.log('\n[STEP 4] Unauth redirect check...');
210
+ const page2 = await ctx.newPage();
211
+ await page2.goto('/');
212
+ await page2.evaluate(() => {
213
+ localStorage.clear();
214
+ sessionStorage.clear();
215
+ });
216
+ await page2.reload({ waitUntil: 'domcontentloaded' });
217
+ await page2.goto(MODULE_ROUTE);
218
+ await page2.waitForTimeout(3_000);
219
+ const loginVisible = await page2.locator(
220
+ 'input[type="password"], [id*="login"], form[action*="login"]'
221
+ ).first().isVisible({ timeout: 5_000 }).catch(() => false);
222
+ console.log(' Login form visible without auth:', loginVisible);
223
+ console.log(' URL after clearing session:', page2.url());
224
+ await page2.close();
225
+
226
+ // ── 5. API calls summary ──────────────────────────────────────────────────
227
+ console.log('\n[STEP 5] API calls captured during session:');
228
+ const unique = [...new Map(apiCalls.map(c => [`${c.method} ${c.url}`, c])).values()];
229
+ if (unique.length === 0) {
230
+ console.log(' (none captured — app may use WASM-level data, not REST calls)');
231
+ } else {
232
+ unique.forEach(c => {
233
+ console.log(` ${c.method} ${c.url}`);
234
+ if (c.body) console.log(` body: ${c.body}`);
235
+ });
236
+ }
237
+
238
+ await browser.close();
239
+ console.log('\n[DONE] Paste key findings into a comment block at the top of the .spec.ts file.');
240
+ }
241
+
242
+ run().catch(err => {
243
+ console.error('FATAL:', err);
244
+ process.exit(1);
245
+ });
@@ -165,6 +165,58 @@ await page.evaluate(
165
165
 
166
166
  ---
167
167
 
168
+ ## Pattern 8: afterAll Cleanup for Data-Mutating Tests
169
+
170
+ When a test assigns, creates, or modifies persistent data (e.g., assigning a role, linking a record),
171
+ always implement `afterAll` cleanup — otherwise the second run finds a different state and fails.
172
+
173
+ ```typescript
174
+ afterAll(async () => {
175
+ // Restore fixture to pre-test state via direct API call
176
+ const token = await getFixtureToken(); // from storageState
177
+ const resp = await fetch(`${API_BASE}/api/{Entity}/Edit`, {
178
+ method: 'PUT',
179
+ headers: { 'Authorization': Bearer ${token}, 'Content-Type': 'application/json' },
180
+ body: JSON.stringify({ /* original state without the assigned record */ }),
181
+ });
182
+ console.log(`[afterAll] Cleanup: ${resp.status}`);
183
+ });
184
+ ```
185
+
186
+ Rule: if beforeAll sets up a fixture by reading state from the API, afterAll must restore that same state.
187
+ Design cleanup from the start — retrofitting it costs as many runs as the test itself.
188
+
189
+ ---
190
+
191
+ ### Pattern 9: Debug Screenshots — Path-bound Artifacts
192
+
193
+ Screenshots MUST always be saved to `agentSettings.screenshotPath` defined in `qa/qa-framework.config.json`. Never save to the workspace root or any ad-hoc path.
194
+
195
+ ```typescript
196
+ // Read screenshotPath from config or use default
197
+ const DIAG_DIR = 'qa/07-automation/e2e/diagnosis';
198
+
199
+ // On test failure — capture state for diagnosis:
200
+ await page.screenshot({ path: `${DIAG_DIR}/${SUBMODULE}-${tcId}-fail-${Date.now()}.png` });
201
+
202
+ // In beforeAll for warm-up verification:
203
+ await page.screenshot({ path: `${DIAG_DIR}/{submodule}-warmup.png` });
204
+
205
+ // In inspection scripts (_inspect-*.js) — always use the same dir:
206
+ await page.screenshot({ path: `${DIAG_DIR}/_inspect-${SUBMODULE}.png`, fullPage: true });
207
+ ```
208
+
209
+ Naming convention: {submodule}-{context}-{optional-timestamp}.png
210
+ When to capture:
211
+
212
+ * End of beforeAll warmup — confirms the app state before tests run
213
+ * Inside catch blocks in beforeAll/beforeEach — documents the failure condition
214
+ * Inside inspection scripts — captures the rendered UI state at each navigation step
215
+ * Never inside individual test() bodies — Playwright's built-in trace (--trace on) already captures this
216
+ * Enforcement: The diagnosis/ folder is gitignored. Screenshots are local-only artifacts, never committed to the repo.
217
+
218
+ ---
219
+
168
220
  ## Coverage Mapping Template
169
221
 
170
222
  Create `COVERAGE-MAPPING.md` in the test directory after automation is complete:
@@ -52,7 +52,9 @@ Category summary:
52
52
  - **F — Network dependency**: external call not mocked; add mock or skip
53
53
  - **G — Environment**: missing env var or wrong base URL; fix config
54
54
  - **H — CI environment**: OS/browser difference; add retry or tag as `@ci-skip`
55
- - **I — Logic error**: test assertion is wrong; fix assertion to match spec
55
+ - **I — Logic error**: test assertion does not match the spec; fix assertion
56
+ to match spec — **not** to match wrong app behavior. If the app is at fault,
57
+ use `test.fail()` instead of correcting the assertion direction.
56
58
 
57
59
  ### Step 3 — Apply fixes in priority order
58
60
 
@@ -64,6 +66,14 @@ For each fix:
64
66
  3. Re-run the full describe block to verify no regression
65
67
  4. Note confidence score (0-100) for each fix
66
68
 
69
+ **Category I special rule — assertion polarity:**
70
+ Before flipping or weakening an assertion, confirm its direction against the
71
+ spec. If the spec says the condition should hold and the app violates it →
72
+ the original assertion was *correct* and the app is broken → use `test.fail()`
73
+ + open a defect. Do NOT invert the assertion.
74
+ A test flipped from failing to passing by inverting its assertion is masking
75
+ a defect - which is worse than a failing test.
76
+
67
77
  ### Step 4 — Confidence scoring
68
78
 
69
79
  Calculate composite confidence:
@@ -58,6 +58,25 @@ await expect(page.locator('.toast')).toHaveText('TEXTO_QUE_NO_EXISTE');
58
58
 
59
59
  ---
60
60
 
61
+ ### Persistence check (for tests that write data)
62
+
63
+ For tests that create, update, or delete records: do not rely solely on a success toast.
64
+ Verify persistence with a follow-up API call or page reload:
65
+
66
+ ```typescript
67
+ // After clicking Guardar:
68
+ await expect(page.locator('.toast')).toHaveText('Guardado exitosamente');
69
+ // Verify via API — a passing toast is a false positive if the save failed silently
70
+ const resp = await page.request.get(`${API_BASE}/api/{Entity}/{id}`);
71
+ const data = await resp.json();
72
+ expect(data.fieldChanged).toBe(expectedValue);
73
+ ```
74
+
75
+ A test that only checks the UI toast is Category B (Wrong assertion) if the spec requires
76
+ verifying the change was persisted.
77
+
78
+ ---
79
+
61
80
  ## Confidence Scoring
62
81
 
63
82
  | Level | Criteria |