@keber/qa-framework 1.5.1 → 1.6.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -1,309 +0,0 @@
1
- # Agent Instructions: Test Stabilization
2
-
3
- > ⚠️ **DEPRECATED**: This file is superseded by `.github/skills/qa-test-stabilization/SKILL.md`.
4
- > Kept as reference during the transition period. Do not update this file.
5
-
6
- **File**: `agent-instructions/04b-test-stabilization.md`
7
- **Purpose**: Instructions for exhaustively reviewing and stabilizing generated Playwright tests to ensure they are logically correct, technically sound, and free of false positives and false negatives before ADO integration.
8
-
9
- ---
10
-
11
- ## Stage Context
12
-
13
- | Field | Value |
14
- |-------|-------|
15
- | Stage number | 3.5 |
16
- | Stage name | Test Stabilization |
17
- | Preceding stage | Stage 4 — Automation Generation (`04-automation-generation.md`) |
18
- | Following stage | Stage 5 — ADO Integration (`05-ado-integration.md`) |
19
- | Can be skipped? | No — ADO integration with unreliable tests produces misleading test run history |
20
- | Required inputs | Generated `.spec.ts` files in `qa/07-automation/e2e/`; approved `05-test-scenarios.md` for the module |
21
- | Produced outputs | Stabilized spec files; `STABILIZATION-REPORT.md` per submodule |
22
- | Exit criterion | Confidence ≥ 90% that every test produces a true positive on failure and a true negative on pass |
23
-
24
- ---
25
-
26
- ## When to use
27
-
28
- Run this stage immediately after **Stage 4 — Automation Generation** and before any ADO
29
- integration or CI pipeline registration. This stage is mandatory regardless of whether tests
30
- are passing or failing after generation.
31
-
32
- Also re-run this stage when:
33
- - Sprint changes introduce significant new test code (more than 3 new spec files)
34
- - A test suite starts flaking in CI after a previously stable period
35
- - A test is suspected of being a false positive (passing despite the feature being broken)
36
- - A test is suspected of being a false negative (failing despite the feature being correct)
37
-
38
- ---
39
-
40
- > **Scope boundary**: This stage operates exclusively on test code (`.spec.ts`, fixtures,
41
- > helpers, `global-setup.ts`, `playwright.config.ts`). The application source code is
42
- > strictly out of scope. Do not modify, patch, or work around application behavior to make
43
- > tests pass — document the mismatch as a defect instead.
44
-
45
- ---
46
-
47
- > **Coverage boundary**: Do not add new tests or increase coverage during this stage. If a
48
- > gap in coverage is identified, note it in the STABILIZATION-REPORT.md for Stage 6
49
- > (Maintenance) to address. The goal is correctness of existing tests, not completeness.
50
-
51
- ---
52
-
53
- ## Step 1 — Establish the Reference Baseline
54
-
55
- Before running any tests, collect the ground truth against which correctness will be judged.
56
-
57
- 1. Open `05-test-scenarios.md` for the target module/submodule. For each TC row, note:
58
- - TC-ID and title
59
- - Priority (P0/P1/P2/P3)
60
- - Expected result (acceptance criterion)
61
- - Automatable status (Yes / Partial / No)
62
-
63
- 2. Open `COVERAGE-MAPPING.md` for the module. For each row:
64
- - Confirm the `.spec.ts` file and test function name exist
65
- - Flag any rows where the spec or function is missing (these are generation gaps, not
66
- stabilization issues — document in the report but do not create new tests)
67
-
68
- 3. If `05-test-scenarios.md` or `COVERAGE-MAPPING.md` are absent or incomplete, stop and
69
- complete Stage 4 before proceeding.
70
-
71
- ---
72
-
73
- ## Step 2 — First Run: Collect Baseline Results
74
-
75
- Run the full suite for the target module without any modifications:
76
-
77
- ```bash
78
- npx playwright test qa/07-automation/e2e/tests/{module-kebab}/ --reporter=list
79
- ```
80
-
81
- Record the outcome of every test:
82
-
83
- | TC-ID | Test title | Result | Notes |
84
- |-------|-----------|--------|-------|
85
- | TC-MOD-SUB-001 | ... | ✅ pass / ❌ fail / ⚠️ flaky / ⏭ skip | Error message if fail |
86
-
87
- Collect trace artifacts for every failing test:
88
-
89
- ```bash
90
- npx playwright test ... --trace=on
91
- ```
92
-
93
- Do NOT make any changes yet. The first run result is the diagnostic baseline.
94
-
95
- ---
96
-
97
- ## Step 3 — Classify Every Failing or Skipped Test
98
-
99
- For each non-passing test, determine its failure category before writing any code.
100
-
101
- ### Failure Categories
102
-
103
- | Category | Definition | Correct action |
104
- |----------|-----------|----------------|
105
- | **A — Test Bug: Wrong selector** | The selector no longer matches the UI element | Fix selector in `.spec.ts` |
106
- | **B — Test Bug: Wrong assertion** | The assertion does not match the specified acceptance criterion | Fix assertion to match spec |
107
- | **C — Test Bug: Wrong flow** | The test steps don't reflect the actual user flow described in the TC | Rewrite steps to match TC |
108
- | **D — Test Bug: Fragile timing** | Test fails intermittently due to missing `await`, race conditions, or missing `waitFor` | Add proper async handling |
109
- | **E — Test Bug: Incorrect data** | Test data is malformed, not unique, or conflicts with QA environment state | Fix data generation or provisioning |
110
- | **F — App Bug** | The test is correct and the application is not behaving as the TC specifies | Do not fix the test; file a defect |
111
- | **G — TC Mismatch** | The spec TC is ambiguous or incorrect vs the actual implemented behavior | Do not fix the test or the app; update `05-test-scenarios.md` and note as a spec defect |
112
- | **H — Infra / Environment** | Test fails due to QA environment unavailability, network, or credentials | Retry in a clean environment; do not classify as a test bug until confirmed |
113
- | **I — Intentional skip** | Test is marked `test.skip()` due to a known defect or pending feature | Verify skip reason is still valid; update `06-defects/open/` reference |
114
-
115
- For each failing test, write your classification in the working STABILIZATION-REPORT.md
116
- before making any change.
117
-
118
- ### Decision Protocol
119
-
120
- ```
121
- Is the test step sequence and assertion derived correctly from 05-test-scenarios.md?
122
- No → Category G (TC Mismatch — fix the spec, not the test)
123
- Yes ↓
124
- Does the application behave as the TC acceptance criterion describes?
125
- No → Category F (App Bug — file a defect, skip the test with a DEF reference)
126
- Yes ↓
127
- Does the failure appear in every run (deterministic)?
128
- No → Category D or H (timing or environment — diagnose further)
129
- Yes ↓
130
- Is the failure caused by a selector, assertion, data, or flow issue in the test code?
131
- → Category A, B, C, or E (Test Bug — fix the test code)
132
- ```
133
-
134
- ---
135
-
136
- ## Step 4 — Fix Iterations
137
-
138
- Apply fixes one category at a time, in this order: E → A → C → B → D.
139
- (Data problems mask selector problems; flow problems mask assertion problems.)
140
-
141
- After each batch of fixes, re-run the affected tests:
142
-
143
- ```bash
144
- npx playwright test qa/07-automation/e2e/tests/{module-kebab}/{file}.spec.ts --reporter=list
145
- ```
146
-
147
- Do not move to the next category until the current category's tests are stable across
148
- **at least two consecutive runs**.
149
-
150
- ### Rules for fixing test code
151
-
152
- - **Selectors**: prefer `getByRole`, `getByLabel`, `getByTestId` — avoid CSS/XPath selectors
153
- that depend on position or class names that aren't semantic
154
- - **Assertions**: every assertion must trace back to a specific acceptance criterion in the TC;
155
- remove or replace any `expect().toBeVisible()` that does not correspond to a TC requirement
156
- - **Waits**: prefer `await expect(locator).toBeVisible()` over `page.waitForTimeout()`;
157
- never use `waitForTimeout` as a fix for timing — diagnose the actual cause
158
- - **Data**: use `EXEC_IDX + RUN_SALT` for unique identifiers (see ISSUE-02 note below);
159
- provision data in `beforeAll`, not inside individual tests
160
- - **Skips**: when skipping for a Category F (App Bug), always include the defect reference:
161
- ```typescript
162
- test.skip(true, 'DEF-{NNN}: {one-line bug description}. Filed {YYYY-MM-DD}.');
163
- ```
164
-
165
- > ⚠️ **EXEC_IDX collision**: Two runs within the same 60-second window produce identical
166
- > `EXEC_IDX` values. For fields requiring guaranteed uniqueness, add a salt:
167
- > ```typescript
168
- > const EXEC_IDX = Math.floor(Date.now() / 60_000) % 100_000;
169
- > const RUN_SALT = Math.floor(Math.random() * 9999).toString().padStart(4, '0');
170
- > ```
171
-
172
- ---
173
-
174
- ## Step 5 — Validate Passing Tests for False Positives
175
-
176
- A passing test is not necessarily a correct test. After fixing failing tests, audit every
177
- **passing** test for false positives.
178
-
179
- For each passing test, perform one of the following checks:
180
-
181
- ### Negation check (preferred)
182
- Temporarily modify the test to assert the opposite of the expected result, then run it.
183
- If it still passes — the test is a false positive (asserting nothing meaningful).
184
-
185
- ```typescript
186
- // Original: expect(toast).toHaveText('Guardado exitosamente');
187
- // Negation check: replace temporarily with:
188
- expect(toast).toHaveText('TEXTO_QUE_NO_EXISTE_JAMAS');
189
- // If this passes → the original assertion is not actually evaluating the toast content
190
- ```
191
-
192
- Restore the original assertion after the check.
193
-
194
- ### Trace review (secondary)
195
- Open the Playwright trace for the passing test and step through it. Verify:
196
- - Every action in the `test()` body corresponds to a step in the TC
197
- - Every `expect()` is actually being evaluated (no skipped assertions due to early returns
198
- or unreachable code)
199
- - The test does not pass because an early `expect()` short-circuits before the meaningful one
200
-
201
- ### API/network verification (for tests touching data persistence)
202
- For tests that create, update, or delete records: after the UI action, verify the result
203
- is persisted, not just that a success toast appeared. A test that only checks the toast
204
- is a false positive if the actual save fails silently.
205
-
206
- ---
207
-
208
- ## Step 6 — Iteration and Convergence
209
-
210
- Repeat Steps 4 and 5 until:
211
-
212
- 1. All deterministic failures are resolved (either fixed or legitimately skipped with a
213
- defect reference)
214
- 2. All passing tests have passed the negation check or trace review
215
- 3. No test produces intermittent results across 3 consecutive runs
216
-
217
- **Confidence scoring**: after each iteration, estimate confidence per test:
218
-
219
- | Confidence | Criteria |
220
- |-----------|---------|
221
- | ✅ High (≥90%) | Passed negation check AND matches TC acceptance criterion exactly |
222
- | ⚠️ Medium (70–89%) | Passes consistently; trace reviewed; no negation check yet |
223
- | ❌ Low (<70%) | Intermittent, unclassified failure, or negation check not attempted |
224
-
225
- The stage is complete when **all tests are ✅ High or legitimately skipped** (Category F
226
- with a defect reference or Category I with a valid skip reason).
227
-
228
- ---
229
-
230
- ## Step 7 — Document Findings
231
-
232
- For each submodule, create or update `STABILIZATION-REPORT.md` in the module's automation
233
- directory:
234
-
235
- ```
236
- qa/07-automation/e2e/tests/{module-kebab}/STABILIZATION-REPORT.md
237
- ```
238
-
239
- ### STABILIZATION-REPORT.md structure
240
-
241
- ```markdown
242
- # Stabilization Report — {MODULE} > {SUBMODULE}
243
-
244
- **Date**: YYYY-MM-DD
245
- **Sprint**: {sprint number or label}
246
- **Analyst**: {agent session name or human name}
247
- **Spec reference**: `qa/{module-key}/{submodule-key}/05-test-scenarios.md`
248
-
249
- ## Summary
250
-
251
- | Metric | Value |
252
- |--------|-------|
253
- | Tests evaluated | N |
254
- | Tests stabilized (now ✅ High confidence) | N |
255
- | Tests skipped — App Bug (Category F) | N |
256
- | Tests skipped — Intentional (Category I) | N |
257
- | Remaining low-confidence tests | N |
258
- | Overall confidence | NN% |
259
-
260
- ## Test-by-Test Record
261
-
262
- | TC-ID | Test title | Baseline result | Final result | Category | Changes made | Confidence |
263
- |-------|-----------|-----------------|-------------|----------|-------------|-----------|
264
- | TC-... | ... | ❌ fail | ✅ pass | A | Fixed selector `#old` → `getByRole('button', { name: '...' })` | ✅ High |
265
-
266
- ## App Bugs Filed
267
-
268
- | DEF-ID | TC-ID | Description | Filed date | ADO WI (if created) |
269
- |--------|-------|-------------|-----------|---------------------|
270
-
271
- ## Spec Defects (Category G)
272
-
273
- | TC-ID | Issue | Recommended `05-test-scenarios.md` update |
274
- |-------|-------|------------------------------------------|
275
-
276
- ## Coverage Gaps Identified (do not fix in this stage)
277
-
278
- - {TC-ID}: test mapped in COVERAGE-MAPPING.md but `.spec.ts` function not found
279
- - ...
280
-
281
- ## Decisions Log
282
-
283
- Chronological record of non-trivial decisions made during stabilization:
284
-
285
- - YYYY-MM-DD: Classified TC-XXX failure as Category F after verifying the save endpoint
286
- returns 500 on duplicate RUTs. App bug, not test bug. DEF-001 filed.
287
- - ...
288
- ```
289
-
290
- ---
291
-
292
- ## Step 8 — Update Upstream Artifacts
293
-
294
- After completing STABILIZATION-REPORT.md:
295
-
296
- 1. **If any Category G issues were found** (TC Mismatch): update `05-test-scenarios.md`
297
- with the corrected acceptance criteria. Note the change with `[REVISED - {YYYY-MM-DD}]`.
298
-
299
- 2. **If any Category F issues were found** (App Bug): create defect files in
300
- `qa/{module-key}/{submodule-key}/06-defects/open/DEF-{NNN}.md` for each bug filed.
301
-
302
- 3. **Update `COVERAGE-MAPPING.md`**: set the `Status` column for each TC:
303
- - `Automated` — test is ✅ High confidence
304
- - `Skipped-Defect` — test is skipped via `test.skip()` with a DEF reference
305
- - `Skipped-Infra` — test is skipped due to environment limitation
306
- - `Manual` — TC is intentionally not automated
307
-
308
- 4. **Update session-summary.md** for the submodule: mark Stage 3.5 as ✅ Complete and
309
- record the overall confidence percentage.
@@ -1,247 +0,0 @@
1
- # Agent Instructions: Azure DevOps Integration
2
-
3
- > ⚠️ **DEPRECATED**: This file is superseded by `.github/skills/qa-ado-integration/SKILL.md`.
4
- > Kept as reference during the transition period. Do not update this file.
5
-
6
- **File**: `agent-instructions/05-ado-integration.md`
7
- **Purpose**: Instructions for an agent to create, configure, and maintain Azure DevOps Test Plans integration with Playwright automation results.
8
-
9
- > **This section is OPTIONAL**. Only use it when `integrations.azureDevOps.enabled = true` in `qa-framework.config.json`.
10
-
11
- ---
12
-
13
- ## Prerequisites
14
-
15
- 1. Azure DevOps PAT stored in `ADO_PAT` environment variable
16
- Required scopes: `Work Items (Read, Write)`, `Test Management (Read, Write)`
17
- 2. `ADO_ORG` and `ADO_PROJECT` environment variables set
18
- 3. PowerShell 5.1+ available
19
- 4. `@alex_neo/playwright-azure-reporter` installed in the E2E package
20
- 5. `qa/08-azure-integration/module-registry.json` exists with module paths and plan IDs
21
-
22
- ---
23
-
24
- ## Step 1: Create ADO Test Plan
25
-
26
- If no Test Plan exists yet:
27
-
28
- ```powershell
29
- # qa/08-azure-integration/scripts/create-testplan-from-mapping.ps1
30
- # Reads ado-ids-mapping-{project}.json and creates the plan + suites + TCs in ADO
31
- # Parameters: -Organization $env:ADO_ORG -Project $env:ADO_PROJECT -Token $env:ADO_PAT
32
- .\qa\08-azure-integration\scripts\create-testplan-from-mapping.ps1 `
33
- -Organization $env:ADO_ORG `
34
- -Project $env:ADO_PROJECT `
35
- -Token $env:ADO_PAT `
36
- -MappingFile .\qa\08-azure-integration\ado-ids-mapping-{project}.json
37
- ```
38
-
39
- After running, the script outputs a `TestPlanId` and `SuiteIds`.
40
- Record these in `qa/08-azure-integration/module-registry.json`.
41
-
42
- ---
43
-
44
- ## Step 2: Inject ADO IDs into Spec Files
45
-
46
- Once Test Cases are created in ADO and have Work Item IDs:
47
-
48
- ```powershell
49
- # Dry run first to preview changes
50
- .\qa\08-azure-integration\scripts\inject-ado-ids.ps1 `
51
- -MappingFile .\qa\08-azure-integration\ado-ids-mapping-{project}.json `
52
- -TestDir .\qa\07-automation\e2e\tests `
53
- -DryRun
54
-
55
- # Apply changes
56
- .\qa\08-azure-integration\scripts\inject-ado-ids.ps1 `
57
- -MappingFile .\qa\08-azure-integration\ado-ids-mapping-{project}.json `
58
- -TestDir .\qa\07-automation\e2e\tests
59
- ```
60
-
61
- This converts test titles from:
62
- ```
63
- '[TC-OPER-CAT-001] Access catalog list @P0'
64
- ```
65
- to:
66
- ```
67
- '[22957] Access catalog list @P0'
68
- ```
69
-
70
- Plus adds annotation:
71
- ```typescript
72
- test.info().annotations.push({ type: 'TestCase', description: '22957' });
73
- ```
74
-
75
- ---
76
-
77
- ## Step 3: Configure playwright-azure-reporter
78
-
79
- In `playwright.config.ts`:
80
-
81
- ```typescript
82
- import type { PlaywrightTestConfig } from '@playwright/test';
83
-
84
- const config: PlaywrightTestConfig = {
85
- reporter: [
86
- ['html'],
87
- ['@alex_neo/playwright-azure-reporter', {
88
- orgUrl: `https://dev.azure.com/${process.env.ADO_ORG}`,
89
- token: process.env.ADO_PAT,
90
- planId: Number(process.env.ADO_PLAN_ID),
91
- projectName: process.env.ADO_PROJECT,
92
- isDisabled: !process.env.CI, // only publish in CI
93
- publishTestResultsMode: 'testRun',
94
- testRunTitle: `QA Run - ${new Date().toISOString()}`,
95
- uploadAttachments: true,
96
- attachmentsType: ['screenshot', 'video', 'trace'],
97
- }],
98
- ],
99
- };
100
- export default config;
101
- ```
102
-
103
- **Note**: Set `isDisabled: !process.env.CI` so the reporter only publishes from CI, not from local runs.
104
-
105
- ---
106
-
107
- ## Step 4: Module Registry
108
-
109
- Maintain `qa/08-azure-integration/module-registry.json`:
110
-
111
- ```json
112
- {
113
- "modules": [
114
- {
115
- "name": "operacion",
116
- "specsPath": "qa/07-automation/e2e/tests/operacion",
117
- "planId": 0,
118
- "suiteId": 0,
119
- "description": "Módulo Operación — 7 submodules"
120
- }
121
- ]
122
- }
123
- ```
124
-
125
- The CI pipeline uses this file to detect which module changed (via `git diff`) and run only the relevant spec.
126
-
127
- ---
128
-
129
- ## Step 5: ADO IDs Mapping File
130
-
131
- Maintain `qa/08-azure-integration/ado-ids-mapping-{project}.json`:
132
-
133
- ```json
134
- {
135
- "planId": 22304,
136
- "module": "operacion",
137
- "specsPath": "qa/07-automation/e2e/tests/operacion",
138
- "suites": [
139
- {
140
- "suiteId": 22956,
141
- "suiteName": "Suite 1.1 - Catálogos de Trabajo",
142
- "testCases": [
143
- { "id": 22957, "title": "Access catalog list @P0", "tags": ["automatable", "playwright"] },
144
- { "id": 22958, "title": "Create labor type @P1", "tags": ["automatable", "playwright"] }
145
- ]
146
- }
147
- ]
148
- }
149
- ```
150
-
151
- ---
152
-
153
- ## Step 6: Configure CI Pipeline
154
-
155
- Template in `qa/08-azure-integration/pipelines/azure-pipeline-qa.yml`:
156
-
157
- ```yaml
158
- trigger:
159
- branches:
160
- include:
161
- - {{CI_TRIGGER_BRANCH}}
162
-
163
- pool:
164
- vmImage: 'ubuntu-latest'
165
-
166
- variables:
167
- - group: qa-secrets # Must contain: QA_USER_EMAIL, QA_USER_PASSWORD, QA_BASE_URL, ADO_PAT
168
-
169
- steps:
170
- - checkout: self
171
- fetchDepth: 2
172
-
173
- - task: NodeTool@0
174
- inputs:
175
- versionSpec: '20.x'
176
- displayName: 'Install Node.js 20'
177
-
178
- - script: |
179
- cd qa/07-automation/e2e
180
- npm install
181
- npx playwright install chromium --with-deps
182
- displayName: 'Install Playwright'
183
-
184
- - script: |
185
- cd qa/07-automation/e2e
186
- npx playwright test
187
- displayName: 'Run Playwright tests'
188
- env:
189
- QA_BASE_URL: $(QA_BASE_URL)
190
- QA_USER_EMAIL: $(QA_USER_EMAIL)
191
- QA_USER_PASSWORD: $(QA_USER_PASSWORD)
192
- ADO_PAT: $(ADO_PAT)
193
- ADO_PLAN_ID: $(ADO_PLAN_ID)
194
- CI: 'true'
195
-
196
- - task: PublishTestResults@2
197
- inputs:
198
- testResultsFormat: 'JUnit'
199
- testResultsFiles: '**/test-results/*.xml'
200
- displayName: 'Publish JUnit results'
201
- condition: always()
202
-
203
- - task: PublishPipelineArtifact@1
204
- inputs:
205
- targetPath: 'qa/07-automation/e2e/playwright-report'
206
- artifact: 'playwright-html-report'
207
- displayName: 'Publish HTML report'
208
- condition: always()
209
- ```
210
-
211
- ---
212
-
213
- ## Syncing ADO TC Titles with Spec Titles
214
-
215
- If TC titles drift between ADO and the spec files:
216
-
217
- ```powershell
218
- .\qa\08-azure-integration\scripts\sync-ado-titles.ps1 `
219
- -Organization $env:ADO_ORG `
220
- -Project $env:ADO_PROJECT `
221
- -Token $env:ADO_PAT `
222
- -MappingFile .\qa\08-azure-integration\ado-ids-mapping-{project}.json `
223
- -DryRun
224
- ```
225
-
226
- This reads ADO TC titles and compares them against the spec. Outputs a diff report. Apply with `-DryRun:$false`.
227
-
228
- ---
229
-
230
- ## Credential Security Rules for ADO
231
-
232
- 1. `ADO_PAT` is **never** committed to any file — always comes from env var or CI variable group
233
- 2. `ADO_PAT` is **never** logged, printed, or written to any markdown file
234
- 3. If a PAT appears in any log or output, revoke it immediately in ADO Portal and generate a new one
235
- 4. Variable group `qa-secrets` in Azure DevOps must be scoped to the pipeline only
236
- 5. PAT minimum scopes: `Work Items (Read, Write)`, `Test Management (Read, Write)` — do not use full access PATs
237
-
238
- ---
239
-
240
- ## Troubleshooting
241
-
242
- | Symptom | Likely Cause | Fix |
243
- |---------|-------------|-----|
244
- | Reporter silently does nothing in CI | `isDisabled: true` or `CI` env not set | Verify `process.env.CI` is `'true'` in pipeline |
245
- | Tests report as "Not Run" in ADO | ADO WI ID not in test title or annotation | Re-run `inject-ado-ids.ps1` |
246
- | TestRun created but no results attached | `publishTestResultsMode` wrong | Use `'testRun'` not `'testCase'` |
247
- | PAT error 401 | PAT expired or wrong scopes | Generate new PAT with correct scopes |
@@ -1,135 +0,0 @@
1
- # Agent Instructions: Maintenance and Updates
2
-
3
- > ⚠️ **DEPRECATED**: This file is superseded by `.github/skills/qa-maintenance/SKILL.md`.
4
- > Kept as reference during the transition period. Do not update this file.
5
-
6
- **File**: `agent-instructions/06-maintenance.md`
7
- **Purpose**: Instructions for updating QA artifacts after application code changes.
8
-
9
- ---
10
-
11
- ## When to use
12
-
13
- - A developer reports a feature change or new feature in a module
14
- - A sprint delivers changes to an already-analyzed module
15
- - A defect is resolved and related `test.skip()` calls need to be reactivated
16
- - The QA environment is updated and existing specs need to be refreshed
17
-
18
- ---
19
-
20
- ## Step 1: Identify What Changed
21
-
22
- Gather from the developer or ticket:
23
- - Which module/submodule was changed?
24
- - Was it a new feature, a modification, or a bug fix?
25
- - Are there new routes, fields, or API endpoints?
26
- - Were any existing behaviors removed or changed?
27
-
28
- If the change scope is unclear, navigate to the affected module in the QA environment and compare against the existing `00-inventory.md`.
29
-
30
- ---
31
-
32
- ## Step 2: Spec Update Rules
33
-
34
- ### A new field was added to a form
35
-
36
- 1. Add the field to `00-inventory.md` under UI Elements table
37
- 2. If the field has validation: add a business rule to `01-business-rules.md` (`RN-{MODULE}-{next-number}`)
38
- 3. Add TCs to `05-test-scenarios.md`:
39
- - TC for happy path with the new field
40
- - TC for required validation (if applicable)
41
- - TC for any new business rule
42
-
43
- ### A field was removed from the UI
44
-
45
- 1. Mark the field in `00-inventory.md` as `[REMOVED - {YYYY-MM-DD}]` — do not delete the row
46
- 2. If automation tests reference the field selector: update or skip them
47
- 3. If a business rule only applied to that field: mark as `[DEPRECATED - {YYYY-MM-DD}]`
48
- 4. Leave TCs in `05-test-scenarios.md` marked as `[DEPRECATED]` for traceability
49
-
50
- ### A workflow was changed
51
-
52
- 1. Update `02-workflows.md` with the new flow
53
- 2. Mark the old version as `[PREVIOUS - {YYYY-MM-DD}]`
54
- 3. Review all TCs that test the affected steps
55
- 4. Update expected results in `05-test-scenarios.md`
56
-
57
- ### A new submodule was added
58
-
59
- Follow the full module analysis process in `00-module-analysis.md`.
60
-
61
- ### A defect was fixed
62
-
63
- 1. Find all `test.skip()` calls that reference the DEF-ID or ADO WI
64
- 2. Remove the `test.skip()` wrapper
65
- 3. Run the test at least 2 times with different EXEC_IDX values to confirm stability
66
- 4. Update `06-defects/DEF-{NNN}.md` (or ADO WI) status to Resolved
67
- 5. If the spec assertion was adapted to document the bug (e.g., `toBeFalsy()` for a wrong default), restore the correct assertion
68
-
69
- ---
70
-
71
- ## Step 3: Automation Update Rules
72
-
73
- ### Selectors changed (redesign, framework update)
74
-
75
- 1. Locate the POM file or spec file that contains the old selector
76
- 2. Check `CORRECTIONS` block at top of spec file (convention for documenting selector audits)
77
- 3. Update the selector
78
- 4. Run the affected test 2+ times to confirm the new selector is stable
79
- 5. Add an entry to the `CORRECTIONS` block noting the change and date
80
-
81
- ### New test suite needed
82
-
83
- Follow `04-automation-generation.md` for the new suite.
84
- Update `COVERAGE-MAPPING.md` with the new TCs.
85
-
86
- ### Test became flaky after app update
87
-
88
- Use the 5-layer debugging methodology:
89
- 1. Environment: is QA env stable?
90
- 2. Timeout: did an operation become slower?
91
- 3. Data: did a seed or precondition change?
92
- 4. Selector: did the element structure change?
93
- 5. System: is this a real defect?
94
-
95
- ---
96
-
97
- ## Step 4: Version the Spec
98
-
99
- At the bottom of each updated spec file, add to the changelog:
100
-
101
- ```markdown
102
- ## Changelog
103
-
104
- | Version | Date | Description |
105
- |---------|------|-------------|
106
- | 1.0 | YYYY-MM-DD | Initial creation |
107
- | 1.1 | YYYY-MM-DD | Added RN-{MODULE}-{NNN}: {rule title}. TC count: N → M |
108
- | 1.2 | YYYY-MM-DD | Removed field {name} from UI inventory (deprecated) |
109
- ```
110
-
111
- ---
112
-
113
- ## Step 5: Update qa/README.md
114
-
115
- After every maintenance session, update the master index:
116
- - TC count (if changed)
117
- - Automation status (if changed)
118
- - Last updated date for the module
119
-
120
- If a **sprint was fully completed** as part of this maintenance pass (all TCs passing after the changes):
121
- 1. Move the completed sprint checklist from `AGENT-NEXT-STEPS.md` to the `## Sprint History` section of `qa/README.md`
122
- 2. Delete the completed sprint block from `AGENT-NEXT-STEPS.md`
123
- 3. Confirm `AGENT-NEXT-STEPS.md` contains **only one active sprint** — never let it accumulate history
124
-
125
- Also update `qa/memory/INDEX.md` if any memory files were created or modified.
126
-
127
- ---
128
-
129
- ## Step 6: Write a Session Summary
130
-
131
- Create `qa/SESSION-SUMMARY-{YYYY-MM-DD}-maintenance.md` documenting:
132
- - What changed
133
- - Which files were updated
134
- - Any new TCs or deprecated TCs
135
- - Any tests that were reactivated or newly skipped