@keber/qa-framework 1.5.1 → 1.6.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +39 -6
- package/README.md +150 -76
- package/docs/architecture.md +49 -44
- package/docs/folder-structure-guide.md +27 -23
- package/docs/installation.md +53 -43
- package/docs/usage-with-agent.md +52 -64
- package/integrations/ado-powershell/pipelines/azure-pipeline-qa.yml +1 -1
- package/integrations/playwright/README.md +1 -1
- package/package.json +1 -1
- package/scripts/init.js +20 -12
- package/scripts/upgrade.js +100 -11
- package/skills/qa-ado-integration/SKILL.md +2 -2
- package/skills/qa-automation/SKILL.md +6 -6
- package/skills/qa-maintenance/SKILL.md +2 -2
- package/templates/automation-scaffold/playwright.config.ts +22 -11
- package/agent-instructions/00-module-analysis.md +0 -266
- package/agent-instructions/01-spec-generation.md +0 -281
- package/agent-instructions/02-test-plan-generation.md +0 -205
- package/agent-instructions/03-test-case-generation.md +0 -150
- package/agent-instructions/04-automation-generation.md +0 -343
- package/agent-instructions/04b-test-stabilization.md +0 -309
- package/agent-instructions/05-ado-integration.md +0 -247
- package/agent-instructions/06-maintenance.md +0 -135
- package/docs/comparison-matrix.md +0 -131
- package/docs/generalization-decisions.md +0 -203
|
@@ -1,131 +0,0 @@
|
|
|
1
|
-
# docs/comparison-matrix.md
|
|
2
|
-
|
|
3
|
-
**Phase 1 Artifact** — Source: analysis of two embedded QA implementations.
|
|
4
|
-
|
|
5
|
-
> **Note**: This document covers the comparative inventory of Repo A (`redacted-repo-web`) and Repo B (`redacted-repo`). It was the primary input for Phase 2 (classification) and Phase 3 (framework design).
|
|
6
|
-
|
|
7
|
-
---
|
|
8
|
-
|
|
9
|
-
## Repository Profiles
|
|
10
|
-
|
|
11
|
-
| Attribute | Repo A (`redacted-repo-web`) | Repo B (`redacted-repo`) |
|
|
12
|
-
|---|---|---|
|
|
13
|
-
| **Application stack** | ASP.NET MVC 5 + Web API 2 + SQL Server | Blazor WebAssembly + .NET + Radzen |
|
|
14
|
-
| **UI component library** | jQuery, Select2, toastr, SweetAlert2, Metronic | Radzen Blazor components |
|
|
15
|
-
| **Authentication** | Form login: RUT + password, `/Seguridad/Login` | Form login: email + password, Blazor identity |
|
|
16
|
-
| **QA environment URL** | `https://redactedURL.com` | `https://redactedURL.com` |
|
|
17
|
-
| **Modules documented** | 1 module (RCL Colación Fría), 7 submodules | 4 modules (Personas, Operación, Sistema, Reportes), 17+ submodules |
|
|
18
|
-
| **Test cases documented** | ~160 TCs across 7 submodules | ~894 TCs across 17+ submodules |
|
|
19
|
-
| **ADO integration level** | Partial (Wiki tools, Plan IDs referenced) | Full (pipeline, reporter, inject-ado-ids.ps1, module-registry.json) |
|
|
20
|
-
| **Playwright suite** | Yes (Sprint 40: 26 P0 tests, 24 pass/2 skip) | Yes (67 automated, 53 pass/2 fail/12 skip) |
|
|
21
|
-
| **Standards folder** | No (`00-standards/` not present) | Yes (naming-conventions, bug-template, TC-template, test-data-guidelines) |
|
|
22
|
-
| **Session retrospectives** | Yes (06-defects used, 2 defects filed) | Extensive (SESSION-SUMMARY-* files at qa root) |
|
|
23
|
-
| **08-azure-integration** | Not present | Fully implemented |
|
|
24
|
-
|
|
25
|
-
---
|
|
26
|
-
|
|
27
|
-
## Item-by-Item Comparison Matrix
|
|
28
|
-
|
|
29
|
-
| Item | Repo A | Repo B | Common? | Different? | Generalizable? | Needs Param? | Out of Framework? |
|
|
30
|
-
|---|:---:|:---:|:---:|:---:|:---:|:---:|:---:|
|
|
31
|
-
| **8-folder qa/ structure (00–08)** | ✅ | ✅ | ✅ | ❌ | ✅ | ❌ | ❌ |
|
|
32
|
-
| **6-file submodule template** | ✅ | ✅ | ✅ | Minor | ✅ | ❌ | ❌ |
|
|
33
|
-
| `00-inventory.md` | ✅ | ✅ | ✅ | ❌ | ✅ | Module code | ❌ |
|
|
34
|
-
| `01-business-rules.md` | ✅ (RN-* IDs) | ✅ (RN-* IDs) | ✅ | ❌ | ✅ | Module code | ❌ |
|
|
35
|
-
| `02-workflows.md` (FL-* IDs) | ✅ | ✅ | ✅ | ❌ | ✅ | Module code | ❌ |
|
|
36
|
-
| `03-roles-permissions.md` | ✅ | ✅ | ✅ | ❌ | ✅ | ❌ | ❌ |
|
|
37
|
-
| `04-test-data.md` | ✅ | ✅ | ✅ | ❌ | ✅ | ❌ | ❌ |
|
|
38
|
-
| `05-test-scenarios.md` (TC-* IDs) | ✅ | ✅ | ✅ | ❌ | ✅ | Module code | ❌ |
|
|
39
|
-
| **Agent instructions: module analysis** | ✅ (766 lines) | ✅ (834 lines) | ✅ | Minor | ✅ | Project URL | ❌ |
|
|
40
|
-
| **Agent instructions: E2E generation** | ✅ (Sprint-specific) | ❌ | ❌ | Major | ✅ (extract pattern) | Module scope | ❌ |
|
|
41
|
-
| **ADO integration instructions** | ❌ | ✅ (AGENT-ADO-INTEGRATION.md) | ❌ | — | ✅ | Org/Project/IDs | ❌ |
|
|
42
|
-
| **QA Structure Guide** | ✅ (994 lines) | ✅ (1018 lines) | ✅ | Minor wording | ✅ | ❌ | ❌ |
|
|
43
|
-
| **00-standards/ folder** | ❌ | ✅ | ❌ | — | ✅ (adopt from B) | ❌ | ❌ |
|
|
44
|
-
| **Naming conventions doc** | Embedded in structure guide | Standalone doc | Partial | Yes | ✅ | Module codes | ❌ |
|
|
45
|
-
| **Bug report template** | ❌ | ✅ | ❌ | — | ✅ | ❌ | ❌ |
|
|
46
|
-
| **Test case template** | ❌ | ✅ | ❌ | — | ✅ | ❌ | ❌ |
|
|
47
|
-
| **Test data guidelines** | ❌ | ✅ | ❌ | — | ✅ | Env var names | ❌ |
|
|
48
|
-
| **session-summary.md pattern** | ❌ | ✅ (2 examples) | ❌ | — | ✅ | ❌ | ❌ |
|
|
49
|
-
| **Multi-role globalSetup (Playwright)** | ✅ (3 roles, sequential) | ✅ (1 role, single) | ✅ (pattern) | Parameters | ✅ | Role count, selectors | ❌ |
|
|
50
|
-
| **storageState auth reuse** | ✅ | ✅ | ✅ | ❌ | ✅ | ❌ | ❌ |
|
|
51
|
-
| **EXEC_IDX / RUN_SLOT date pattern** | ✅ | ❌ (uses Date.now() only) | ❌ | Major | ✅ (adopt from A) | ❌ | ❌ |
|
|
52
|
-
| **3-layer email validation strategy** | ✅ | ❌ | ❌ | — | ✅ (domain-agnostic) | ❌ | Optional |
|
|
53
|
-
| **playwright-azure-reporter** | ❌ | ✅ | ❌ | — | ✅ | planId, suiteId | Optional |
|
|
54
|
-
| **inject-ado-ids.ps1** | ❌ | ✅ | ❌ | — | ✅ | ❌ | Optional |
|
|
55
|
-
| **ADO pipeline (azure-pipeline-qa.yml)** | ❌ | ✅ (367 lines) | ❌ | — | ✅ | Org, project, branch | Optional |
|
|
56
|
-
| **module-registry.json** | ❌ | ✅ | ❌ | — | ✅ | Module paths/IDs | Optional |
|
|
57
|
-
| **ado-ids-mapping.json** | ❌ | ✅ | ❌ | — | ✅ | All IDs | Optional |
|
|
58
|
-
| **Debug artifact policy** | ❌ | ✅ (DEBUG-ARTIFACTS-POLICY.md) | ❌ | — | ✅ | ❌ | ❌ |
|
|
59
|
-
| **UI as source of truth principle** | Implied | ✅ (UI_COMO_FUENTE_DE_VERDAD.md) | ✅ (implicit) | Documentation | ✅ | ❌ | ❌ |
|
|
60
|
-
| **TC origin classification** | ❌ | ✅ (UI-OBSERVADO/PENDIENTE-CODE/BLOQUEADO-PERMISOS) | ❌ | — | ✅ | ❌ | ❌ |
|
|
61
|
-
| **BLAZOR_WASM_AUTOMATION.md** | ❌ | ✅ | ❌ | — | Partial | UI library | Optional |
|
|
62
|
-
| **RADZEN_COMPONENTS_PLAYWRIGHT.md** | ❌ | ✅ | ❌ | — | Partial | UI library | Optional (Blazor adapter) |
|
|
63
|
-
| **DB schema documentation pattern** | ✅ (Mermaid ER) | ❌ | ❌ | — | ✅ (pattern only) | DB naming | Optional |
|
|
64
|
-
| **Correctness criteria** | ✅ (≥2 passes with different EXEC_IDX) | ❌ | ❌ | — | ✅ | ❌ | ❌ |
|
|
65
|
-
| **5-layer debugging methodology** | ✅ | ❌ | ❌ | — | ✅ | ❌ | ❌ |
|
|
66
|
-
| **Page Object Model (POM) pattern** | ✅ (4 POMs) | Inline (no POMs) | Partial | Approach | ✅ (recommend POMs) | ❌ | ❌ |
|
|
67
|
-
| **06-defects/ directory** | ✅ (2 active DEFs used) | ✅ (folder only, no active DEFs) | ✅ | Usage | Conditional | ❌ | Optional |
|
|
68
|
-
| **Seed specs pattern** | ❌ | ✅ (seeds/*.spec.ts excluded from runs) | ❌ | — | ✅ | ❌ | Optional |
|
|
69
|
-
| **automation-feasibility-analysis.md** | ❌ | ✅ | ❌ | — | ✅ | ❌ | ❌ |
|
|
70
|
-
| **Coverage mapping (COVERAGE-MAPPING.md)** | ❌ | ✅ | ❌ | — | ✅ | ADO IDs | Optional |
|
|
71
|
-
| **Project-specific URLs** | ✅ (redacted) | ✅ (redacted) | Both | Different | ❌ (parameterize) | QA_BASE_URL | ❌ |
|
|
72
|
-
| **ADO org / project names** | redacted-organization / redacted-project | redacted-organization / redacted-project | Org same | Project different | ❌ (parameterize) | ADO_ORG, ADO_PROJECT | ❌ |
|
|
73
|
-
| **Login selectors** | `#m_login_signin_submit` (Metronic) | `#email-input`, `#password-input` (Blazor) | Different | Major | ❌ (parameterize) | LOGIN_EMAIL_SELECTOR, etc. | ❌ |
|
|
74
|
-
| **Application-specific form selectors** | `#tbl_redacted`, `#redacted` | `.rz-data-row`, `input.rz-textbox` | None | Major | ❌ | ❌ | Yes (project adapters) |
|
|
75
|
-
| **Test user DNIs / real personal data** | DNIs in test-data.ts, personas.md | None (uses env vars) | None | Major | ❌ | ❌ | Yes (project data) |
|
|
76
|
-
| **Module-specific business data** | redacted | redacted | None | Major | ❌ | ❌ | Yes (project data) |
|
|
77
|
-
| **Sprint references** | redacted sprint | No sprints (session-based) | None | Different | ❌ | ❌ | Yes (project context) |
|
|
78
|
-
| **Developer names** | redacted names | None | None | — | ❌ | ❌ | Yes (project context) |
|
|
79
|
-
| **Internal SOAP email service** | EnvioMailSoapClient | None | None | — | ❌ | ❌ | Yes |
|
|
80
|
-
|
|
81
|
-
---
|
|
82
|
-
|
|
83
|
-
## Summary Counts
|
|
84
|
-
|
|
85
|
-
| Category | Count |
|
|
86
|
-
|---|---|
|
|
87
|
-
| Items common to both repos | 14 |
|
|
88
|
-
| Items only in Repo A, generalizable | 8 |
|
|
89
|
-
| Items only in Repo B, generalizable | 16 |
|
|
90
|
-
| Items that need parameterization | 12 |
|
|
91
|
-
| Items that are purely project-specific (out of framework) | 10+ |
|
|
92
|
-
| Optional/plugin items | 9 |
|
|
93
|
-
|
|
94
|
-
---
|
|
95
|
-
|
|
96
|
-
## Key Differences That Create Design Tensions
|
|
97
|
-
|
|
98
|
-
### 1. Login credential type: RUT vs email
|
|
99
|
-
|
|
100
|
-
- Repo A uses RUT (Chilean national ID) as username
|
|
101
|
-
- Repo B uses email as username
|
|
102
|
-
- **Resolution**: Parameterize via `QA_USER_EMAIL` and `QA_USER_PASSWORD` env vars; the login selector set is a project config parameter in `qa-framework.config.json`
|
|
103
|
-
|
|
104
|
-
### 2. Single-role vs multi-role globalSetup
|
|
105
|
-
|
|
106
|
-
- Repo A: 3 roles (solicitante, encargado, admin) saved sequentially to `.auth/{role}.json`
|
|
107
|
-
- Repo B: 1 role saved to `.auth/session.json`
|
|
108
|
-
- **Resolution**: Scaffold supports both. Config parameter `testUsers[]` drives how many roles are set up. Template defaults to single-role; multi-role is documented as an extension.
|
|
109
|
-
|
|
110
|
-
### 3. ADO integration depth
|
|
111
|
-
|
|
112
|
-
- Repo A: ADO IDs referenced in comments; no pipeline or reporter
|
|
113
|
-
- Repo B: Full pipeline, playwright-azure-reporter, inject-ado-ids.ps1, module-registry.json
|
|
114
|
-
- **Resolution**: ADO integration is a fully optional plugin (`integrations/ado-powershell/`). The framework core does not require it.
|
|
115
|
-
|
|
116
|
-
### 4. Standards folder
|
|
117
|
-
|
|
118
|
-
- Repo B has a mature `00-standards/` folder absent from Repo A
|
|
119
|
-
- **Resolution**: Adopt the standards from Repo B as the framework baseline, since they are the more complete and generalized set.
|
|
120
|
-
|
|
121
|
-
### 5. `06-defects/` directory
|
|
122
|
-
|
|
123
|
-
- Repo A: Used actively (2 bugs filed, test skips reference DEF-IDs)
|
|
124
|
-
- Repo B: Folder exists but no active defect files
|
|
125
|
-
- **Resolution**: See `docs/generalization-decisions.md` §7 for the full evaluation. Decision: **keep as optional, not required by framework core**.
|
|
126
|
-
|
|
127
|
-
### 6. Test naming convention
|
|
128
|
-
|
|
129
|
-
- Repo A: `[Nx] Title @Pp` (sprint-relative sequential numbering)
|
|
130
|
-
- Repo B: `[ADO_ID] Title` (ADO Work Item ID prefix)
|
|
131
|
-
- **Resolution**: Framework uses `[{ID}] {title} @{priority}` where `{ID}` defaults to a local TC-ID (e.g., TC-MOD-SUB-001) and is replaced with ADO WI ID when ADO integration is enabled.
|
|
@@ -1,203 +0,0 @@
|
|
|
1
|
-
# docs/generalization-decisions.md
|
|
2
|
-
|
|
3
|
-
**Phase 2 Artifact** — Classification and decision log for every element found during discovery.
|
|
4
|
-
|
|
5
|
-
Every decision in this document follows the format:
|
|
6
|
-
|
|
7
|
-
> **Decision ID** | **Item** | **Source** | **Decision** | **Rationale** | **What was discarded**
|
|
8
|
-
|
|
9
|
-
---
|
|
10
|
-
|
|
11
|
-
## §1 — 8-Folder QA Directory Structure
|
|
12
|
-
|
|
13
|
-
- **Item**: Numbered folder hierarchy `00-guides/` through `08-azure-integration/`
|
|
14
|
-
- **Source**: Both repos (identical structure)
|
|
15
|
-
- **Decision**: ✅ **Framework core**
|
|
16
|
-
- **Rationale**: Both repos independently converged on this same structure. It provides a clear, numbered namespace that agents can navigate predictably. The numbers prevent alphabetical re-ordering from breaking conventions.
|
|
17
|
-
- **What was discarded**: Nothing. Structure adopted as-is.
|
|
18
|
-
|
|
19
|
-
---
|
|
20
|
-
|
|
21
|
-
## §2 — 6-File Submodule Template Set
|
|
22
|
-
|
|
23
|
-
- **Item**: `00-inventory.md` → `05-test-scenarios.md`
|
|
24
|
-
- **Source**: Both repos (Repo B more consistently follows the pattern)
|
|
25
|
-
- **Decision**: ✅ **Framework core** (reusable template)
|
|
26
|
-
- **Rationale**: This is the most consistently-replicated pattern across both repos and the most clearly generalizable unit of work. Every submodule follows this structure regardless of domain.
|
|
27
|
-
- **Naming difference**: Repo A names file 1 `01-business-rules.md`; Repo B sometimes uses `01-functional-analysis.md`. **Resolution**: Standardize as `01-business-rules.md`. Functional analysis is a technique to produce business rules, not the output itself.
|
|
28
|
-
- **What was discarded**: `01-functional-analysis.md` naming variant.
|
|
29
|
-
|
|
30
|
-
---
|
|
31
|
-
|
|
32
|
-
## §3 — Agent Instructions: Module Analysis
|
|
33
|
-
|
|
34
|
-
- **Item**: `AGENT-INSTRUCTIONS-MODULE-ANALYSIS.md` — 766 lines (A), 834 lines (B)
|
|
35
|
-
- **Source**: Both repos (conceptually identical, slightly different phrasings)
|
|
36
|
-
- **Decision**: ✅ **Agent instructions** (generalized, project-specific refs removed)
|
|
37
|
-
- **Rationale**: Both are 4-phase instructions (Prepare → Explore → Document → Review). The core process is identical. Project-specific items removed: hardcoded URLs, project names, session names.
|
|
38
|
-
- **What was discarded**:
|
|
39
|
-
- Repo A: `https://redactedURL.com`, `redacted@mail.com`, `playwright-cli MCP tool reference`
|
|
40
|
-
- Repo B: `https://redactedURL.com`, `redacted-project` session name
|
|
41
|
-
- Replaced with: `{{QA_BASE_URL}}`, `{{QA_USER_EMAIL}}` placeholders
|
|
42
|
-
- **Adopted from Repo B** (merged): TC origin classification (UI-OBSERVADO / PENDIENTE-CODE / BLOQUEADO-PERMISOS), 50–85 TC target per submodule, 3-condition TC validity rule (VISIBLE + ACCESIBLE + OBSERVABLE)
|
|
43
|
-
|
|
44
|
-
---
|
|
45
|
-
|
|
46
|
-
## §4 — Agent Instructions: E2E Automation
|
|
47
|
-
|
|
48
|
-
- **Item**: `AGENT-INSTRUCTIONS-E2E-PLAYWRIGHT-SPRINT40.md` (Repo A only)
|
|
49
|
-
- **Source**: Repo A — sprint-specific, 961 lines
|
|
50
|
-
- **Decision**: ✅ **Agent instructions** + **Optional integration** (heavily refactored)
|
|
51
|
-
- **Rationale**: The document contains highly valuable generalizable patterns (multi-role auth, EXEC_IDX date strategy, 3-layer email validation, skip+DEF pattern, storageState) embedded within Sprint 40-specific content. The generalizable patterns were extracted; the sprint-specific content (task IDs, routes, form selectors) was discarded.
|
|
52
|
-
- **What was discarded**: Azure Task IDs (20111, 21944–21949, 20528–20529), all `/RCL/`, `/Seguridad/` route references, all Sprint 40 test counts and scope, specific ADO Plan/Suite IDs.
|
|
53
|
-
- **What was extracted**: The 18-section structure was collapsed into a generalized `04-automation-generation.md` agent instruction with parameterized sections.
|
|
54
|
-
|
|
55
|
-
---
|
|
56
|
-
|
|
57
|
-
## §5 — Standards Folder (`00-standards/`)
|
|
58
|
-
|
|
59
|
-
- **Item**: Naming conventions, bug template, TC template, test-data guidelines
|
|
60
|
-
- **Source**: Repo B only
|
|
61
|
-
- **Decision**: ✅ **Framework core** — adopt from Repo B as the baseline
|
|
62
|
-
- **Rationale**: Repo B has a full and well-designed standards set absent from Repo A. These are fully generalizable with minor parameterization (module/submodule codes).
|
|
63
|
-
- **What was discarded**:
|
|
64
|
-
- Repo B has `admin.qakeber.com` hardcoded in bug-report template and test-data guidelines. Replaced with `{{QA_ADMIN_EMAIL}}`.
|
|
65
|
-
- Module codes (AUTH, PERS, USER, AGRI, OPER, REP) were kept as example values in templates but removed from any "authoritative" list.
|
|
66
|
-
|
|
67
|
-
---
|
|
68
|
-
|
|
69
|
-
## §6 — Playwright Integration
|
|
70
|
-
|
|
71
|
-
- **Item**: `playwright.config.ts`, `global-setup.ts`, `.env.example`, `fixtures/auth.ts`, page objects, spec files
|
|
72
|
-
- **Source**: Both repos
|
|
73
|
-
- **Decision**: ✅ **Optional integration** (playwright adapter) + ✅ **Reusable templates** for config files
|
|
74
|
-
- **Rationale**: Playwright is the automation framework of choice in both repos but must remain optional. Projects may use a different test runner (Jest, Cypress). The configuration patterns (storageState, globalSetup, testIgnore for seeds, CI vs local workers) are generalizable as templates.
|
|
75
|
-
- **What was discarded**:
|
|
76
|
-
- All project-specific selectors (form fields, route paths, element IDs)
|
|
77
|
-
- All hardcoded URLs
|
|
78
|
-
- Sprint-specific npm scripts (`sprint40`, `sprint40:p0`)
|
|
79
|
-
- Application-specific form constants from `fixtures/test-data.ts`
|
|
80
|
-
- All page objects (too application-specific)
|
|
81
|
-
- **What was templated**:
|
|
82
|
-
- `playwright.config.ts` — with `{{...}}` placeholders for baseURL, test path, ADO credentials
|
|
83
|
-
- `global-setup.ts` — generic login flow with configurable selectors
|
|
84
|
-
- `.env.example` — with all credential variables documented
|
|
85
|
-
- `fixtures/auth.ts` — generic multi-role fixture
|
|
86
|
-
|
|
87
|
-
---
|
|
88
|
-
|
|
89
|
-
## §7 — `06-defects/` Directory
|
|
90
|
-
|
|
91
|
-
**This is the evaluation specifically requested in the brief.**
|
|
92
|
-
|
|
93
|
-
- **Item**: The `06-defects/` folder and its defect-tracking purpose
|
|
94
|
-
- **Source**: Repo A (actively used: DEF-001, DEF-002), Repo B (folder exists, no files)
|
|
95
|
-
- **Analysis**:
|
|
96
|
-
|
|
97
|
-
| Criteria | Finding |
|
|
98
|
-
|---|---|
|
|
99
|
-
| Is real value demonstrated? | Yes, in Repo A: DEF-* IDs are referenced from `test.skip()`, linking automation state to bug state. This provides traceability from failing test → known defect. |
|
|
100
|
-
| Is it redundant with ADO? | Partially. ADO Work Items serve the same purpose in organizations that use ADO. However, not all projects use ADO. |
|
|
101
|
-
| Is the format standardized? | No. In Repo A: freeform Markdown. Repo B has a bug-report template in `00-standards/`. |
|
|
102
|
-
| Does it create maintenance burden? | Yes, if not kept in sync with ADO. Two sources of truth for defects is a known anti-pattern. |
|
|
103
|
-
|
|
104
|
-
- **Decision**: ✅ **Keep as optional** — not required by framework core, but recommended when:
|
|
105
|
-
1. ADO is not in use (lightweight alternative)
|
|
106
|
-
2. The project needs offline/local defect tracking during sprint stabilization
|
|
107
|
-
3. The team wants to document the `test.skip() → DEF-* → reactivation path` explicitly
|
|
108
|
-
|
|
109
|
-
- **Recommended evolution**: When ADO is enabled, `06-defects/` becomes a cache/mirror of ADO bugs relevant to QA automation. The defect file links to the ADO WI, rather than replacing it.
|
|
110
|
-
|
|
111
|
-
- **What would be removed**: If this folder is removed from a project, the `test.skip('DEF-XXX: ...')` convention should still be kept in spec files; the DEF ID should then reference the ADO WI ID directly.
|
|
112
|
-
|
|
113
|
-
---
|
|
114
|
-
|
|
115
|
-
## §8 — Azure DevOps Integration
|
|
116
|
-
|
|
117
|
-
- **Item**: Pipeline YAML, `playwright-azure-reporter`, PowerShell scripts, `module-registry.json`, `ado-ids-mapping.json`
|
|
118
|
-
- **Source**: Repo B (fully implemented), Repo A (partial — referenced but not implemented)
|
|
119
|
-
- **Decision**: ✅ **Optional integration** (ado-powershell adapter + playwright-azure-reporter adapter)
|
|
120
|
-
- **Rationale**: ADO integration provides real value (E2E result sync with Test Plans, CI pipeline, WI ID injection) but creates a strong dependency on a specific vendor (Azure DevOps). The framework must work without it.
|
|
121
|
-
- **What was kept**: The full set of scripts from Repo B — `inject-ado-ids.ps1`, `create-testplan-from-mapping.ps1`, `sync-ado-titles.ps1` — generalized by removing hardcoded org/project/ID values.
|
|
122
|
-
- **What was parameterized**: `ADO_ORG`, `ADO_PROJECT`, `ADO_PAT`, `planId`, `suiteId`, variable group name.
|
|
123
|
-
- **What was marked as stub**: The Repo B pipeline references a specific trigger branch (`keber.flores/qa-test/qa-specs`). This is replaced with a parameterized `{{CI_TRIGGER_BRANCH}}` in the template.
|
|
124
|
-
|
|
125
|
-
---
|
|
126
|
-
|
|
127
|
-
## §9 — Blazor-Specific Automation Guides
|
|
128
|
-
|
|
129
|
-
- **Item**: `BLAZOR_WASM_AUTOMATION.md`, `RADZEN_COMPONENTS_PLAYWRIGHT.md`, `RETROSPECTIVA_CASCADA_2026-02.md`
|
|
130
|
-
- **Source**: Repo B only
|
|
131
|
-
- **Decision**: ✅ **Optional integration** (blazor-radzen adapter) for reusable helper functions; ❌ **Out of framework** for retrospective
|
|
132
|
-
- **Rationale**: The Blazor/Radzen automation patterns (`openDropdownAndSelectFirst()`, `toBeAttached()`, `waitForFunction()` for popup detection) are highly reusable across any Radzen Blazor project. However, they are not generic enough for the framework core. The retrospective is project-specific debug log.
|
|
133
|
-
- **What was extracted**: `openDropdownAndSelectFirst()` function moved to `integrations/blazor-radzen/helpers.ts` (stub — marked as `ADAPTER`).
|
|
134
|
-
- **What was discarded**: Retrospective content, specific route references in cascade-dropdown backtrack.
|
|
135
|
-
|
|
136
|
-
---
|
|
137
|
-
|
|
138
|
-
## §10 — EXEC_IDX / RUN_SLOT Date Generation Pattern
|
|
139
|
-
|
|
140
|
-
- **Item**: `Math.floor(Date.now() / 60_000) % 100_000` (EXEC_IDX), `Math.floor(Date.now()/300_000)` (RUN_SLOT)
|
|
141
|
-
- **Source**: Repo A (primary), Repo B (uses raw `Date.now()` without the slot pattern)
|
|
142
|
-
- **Decision**: ✅ **Framework core** (general convention, extracted to template)
|
|
143
|
-
- **Rationale**: This pattern solves a real cross-project problem: parallel or repeated test runs creating data collisions. It avoids the need for test teardown and is independently applicable to any project with CRUD tests.
|
|
144
|
-
- **Action**: Documented in `agent-instructions/04-automation-generation.md` as a required pattern for data-producing tests. Moved to `templates/automation-scaffold/fixtures/test-helpers.ts`.
|
|
145
|
-
|
|
146
|
-
---
|
|
147
|
-
|
|
148
|
-
## §11 — Naming Convention: Test Titles
|
|
149
|
-
|
|
150
|
-
- **Item**: Test title format — `[Nx] Title @Pp` (Repo A) vs `[ADO_ID] Title` (Repo B)
|
|
151
|
-
- **Source**: Both repos, incompatible
|
|
152
|
-
- **Decision**: ✅ **Framework core** (adopt unified convention)
|
|
153
|
-
- **Unified convention**: `[{LOCAL_ID}] {title} @{priority}` — e.g. `[TC-OPER-CAT-001] Access catalog list @P0`
|
|
154
|
-
- **When ADO is enabled**: Local ID is replaced with ADO WI ID by `inject-ado-ids.ps1`, becoming `[22957] Access catalog list @P0`
|
|
155
|
-
- **Rationale**: The local ID format preserves traceability without requiring ADO. The ADO ID injection is an idempotent upgrade step, not a prerequisite.
|
|
156
|
-
|
|
157
|
-
---
|
|
158
|
-
|
|
159
|
-
## §12 — Page Object Model (POM) vs Inline Automation
|
|
160
|
-
|
|
161
|
-
- **Item**: POMs (Repo A) vs inline selector logic (Repo B)
|
|
162
|
-
- **Source**: Repo A has 5 POMs; Repo B uses inline `page.locator()` calls in spec files
|
|
163
|
-
- **Decision**: ✅ **Framework recommendation: use POMs** for production automation; inline is acceptable for exploratory scripts
|
|
164
|
-
- **Rationale**: POMs provide selector encapsulation and reuse across suites. Repo A's approach led to smaller spec files and easier maintenance. Repo B's inline approach was faster to write but led to selector duplication. The framework templates use POMs, but the agent instruction notes that inline is acceptable for single-use diagnostic specs.
|
|
165
|
-
|
|
166
|
-
---
|
|
167
|
-
|
|
168
|
-
## §13 — `SITEMAP_TERRAVIX_QA.md` / `ui-menu-map.md` Pattern
|
|
169
|
-
|
|
170
|
-
- **Item**: Auto-generated application sitemap from a real Playwright session
|
|
171
|
-
- **Source**: Both repos (Repo B: SITEMAP file, Repo A: auto-generated ui-menu-map.md)
|
|
172
|
-
- **Decision**: ✅ **Reusable template** — the `ui-menu-map.spec.ts` script extracted and generalized
|
|
173
|
-
- **Rationale**: Generating a live sitemap from a Playwright session is a highly valuable, universally applicable pattern. It provides the ground truth for module analysis and access-control testing.
|
|
174
|
-
- **What was parameterized**: `BASE_URL` env var, output path in `qa/01-specifications/shared/ui-menu-map.md`.
|
|
175
|
-
|
|
176
|
-
---
|
|
177
|
-
|
|
178
|
-
## §14 — Session Summary Pattern
|
|
179
|
-
|
|
180
|
-
- **Item**: `SESSION-SUMMARY-YYYY-MM-DD.md` files at qa/ root (Repo B)
|
|
181
|
-
- **Source**: Repo B only
|
|
182
|
-
- **Decision**: ✅ **Reusable template**
|
|
183
|
-
- **Rationale**: Provides a living log of what was produced in each QA working session. Useful for agent continuity (can resume from a known state) and for traceability.
|
|
184
|
-
- **What was discarded**: Specific content (Personas module, 19 files created, etc.)
|
|
185
|
-
|
|
186
|
-
---
|
|
187
|
-
|
|
188
|
-
## §15 — `06-defects/` vs ADO Work Items Decision Tree
|
|
189
|
-
|
|
190
|
-
For completeness, here is the recommended decision tree:
|
|
191
|
-
|
|
192
|
-
```
|
|
193
|
-
Is Azure DevOps enabled in this project?
|
|
194
|
-
├── YES → Use ADO Work Items for ALL defects.
|
|
195
|
-
│ Keep 06-defects/ as a lightweight reference cache only.
|
|
196
|
-
│ Each DEF-*.md file = summary + link to ADO WI URL.
|
|
197
|
-
│ test.skip() references ADO WI ID, not DEF-ID.
|
|
198
|
-
│
|
|
199
|
-
└── NO → Use 06-defects/ as the primary defect tracker.
|
|
200
|
-
Follow the bug-report-template.md format.
|
|
201
|
-
test.skip() references DEF-ID.
|
|
202
|
-
When ADO is enabled later: migrate IDs.
|
|
203
|
-
```
|