@keber/qa-framework 1.5.1 → 1.6.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +39 -6
- package/README.md +150 -76
- package/docs/architecture.md +49 -44
- package/docs/folder-structure-guide.md +27 -23
- package/docs/installation.md +53 -43
- package/docs/usage-with-agent.md +52 -64
- package/integrations/ado-powershell/pipelines/azure-pipeline-qa.yml +1 -1
- package/integrations/playwright/README.md +1 -1
- package/package.json +1 -1
- package/scripts/init.js +20 -12
- package/scripts/upgrade.js +100 -11
- package/skills/qa-ado-integration/SKILL.md +2 -2
- package/skills/qa-automation/SKILL.md +6 -6
- package/skills/qa-maintenance/SKILL.md +2 -2
- package/templates/automation-scaffold/playwright.config.ts +22 -11
- package/agent-instructions/00-module-analysis.md +0 -266
- package/agent-instructions/01-spec-generation.md +0 -281
- package/agent-instructions/02-test-plan-generation.md +0 -205
- package/agent-instructions/03-test-case-generation.md +0 -150
- package/agent-instructions/04-automation-generation.md +0 -343
- package/agent-instructions/04b-test-stabilization.md +0 -309
- package/agent-instructions/05-ado-integration.md +0 -247
- package/agent-instructions/06-maintenance.md +0 -135
- package/docs/comparison-matrix.md +0 -131
- package/docs/generalization-decisions.md +0 -203
|
@@ -63,7 +63,7 @@ After sync, each TC file must have `adoId: <NNN>` in its YAML frontmatter block.
|
|
|
63
63
|
|
|
64
64
|
### Step 4 — Configure Playwright ADO reporter
|
|
65
65
|
|
|
66
|
-
See `references/scripts-and-config.md` for the reporter config block to add to `qa/07-automation/playwright.config.ts`.
|
|
66
|
+
See `references/scripts-and-config.md` for the reporter config block to add to `qa/07-automation/e2e/playwright.config.ts`.
|
|
67
67
|
|
|
68
68
|
Required fields:
|
|
69
69
|
- `orgUrl`, `projectName`, `planId`, `runName`
|
|
@@ -100,4 +100,4 @@ Update `qa/08-azure-integration/module-registry.json` with:
|
|
|
100
100
|
- ADO Test Plan populated with test cases
|
|
101
101
|
- `qa/03-test-cases/automated/TC-*.md` — ADO IDs injected
|
|
102
102
|
- `qa/08-azure-integration/module-registry.json` updated
|
|
103
|
-
- `qa/07-automation/playwright.config.ts` updated with reporter config
|
|
103
|
+
- `qa/07-automation/e2e/playwright.config.ts` updated with reporter config
|
|
@@ -14,7 +14,7 @@ description: >
|
|
|
14
14
|
|
|
15
15
|
**Stage**: 5 — Automation
|
|
16
16
|
**Prerequisite**: All specs for the target submodule are complete and contain no `PENDING-CODE` sections
|
|
17
|
-
**Output**: `qa/07-automation/e2e/{module}/{submodule}.spec.ts` + supporting files
|
|
17
|
+
**Output**: `qa/07-automation/e2e/tests/{module}/{submodule}.spec.ts` + supporting files
|
|
18
18
|
**Next stage**: Stage 6 — Maintenance (`qa-maintenance`) or Stage 5b — Stabilization (`qa-test-stabilization`) if tests fail
|
|
19
19
|
|
|
20
20
|
> **Pipeline rule**: Never automate a TC whose spec contains `PENDING-CODE`. Resolve spec gaps first.
|
|
@@ -24,8 +24,8 @@ description: >
|
|
|
24
24
|
## Inputs Required
|
|
25
25
|
|
|
26
26
|
1. `qa/01-specifications/{module}/` — full spec set for target submodule
|
|
27
|
-
2. `qa/07-automation/playwright.config.ts` — confirm project name maps to submodule tag
|
|
28
|
-
3. `qa/07-automation/fixtures/auth.ts` — identify available auth fixtures
|
|
27
|
+
2. `qa/07-automation/e2e/playwright.config.ts` — confirm project name maps to submodule tag
|
|
28
|
+
3. `qa/07-automation/e2e/fixtures/auth.ts` — identify available auth fixtures
|
|
29
29
|
4. TC list (from test plan or Stage 4) — defines which TCs to automate in this session
|
|
30
30
|
5. `qa/qa-framework.config.json` → screenshotPath, automationRoot
|
|
31
31
|
|
|
@@ -38,7 +38,7 @@ description: >
|
|
|
38
38
|
For every new submodule, run a dedicated inspection script before writing tests.
|
|
39
39
|
**Template**: `references/dom-inspection-template.js` — copy, set the 4 constants at the top, run.
|
|
40
40
|
|
|
41
|
-
1. Copy `references/dom-inspection-template.js` → `qa/07-automation/_inspect-{submodule}.js`
|
|
41
|
+
1. Copy `references/dom-inspection-template.js` → `qa/07-automation/e2e/_inspect-{submodule}.js`
|
|
42
42
|
2. Set the 4 constants at the top of the script:
|
|
43
43
|
- `MODULE_ROUTE` — the submodule's URL path (e.g. `'/Users'`, `'/Products/list'`)
|
|
44
44
|
- `APP_SHELL_SEL` — a selector that confirms the SPA has loaded (nav, sidebar, app shell)
|
|
@@ -150,7 +150,7 @@ See `references/patterns.md` for full TypeScript implementations.
|
|
|
150
150
|
|
|
151
151
|
## Outputs
|
|
152
152
|
|
|
153
|
-
- `qa/07-automation/e2e/{module}/{submodule}.spec.ts`
|
|
153
|
+
- `qa/07-automation/e2e/tests/{module}/{submodule}.spec.ts`
|
|
154
154
|
- `qa/07-automation/e2e/page-objects/{SubmoduleName}Page.ts` (when POM criteria met)
|
|
155
|
-
- `qa/07-automation/playwright.config.ts` (updated or confirmed)
|
|
155
|
+
- `qa/07-automation/e2e/playwright.config.ts` (updated or confirmed)
|
|
156
156
|
- `qa/README.md` automation status updated
|
|
@@ -24,7 +24,7 @@ description: >
|
|
|
24
24
|
|
|
25
25
|
1. Description of the change — which submodule, what changed, and why
|
|
26
26
|
2. `qa/01-specifications/{module}/` — specs affected by the change
|
|
27
|
-
3. `qa/07-automation/e2e/{module}/` — automation affected by the change
|
|
27
|
+
3. `qa/07-automation/e2e/tests/{module}/` — automation affected by the change
|
|
28
28
|
4. Optionally: ADO work item or defect report describing the change
|
|
29
29
|
|
|
30
30
|
---
|
|
@@ -105,6 +105,6 @@ change-summary: {one-line description of what changed}
|
|
|
105
105
|
|
|
106
106
|
- Updated `qa/01-specifications/{module}/*.md` files
|
|
107
107
|
- Updated `qa/03-test-cases/automated/TC-*.md` files
|
|
108
|
-
- Updated `qa/07-automation/e2e/{module}/*.spec.ts`
|
|
108
|
+
- Updated `qa/07-automation/e2e/tests/{module}/*.spec.ts`
|
|
109
109
|
- Updated `qa/README.md`
|
|
110
110
|
- Updated `AGENT-NEXT-STEPS.md` (or sprint moved to history if complete)
|
|
@@ -13,21 +13,32 @@ for (const key of required) {
|
|
|
13
13
|
}
|
|
14
14
|
|
|
15
15
|
// Optional: Azure DevOps reporter configuration
|
|
16
|
-
//
|
|
16
|
+
// To enable: set env vars ADO_ORG, ADO_PROJECT, ADO_PAT, ADO_PLAN_ID and CI=true,
|
|
17
|
+
// then uncomment the reporter entry in the `reporter` array below.
|
|
18
|
+
// Install: npm install @alex_neo/playwright-azure-reporter --save-dev
|
|
17
19
|
// import { AzureReporter } from '@alex_neo/playwright-azure-reporter';
|
|
18
|
-
// const
|
|
19
|
-
//
|
|
20
|
-
//
|
|
21
|
-
//
|
|
22
|
-
//
|
|
23
|
-
//
|
|
24
|
-
//
|
|
25
|
-
//
|
|
20
|
+
// const adoReporterConfig = {
|
|
21
|
+
// orgUrl: `https://dev.azure.com/${process.env.ADO_ORG}`,
|
|
22
|
+
// // In CI it uses System.AccessToken (pipeline's OAuth, not subject to Conditional Access Policy).
|
|
23
|
+
// // Locally, if you want to test the reporter, you can set ADO_PAT manually.
|
|
24
|
+
// token: process.env.SYSTEM_ACCESSTOKEN ?? process.env.ADO_PAT,
|
|
25
|
+
// planId: Number(process.env.ADO_PLAN_ID),
|
|
26
|
+
// projectName: process.env.ADO_PROJECT!,
|
|
27
|
+
// testRunTitle: `[Auto] Sprint {{NNN}} — ${new Date().toISOString().slice(0, 10)}`,
|
|
28
|
+
// publishTestResultsMode: 'testRun' as const,
|
|
29
|
+
// uploadAttachments: true,
|
|
30
|
+
// attachmentsType: ['screenshot', 'video', 'trace'] as const,
|
|
31
|
+
// isDisabled: !process.env.CI, // only publishes when CI=true
|
|
32
|
+
// autoMarkTestCasesAsAutomated: {
|
|
33
|
+
// enabled: true,
|
|
34
|
+
// updateAutomatedTestName: true, // saves test title in AutomatedTestName
|
|
35
|
+
// updateAutomatedTestStorage: true, // saves spec file name in AutomatedTestStorage
|
|
36
|
+
// },
|
|
26
37
|
// };
|
|
27
38
|
|
|
28
39
|
export default defineConfig({
|
|
29
40
|
// ------ Test discovery ------
|
|
30
|
-
testDir: '
|
|
41
|
+
testDir: './tests',
|
|
31
42
|
// Use any subdir pattern your project standardizes on, e.g.:
|
|
32
43
|
// testMatch: ['**/*.spec.ts'],
|
|
33
44
|
|
|
@@ -44,7 +55,7 @@ export default defineConfig({
|
|
|
44
55
|
['html', { open: 'never' }],
|
|
45
56
|
['list'],
|
|
46
57
|
// Uncomment for ADO:
|
|
47
|
-
// ['@alex_neo/playwright-azure-reporter',
|
|
58
|
+
// ['@alex_neo/playwright-azure-reporter', adoReporterConfig],
|
|
48
59
|
],
|
|
49
60
|
|
|
50
61
|
// ------ Global settings ------
|
|
@@ -1,266 +0,0 @@
|
|
|
1
|
-
# Agent Instructions: Module Analysis
|
|
2
|
-
|
|
3
|
-
> ⚠️ **DEPRECATED**: This file is superseded by `.github/skills/qa-module-analysis/SKILL.md`.
|
|
4
|
-
> Kept as reference during the transition period. Do not update this file.
|
|
5
|
-
|
|
6
|
-
**File**: `agent-instructions/00-module-analysis.md`
|
|
7
|
-
**Purpose**: Instructions for an AI agent to perform complete functional analysis of an application module and produce the 6-file specification set.
|
|
8
|
-
|
|
9
|
-
---
|
|
10
|
-
|
|
11
|
-
## When to use these instructions
|
|
12
|
-
|
|
13
|
-
Use this instruction set when:
|
|
14
|
-
- Analyzing a new module for the first time
|
|
15
|
-
- A module has been updated and its spec needs to be refreshed
|
|
16
|
-
- Building the initial `01-specifications/` tree for a project
|
|
17
|
-
|
|
18
|
-
---
|
|
19
|
-
|
|
20
|
-
## Prerequisites (Agent must verify before starting)
|
|
21
|
-
|
|
22
|
-
1. Confirm that `qa-framework.config.json` exists and contains `project.qaBaseUrl`
|
|
23
|
-
2. Confirm that `qa/00-standards/naming-conventions.md` is available
|
|
24
|
-
3. Confirm that the Playwright CLI (playwright-mcp or equivalent) is available for browser interaction
|
|
25
|
-
4. Ask the user for:
|
|
26
|
-
- The module name and URL (or route prefix) to analyze
|
|
27
|
-
- The submodules to analyze (if known; otherwise discover from the UI)
|
|
28
|
-
- QA user credentials (handle via env vars — never store in markdown)
|
|
29
|
-
|
|
30
|
-
**⚠️ SECURITY**: Never write real credentials to any file. If credentials appear in any output, replace immediately with `<PLACEHOLDER>`.
|
|
31
|
-
|
|
32
|
-
---
|
|
33
|
-
|
|
34
|
-
## Phase 1 — Preparation
|
|
35
|
-
|
|
36
|
-
### 1.1 Review existing specs
|
|
37
|
-
|
|
38
|
-
Check if a `qa/01-specifications/module-{name}/` directory already exists.
|
|
39
|
-
|
|
40
|
-
- If it does: read all existing files to understand what is already documented. Only re-analyze what has changed.
|
|
41
|
-
- If it does not: create the directory structure.
|
|
42
|
-
|
|
43
|
-
### 1.2 Generate module code
|
|
44
|
-
|
|
45
|
-
If no module code exists yet, allocate a unique 2–6 uppercase letter code for the module.
|
|
46
|
-
Check `qa/00-standards/naming-conventions.md` for existing codes to avoid conflicts.
|
|
47
|
-
|
|
48
|
-
Example: module "Operación" → code `OPER`; submodule "Catálogos" → code `CAT`
|
|
49
|
-
|
|
50
|
-
### 1.3 Plan the session
|
|
51
|
-
|
|
52
|
-
Identify all submodules in scope. A submodule typically corresponds to:
|
|
53
|
-
- A distinct menu item or section in the application
|
|
54
|
-
- A distinct CRUD entity (users, products, orders, etc.)
|
|
55
|
-
- A distinct workflow (approval, registration, reporting)
|
|
56
|
-
|
|
57
|
-
Document the planned submodules in `qa/01-specifications/module-{name}/README.md` before analysis begins.
|
|
58
|
-
|
|
59
|
-
---
|
|
60
|
-
|
|
61
|
-
## Phase 2 — Exploration
|
|
62
|
-
|
|
63
|
-
For each submodule, perform the following exploration using browser automation (Playwright CLI):
|
|
64
|
-
|
|
65
|
-
### 2.1 Navigation and route mapping
|
|
66
|
-
|
|
67
|
-
```
|
|
68
|
-
Navigate to {{QA_BASE_URL}} and log in with the appropriate role
|
|
69
|
-
Navigate to the submodule URL
|
|
70
|
-
Record:
|
|
71
|
-
- Full URL of each page/view
|
|
72
|
-
- Page title
|
|
73
|
-
- Navigation breadcrumb
|
|
74
|
-
```
|
|
75
|
-
|
|
76
|
-
### 2.2 UI inventory
|
|
77
|
-
|
|
78
|
-
For each screen in the submodule:
|
|
79
|
-
|
|
80
|
-
```
|
|
81
|
-
Record all:
|
|
82
|
-
- Input fields (name, type, required/optional, validation rules visible in UI)
|
|
83
|
-
- Dropdowns (static or dynamic/AJAX, options if static)
|
|
84
|
-
- Buttons (action, what they trigger)
|
|
85
|
-
- Tables/grids (columns, default sorting, pagination)
|
|
86
|
-
- Modals (trigger condition, fields, confirmation behavior)
|
|
87
|
-
- Status indicators / badges / alerts
|
|
88
|
-
- File upload / download controls
|
|
89
|
-
```
|
|
90
|
-
|
|
91
|
-
### 2.3 API endpoint discovery
|
|
92
|
-
|
|
93
|
-
```
|
|
94
|
-
Observe network requests during key interactions (create, read, update, delete)
|
|
95
|
-
Record:
|
|
96
|
-
- Endpoint path (e.g., /api/GetUsers, api/CreateUser)
|
|
97
|
-
- HTTP method
|
|
98
|
-
- Request payload shape (field names)
|
|
99
|
-
- Response shape (key fields)
|
|
100
|
-
```
|
|
101
|
-
|
|
102
|
-
### 2.4 Role-based access testing
|
|
103
|
-
|
|
104
|
-
```
|
|
105
|
-
For each configured test user role:
|
|
106
|
-
Log in as that role
|
|
107
|
-
Navigate to the submodule
|
|
108
|
-
Record: can access? (YES / NO / PERMISSION_ERROR)
|
|
109
|
-
Record: which actions are available vs hidden
|
|
110
|
-
```
|
|
111
|
-
|
|
112
|
-
### 2.5 Workflow discovery
|
|
113
|
-
|
|
114
|
-
```
|
|
115
|
-
Execute the primary happy path with appropriate test data
|
|
116
|
-
Record each step
|
|
117
|
-
Identify branch points (e.g., approve vs reject)
|
|
118
|
-
Record state transitions
|
|
119
|
-
```
|
|
120
|
-
|
|
121
|
-
### 2.6 Business rule discovery
|
|
122
|
-
|
|
123
|
-
```
|
|
124
|
-
Attempt invalid inputs for each validation
|
|
125
|
-
Record: validation message text, field that shows the error
|
|
126
|
-
Attempt boundary conditions (empty, max length, duplicate, etc.)
|
|
127
|
-
Record: system behavior
|
|
128
|
-
```
|
|
129
|
-
|
|
130
|
-
---
|
|
131
|
-
|
|
132
|
-
## Phase 3 — Documentation (6-file output)
|
|
133
|
-
|
|
134
|
-
Produce the following files for each submodule. Use the templates in `qa/00-standards/` and `templates/specification/`.
|
|
135
|
-
|
|
136
|
-
### File 00: `00-inventory.md`
|
|
137
|
-
|
|
138
|
-
Contents:
|
|
139
|
-
- Module header (name, code, URL, deployment date if known)
|
|
140
|
-
- Submodule header (name, code, routes)
|
|
141
|
-
- UI element inventory table
|
|
142
|
-
- Element name | Type | Required | Notes
|
|
143
|
-
- API endpoint table
|
|
144
|
-
- Method | Path | Purpose
|
|
145
|
-
- Key identifiers and selectors observed in the UI
|
|
146
|
-
|
|
147
|
-
### File 01: `01-business-rules.md`
|
|
148
|
-
|
|
149
|
-
Contents:
|
|
150
|
-
- Each rule as `RN-{MODULE}-{NNN}`: Description
|
|
151
|
-
- Rule type: Validation / Access Control / State Machine / Calculation / Integration
|
|
152
|
-
- Trigger: what user action triggers this rule
|
|
153
|
-
- Error message: exact text shown when rule is violated (if applicable)
|
|
154
|
-
|
|
155
|
-
Aim for 5–15 business rules per submodule.
|
|
156
|
-
|
|
157
|
-
### File 02: `02-workflows.md`
|
|
158
|
-
|
|
159
|
-
Contents:
|
|
160
|
-
- Each major workflow as `FL-{MODULE}-{NNN}`: Title
|
|
161
|
-
- Use ASCII flowchart or Mermaid block diagram
|
|
162
|
-
- Cover: happy path + main alternative paths + error paths
|
|
163
|
-
|
|
164
|
-
### File 03: `03-roles-permissions.md`
|
|
165
|
-
|
|
166
|
-
Contents:
|
|
167
|
-
- Role × Feature matrix table
|
|
168
|
-
- Row: test user role
|
|
169
|
-
- Column: feature/action (view, create, edit, delete, approve, export, etc.)
|
|
170
|
-
- Cell: ✅ / ❌ / ⚠️ (limited)
|
|
171
|
-
- Notes on role inheritance or conditional access
|
|
172
|
-
|
|
173
|
-
### File 04: `04-test-data.md`
|
|
174
|
-
|
|
175
|
-
Contents:
|
|
176
|
-
- Prerequisites: what data must exist before tests run
|
|
177
|
-
- Data shapes: for each key test scenario, what data is needed
|
|
178
|
-
- DO NOT include real user credentials
|
|
179
|
-
- Use env var references for any credential: `process.env.QA_USER_EMAIL`
|
|
180
|
-
- Data dependencies between submodules (if any)
|
|
181
|
-
|
|
182
|
-
### File 05: `05-test-scenarios.md`
|
|
183
|
-
|
|
184
|
-
Contents:
|
|
185
|
-
- Summary table at top: total TCs, by priority, by type
|
|
186
|
-
- For each TC:
|
|
187
|
-
- ID: `TC-{MODULE}-{SUBMODULE}-{NNN}`
|
|
188
|
-
- Title: brief action description
|
|
189
|
-
- Priority: P0 / P1 / P2 / P3
|
|
190
|
-
- Type: Functional / Regression / Security / Negative / Integration
|
|
191
|
-
- Origin: `UI-OBSERVED` / `PENDING-CODE` / `BLOCKED-PERMISSIONS`
|
|
192
|
-
- Preconditions
|
|
193
|
-
- Steps (numbered)
|
|
194
|
-
- Expected result
|
|
195
|
-
- Automation feasibility: Yes / Partial / No
|
|
196
|
-
|
|
197
|
-
Target: **50–85 TCs per submodule** is a well-analyzed submodule.
|
|
198
|
-
|
|
199
|
-
#### TC coverage must include:
|
|
200
|
-
- ✅ Access control (each role: can access / cannot access)
|
|
201
|
-
- ✅ Happy path (primary workflow end-to-end)
|
|
202
|
-
- ✅ Negative scenarios (required field empty, invalid format, duplicate, etc.)
|
|
203
|
-
- ✅ State transitions (create → edit → delete; pending → approved → rejected)
|
|
204
|
-
- ✅ Cross-module dependencies (if any)
|
|
205
|
-
- ✅ Export/download (if feature exists)
|
|
206
|
-
- ✅ Pagination/search (if feature exists)
|
|
207
|
-
|
|
208
|
-
TC origin rules:
|
|
209
|
-
- Only write `UI-OBSERVED` if you directly observed the behavior in the QA environment
|
|
210
|
-
- If a feature is documented but not visible in the QA environment: `PENDING-CODE`
|
|
211
|
-
- If you couldn't test it due to permission restrictions: `BLOCKED-PERMISSIONS`
|
|
212
|
-
|
|
213
|
-
---
|
|
214
|
-
|
|
215
|
-
## Phase 4 — Review and Session Summary
|
|
216
|
-
|
|
217
|
-
### 4.1 Completeness checklist
|
|
218
|
-
|
|
219
|
-
After completing all submodule files, verify:
|
|
220
|
-
|
|
221
|
-
- [ ] All menu items in the module have a corresponding submodule folder
|
|
222
|
-
- [ ] All TCs marked `UI-OBSERVED` were actually observed in this session
|
|
223
|
-
- [ ] `01-business-rules.md` covers all visible validation behaviors
|
|
224
|
-
- [ ] `03-roles-permissions.md` reflects the access observed for each role
|
|
225
|
-
- [ ] No credentials appear in any file
|
|
226
|
-
- [ ] All TC IDs are unique across the module
|
|
227
|
-
- [ ] Module README lists all submodules with their codes and TC counts
|
|
228
|
-
|
|
229
|
-
### 4.2 Write session summary
|
|
230
|
-
|
|
231
|
-
Create `qa/SESSION-SUMMARY-{YYYY-MM-DD}.md` with:
|
|
232
|
-
|
|
233
|
-
```markdown
|
|
234
|
-
# Session Summary — {YYYY-MM-DD}
|
|
235
|
-
|
|
236
|
-
## Module analyzed
|
|
237
|
-
{module-name} ({module-code})
|
|
238
|
-
|
|
239
|
-
## Submodules documented
|
|
240
|
-
| Submodule | Code | TC count | Notes |
|
|
241
|
-
|-----------|------|----------|-------|
|
|
242
|
-
| {name} | {code} | {N} | {any notable blocking issues} |
|
|
243
|
-
|
|
244
|
-
## Files created
|
|
245
|
-
- {list of all files created with their paths}
|
|
246
|
-
|
|
247
|
-
## Blockers
|
|
248
|
-
- {any features that couldn't be analyzed and why}
|
|
249
|
-
|
|
250
|
-
## Observations for next session
|
|
251
|
-
- {what to tackle next}
|
|
252
|
-
```
|
|
253
|
-
|
|
254
|
-
---
|
|
255
|
-
|
|
256
|
-
## Quality Standards
|
|
257
|
-
|
|
258
|
-
A well-executed module analysis session:
|
|
259
|
-
|
|
260
|
-
1. Takes between 2-6 hours depending on module complexity
|
|
261
|
-
2. Produces 6 files per submodule (not more, not less)
|
|
262
|
-
3. Has no hardcoded credentials in any file
|
|
263
|
-
4. Has TC IDs that follow the `TC-{MODULE}-{SUBMODULE}-{NNN}` pattern
|
|
264
|
-
5. Has business rules that are verifiable in automated tests
|
|
265
|
-
6. Has at least 1 P0 TC per submodule (the primary happy path)
|
|
266
|
-
7. Does not invent TCs for behavior that wasn't observed in the QA environment
|
|
@@ -1,281 +0,0 @@
|
|
|
1
|
-
# Agent Instructions: Specification Generation
|
|
2
|
-
|
|
3
|
-
> ⚠️ **DEPRECATED**: This file is superseded by `.github/skills/qa-spec-generation/SKILL.md`.
|
|
4
|
-
> Kept as reference during the transition period. Do not update this file.
|
|
5
|
-
|
|
6
|
-
**File**: `agent-instructions/01-spec-generation.md`
|
|
7
|
-
**Purpose**: Detailed instructions for generating the 6-file submodule specification set from a completed module analysis or from existing source material (UI observation, requirements docs, developer input).
|
|
8
|
-
|
|
9
|
-
---
|
|
10
|
-
|
|
11
|
-
## When to use
|
|
12
|
-
|
|
13
|
-
- After Phase 2 (Exploration) of module analysis is complete
|
|
14
|
-
- When updating specs after a code change
|
|
15
|
-
- When a spec file is missing or incomplete
|
|
16
|
-
|
|
17
|
-
---
|
|
18
|
-
|
|
19
|
-
## Inputs Required
|
|
20
|
-
|
|
21
|
-
Before generating specs, gather:
|
|
22
|
-
|
|
23
|
-
1. **Module exploration notes** — from the analysis session
|
|
24
|
-
2. **Module code** — from `qa/00-standards/naming-conventions.md` or allocate a new one
|
|
25
|
-
3. **Submodule name** and its kebab-case directory name
|
|
26
|
-
4. **QA environment observations** — what you actually saw in the UI
|
|
27
|
-
|
|
28
|
-
---
|
|
29
|
-
|
|
30
|
-
## File-by-File Generation Rules
|
|
31
|
-
|
|
32
|
-
### `00-inventory.md` — UI and API Inventory
|
|
33
|
-
|
|
34
|
-
**Header block** (always required):
|
|
35
|
-
|
|
36
|
-
```markdown
|
|
37
|
-
# MODULE: {Module Display Name} — Submodule: {Submodule Display Name}
|
|
38
|
-
|
|
39
|
-
| Field | Value |
|
|
40
|
-
|-------|-------|
|
|
41
|
-
| Module code | {MODULE_CODE} |
|
|
42
|
-
| Submodule code | {SUBMODULE_CODE} |
|
|
43
|
-
| Primary URL | {{QA_BASE_URL}}/path/to/submodule |
|
|
44
|
-
| Status | Active / Deprecated / In Development |
|
|
45
|
-
| Last updated | YYYY-MM-DD |
|
|
46
|
-
```
|
|
47
|
-
|
|
48
|
-
**UI element table**:
|
|
49
|
-
|
|
50
|
-
```markdown
|
|
51
|
-
## UI Elements
|
|
52
|
-
|
|
53
|
-
| Element | Type | Required | Notes |
|
|
54
|
-
|---------|------|----------|-------|
|
|
55
|
-
| {field-name} | text / select / checkbox / date | Yes/No | {validation rule} |
|
|
56
|
-
```
|
|
57
|
-
|
|
58
|
-
**API endpoints table**:
|
|
59
|
-
|
|
60
|
-
```markdown
|
|
61
|
-
## API Endpoints
|
|
62
|
-
|
|
63
|
-
| Method | Path | Purpose |
|
|
64
|
-
|--------|------|---------|
|
|
65
|
-
| GET | /api/{endpoint} | {description} |
|
|
66
|
-
| POST | /api/{endpoint} | {description} |
|
|
67
|
-
```
|
|
68
|
-
|
|
69
|
-
**Do NOT include**: credentials, personal data, real user IDs.
|
|
70
|
-
|
|
71
|
-
---
|
|
72
|
-
|
|
73
|
-
### `01-business-rules.md` — Business Rules
|
|
74
|
-
|
|
75
|
-
**Format for each rule**:
|
|
76
|
-
|
|
77
|
-
```markdown
|
|
78
|
-
### RN-{MODULE}-{NNN}: {Rule Title}
|
|
79
|
-
|
|
80
|
-
- **Type**: Validation / Access Control / State Machine / Calculation / Integration
|
|
81
|
-
- **Trigger**: {What user action triggers this rule}
|
|
82
|
-
- **Behavior**: {What the system does when the rule is evaluated}
|
|
83
|
-
- **Error message** (if validation): "{exact text as shown in UI}"
|
|
84
|
-
- **Notes**: {any special circumstances}
|
|
85
|
-
```
|
|
86
|
-
|
|
87
|
-
**Rule identification tips**:
|
|
88
|
-
- Every required field → at least 1 RN (field is required)
|
|
89
|
-
- Every unique constraint → 1 RN (duplicate not allowed)
|
|
90
|
-
- Every state transition → 1 RN (can only approve if status=pending)
|
|
91
|
-
- Every role restriction → 1 RN (only role X can see this)
|
|
92
|
-
- Every auto-calculated field → 1 RN (total = quantity × price)
|
|
93
|
-
|
|
94
|
-
**Minimum**: 5 rules per submodule. **Typical**: 8–15.
|
|
95
|
-
|
|
96
|
-
---
|
|
97
|
-
|
|
98
|
-
### `02-workflows.md` — User Workflows
|
|
99
|
-
|
|
100
|
-
**Format for each workflow**:
|
|
101
|
-
|
|
102
|
-
```markdown
|
|
103
|
-
### FL-{MODULE}-{NNN}: {Workflow Title}
|
|
104
|
-
|
|
105
|
-
**Actor**: {Role who performs this workflow}
|
|
106
|
-
**Trigger**: {What initiates the workflow}
|
|
107
|
-
**Precondition**: {What must be true before this starts}
|
|
108
|
-
|
|
109
|
-
**Flow**:
|
|
110
|
-
```
|
|
111
|
-
[Start]
|
|
112
|
-
│
|
|
113
|
-
▼
|
|
114
|
-
[Step 1: action]
|
|
115
|
-
│
|
|
116
|
-
├─ [Alternative: condition] ──► [Alternative path]
|
|
117
|
-
│
|
|
118
|
-
▼
|
|
119
|
-
[Step 2: action]
|
|
120
|
-
│
|
|
121
|
-
▼
|
|
122
|
-
[End: outcome]
|
|
123
|
-
```
|
|
124
|
-
|
|
125
|
-
**Postcondition**: {State of the system after the workflow completes}
|
|
126
|
-
```
|
|
127
|
-
|
|
128
|
-
Cover at minimum:
|
|
129
|
-
- Primary happy path (end-to-end main scenario)
|
|
130
|
-
- Rejection / cancellation path (if applicable)
|
|
131
|
-
- Error path (validation failure, system error)
|
|
132
|
-
|
|
133
|
-
---
|
|
134
|
-
|
|
135
|
-
### `03-roles-permissions.md` — Role Matrix
|
|
136
|
-
|
|
137
|
-
**Format**:
|
|
138
|
-
|
|
139
|
-
```markdown
|
|
140
|
-
# Roles and Permissions — {Submodule}
|
|
141
|
-
|
|
142
|
-
## Access Matrix
|
|
143
|
-
|
|
144
|
-
| Feature / Action | {Role 1} | {Role 2} | {Role 3} |
|
|
145
|
-
|-----------------|---------|---------|---------|
|
|
146
|
-
| View list | ✅ | ✅ | ❌ |
|
|
147
|
-
| Create new | ✅ | ❌ | ❌ |
|
|
148
|
-
| Edit | ✅ | ❌ | ❌ |
|
|
149
|
-
| Delete | ✅ | ❌ | ❌ |
|
|
150
|
-
| Approve | ❌ | ✅ | ❌ |
|
|
151
|
-
| Export | ✅ | ✅ | ❌ |
|
|
152
|
-
|
|
153
|
-
## Test User Reference
|
|
154
|
-
|
|
155
|
-
| Role | Env var (email) | Env var (password) |
|
|
156
|
-
|------|----------------|-------------------|
|
|
157
|
-
| {Role 1} | `QA_USER_{ROLE}_EMAIL` | `QA_USER_{ROLE}_PASSWORD` |
|
|
158
|
-
|
|
159
|
-
**Notes**: {any conditional access rules, e.g., "Admin can only edit own records"}
|
|
160
|
-
```
|
|
161
|
-
|
|
162
|
-
**Do NOT include**: actual email addresses, passwords, or personal identifiers.
|
|
163
|
-
|
|
164
|
-
---
|
|
165
|
-
|
|
166
|
-
### `04-test-data.md` — Test Data Requirements
|
|
167
|
-
|
|
168
|
-
**Format**:
|
|
169
|
-
|
|
170
|
-
```markdown
|
|
171
|
-
# Test Data — {Submodule}
|
|
172
|
-
|
|
173
|
-
## Prerequisites
|
|
174
|
-
|
|
175
|
-
The following data must exist in the QA environment before tests run:
|
|
176
|
-
|
|
177
|
-
| Item | Description | Source |
|
|
178
|
-
|------|-------------|--------|
|
|
179
|
-
| {entity} | {description} | QA seed / pre-existing |
|
|
180
|
-
|
|
181
|
-
## Data shapes for key scenarios
|
|
182
|
-
|
|
183
|
-
### Scenario: {happy path}
|
|
184
|
-
- Field A: {value type, e.g., "any valid text, max 100 chars"}
|
|
185
|
-
- Field B: {value type}
|
|
186
|
-
- User: role `{ROLE_NAME}`, credentials from `QA_USER_{ROLE}_EMAIL` env var
|
|
187
|
-
|
|
188
|
-
### Scenario: {negative — duplicate}
|
|
189
|
-
- Precondition: {entity} with the same key already exists
|
|
190
|
-
- Field A: {same value as existing record}
|
|
191
|
-
- Expected: {system rejection behavior}
|
|
192
|
-
|
|
193
|
-
## Dynamic data generation
|
|
194
|
-
|
|
195
|
-
For tests that create new records, use the EXEC_IDX pattern to avoid collisions:
|
|
196
|
-
|
|
197
|
-
```typescript
|
|
198
|
-
const EXEC_IDX = Math.floor(Date.now() / 60_000) % 100_000;
|
|
199
|
-
const uniqueTitle = `Test-${EXEC_IDX}`;
|
|
200
|
-
```
|
|
201
|
-
|
|
202
|
-
## Data isolation rules
|
|
203
|
-
|
|
204
|
-
1. Each test must operate on its own data (no shared mutable state between tests)
|
|
205
|
-
2. Tests that require pre-existing data must provision it in `beforeAll`
|
|
206
|
-
3. Provisioned data does not need to be cleaned up if EXEC_IDX-based names are used
|
|
207
|
-
```
|
|
208
|
-
|
|
209
|
-
---
|
|
210
|
-
|
|
211
|
-
### `05-test-scenarios.md` — Test Cases
|
|
212
|
-
|
|
213
|
-
**Header block** (always required):
|
|
214
|
-
|
|
215
|
-
```markdown
|
|
216
|
-
# Test Scenarios — {Module}: {Submodule}
|
|
217
|
-
|
|
218
|
-
**Module code**: {MODULE_CODE}
|
|
219
|
-
**Submodule code**: {SUBMODULE_CODE}
|
|
220
|
-
**Last updated**: YYYY-MM-DD
|
|
221
|
-
|
|
222
|
-
## Summary
|
|
223
|
-
|
|
224
|
-
| Priority | Count | Automated | Manual |
|
|
225
|
-
|----------|-------|-----------|--------|
|
|
226
|
-
| P0 | {N} | {N} | {N} |
|
|
227
|
-
| P1 | {N} | {N} | {N} |
|
|
228
|
-
| P2 | {N} | {N} | {N} |
|
|
229
|
-
| P3 | {N} | {N} | {N} |
|
|
230
|
-
| **Total** | **{N}** | **{N}** | **{N}** |
|
|
231
|
-
```
|
|
232
|
-
|
|
233
|
-
**Format for each test case**:
|
|
234
|
-
|
|
235
|
-
```markdown
|
|
236
|
-
### TC-{MODULE}-{SUBMODULE}-{NNN}: {TC Title}
|
|
237
|
-
|
|
238
|
-
| Field | Value |
|
|
239
|
-
|-------|-------|
|
|
240
|
-
| Priority | P0 / P1 / P2 / P3 |
|
|
241
|
-
| Type | Functional / Negative / Regression / Security / Integration |
|
|
242
|
-
| Origin | UI-OBSERVED / PENDING-CODE / BLOCKED-PERMISSIONS |
|
|
243
|
-
| Automation | Yes / Partial / No |
|
|
244
|
-
| Playwright | (leave blank until automation is written; then: `tests/{module}/file.spec.ts`) |
|
|
245
|
-
|
|
246
|
-
**Preconditions**:
|
|
247
|
-
- {precondition 1}
|
|
248
|
-
|
|
249
|
-
**Steps**:
|
|
250
|
-
1. {step 1}
|
|
251
|
-
2. {step 2}
|
|
252
|
-
3. {step 3}
|
|
253
|
-
|
|
254
|
-
**Expected result**: {what should happen}
|
|
255
|
-
|
|
256
|
-
**Notes**: {any special considerations, DEF references, skip conditions}
|
|
257
|
-
```
|
|
258
|
-
|
|
259
|
-
**Mandatory coverage categories** (must have at least 1 TC per category where applicable):
|
|
260
|
-
|
|
261
|
-
- [ ] Access: unauthenticated user redirected to login
|
|
262
|
-
- [ ] Access: role without permission receives error / empty page
|
|
263
|
-
- [ ] Happy path: primary workflow with valid data succeeds
|
|
264
|
-
- [ ] Negative: required field missing → validation error shown
|
|
265
|
-
- [ ] Negative: duplicate record → system rejects with error message
|
|
266
|
-
- [ ] Negative: invalid format → system rejects with format error
|
|
267
|
-
- [ ] State transition: state changes correctly on action
|
|
268
|
-
- [ ] Export / download: if feature exists, file is generated
|
|
269
|
-
- [ ] Pagination / search: if feature exists, filtering works correctly
|
|
270
|
-
|
|
271
|
-
---
|
|
272
|
-
|
|
273
|
-
## Common Anti-Patterns to Avoid
|
|
274
|
-
|
|
275
|
-
| Anti-pattern | Why it's wrong | Correct approach |
|
|
276
|
-
|---|---|---|
|
|
277
|
-
| TC for a feature not in QA env | Creates invalid test expectations | Mark as `PENDING-CODE`, don't write test steps |
|
|
278
|
-
| TC with a real user's DNI in steps | Security issue | Use env var reference |
|
|
279
|
-
| TC with "click the X button" without knowing the element exists | Assumes UI without observation | Verify element exists first |
|
|
280
|
-
| 100+ TCs per submodule by copy-pasting variations | Bloats the spec without value | Merge near-identical cases, use parameterization |
|
|
281
|
-
| TC with "verify system works correctly" as expected result | Untestable | Write observable, measurable expected result |
|