@keber/qa-framework 1.11.3 → 1.12.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +2 -2
- package/docs/architecture.md +30 -26
- package/docs/decisions/lane-heartbeat-caller.md +86 -0
- package/docs/final-report.md +23 -23
- package/docs/folder-structure-guide.md +100 -80
- package/docs/installation.md +35 -7
- package/docs/skills-architecture.md +23 -23
- package/docs/spec-driven-philosophy.md +9 -9
- package/docs/usage-with-agent.md +83 -9
- package/docs/visuals/diag-01-ejecutivo-1.png +0 -0
- package/docs/visuals/diag-02-tactico-1.png +0 -0
- package/docs/visuals/diag-03-tecnico-1.png +0 -0
- package/docs/visuals/info-01-ejecutivo-1.png +0 -0
- package/docs/visuals/info-02-tactico-1.png +0 -0
- package/docs/visuals/info-03-tecnico-1.png +0 -0
- package/docs/visuals/pres-01-ejecutivo-1.png +0 -0
- package/docs/visuals/pres-02-tactico-1.png +0 -0
- package/docs/visuals/pres-03-tecnico-1.png +0 -0
- package/docs/visuals/qa-01-problema-1.png +0 -0
- package/docs/visuals/qa-02-que-es-1.png +0 -0
- package/docs/visuals/qa-03-pipeline-1.png +0 -0
- package/docs/visuals/qa-04-6-archivos-1.png +0 -0
- package/docs/visuals/qa-05-humano-1.png +0 -0
- package/docs/visuals/qa-06-agente-1.png +0 -0
- package/docs/visuals/qa-07-skills-arch-1.png +0 -0
- package/docs/visuals/qa-08-traceability-1.png +0 -0
- package/docs/visuals/qa-en-01-problem-1.png +0 -0
- package/docs/visuals/qa-en-02-what-is-1.png +0 -0
- package/docs/visuals/qa-en-03-pipeline-1.png +0 -0
- package/docs/visuals/qa-en-04-6files-1.png +0 -0
- package/docs/visuals/qa-en-05-human-1.png +0 -0
- package/docs/visuals/qa-en-06-agent-1.png +0 -0
- package/docs/visuals/qa-en-07-skills-arch-1.png +0 -0
- package/docs/visuals/qa-en-08-traceability-1.png +0 -0
- package/examples/module-example/suppliers/00-inventory.md +1 -1
- package/examples/module-example/suppliers/suppliers-create.spec.ts +8 -8
- package/integrations/ado-powershell/README.md +1 -1
- package/integrations/ado-powershell/scripts/sync-ado-titles.ps1 +2 -2
- package/integrations/playwright/README.md +4 -4
- package/package.json +7 -3
- package/qa-framework.config.json +23 -2
- package/scripts/check-forbidden-chars.js +187 -0
- package/scripts/check-spanish-accents.js +159 -0
- package/scripts/cli.js +1 -1
- package/scripts/generate.js +2 -2
- package/scripts/init.js +55 -7
- package/scripts/lib/claude-agents.js +78 -0
- package/scripts/lib/claude-commands.js +56 -0
- package/scripts/sync-version.js +36 -0
- package/scripts/upgrade.js +356 -20
- package/scripts/validate.js +17 -17
- package/skills/qa-ado-integration/SKILL.md +19 -19
- package/skills/qa-ado-integration/references/scripts-and-config.md +6 -6
- package/skills/qa-automation/SKILL.md +49 -27
- package/skills/qa-automation/references/config-checklist.md +4 -4
- package/skills/qa-automation/references/dom-inspection-template.js +7 -7
- package/skills/qa-automation/references/patterns.md +20 -20
- package/skills/qa-automation/references/pom-template.md +161 -0
- package/skills/qa-maintenance/SKILL.md +11 -11
- package/skills/qa-maintenance/references/update-rules.md +8 -8
- package/skills/qa-module-analysis/SKILL.md +20 -19
- package/skills/qa-module-analysis/references/exploration-checklist.md +4 -4
- package/skills/qa-module-analysis/references/output-verification.md +69 -0
- package/skills/qa-module-analysis/references/spec-file-formats.md +14 -14
- package/skills/qa-spec-generation/SKILL.md +13 -7
- package/skills/qa-test-cases/SKILL.md +11 -81
- package/skills/qa-test-cases/references/test-case-template.md +5 -5
- package/skills/qa-test-plan/SKILL.md +22 -86
- package/skills/qa-test-plan/references/plan-de-pruebas-template.md +24 -24
- package/skills/qa-test-stabilization/SKILL.md +59 -30
- package/skills/qa-test-stabilization/references/classification-protocol.md +35 -35
- package/templates/agent-next-steps.md +10 -10
- package/templates/agents/qa-analisis.md +142 -0
- package/templates/agents/qa-asesoria.md +48 -0
- package/templates/agents/qa-informe-resultados.md +234 -0
- package/templates/agents/qa-plan.md +131 -0
- package/templates/automation-scaffold/.env.example +36 -6
- package/templates/automation-scaffold/fixtures/auth.ts +27 -5
- package/templates/automation-scaffold/fixtures/base.ts +23 -1
- package/templates/automation-scaffold/fixtures/test-helpers.ts +1 -1
- package/templates/automation-scaffold/global-setup.ts +117 -48
- package/templates/automation-scaffold/playwright.config.ts +72 -32
- package/templates/automation-scaffold/scripts/global-setup-guards.js +184 -0
- package/templates/automation-scaffold/scripts/lane-config.js +269 -0
- package/templates/automation-scaffold/scripts/lane-lock.js +589 -0
- package/templates/defect-report.md +17 -3
- package/templates/execution-report.md +9 -9
- package/templates/integration-scaffold/.env.example +1 -1
- package/templates/integration-scaffold/global-setup.ts +21 -1
- package/templates/integration-scaffold/playwright.config.ts +15 -1
- package/templates/qa-framework.instructions.md +8 -7
- package/templates/qa-framework.rules.md +53 -0
- package/templates/qa-readme.md +8 -8
- package/templates/session-summary.md +2 -2
- package/templates/specification/00-inventory.md +3 -3
- package/templates/specification/01-business-rules.md +6 -6
- package/templates/specification/02-workflows.md +1 -1
- package/templates/specification/03-roles-permissions.md +1 -1
- package/templates/specification/04-test-data.md +8 -8
- package/templates/specification/05-test-scenarios.md +5 -5
- package/templates/test-case.md +3 -3
- package/templates/test-data-guidelines.md +7 -7
- package/templates/test-plan-sprint.md +13 -13
- package/templates/test-plan.md +3 -3
- package/skills/qa-test-plan/references/priority-and-feasibility.md +0 -159
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
# docs/folder-structure-guide.md
|
|
2
2
|
|
|
3
|
-
## `qa/` Directory Structure
|
|
3
|
+
## `qa/` Directory Structure - Full Reference Guide
|
|
4
4
|
|
|
5
5
|
This document explains the purpose, contents, and conventions for every folder in the `qa/` directory.
|
|
6
6
|
|
|
@@ -10,25 +10,28 @@ This document explains the purpose, contents, and conventions for every folder i
|
|
|
10
10
|
|
|
11
11
|
```
|
|
12
12
|
qa/
|
|
13
|
-
├── README.md
|
|
14
|
-
├── AGENT-NEXT-STEPS.md
|
|
15
|
-
├── QA-STRUCTURE-GUIDE.md
|
|
16
|
-
├── qa-framework.config.json
|
|
13
|
+
├── README.md <- Living master index (project-maintained)
|
|
14
|
+
├── AGENT-NEXT-STEPS.md <- Active sprint queue for the agent (project-maintained)
|
|
15
|
+
├── QA-STRUCTURE-GUIDE.md <- Copy of this guide (installed by framework)
|
|
16
|
+
├── qa-framework.config.json <- Project configuration
|
|
17
17
|
│
|
|
18
|
-
├── 00-standards/
|
|
19
|
-
├── 01-specifications/
|
|
20
|
-
├── 02-test-plans/
|
|
21
|
-
├── 03-test-cases/
|
|
22
|
-
├── 04-test-data/
|
|
23
|
-
├── 05-test-execution/
|
|
24
|
-
├── 06-defects/
|
|
25
|
-
├── 07-automation/
|
|
26
|
-
├── 08-azure-integration/
|
|
27
|
-
└── memory/
|
|
18
|
+
├── 00-standards/ <- Naming conventions and artifact templates
|
|
19
|
+
├── 01-specifications/ <- Functional specifications per module
|
|
20
|
+
├── 02-test-plans/ <- Plan de Pruebas per sprint/module (plan + TC table + steps)
|
|
21
|
+
├── 03-test-cases/ <- Standalone test case documents
|
|
22
|
+
├── 04-test-data/ <- Test data definitions and factories
|
|
23
|
+
├── 05-test-execution/ <- Execution reports and results
|
|
24
|
+
├── 06-defects/ <- Defect tracking (optional)
|
|
25
|
+
├── 07-automation/ <- Automation code (Playwright, etc.)
|
|
26
|
+
├── 08-azure-integration/ <- Azure DevOps integration (optional)
|
|
27
|
+
└── memory/ <- Project QA learnings (project-maintained)
|
|
28
28
|
```
|
|
29
29
|
|
|
30
30
|
Agent skills live in `.github/skills/` (not inside `qa/`). See the `.github/skills/` section below.
|
|
31
31
|
|
|
32
|
+
`init`/`upgrade` also generate `.claude/rules/qa-framework.md` and `.claude/commands/qa-*.md`
|
|
33
|
+
(Claude Code equivalents of the Copilot instructions/skills - see the section below).
|
|
34
|
+
|
|
32
35
|
---
|
|
33
36
|
|
|
34
37
|
## Dual-Track Pipeline
|
|
@@ -37,35 +40,35 @@ All pipeline paths share the same input: `01-specifications/`. From there, work
|
|
|
37
40
|
two tracks that can run in parallel or independently depending on team maturity.
|
|
38
41
|
|
|
39
42
|
```
|
|
40
|
-
01-specifications/ (source of truth
|
|
43
|
+
01-specifications/ (source of truth - what the app does and what must be tested)
|
|
41
44
|
│
|
|
42
45
|
├─── Automation track ──────────────────────────────────────────────────
|
|
43
46
|
│ │
|
|
44
47
|
│ ▼
|
|
45
|
-
│ 02-test-plans/sprints/Sprint-{N}/
|
|
48
|
+
│ 02-test-plans/sprints/Sprint-{N}/ <- Plan de Pruebas (plan + TC table + steps)
|
|
46
49
|
│ │ Stage 4 expands steps if needed
|
|
47
50
|
│ ▼
|
|
48
|
-
│ 07-automation/e2e/tests/
|
|
51
|
+
│ 07-automation/e2e/tests/ <- Playwright .spec.ts files
|
|
49
52
|
│ │
|
|
50
53
|
│ ▼
|
|
51
|
-
│ 05-test-execution/automated/
|
|
54
|
+
│ 05-test-execution/automated/ <- execution reports
|
|
52
55
|
│ │
|
|
53
56
|
│ ▼
|
|
54
|
-
│ 08-azure-integration/
|
|
57
|
+
│ 08-azure-integration/ <- ADO sync via playwright-azure-reporter
|
|
55
58
|
│
|
|
56
59
|
└─── Manual track ──────────────────────────────────────────────────────
|
|
57
60
|
│
|
|
58
61
|
▼
|
|
59
|
-
02-test-plans/sprints/Sprint-{N}/
|
|
62
|
+
02-test-plans/sprints/Sprint-{N}/ <- same Plan de Pruebas (shared artifact)
|
|
60
63
|
│
|
|
61
64
|
▼
|
|
62
|
-
03-test-cases/ (optional)
|
|
65
|
+
03-test-cases/ (optional) <- standalone TC-*.md for complex/reusable TCs
|
|
63
66
|
│
|
|
64
67
|
▼
|
|
65
|
-
05-test-execution/manual/
|
|
68
|
+
05-test-execution/manual/ <- manual run evidence
|
|
66
69
|
│
|
|
67
70
|
▼
|
|
68
|
-
08-azure-integration/
|
|
71
|
+
08-azure-integration/ <- ADO sync via ado-integration skill
|
|
69
72
|
(reads Plan de Pruebas, creates TCs with steps)
|
|
70
73
|
```
|
|
71
74
|
|
|
@@ -73,21 +76,21 @@ two tracks that can run in parallel or independently depending on team maturity.
|
|
|
73
76
|
|
|
74
77
|
| Scenario | Entry point | Tracks active |
|
|
75
78
|
|---|---|---|
|
|
76
|
-
| New system, no prior QA | Module analysis
|
|
77
|
-
| Sprint with ADO stories | Stories
|
|
78
|
-
| Maintenance/continuous improvement | Changed module
|
|
79
|
+
| New system, no prior QA | Module analysis -> spec generation | Automation (primary), Manual (as needed) |
|
|
80
|
+
| Sprint with ADO stories | Stories -> spec generation -> test plan | Both, driven by sprint scope |
|
|
81
|
+
| Maintenance/continuous improvement | Changed module -> maintenance skill | Automation (update existing tests) |
|
|
79
82
|
|
|
80
83
|
**Track selection guidance:**
|
|
81
84
|
- Use **automation track** when the goal is regression coverage and CI integration.
|
|
82
85
|
- Use **manual track** when: (a) team has manual testers who need step-by-step documents; (b) auditability or traceability to external standards is required; (c) features are not yet automatable.
|
|
83
|
-
- Both tracks use the same TC-IDs from `01-specifications/*/05-test-scenarios.md`
|
|
86
|
+
- Both tracks use the same TC-IDs from `01-specifications/*/05-test-scenarios.md` - traceability is preserved regardless of which track executes a given TC.
|
|
84
87
|
- `04-test-data/` is optional in both tracks. Use it when test data factories or seeders are shared across multiple modules. Per-submodule data belongs in `01-specifications/*/04-test-data.md`.
|
|
85
88
|
|
|
86
89
|
---
|
|
87
90
|
|
|
88
91
|
## .github/skills/
|
|
89
92
|
|
|
90
|
-
**Purpose**: 3-layer QA pipeline skills installed by the framework. Loaded by the agent on demand
|
|
93
|
+
**Purpose**: 3-layer QA pipeline skills installed by the framework. Loaded by the agent on demand - only the relevant skill for the current task is loaded, not all skills at once.
|
|
91
94
|
|
|
92
95
|
**Installed by framework** via `npm install @keber/qa-framework`:
|
|
93
96
|
|
|
@@ -103,8 +106,25 @@ two tracks that can run in parallel or independently depending on team maturity.
|
|
|
103
106
|
| `qa-maintenance/` | 6 | Updating specs and tests after app changes |
|
|
104
107
|
|
|
105
108
|
Each skill folder contains:
|
|
106
|
-
- `SKILL.md`
|
|
107
|
-
- `references/`
|
|
109
|
+
- `SKILL.md` - process outline (~400-600 tokens, always loaded when skill applies)
|
|
110
|
+
- `references/` - detailed templates and code patterns (loaded on demand)
|
|
111
|
+
|
|
112
|
+
---
|
|
113
|
+
|
|
114
|
+
## .claude/commands/ and .claude/rules/
|
|
115
|
+
|
|
116
|
+
**Purpose**: Claude Code native equivalents of `.github/instructions/qa-framework.instructions.md`
|
|
117
|
+
and `.github/skills/`, generated by `init`/`upgrade` in parallel with the Copilot artifacts
|
|
118
|
+
(no agent detection - both are always generated).
|
|
119
|
+
|
|
120
|
+
| File | Role |
|
|
121
|
+
|---|---|
|
|
122
|
+
| `.claude/rules/qa-framework.md` | Same 11 agent behavior rules + pipeline table as the Copilot instructions file. No frontmatter, so it loads unconditionally in every Claude Code session (equivalent to `applyTo: '**'`). |
|
|
123
|
+
| `.claude/commands/qa-{name}.md` | One thin wrapper per skill folder under `.github/skills/`, discovered dynamically (not hardcoded). Each wrapper reads: `Read the full skill at .github/skills/qa-{name}/SKILL.md FIRST, then follow its instructions exactly`, plus the stage prerequisite. Invoked as `/qa-{name}` in a Claude Code session. |
|
|
124
|
+
|
|
125
|
+
This is a "thin wrapper" pattern: the command never duplicates skill content, it only
|
|
126
|
+
points back at `SKILL.md` as the single source of truth. Updating a skill automatically
|
|
127
|
+
updates every command that references it - no regeneration needed, zero drift possible.
|
|
108
128
|
|
|
109
129
|
---
|
|
110
130
|
|
|
@@ -126,25 +146,25 @@ Each skill folder contains:
|
|
|
126
146
|
|
|
127
147
|
## 01-specifications/
|
|
128
148
|
|
|
129
|
-
**Purpose**: Functional specifications
|
|
149
|
+
**Purpose**: Functional specifications - the primary source of truth for what the application does and what must be tested.
|
|
130
150
|
|
|
131
151
|
**Structure**:
|
|
132
152
|
```
|
|
133
153
|
01-specifications/
|
|
134
|
-
├── README.md
|
|
135
|
-
├── shared/
|
|
136
|
-
│ ├── ui-menu-map.md
|
|
137
|
-
│ └── Instrucciones-analisis.md
|
|
154
|
+
├── README.md <- Index of all modules
|
|
155
|
+
├── shared/ <- Shared artifacts (sitemap, shared templates)
|
|
156
|
+
│ ├── ui-menu-map.md <- Auto-generated from playwright session
|
|
157
|
+
│ └── Instrucciones-analisis.md <- Checklist template for module analysis
|
|
138
158
|
│
|
|
139
159
|
└── module-{module-name}/
|
|
140
|
-
├── README.md
|
|
160
|
+
├── README.md <- Module index (submodule table + E2E flow)
|
|
141
161
|
└── submodule-{name}/
|
|
142
|
-
├── 00-inventory.md
|
|
143
|
-
├── 01-business-rules.md
|
|
144
|
-
├── 02-workflows.md
|
|
145
|
-
├── 03-roles-permissions.md
|
|
146
|
-
├── 04-test-data.md
|
|
147
|
-
└── 05-test-scenarios.md
|
|
162
|
+
├── 00-inventory.md <- UI elements, routes, APIs, fields
|
|
163
|
+
├── 01-business-rules.md <- RN-* rules the system enforces
|
|
164
|
+
├── 02-workflows.md <- FL-* user flow diagrams (Mermaid/ASCII)
|
|
165
|
+
├── 03-roles-permissions.md <- Role matrix
|
|
166
|
+
├── 04-test-data.md <- Data shapes and prerequisites
|
|
167
|
+
└── 05-test-scenarios.md <- TC-* test cases with priority and steps
|
|
148
168
|
```
|
|
149
169
|
|
|
150
170
|
**Naming rules**:
|
|
@@ -153,7 +173,7 @@ Each skill folder contains:
|
|
|
153
173
|
- Module code: 2-6 uppercase letters (e.g., `OPER`, `PERS`, `ARCF`)
|
|
154
174
|
- TC ID: `TC-{MODULE}-{SUBMODULE}-{3-digit}` (e.g., `TC-OPER-CAT-001`)
|
|
155
175
|
|
|
156
|
-
**TC target per submodule**: 50
|
|
176
|
+
**TC target per submodule**: 50-85 test cases is considered a well-analyzed submodule.
|
|
157
177
|
|
|
158
178
|
---
|
|
159
179
|
|
|
@@ -166,8 +186,8 @@ Each skill folder contains:
|
|
|
166
186
|
├── README.md
|
|
167
187
|
└── sprints/
|
|
168
188
|
└── Sprint-{NNN}/
|
|
169
|
-
├── Plan-de-Pruebas-{proyecto}-Sprint-{NNN}-{modulo}.md
|
|
170
|
-
└── Plan-de-Pruebas-{proyecto}-Sprint-{NNN}.md
|
|
189
|
+
├── Plan-de-Pruebas-{proyecto}-Sprint-{NNN}-{modulo}.md <- one per module
|
|
190
|
+
└── Plan-de-Pruebas-{proyecto}-Sprint-{NNN}.md <- consolidado (generated by @keber/ado-qa, not by agent)
|
|
171
191
|
```
|
|
172
192
|
|
|
173
193
|
**Naming convention**: `Plan-de-Pruebas-{proyecto}-Sprint-{NNN}-{modulo}.md`
|
|
@@ -179,7 +199,7 @@ Each skill folder contains:
|
|
|
179
199
|
- Header (project, sprint, dates, ADO Test Plan link)
|
|
180
200
|
- Scope, strategy, preconditions, test data
|
|
181
201
|
- **Tabla de Pruebas**: TC-ID | Suite | Título | Descripción | Steps (numbered `<br>`-separated) | Resultado Esperado | Confirma (Task ID or N/A) | Tipo | Prioridad
|
|
182
|
-
- Matriz de trazabilidad (Task
|
|
202
|
+
- Matriz de trazabilidad (Task <-> N)
|
|
183
203
|
- Automation notes
|
|
184
204
|
|
|
185
205
|
**Legacy migration**: `upgrade` Section 6 moves `automated/` and `manual/` subdirectories to `sprints/legacy-*` non-destructively.
|
|
@@ -194,7 +214,7 @@ Each skill folder contains:
|
|
|
194
214
|
|
|
195
215
|
```
|
|
196
216
|
03-test-cases/
|
|
197
|
-
├── README.md
|
|
217
|
+
├── README.md <- explains optional/legacy status
|
|
198
218
|
└── TC-{ID}-{title}.md
|
|
199
219
|
```
|
|
200
220
|
|
|
@@ -212,10 +232,10 @@ Each skill folder contains:
|
|
|
212
232
|
```
|
|
213
233
|
04-test-data/
|
|
214
234
|
├── README.md
|
|
215
|
-
├── users.md
|
|
216
|
-
├── fixtures/
|
|
217
|
-
├── factories/
|
|
218
|
-
└── seeders/
|
|
235
|
+
├── users.md <- Test user definitions (roles, env var references, NOT passwords)
|
|
236
|
+
├── fixtures/ <- Static data files (JSON/Markdown)
|
|
237
|
+
├── factories/ <- Dynamic data generation patterns (Markdown or TS files)
|
|
238
|
+
└── seeders/ <- DB/API seed scripts for complex test setup
|
|
219
239
|
```
|
|
220
240
|
|
|
221
241
|
**Critical rules**:
|
|
@@ -228,14 +248,14 @@ Each skill folder contains:
|
|
|
228
248
|
|
|
229
249
|
## 05-test-execution/
|
|
230
250
|
|
|
231
|
-
**Purpose**: Evidence of test runs
|
|
251
|
+
**Purpose**: Evidence of test runs - reports, results, and screenshots.
|
|
232
252
|
|
|
233
253
|
```
|
|
234
254
|
05-test-execution/
|
|
235
255
|
├── README.md
|
|
236
256
|
├── automated/
|
|
237
|
-
│ ├── {YYYY-MM-DD_HH-MM-SS_desc}.md
|
|
238
|
-
│ └── test-results/
|
|
257
|
+
│ ├── {YYYY-MM-DD_HH-MM-SS_desc}.md <- Execution report (human-readable summary)
|
|
258
|
+
│ └── test-results/ <- Playwright output (gitignored if large)
|
|
239
259
|
│ └── .last-run.json
|
|
240
260
|
└── manual/
|
|
241
261
|
├── exploratory/
|
|
@@ -271,8 +291,8 @@ Each skill folder contains:
|
|
|
271
291
|
- `resolved/` - closed by fix or formal decision
|
|
272
292
|
|
|
273
293
|
**Decision guide**:
|
|
274
|
-
- ADO enabled
|
|
275
|
-
- ADO disabled
|
|
294
|
+
- ADO enabled -> use ADO Work Items. `06-defects/` contains lightweight references only
|
|
295
|
+
- ADO disabled -> use this folder as primary defect tracker
|
|
276
296
|
- Either way: `test.skip()` references a defect ID so the skip is traceable
|
|
277
297
|
|
|
278
298
|
See `docs/generalization-decisions.md §7` for the full evaluation of this folder.
|
|
@@ -325,21 +345,21 @@ See `docs/generalization-decisions.md §7` for the full evaluation of this folde
|
|
|
325
345
|
```
|
|
326
346
|
08-azure-integration/
|
|
327
347
|
├── README.md
|
|
328
|
-
├── AGENT-ADO-INTEGRATION.md
|
|
329
|
-
├── PLAYWRIGHT-AZURE-REPORTER-PLAN.md
|
|
330
|
-
├── module-registry.json
|
|
331
|
-
├── ado-ids-mapping-{project}.json
|
|
348
|
+
├── AGENT-ADO-INTEGRATION.md <- Agent instructions for ADO operations
|
|
349
|
+
├── PLAYWRIGHT-AZURE-REPORTER-PLAN.md <- Reporter setup and migration log
|
|
350
|
+
├── module-registry.json <- Module -> spec path -> ADO plan/suite IDs
|
|
351
|
+
├── ado-ids-mapping-{project}.json <- Complete TC -> ADO WI ID registry
|
|
332
352
|
├── pipelines/
|
|
333
|
-
│ └── azure-pipeline-qa.yml
|
|
353
|
+
│ └── azure-pipeline-qa.yml <- CI/CD pipeline definition
|
|
334
354
|
└── scripts/
|
|
335
|
-
├── inject-ado-ids.ps1
|
|
336
|
-
├── create-testplan-from-mapping.ps1
|
|
337
|
-
└── sync-ado-titles.ps1
|
|
355
|
+
├── inject-ado-ids.ps1 <- Injects [ID] prefix into spec files
|
|
356
|
+
├── create-testplan-from-mapping.ps1 <- Creates ADO plan from mapping JSON
|
|
357
|
+
└── sync-ado-titles.ps1 <- Syncs spec titles with ADO TC titles
|
|
338
358
|
```
|
|
339
359
|
|
|
340
360
|
---
|
|
341
361
|
|
|
342
|
-
## `qa/README.md`
|
|
362
|
+
## `qa/README.md` - The Living Index
|
|
343
363
|
|
|
344
364
|
The `qa/README.md` is the primary human-readable project status document. It should be updated whenever:
|
|
345
365
|
- A new module is analyzed
|
|
@@ -352,46 +372,46 @@ Minimum contents:
|
|
|
352
372
|
- Quick-start commands for common tasks
|
|
353
373
|
- Active blockers section
|
|
354
374
|
- Last execution results summary
|
|
355
|
-
- **`## Sprint History`** section
|
|
375
|
+
- **`## Sprint History`** section - completed sprint checklists moved here from `AGENT-NEXT-STEPS.md`
|
|
356
376
|
|
|
357
377
|
Template: `templates/qa-readme.md`
|
|
358
378
|
|
|
359
379
|
---
|
|
360
380
|
|
|
361
|
-
## `qa/AGENT-NEXT-STEPS.md`
|
|
381
|
+
## `qa/AGENT-NEXT-STEPS.md` - Sprint Queue
|
|
362
382
|
|
|
363
383
|
The `AGENT-NEXT-STEPS.md` is the agent's task queue for the current sprint. It is read by the agent at the start of every conversation.
|
|
364
384
|
|
|
365
385
|
**Design principles (SRP + KISS)**:
|
|
366
|
-
- Contains **one active sprint only**
|
|
386
|
+
- Contains **one active sprint only** - no history, no completed checklists
|
|
367
387
|
- Three sections maximum: module status table, active sprint checklist, context references
|
|
368
|
-
- Does not repeat standing instructions already in `
|
|
388
|
+
- Does not repeat standing instructions already in `qa-framework.instructions.md`
|
|
369
389
|
- Detailed environment notes and patterns belong in `qa/memory/`, referenced from here
|
|
370
390
|
|
|
371
|
-
When a sprint completes, the agent **moves** the completed checklist to `qa/README.md
|
|
391
|
+
When a sprint completes, the agent **moves** the completed checklist to `qa/README.md -> ## Sprint History` and **deletes** that section from this file.
|
|
372
392
|
|
|
373
393
|
Template: `templates/agent-next-steps.md`
|
|
374
394
|
|
|
375
395
|
---
|
|
376
396
|
|
|
377
|
-
## `qa/memory/`
|
|
397
|
+
## `qa/memory/` - Project QA Learnings
|
|
378
398
|
|
|
379
399
|
Project-maintained folder for accumulated knowledge about the application under test: environment quirks, DOM patterns, Playwright gotchas, sprint-specific discoveries.
|
|
380
400
|
|
|
381
401
|
```
|
|
382
402
|
qa/memory/
|
|
383
|
-
├── INDEX.md
|
|
384
|
-
├── {technology}-patterns.md
|
|
385
|
-
├── {sprint-name}-discovery.md
|
|
386
|
-
└── {sprint-name}-lessons.md
|
|
403
|
+
├── INDEX.md <- Required entry point - index of all files
|
|
404
|
+
├── {technology}-patterns.md <- Framework/stack-specific patterns (e.g., playwright-blazor-wasm-patterns.md)
|
|
405
|
+
├── {sprint-name}-discovery.md <- DOM/UI findings for an in-progress sprint
|
|
406
|
+
└── {sprint-name}-lessons.md <- Retrospective patterns after a sprint completes
|
|
387
407
|
```
|
|
388
408
|
|
|
389
|
-
### `qa/memory/INDEX.md`
|
|
409
|
+
### `qa/memory/INDEX.md` - Required
|
|
390
410
|
|
|
391
411
|
The `INDEX.md` is the **gate** that the agent reads before loading any other memory file. Its purpose is to let the agent load only what is relevant to the current task, rather than loading all files.
|
|
392
412
|
|
|
393
413
|
```markdown
|
|
394
|
-
# Memory Index
|
|
414
|
+
# Memory Index - {PROJECT_NAME}
|
|
395
415
|
|
|
396
416
|
| File | Topic | When to load |
|
|
397
417
|
|---|---|---|
|
|
@@ -403,7 +423,7 @@ The `INDEX.md` is the **gate** that the agent reads before loading any other mem
|
|
|
403
423
|
**Rules for `qa/memory/`**:
|
|
404
424
|
- Always update `INDEX.md` when adding or modifying a memory file
|
|
405
425
|
- Sprint-specific discovery files (`sprint-N-discovery.md`) are for the duration of that sprint; consolidate long-term patterns into a technology-specific file afterwards
|
|
406
|
-
- Never load all memory files unconditionally
|
|
426
|
+
- Never load all memory files unconditionally - always read `INDEX.md` first and select by relevance
|
|
407
427
|
|
|
408
428
|
---
|
|
409
429
|
|
|
@@ -414,4 +434,4 @@ Optionally, place `SESSION-SUMMARY-YYYY-MM-DD.md` at the `qa/` root after each m
|
|
|
414
434
|
- Record any blockers encountered
|
|
415
435
|
- Provide the starting point for the next session
|
|
416
436
|
|
|
417
|
-
This is especially valuable for agents
|
|
437
|
+
This is especially valuable for agents - a session summary provides continuity across multiple chat sessions.
|
package/docs/installation.md
CHANGED
|
@@ -13,7 +13,7 @@
|
|
|
13
13
|
|
|
14
14
|
---
|
|
15
15
|
|
|
16
|
-
## Option A
|
|
16
|
+
## Option A - npm install (recommended)
|
|
17
17
|
|
|
18
18
|
```bash
|
|
19
19
|
npm install --save-dev @keber/qa-framework
|
|
@@ -57,7 +57,7 @@ qa/
|
|
|
57
57
|
|
|
58
58
|
---
|
|
59
59
|
|
|
60
|
-
## Option B
|
|
60
|
+
## Option B - Clone or copy (no npm registry)
|
|
61
61
|
|
|
62
62
|
```bash
|
|
63
63
|
# From your project root
|
|
@@ -69,7 +69,7 @@ node tools/qa-framework/scripts/cli.js init
|
|
|
69
69
|
|
|
70
70
|
---
|
|
71
71
|
|
|
72
|
-
## Option C
|
|
72
|
+
## Option C - Manual scaffold (advanced)
|
|
73
73
|
|
|
74
74
|
If you prefer to control exactly what gets created:
|
|
75
75
|
|
|
@@ -156,9 +156,37 @@ Edit `qa/qa-framework.config.json` (or `qa-framework.config.json` at project roo
|
|
|
156
156
|
**Never** put `testPlanId`, `suiteId`, or `ADO_PAT` in the config file directly.
|
|
157
157
|
Use environment variables:
|
|
158
158
|
|
|
159
|
-
- `ADO_PAT`
|
|
160
|
-
|
|
161
|
-
|
|
159
|
+
- `ADO_PAT` - Personal Access Token
|
|
160
|
+
|
|
161
|
+
#### Optional: ANALISIS/PLAN agent mode (Claude Code only)
|
|
162
|
+
|
|
163
|
+
Gated by `integrations.azureDevOps.sprintCycle.enabled`. When `true`, `init`/`upgrade`
|
|
164
|
+
generate four Claude Code subagents (`.claude/agents/qa-analisis.md`, `qa-plan.md`,
|
|
165
|
+
`qa-asesoria.md`, `qa-informe-resultados.md`) for a sprint-centric manual testing
|
|
166
|
+
workflow parallel to the 6-stage pipeline. See
|
|
167
|
+
[docs/usage-with-agent.md](usage-with-agent.md) for details.
|
|
168
|
+
|
|
169
|
+
```json
|
|
170
|
+
{
|
|
171
|
+
"integrations": {
|
|
172
|
+
"azureDevOps": {
|
|
173
|
+
"enabled": true,
|
|
174
|
+
"sprintCycle": {
|
|
175
|
+
"enabled": true,
|
|
176
|
+
"sprintDurationDays": 8,
|
|
177
|
+
"manualTestingTimeboxDays": 2,
|
|
178
|
+
"dateFormat": "dd-mm-aaaa",
|
|
179
|
+
"timezone": "America/Santiago"
|
|
180
|
+
}
|
|
181
|
+
}
|
|
182
|
+
}
|
|
183
|
+
}
|
|
184
|
+
```
|
|
185
|
+
|
|
186
|
+
All `sprintCycle` fields are optional; omitted fields fall back to neutral framework
|
|
187
|
+
defaults (8-day sprint, 2-day manual testing timebox, `dd-mm-aaaa`, `America/Santiago`).
|
|
188
|
+
- `ADO_PLAN_ID` - Test Plan ID (can also go in `module-registry.json`)
|
|
189
|
+
- `ADO_SUITE_ID` - Suite ID
|
|
162
190
|
|
|
163
191
|
---
|
|
164
192
|
|
|
@@ -244,6 +272,6 @@ The `upgrade` command:
|
|
|
244
272
|
1. Checks the current framework version in your project
|
|
245
273
|
2. Shows a diff of changed template and instruction files
|
|
246
274
|
3. Prompts before overwriting any file that has local modifications
|
|
247
|
-
4. Never touches `01-specifications/`, `02-test-plans/`, `03-test-cases/`, `04-test-data/`, `05-test-execution/`, `06-defects/`
|
|
275
|
+
4. Never touches `01-specifications/`, `02-test-plans/`, `03-test-cases/`, `04-test-data/`, `05-test-execution/`, `06-defects/` - only framework-owned files are updated
|
|
248
276
|
|
|
249
277
|
See `MIGRATION-NOTES.md` for version-specific migration instructions.
|
|
@@ -16,9 +16,9 @@ The 3-layer skill architecture reduces this to ~500 tokens for a typical task, l
|
|
|
16
16
|
|
|
17
17
|
| Layer | Location | Size | Purpose |
|
|
18
18
|
|---|---|---|---|
|
|
19
|
-
| 1
|
|
20
|
-
| 2
|
|
21
|
-
| 3
|
|
19
|
+
| 1 - Descriptor | YAML frontmatter in `SKILL.md` | ~50 tokens | Name + description; used for relevance detection |
|
|
20
|
+
| 2 - Process outline | `SKILL.md` body | ~400-600 tokens | Step-by-step process without code; always loaded when skill applies |
|
|
21
|
+
| 3 - Reference material | `references/*.md` | ~500-1500 tokens each | Detailed templates, code patterns, decision trees; loaded on demand |
|
|
22
22
|
|
|
23
23
|
The agent reads Layer 2 to understand what to do, then loads only the specific `references/` files it needs for the current step.
|
|
24
24
|
|
|
@@ -26,7 +26,7 @@ The agent reads Layer 2 to understand what to do, then loads only the specific `
|
|
|
26
26
|
|
|
27
27
|
## Pipeline Sequence
|
|
28
28
|
|
|
29
|
-
Skills map to the QA pipeline stages defined in `copilot-instructions.md`. The file acts as a **pipeline sequencer**
|
|
29
|
+
Skills map to the QA pipeline stages defined in `copilot-instructions.md`. The file acts as a **pipeline sequencer** - it defines which stage comes before which, and what the prerequisite is. Skills are the implementation of each stage.
|
|
30
30
|
|
|
31
31
|
```
|
|
32
32
|
Stage 1: qa-module-analysis (no prerequisite)
|
|
@@ -61,12 +61,12 @@ Optional (any time after Stage 4):
|
|
|
61
61
|
|---|---|---|
|
|
62
62
|
| `qa-module-analysis` | Stage 1 process (4 phases) | `exploration-checklist.md`, `spec-file-formats.md` |
|
|
63
63
|
| `qa-spec-generation` | Stage 2 rules + anti-patterns | `spec-file-formats.md` (shared with qa-module-analysis) |
|
|
64
|
-
| `qa-test-plan` | Stage 3
|
|
65
|
-
| `qa-test-cases` | Stage 4
|
|
66
|
-
| `qa-automation` | Stage 5
|
|
67
|
-
| `qa-test-stabilization` | Stage 5b
|
|
68
|
-
| `qa-ado-integration` | Optional ADO sync
|
|
69
|
-
| `qa-maintenance` | Stage 6
|
|
64
|
+
| `qa-test-plan` | Stage 3 - Plan de Pruebas generation | `plan-de-pruebas-template.md` (includes priority + feasibility rules) |
|
|
65
|
+
| `qa-test-cases` | Stage 4 - TC document rules | `test-case-template.md` |
|
|
66
|
+
| `qa-automation` | Stage 5 - patterns + completion checklist | `patterns.md`, `config-checklist.md` |
|
|
67
|
+
| `qa-test-stabilization` | Stage 5b - 8 steps + confidence scoring | `classification-protocol.md` (includes report template) |
|
|
68
|
+
| `qa-ado-integration` | Optional ADO sync - 6 steps | `scripts-and-config.md` |
|
|
69
|
+
| `qa-maintenance` | Stage 6 - update rules | `update-rules.md` |
|
|
70
70
|
|
|
71
71
|
---
|
|
72
72
|
|
|
@@ -85,7 +85,7 @@ When a consumer project runs `npm install @keber/qa-framework`, the `postinstall
|
|
|
85
85
|
SKILL.md
|
|
86
86
|
...
|
|
87
87
|
|
|
88
|
-
|
|
88
|
+
-> copies to ->
|
|
89
89
|
|
|
90
90
|
{consumer-project}/.github/skills/
|
|
91
91
|
qa-module-analysis/
|
|
@@ -96,7 +96,7 @@ When a consumer project runs `npm install @keber/qa-framework`, the `postinstall
|
|
|
96
96
|
...
|
|
97
97
|
```
|
|
98
98
|
|
|
99
|
-
The copy uses `writeIfMissing`
|
|
99
|
+
The copy uses `writeIfMissing` - it will not overwrite files that already exist in the consumer project. This allows consumer projects to customize their `.github/skills/` without losing changes on re-install.
|
|
100
100
|
|
|
101
101
|
---
|
|
102
102
|
|
|
@@ -112,7 +112,7 @@ Consumer projects can extend the skills by adding skill folders directly to `.gi
|
|
|
112
112
|
Describe when this skill should be triggered.
|
|
113
113
|
---
|
|
114
114
|
```
|
|
115
|
-
2. Add the skill body following the same 4-section structure: prerequisite
|
|
115
|
+
2. Add the skill body following the same 4-section structure: prerequisite -> steps -> rules -> outputs
|
|
116
116
|
3. Optionally add `references/*.md` for detailed templates
|
|
117
117
|
4. Reference the skill from `copilot-instructions.md` if it's part of your pipeline
|
|
118
118
|
|
|
@@ -124,13 +124,13 @@ Project-specific skills are not overwritten by `npm install` updates.
|
|
|
124
124
|
|
|
125
125
|
| Former file | New skill | Token reduction |
|
|
126
126
|
|---|---|---|
|
|
127
|
-
| `agent-instructions/00-module-analysis.md` | `.github/skills/qa-module-analysis/SKILL.md` | ~2,500
|
|
128
|
-
| `agent-instructions/01-spec-generation.md` | `.github/skills/qa-spec-generation/SKILL.md` | ~2,800
|
|
129
|
-
| `agent-instructions/02-test-plan-generation.md` | `.github/skills/qa-test-plan/SKILL.md` | ~2,200
|
|
130
|
-
| `agent-instructions/03-test-case-generation.md` | `.github/skills/qa-test-cases/SKILL.md` | ~2,100
|
|
131
|
-
| `agent-instructions/04-automation-generation.md` | `.github/skills/qa-automation/SKILL.md` | ~3,500
|
|
132
|
-
| `agent-instructions/04b-test-stabilization.md` | `.github/skills/qa-test-stabilization/SKILL.md` | ~3,800
|
|
133
|
-
| `agent-instructions/05-ado-integration.md` | `.github/skills/qa-ado-integration/SKILL.md` | ~2,400
|
|
134
|
-
| `agent-instructions/06-maintenance.md` | `.github/skills/qa-maintenance/SKILL.md` | ~1,700
|
|
135
|
-
|
|
136
|
-
**Total context reduction**: ~21,000 tokens
|
|
127
|
+
| `agent-instructions/00-module-analysis.md` | `.github/skills/qa-module-analysis/SKILL.md` | ~2,500 -> ~500 + on-demand references |
|
|
128
|
+
| `agent-instructions/01-spec-generation.md` | `.github/skills/qa-spec-generation/SKILL.md` | ~2,800 -> ~400 + shared references |
|
|
129
|
+
| `agent-instructions/02-test-plan-generation.md` | `.github/skills/qa-test-plan/SKILL.md` | ~2,200 -> ~400 + test-plan-template |
|
|
130
|
+
| `agent-instructions/03-test-case-generation.md` | `.github/skills/qa-test-cases/SKILL.md` | ~2,100 -> ~400 + tc-template |
|
|
131
|
+
| `agent-instructions/04-automation-generation.md` | `.github/skills/qa-automation/SKILL.md` | ~3,500 -> ~500 + patterns + config |
|
|
132
|
+
| `agent-instructions/04b-test-stabilization.md` | `.github/skills/qa-test-stabilization/SKILL.md` | ~3,800 -> ~500 + classification |
|
|
133
|
+
| `agent-instructions/05-ado-integration.md` | `.github/skills/qa-ado-integration/SKILL.md` | ~2,400 -> ~400 + scripts |
|
|
134
|
+
| `agent-instructions/06-maintenance.md` | `.github/skills/qa-maintenance/SKILL.md` | ~1,700 -> ~400 + update-rules |
|
|
135
|
+
|
|
136
|
+
**Total context reduction**: ~21,000 tokens -> ~3,500 tokens (Layer 2 only) + selective Layer 3 loading
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
# docs/spec-driven-philosophy.md
|
|
2
2
|
|
|
3
|
-
## Spec-Driven Automated Testing
|
|
3
|
+
## Spec-Driven Automated Testing - Core Philosophy
|
|
4
4
|
|
|
5
5
|
This document explains the methodology that underpins the entire `qa-framework`.
|
|
6
6
|
|
|
@@ -34,7 +34,7 @@ A test case with origin `PENDING-CODE` or `BLOCKED-PERMISSIONS` must be in `test
|
|
|
34
34
|
## The Spec-Before-Automation Rule
|
|
35
35
|
|
|
36
36
|
```
|
|
37
|
-
Specification exists FIRST
|
|
37
|
+
Specification exists FIRST -> Automation comes second
|
|
38
38
|
```
|
|
39
39
|
|
|
40
40
|
Automation artifacts (spec files, page objects) always reference a spec TC-ID. It is never acceptable to write a Playwright test without a corresponding TC in `05-test-scenarios.md`.
|
|
@@ -90,10 +90,10 @@ E2E tests should NOT try to cover:
|
|
|
90
90
|
|
|
91
91
|
| Priority | Meaning | Target in automation |
|
|
92
92
|
|---|---|---|
|
|
93
|
-
| **P0** | Critical
|
|
94
|
-
| **P1** | High
|
|
95
|
-
| **P2** | Medium
|
|
96
|
-
| **P3** | Low
|
|
93
|
+
| **P0** | Critical - system unusable without this | Must be automated |
|
|
94
|
+
| **P1** | High - significant impact if broken | Should be automated |
|
|
95
|
+
| **P2** | Medium - moderate impact | Automate if feasible |
|
|
96
|
+
| **P3** | Low - minor impact | Manual testing acceptable |
|
|
97
97
|
|
|
98
98
|
In the automation suite, P0 tests form the **smoke suite** that runs on every CI build.
|
|
99
99
|
P1 tests run on scheduled runs or before each release.
|
|
@@ -165,6 +165,6 @@ This framework supports both:
|
|
|
165
165
|
| Maintenance after code change | Agent + Human review |
|
|
166
166
|
|
|
167
167
|
The handoff between agent and human should always happen at a known checkpoint:
|
|
168
|
-
- After the spec set is produced
|
|
169
|
-
- After automation is written
|
|
170
|
-
- After a defect is filed
|
|
168
|
+
- After the spec set is produced -> human reviews before automation is written
|
|
169
|
+
- After automation is written -> human reviews coverage mapping
|
|
170
|
+
- After a defect is filed -> human decides priority and fix approach
|