@keber/qa-framework 1.5.2 → 1.6.2

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md CHANGED
@@ -33,21 +33,80 @@ has been removed or parameterized. See `docs/comparison-matrix.md` and `docs/gen
33
33
 
34
34
  ---
35
35
 
36
- ## [Unreleased]
36
+ ## [1.6.2] - 2026-05-05
37
37
 
38
38
  ### Added
39
39
 
40
- - **Stage 3.5 — Test Stabilization** (`agent-instructions/04b-test-stabilization.md`) — new agent
41
- instruction file for exhaustively reviewing and stabilizing generated Playwright tests before
42
- ADO integration. Covers baseline collection, failure classification (9 categories), false-positive
43
- validation, confidence scoring (≥90% exit threshold), `STABILIZATION-REPORT.md` schema, and
44
- upstream artifact update rules. Sits between Stage 4 (Automation Generation) and Stage 5
45
- (ADO Integration) and is mandatory before any CI pipeline registration.
40
+ - **`tests/helpers/debug/`** — diagnostic scripts directory created by `init` and `upgrade` inside
41
+ `07-automation/e2e/tests/`. Scripts here are excluded from test runs via `testIgnore`.
42
+ - **`tests/seeds/`** — data seeding specs directory created by `init` and `upgrade` inside
43
+ `07-automation/e2e/tests/`. Excluded from test runs via `testIgnore`.
44
+ - **`testIgnore`** in `playwright.config.ts` template — excludes `**/helpers/debug/**` and
45
+ `**/seeds/**` from Playwright test discovery.
46
+ - **`upgrade` step 5e** — patches existing `playwright.config.ts` in `e2e/`: replaces
47
+ `testDir: '.'` with `'./tests'`, injects `testIgnore` if missing, creates `helpers/debug/`
48
+ and `seeds/` if absent.
49
+ - **Dual-Track Pipeline** section in `docs/folder-structure-guide.md` and `qa/QA-STRUCTURE-GUIDE.md`
50
+ — documents the automation track and manual track as parallel paths sharing `01-specifications/`
51
+ as source of truth. Includes entry point table (3 scenarios) and track selection guidance.
52
+ Clarifies that `03-test-cases/` is the manual track artifact and `04-test-data/` is optional.
53
+
54
+ ### Fixed
55
+
56
+ - `qa/QA-STRUCTURE-GUIDE.md` — removed stale reference to deleted `docs/generalization-decisions.md`
57
+ from the `06-defects/` section.
58
+ - `README.md` — pipeline table updated with Track column; `03-test-cases/` and `04-test-data/`
59
+ annotated as manual-track / optional in the directory tree.
60
+
61
+ ---
62
+
63
+ ## [1.6.0] - 2026-04-29
64
+
65
+ ### Changed (breaking - automated by `upgrade`)
66
+
67
+ - **`07-automation/` restructured**: the Playwright project root moves from `07-automation/` to
68
+ `07-automation/e2e/`. Affected files: `playwright.config.ts`, `global-setup.ts`, `package.json`,
69
+ `.env.example`, `fixtures/`. Spec stubs move from `07-automation/e2e/{module}/` to
70
+ `07-automation/e2e/tests/{module}/`. Run `npx qa-framework upgrade` to migrate automatically.
71
+ - `testDir` in `playwright.config.ts` template is now `'./tests'` (was `'.'`).
72
+ - `QA_DIR` in the Azure Pipeline template now points to `qa/07-automation/e2e` (was `qa/07-automation`).
73
+ - CI commands: `cd qa/07-automation/e2e && npm install && npx playwright install chromium`.
74
+
75
+ ### Removed
76
+
77
+ - `agent-instructions/` directory (legacy monolithic per-task instruction files) — superseded by `skills/`
78
+ architecture since v1.5.0. All behavior is now in `skills/*/SKILL.md` + `references/`.
79
+ - `docs/comparison-matrix.md` — superseded by `docs/skills-architecture.md`.
80
+ - `docs/generalization-decisions.md` — internal design notes; no longer relevant.
81
+
82
+ ### Documentation
83
+
84
+ - `docs/architecture.md` — updated for v1.6.0: correct 07-automation/e2e/ tree, removed blazor-radzen
85
+ stub, added `.github/skills/` and `memory/` to target project box.
86
+ - `docs/installation.md` — rewritten: correct `npm install` command, no interactive prompts, correct
87
+ directory tree with e2e/integration/load, added `upgrade` command section.
88
+ - `docs/usage-with-agent.md` — rewritten: all `00-guides/` references replaced with `.github/skills/`,
89
+ session workflows updated to reflect skills model, DO/DON'T list corrected (e2e/ is now correct run location).
90
+ - `README.md` — version bumped, `agent-instructions/` entry removed from package tree.
91
+
92
+ ### Added
93
+
94
+ - `07-automation/integration/` — placeholder for API/integration tests (k6, JMeter, Azure Load Testing);
95
+ created on `init` and `upgrade`.
96
+ - `07-automation/load/` — placeholder for load and performance tests; created on `init` and `upgrade`.
97
+ - **`scripts/upgrade.js` migration section 5** — detects old v1.5.x structure and relocates all
98
+ affected files non-destructively (moves only when destination does not exist; warns on conflict).
99
+ - **Stage 5b - Test Stabilization** (`skills/qa-test-stabilization/`) — skill for exhaustively
100
+ reviewing and stabilizing generated Playwright tests before ADO integration. Covers baseline
101
+ collection, failure classification (9 categories), false-positive validation, confidence scoring
102
+ (>=90% exit threshold), `STABILIZATION-REPORT.md` schema, and upstream artifact update rules.
46
103
 
47
104
  ### Planned
48
105
 
49
106
  - `qa-framework generate spec <module-name>` — scaffold a full spec set from CLI
50
107
  - `qa-framework validate` — check `qa/` structure for compliance with conventions
51
- - `qa-framework upgrade` — migrate old embedded structure to package-based layout
52
108
  - Integration: `jest` support alongside Playwright
53
109
  - Integration: GitHub Actions pipeline template (alongside existing Azure Pipelines template)
110
+
111
+ ---
112
+
package/README.md CHANGED
@@ -1,16 +1,17 @@
1
1
  # qa-framework
2
2
 
3
3
  > Reusable, installable, agent-oriented QA framework for spec-driven automated testing.
4
+ > Current version: **v1.6.2**
4
5
 
5
6
  ---
6
7
 
7
8
  ## What is this?
8
9
 
9
- `qa-framework` is a methodology package that provides:
10
+ `@keber/qa-framework` is a methodology package that provides:
10
11
 
11
12
  - **A standardized `qa/` directory structure** for any project
12
13
  - **Reusable templates** for specifications, test plans, test cases, defect reports, and execution reports
13
- - **Agent instructions** (for GitHub Copilot and equivalent IDE agents) that enable AI-assisted QA work
14
+ - **Agent skills** (for GitHub Copilot and equivalent IDE agents) that enable AI-assisted QA work via a 3-layer skills architecture
14
15
  - **Optional integrations** for Playwright and Azure DevOps
15
16
  - **Bootstrap commands** to scaffold the structure into any project in seconds
16
17
 
@@ -20,7 +21,7 @@ This framework was designed to be:
20
21
  |---|---|
21
22
  | **Reusable** | Not tied to any specific project, domain, or tech stack |
22
23
  | **Decoupled** | Core is independent from Azure DevOps, Playwright, or any other tool |
23
- | **Agent-oriented** | Instructions are written for IDE agents (Copilot, etc.) to consume |
24
+ | **Agent-oriented** | Skills are written for IDE agents (Copilot, etc.) to consume with minimal token overhead |
24
25
  | **Spec-driven** | Every automation artifact traces back to a specification |
25
26
  | **Extensible** | Adapters/plugins for optional integrations are well-separated |
26
27
 
@@ -34,17 +35,19 @@ This framework was designed to be:
34
35
  npm install @keber/qa-framework
35
36
  ```
36
37
 
37
- The postinstall script scaffolds the `qa/` structure automatically. No need to run `init` manually.
38
+ The `postinstall` script runs `init --skip-if-exists` automatically, scaffolding the `qa/`
39
+ structure on first install without overwriting existing files.
38
40
 
39
41
  ### Using with an AI agent (GitHub Copilot, Claude, etc.)
40
42
 
41
43
  Use this prompt in your IDE agent chat:
42
44
 
43
45
  ```
44
- Install @keber/qa-framework, check it's readme file and set up the project for integration with azure devops
46
+ Install @keber/qa-framework, check its readme file and set up the project for integration with azure devops
45
47
  ```
46
48
 
47
- The install creates `qa/AGENT-NEXT-STEPS.md` with the exact steps the agent needs to complete the setup (optional integrations, module configuration, credentials, etc.).
49
+ The install creates `qa/AGENT-NEXT-STEPS.md` with the exact steps the agent needs to complete
50
+ the setup (optional integrations, module configuration, credentials, etc.).
48
51
 
49
52
  ### Using manually (without an agent)
50
53
 
@@ -53,13 +56,25 @@ npm install @keber/qa-framework
53
56
  # then read qa/AGENT-NEXT-STEPS.md for next steps
54
57
  ```
55
58
 
56
- The `init` command will:
59
+ `init` will:
57
60
 
58
61
  1. Create the `qa/` directory tree in your project root
59
- 2. Copy all base templates and agent instructions into place
62
+ 2. Copy all base templates into place
60
63
  3. Create `qa/qa-framework.config.json` for project-specific settings
61
- 4. Create `.github/copilot-instructions.md` with QA agent behavior rules
62
- 5. Create `qa/AGENT-NEXT-STEPS.md` with follow-up steps
64
+ 4. Generate `.github/copilot-instructions.md` with QA agent behavior rules (11 rules + ADO detection)
65
+ 5. Copy the `skills/` set to `.github/skills/` so the agent can load them on demand
66
+ 6. Create `qa/AGENT-NEXT-STEPS.md` with follow-up steps
67
+
68
+ ### Upgrading
69
+
70
+ After `npm update @keber/qa-framework`, run:
71
+
72
+ ```bash
73
+ npx qa-framework upgrade
74
+ ```
75
+
76
+ This overwrites **only** framework-owned files (skills, `copilot-instructions.md`,
77
+ `QA-STRUCTURE-GUIDE.md`) and never touches your specs, test plans, automation code, or memory.
63
78
 
64
79
  ---
65
80
 
@@ -67,33 +82,32 @@ The `init` command will:
67
82
 
68
83
  ```
69
84
  qa-framework/
70
- ├── README.md ← This file
71
- ├── CHANGELOG.md ← Version history
72
- ├── MIGRATION-NOTES.md ← How to migrate from embedded to package
73
- ├── package.json ← npm descriptor
74
- ├── qa-framework.config.json ← Example config (copy to project)
85
+ ├── README.md <- This file
86
+ ├── CHANGELOG.md <- Version history
87
+ ├── package.json <- npm descriptor
88
+ ├── qa-framework.config.json <- Example config (copy to project)
75
89
  │
76
- ├── docs/ ← Framework documentation
77
- │ ├── architecture.md ← Design decisions and component map
78
- │ ├── comparison-matrix.md ← Source-repo analysis (Phase 1 artifact)
79
- │ ├── generalization-decisions.md ← What was abstracted and why (Phase 2 artifact)
80
- │ ├── installation.md ← Detailed installation guide
81
- │ ├── usage-with-agent.md ← How to use the framework with an IDE agent
82
- │ ├── spec-driven-philosophy.md ← Core QA methodology
83
- │ └── folder-structure-guide.md ← Explanation of every qa/ folder
90
+ ├── docs/ <- Framework documentation
91
+ │ ├── architecture.md <- Design decisions and component map
92
+ │ ├── skills-architecture.md <- 3-layer skills model explanation
93
+ │ ├── installation.md <- Detailed installation guide
94
+ │ ├── usage-with-agent.md <- How to use the framework with an IDE agent
95
+ │ ├── spec-driven-philosophy.md <- Core QA methodology
96
+ │ └── folder-structure-guide.md <- Explanation of every qa/ folder
84
97
  │
85
- ├── agent-instructions/ ← Instructions for AI agents, per task
86
- │ ├── 00-module-analysis.md
87
- │ ├── 01-spec-generation.md
88
- │ ├── 02-test-plan-generation.md
89
- │ ├── 03-test-case-generation.md
90
- │ ├── 04-automation-generation.md
91
- │ ├── 04b-test-stabilization.md
92
- │ ├── 05-ado-integration.md
93
- │ └── 06-maintenance.md
98
+ ├── skills/ <- Agent skills (3-layer model; copied to .github/skills/ on init)
99
+ │ ├── qa-module-analysis/ <- Stage 1: analyze a module, produce 6-file spec set
100
+ │ ├── qa-spec-generation/ <- Stage 2: generate/update the 6-file spec set
101
+ │ ├── qa-test-plan/ <- Stage 3: generate a test plan
102
+ │ ├── qa-test-cases/ <- Stage 4: generate individual TC-{ID}.md files
103
+ │ ├── qa-automation/ <- Stage 5: implement Playwright automation
104
+ │ ├── qa-test-stabilization/ <- Stage 5b: stabilize failing tests
105
+ │ ├── qa-maintenance/ <- Stage 6: update artifacts after app changes
106
+ │ └── qa-ado-integration/ <- Optional: sync with Azure DevOps
107
+ │ (each skill contains SKILL.md + references/*.md)
94
108
  │
95
- ├── templates/ ← Reusable file templates
96
- │ ├── specification/ ← 6-file submodule set
109
+ ├── templates/ <- Reusable file templates
110
+ │ ├── specification/ <- 6-file submodule set
97
111
  │ │ ├── 00-inventory.md
98
112
  │ │ ├── 01-business-rules.md
99
113
  │ │ ├── 02-workflows.md
@@ -105,15 +119,17 @@ qa-framework/
105
119
  │ ├── execution-report.md
106
120
  │ ├── defect-report.md
107
121
  │ ├── session-summary.md
108
- │ └── automation-scaffold/ ← Files to bootstrap a Playwright E2E project
122
+ │ ├── qa-readme.md <- Template for qa/README.md
123
+ │ └── automation-scaffold/ <- Files to bootstrap a Playwright E2E project
109
124
  │ ├── package.json
110
125
  │ ├── playwright.config.ts
111
126
  │ ├── global-setup.ts
112
127
  │ ├── .env.example
113
128
  │ └── fixtures/
114
- │ └── auth.ts
129
+ │ ├── auth.ts
130
+ │ └── test-helpers.ts
115
131
  │
116
- ├── integrations/ ← Optional integration adapters
132
+ ├── integrations/ <- Optional integration adapters
117
133
  │ ├── playwright/
118
134
  │ │ └── README.md
119
135
  │ ├── playwright-azure-reporter/
@@ -125,16 +141,17 @@ qa-framework/
125
141
  │ ├── create-testplan-from-mapping.ps1
126
142
  │ └── sync-ado-titles.ps1
127
143
  │
128
- ├── examples/ ← Reference examples (non-project-specific)
144
+ ├── examples/ <- Reference examples (non-project-specific)
129
145
  │ └── module-example/
130
146
  │ ├── README.md
131
- │ └── submodule-example/ ← Full 6-file spec example
147
+ │ └── submodule-example/ <- Full 6-file spec example
132
148
  │
133
- └── scripts/ ← CLI commands
134
- ├── cli.js ← Entry point (qa-framework <command>)
135
- ├── init.js ← Scaffold qa/ tree
136
- ├── generate.js ← Generate individual artifact
137
- └── validate.js ← Check structure compliance
149
+ └── scripts/ <- CLI commands
150
+ ├── cli.js <- Entry point (qa-framework <command>)
151
+ ├── init.js <- Scaffold qa/ tree + skills + copilot-instructions
152
+ ├── upgrade.js <- Refresh framework-owned files after npm update
153
+ ├── generate.js <- Generate individual artifact from template
154
+ └── validate.js <- Check structure compliance
138
155
  ```
139
156
 
140
157
  ---
@@ -145,46 +162,102 @@ When you run `qa-framework init`, this structure is created inside your project:
145
162
 
146
163
  ```
147
164
  qa/
148
- ├── README.md ← Living index of all QA artifacts
149
- ├── QA-STRUCTURE-GUIDE.md ← Local copy of the structure guide
165
+ ├── README.md <- Living index of all QA artifacts
166
+ ├── QA-STRUCTURE-GUIDE.md <- Local copy of the structure guide (framework-owned)
167
+ ├── AGENT-NEXT-STEPS.md <- Active sprint checklist for the agent
150
168
  │
151
- ├── 00-guides/ ← Process guides and agent instructions
152
- ├── 00-standards/ ← Naming conventions, templates, data policies
153
- ├── 01-specifications/ ← Functional specs by module/submodule
154
- ├── 02-test-plans/ ← Test plans (automated + manual)
155
- ├── 03-test-cases/ ← Explicit test case documents
156
- ├── 04-test-data/ ← Test data, fixtures, factories
157
- ├── 05-test-execution/ ← Execution reports and results
158
- ├── 06-defects/ ← Defect tracking (optional)
159
- ├── 07-automation/ ← Automation code and config
160
- └── 08-azure-integration/ ← Azure DevOps integration (optional)
169
+ ├── 00-standards/ <- Naming conventions, templates, data policies
170
+ ├── 01-specifications/ <- Functional specs (used by generate command)
171
+ ├── {module}/{submodule}/ <- 6-file spec sets created per module/submodule in config
172
+ ├── 02-test-plans/ <- Test plans (automated + manual)
173
+ ├── 03-test-cases/ <- TC-*.md step-by-step docs (manual track; optional if fully automated)
174
+ ├── 04-test-data/ <- Shared test data factories/seeders (optional — per-module data lives in 01-specs)
175
+ ├── 05-test-execution/ <- Execution reports and results
176
+ ├── 06-defects/open|resolved/ <- Defect tracking
177
+ ├── 07-automation/ <- Playwright automation code and config
178
+ │ ├── e2e/ <- Playwright E2E project (self-contained)
179
+ │ │ ├── playwright.config.ts
180
+ │ │ ├── package.json
181
+ │ │ ├── global-setup.ts
182
+ │ │ ├── .env.example
183
+ │ │ ├── fixtures/auth.ts
184
+ │ │ ├── fixtures/test-helpers.ts
185
+ │ │ └── tests/{module}/ <- Generated .spec.ts stubs per submodule
186
+ │ ├── integration/ <- API / integration tests (k6, JMeter...)
187
+ │ └── load/ <- Load / performance tests (future)
188
+ ├── 08-azure-integration/ <- Azure DevOps integration (optional)
189
+ └── memory/ <- Agent learnings and session notes
190
+ └── INDEX.md
191
+ ```
192
+
193
+ Additionally, `init` writes to the project root:
194
+
195
+ ```
196
+ .github/
197
+ ├── copilot-instructions.md <- Generated agent rules (11 rules + ADO detection)
198
+ └── skills/ <- Copied from package skills/ (one folder per pipeline stage)
161
199
  ```
162
200
 
163
201
  See [docs/folder-structure-guide.md](docs/folder-structure-guide.md) for a full explanation of every folder.
164
202
 
165
203
  ---
166
204
 
167
- ## Agent Instructions
205
+ ## Skills Architecture
168
206
 
169
- The `agent-instructions/` folder contains Markdown documents designed for IDE agents
170
- (such as GitHub Copilot in VS Code). Each file covers one specific task:
207
+ The `skills/` folder contains agent skills organized in a **3-layer model** that minimizes
208
+ token overhead compared to the legacy monolithic per-task instruction file approach.
171
209
 
172
- | File | Agent Task |
173
- |---|---|
174
- | `00-module-analysis.md` | Analyze a module and produce the 6-file spec set |
175
- | `01-spec-generation.md` | Generate functional specifications |
176
- | `02-test-plan-generation.md` | Create a test plan for a module |
177
- | `03-test-case-generation.md` | Generate detailed test cases |
178
- | `04-automation-generation.md` | Write Playwright E2E tests |
179
- | `04b-test-stabilization.md` | Exhaustively review and stabilize generated tests (Stage 3.5) |
180
- | `05-ado-integration.md` | Sync with Azure DevOps Test Plans |
181
- | `06-maintenance.md` | Update QA artifacts after code changes |
182
-
183
- To use: copy the relevant instruction into a Copilot/agent chat, or reference it from a
184
- `.github/copilot-instructions.md` or equivalent workspace instruction file.
210
+ | Layer | Location | Purpose |
211
+ |---|---|---|
212
+ | 1 - Descriptor | YAML frontmatter in `SKILL.md` | Name + description; used for relevance detection (~50 tokens) |
213
+ | 2 - Process outline | `SKILL.md` body | Step-by-step process; always loaded when the skill applies (~400-600 tokens) |
214
+ | 3 - Reference material | `references/*.md` | Detailed templates, code patterns, decision trees; loaded on demand (~500-1500 tokens each) |
215
+
216
+ On `init`, the full `skills/` tree is copied to `.github/skills/` so the agent can load
217
+ them via `read_file` without touching `node_modules`.
218
+
219
+ ### QA Pipeline
220
+
221
+ Skills map to the pipeline stages enforced by `.github/copilot-instructions.md`:
222
+
223
+ | Stage | Skill | Task | Track | Prerequisite |
224
+ |---|---|---|---|---|
225
+ | 1 | `qa-module-analysis` | Analyze module, produce 6-file spec set | Both | None |
226
+ | 2 | `qa-spec-generation` | Generate/update 6-file spec set | Both | `00-inventory.md` exists |
227
+ | 3 | `qa-test-plan` | Generate test plan | Both | `05-test-scenarios.md` exists |
228
+ | 4 | `qa-test-cases` | Generate TC-{ID}.md files | Manual | Test plan exists |
229
+ | 5 | `qa-automation` | Implement Playwright tests | Automation | Specs approved, no PENDING-CODE |
230
+ | 5b | `qa-test-stabilization` | Stabilize failing tests | Automation | Failing tests exist |
231
+ | 6 | `qa-maintenance` | Update artifacts after app changes | Both | Delivered change |
232
+ | - | `qa-ado-integration` | Sync with Azure DevOps Test Plans | Both | ADO enabled in config |
233
+
234
+ See [docs/skills-architecture.md](docs/skills-architecture.md) for the full design rationale.
185
235
 
186
236
  ---
187
237
 
238
+ ## CLI Reference
239
+
240
+ ```
241
+ qa-framework <command> [options]
242
+
243
+ Commands:
244
+ init [--config <path>] Scaffold qa/ structure from config file
245
+ upgrade [--dry-run] Refresh framework-owned files after npm update
246
+ generate <artifact> Generate from template
247
+ artifact: spec | test-plan | test-case |
248
+ execution-report | defect-report |
249
+ session-summary
250
+ validate [--strict] Validate qa/ structure and naming conventions
251
+
252
+ Options:
253
+ --help, -h Show help
254
+ --version, -v Show version
255
+ ```
256
+
257
+ **upgrade** overwrites only framework-owned files (skills, `copilot-instructions.md`,
258
+ `QA-STRUCTURE-GUIDE.md`) and never touches project-owned files. Run `--dry-run` to preview
259
+ without writing.
260
+
188
261
  ## Configuration
189
262
 
190
263
  `qa-framework init` now bootstraps a default config automatically at `qa/qa-framework.config.json` if none exists.
@@ -214,10 +287,11 @@ See [docs/installation.md](docs/installation.md) for full config documentation.
214
287
  ## Design Principles
215
288
 
216
289
  1. **UI is the source of truth.** Test cases are defined from what is observable in the running QA environment, not from source code or requirements docs alone.
217
- 2. **Spec before automation.** Every automated test traces back to a written specification (TC-ID → spec file).
290
+ 2. **Spec before automation.** Every automated test traces back to a written specification (TC-ID -> spec file).
218
291
  3. **Credentials never in documentation.** All credentials are handled via environment variables and sanitized from markdown with `<PLACEHOLDER>`.
219
292
  4. **Agent-readable conventions.** File names, folder names, and IDs follow strict, predictable patterns so agents can navigate without guessing.
220
293
  5. **Optional integrations don't pollute the core.** Azure DevOps and Playwright configuration live in separate, opt-in sections.
294
+ 6. **Assertion polarity.** A test must assert what the spec requires, not what the app currently does. A failing test that exposes a real defect is more valuable than a passing test that hides one.
221
295
 
222
296
  ---
223
297
 
@@ -225,8 +299,8 @@ See [docs/installation.md](docs/installation.md) for full config documentation.
225
299
 
226
300
  This framework follows [Semantic Versioning](https://semver.org/):
227
301
 
228
- - **MAJOR**: breaking changes to the `qa/` directory structure or agent instruction APIs
229
- - **MINOR**: new templates, new optional integrations, new agent instruction files
302
+ - **MAJOR**: breaking changes to the `qa/` directory structure or skill APIs
303
+ - **MINOR**: new templates, new optional integrations, new skills
230
304
  - **PATCH**: bug fixes in CLI scripts, documentation corrections
231
305
 
232
306
  See [CHANGELOG.md](CHANGELOG.md) for detailed history.
@@ -241,4 +315,4 @@ This framework is designed to be forked and adapted. If you need a project-speci
241
315
  2. Place your adapter config and scripts there
242
316
  3. Reference it from `qa-framework.config.json` under `integrations`
243
317
 
244
- Do **not** put project-specific configuration inside `agent-instructions/` or `templates/`. Those must remain generic.
318
+ Do **not** put project-specific configuration inside `skills/` or `templates/`. Those must remain generic.
@@ -2,8 +2,8 @@
2
2
 
3
3
  ## Framework Architecture
4
4
 
5
- **Version**: 1.0.0
6
- **Date**: 2026-03-04
5
+ **Version**: 1.6.0
6
+ **Date**: 2026-04-29
7
7
 
8
8
  ---
9
9
 
@@ -27,16 +27,17 @@
27
27
  │ │ FRAMEWORK CORE │ │
28
28
  │ │ │ │
29
29
  │ │ ┌────────────────┐ ┌──────────────────┐ ┌─────────────────────┐ │ │
30
- │ │ │ qa/ structure │ │ Agent │ │ Standards & │ │ │
31
- │ │ │ (9 folders) │ │ instructions │ │ Naming conventions │ │ │
32
- │ │ │ │ │ (7 files) │ │ │ │ │
30
+ │ │ │ qa/ structure │ │ Agent skills │ │ Standards & │ │ │
31
+ │ │ │ (10 folders) │ │ (8 skill sets, │ │ Naming conventions │ │ │
32
+ │ │ │ │ │ 3-layer model) │ │ │ │ │
33
33
  │ │ └────────────────┘ └──────────────────┘ └─────────────────────┘ │ │
34
34
  │ │ │ │
35
35
  │ │ ┌────────────────┐ ┌──────────────────┐ ┌─────────────────────┐ │ │
36
36
  │ │ │ 6-file │ │ CLI commands │ │ Config schema │ │ │
37
- │ │ │ submodule │ │ (init/generate/ │ │ (qa-framework. │ │ │
38
- │ │ │ templates │ │ validate) │ │ config.json) │ │ │
39
- │ │ └────────────────┘ └──────────────────┘ └─────────────────────┘ │ │
37
+ │ │ │ submodule │ │ (init/upgrade/ │ │ (qa-framework. │ │ │
38
+ │ │ │ templates │ │ generate/ │ │ config.json) │ │ │
39
+ │ │ └────────────────┘ │ validate) │ └─────────────────────┘ │ │
40
+ │ │ └──────────────────┘ │ │
40
41
  │ └──────────────────────────────────────────────────────────────────────┘ │
41
42
  │ │
42
43
  │ ┌──────────────────────────────────────────────────────────────────────┐ │
@@ -44,18 +45,18 @@
44
45
  │ │ │ │
45
46
  │ │ ┌─────────────────────────┐ ┌──────────────────────────────┐ │ │
46
47
  │ │ │ playwright/ │ │ playwright-azure-reporter/ │ │ │
47
- │ │ │ - playwright.config.ts │ │ - reporter config template │ │ │
48
- │ │ │ - global-setup.ts │ │ - ado-ids-mapping.json │ │ │
49
- │ │ │ - fixtures/auth.ts │ │ - inject-ado-ids.ps1 │ │ │
50
- │ │ │ - .env.example │ │ - module-registry.json │ │ │
48
+ │ │ │ - e2e/playwright.cfg │ │ - reporter config template │ │ │
49
+ │ │ │ - e2e/global-setup.ts │ │ - ado-ids-mapping.json │ │ │
50
+ │ │ │ - e2e/fixtures/ │ │ - inject-ado-ids.ps1 │ │ │
51
+ │ │ │ - e2e/.env.example │ │ - module-registry.json │ │ │
51
52
  │ │ └─────────────────────────┘ └──────────────────────────────┘ │ │
52
53
  │ │ │ │
53
- │ │ ┌─────────────────────────┐ ┌──────────────────────────────┐ │ │
54
- │ │ │ ado-powershell/ │ │ blazor-radzen/ (stub) │ │ │
55
- │ │ │ - create-testplan.ps1 │ │ - openDropdownAndSelect() │ │ │
56
- │ │ │ - sync-ado-titles.ps1 │ │ - waitForBlazorRender() │ │ │
57
- │ │ │ - azure-pipeline.yml │ │ - toBeAttached() guidance │ │ │
58
- │ │ └─────────────────────────┘ └──────────────────────────────┘ │ │
54
+ │ │ ┌─────────────────────────┐ │ │
55
+ │ │ │ ado-powershell/ │ │ │
56
+ │ │ │ - create-testplan.ps1 │ │ │
57
+ │ │ │ - sync-ado-titles.ps1 │ │ │
58
+ │ │ │ - azure-pipeline.yml │ │ │
59
+ │ │ └─────────────────────────┘ │ │
59
60
  │ └──────────────────────────────────────────────────────────────────────┘ │
60
61
  └─────────────────────────────────────────────────────────────────────────────┘
61
62
 
@@ -65,20 +66,26 @@
65
66
  │ TARGET PROJECT │
66
67
  │ │
67
68
  │ qa/ │
68
- │ ├── README.md ← living index (project-specific) │
69
- │ ├── QA-STRUCTURE-GUIDE.md ← copy from framework │
70
- │ ├── qa-framework.config.json ← project config (parameterized) │
69
+ │ ├── README.md <- living index (project-specific) │
70
+ │ ├── AGENT-NEXT-STEPS.md <- active sprint queue (project-specific) │
71
+ │ ├── QA-STRUCTURE-GUIDE.md <- copy from framework (framework-owned) │
72
+ │ ├── qa-framework.config.json <- project config (parameterized) │
71
73
  │ │ │
72
- │ ├── 00-guides/ ← agent instructions (copied from pkg) │
73
- │ ├── 00-standards/ ← naming + templates (copied from pkg) │
74
- │ ├── 01-specifications/ ← generated per module (project work) │
75
- │ ├── 02-test-plans/ ← generated (project work) │
76
- │ ├── 03-test-cases/ ← generated (project work) │
77
- │ ├── 04-test-data/ ← project data (project work) │
78
- │ ├── 05-test-execution/ ← execution reports (project work) │
79
- │ ├── 06-defects/ ← optional defect cache │
80
- │ ├── 07-automation/ ← Playwright code (project work) │
81
- │ └── 08-azure-integration/ ← optional ADO integration │
74
+ │ ├── 00-standards/ <- naming + templates (copied from pkg) │
75
+ │ ├── 01-specifications/ <- generated per module (project work) │
76
+ │ ├── 02-test-plans/ <- generated (project work) │
77
+ │ ├── 03-test-cases/ <- generated (project work) │
78
+ │ ├── 04-test-data/ <- project data (project work) │
79
+ │ ├── 05-test-execution/ <- execution reports (project work) │
80
+ │ ├── 06-defects/open|resolved/ <- optional defect cache │
81
+ │ ├── 07-automation/e2e/ <- Playwright project (self-contained) │
82
+ │ ├── 07-automation/integration/ <- API/integration tests │
83
+ │ ├── 07-automation/load/ <- load/performance tests │
84
+ │ ├── 08-azure-integration/ <- optional ADO integration │
85
+ │ └── memory/ <- agent learnings (project-maintained) │
86
+ │ │
87
+ │ .github/copilot-instructions.md <- generated (framework-owned) │
88
+ │ .github/skills/ <- copied from package (framework-owned) │
82
89
  └─────────────────────────────────────────────────────────────────────────────┘
83
90
  ```
84
91
 
@@ -107,7 +114,7 @@ Module analysis (agent)
107
114
  │
108
115
  ▼
109
116
  07-automation/e2e/tests/{module}/
110
- *.spec.ts ←── test title contains TC-ID
117
+ *.spec.ts <--- test title contains TC-ID
111
118
  │
112
119
  ┌────────────────────┘
113
120
  │
@@ -129,21 +136,25 @@ Module analysis (agent)
129
136
 
130
137
  Installed always. Contains:
131
138
 
132
- - `qa/` directory skeleton (9 folders)
133
- - `00-guides/` — agent instructions
139
+ - `qa/` directory skeleton (10 folders)
140
+ - `.github/skills/` — 8 agent skill sets (3-layer model: SKILL.md + references/)
134
141
  - `00-standards/` — naming conventions, templates
135
142
  - `QA-STRUCTURE-GUIDE.md`
143
+ - `.github/copilot-instructions.md` — generated pipeline sequencer
136
144
  - `qa-framework.config.json` schema
137
145
 
138
146
  ### Layer 2 — Playwright Integration (opt-in)
139
147
 
140
- Installed when `integrations.playwright.enabled = true`:
148
+ Installed always (scaffold is always created by `init`):
141
149
 
142
150
  - `qa/07-automation/e2e/playwright.config.ts`
143
151
  - `qa/07-automation/e2e/global-setup.ts`
144
152
  - `qa/07-automation/e2e/.env.example`
145
153
  - `qa/07-automation/e2e/fixtures/auth.ts`
154
+ - `qa/07-automation/e2e/fixtures/test-helpers.ts`
146
155
  - `qa/07-automation/e2e/package.json`
156
+ - `qa/07-automation/integration/README.md` (placeholder)
157
+ - `qa/07-automation/load/README.md` (placeholder)
147
158
 
148
159
  ### Layer 3 — Azure DevOps Integration (opt-in)
149
160
 
@@ -155,13 +166,6 @@ Installed when `integrations.azureDevOps.enabled = true`:
155
166
  - PowerShell scripts (inject, create-plan, sync)
156
167
  - Azure Pipeline YAML template
157
168
 
158
- ### Layer 4 — Framework Adapters (opt-in per adapter)
159
-
160
- Technology-specific extensions. Currently available:
161
-
162
- - `blazor-radzen` — Playwright helpers for Radzen Blazor apps (STUB - v1.1 planned)
163
- - `aspnet-mvc` — Select2/jQuery datatable helpers (STUB - v1.1 planned)
164
-
165
169
  ---
166
170
 
167
171
  ## Configuration Architecture
@@ -188,10 +192,11 @@ The framework uses **Semantic Versioning**:
188
192
 
189
193
  | Change type | Version bump |
190
194
  |---|---|
191
- | New folder added to `qa/` tree | MAJOR (breaks existing structure) |
195
+ | New folder added to `qa/` tree or restructured | MAJOR |
192
196
  | New template file | MINOR |
193
197
  | New integration adapter | MINOR |
194
- | Agent instruction update | PATCH |
198
+ | New skill or skill update | MINOR |
199
+ | CLI command update | PATCH |
195
200
  | Documentation correction | PATCH |
196
201
 
197
202
  **Upgrade path**: When a new MAJOR version changes the `qa/` structure, the `MIGRATION-NOTES.md` in the package provides a step-by-step migration script. Projects should pin their framework version in `package.json` and upgrade deliberately.