@keber/qa-framework 1.6.2 → 1.7.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +37 -0
- package/README.md +17 -8
- package/docs/installation.md +2 -2
- package/package.json +1 -1
- package/scripts/init.js +11 -49
- package/scripts/upgrade.js +42 -40
- package/templates/qa-framework.instructions.md +45 -0
package/CHANGELOG.md
CHANGED
|
@@ -33,6 +33,43 @@ has been removed or parameterized. See `docs/comparison-matrix.md` and `docs/gen
|
|
|
33
33
|
|
|
34
34
|
---
|
|
35
35
|
|
|
36
|
+
## [1.7.1] - 2026-05-06
|
|
37
|
+
|
|
38
|
+
### Fixed
|
|
39
|
+
|
|
40
|
+
- **Agent instructions template extracted** — `templates/qa-framework.instructions.md` is now
|
|
41
|
+
the single source of truth for QA agent rules. Both `init` and `upgrade` read and interpolate
|
|
42
|
+
this file instead of maintaining separate hardcoded strings, eliminating the rule divergence
|
|
43
|
+
that caused `upgrade` to deploy only 7 rules instead of 11.
|
|
44
|
+
- `upgrade.js` — removed `buildCopilotInstructions()` function (dead code after template extraction).
|
|
45
|
+
|
|
46
|
+
---
|
|
47
|
+
|
|
48
|
+
## [1.7.0] - 2026-05-06
|
|
49
|
+
|
|
50
|
+
### Changed
|
|
51
|
+
|
|
52
|
+
- **Agent instructions moved to `.github/instructions/qa-framework.instructions.md`** — the
|
|
53
|
+
framework no longer writes to `.github/copilot-instructions.md`. The new file is a VS Code
|
|
54
|
+
`*.instructions.md` file with `applyTo: '**'` frontmatter, which Copilot loads automatically.
|
|
55
|
+
This eliminates the conflict between framework-owned rules and project-owned custom instructions:
|
|
56
|
+
both files coexist independently and the framework file is freely upgradeable.
|
|
57
|
+
|
|
58
|
+
### Added
|
|
59
|
+
|
|
60
|
+
- **`upgrade` step 2b — automatic migration of old `copilot-instructions.md`**:
|
|
61
|
+
- If the file contains only QA Framework content (any previous version) → the file is deleted.
|
|
62
|
+
- If the file contains custom instructions and a QA Framework section → the QA section is stripped and the custom content is preserved.
|
|
63
|
+
- If the file contains no QA Framework content → the file is left untouched.
|
|
64
|
+
The detection marker is the heading `# QA Framework Instructions`, consistent across all versions.
|
|
65
|
+
|
|
66
|
+
### Migration
|
|
67
|
+
|
|
68
|
+
Run `npx qa-framework upgrade`. No manual steps required.
|
|
69
|
+
See `MIGRATION-NOTES.md` — section "Upgrading from a version that used `.github/copilot-instructions.md`".
|
|
70
|
+
|
|
71
|
+
---
|
|
72
|
+
|
|
36
73
|
## [1.6.2] - 2026-05-05
|
|
37
74
|
|
|
38
75
|
### Added
|
package/README.md
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
# qa-framework
|
|
2
2
|
|
|
3
3
|
> Reusable, installable, agent-oriented QA framework for spec-driven automated testing.
|
|
4
|
-
> Current version: **v1.
|
|
4
|
+
> Current version: **v1.7.1**
|
|
5
5
|
|
|
6
6
|
---
|
|
7
7
|
|
|
@@ -61,7 +61,7 @@ npm install @keber/qa-framework
|
|
|
61
61
|
1. Create the `qa/` directory tree in your project root
|
|
62
62
|
2. Copy all base templates into place
|
|
63
63
|
3. Create `qa/qa-framework.config.json` for project-specific settings
|
|
64
|
-
4. Generate `.github/
|
|
64
|
+
4. Generate `.github/instructions/qa-framework.instructions.md` with QA agent behavior rules (11 rules + ADO detection)
|
|
65
65
|
5. Copy the `skills/` set to `.github/skills/` so the agent can load them on demand
|
|
66
66
|
6. Create `qa/AGENT-NEXT-STEPS.md` with follow-up steps
|
|
67
67
|
|
|
@@ -73,9 +73,13 @@ After `npm update @keber/qa-framework`, run:
|
|
|
73
73
|
npx qa-framework upgrade
|
|
74
74
|
```
|
|
75
75
|
|
|
76
|
-
This overwrites **only** framework-owned files (skills, `
|
|
76
|
+
This overwrites **only** framework-owned files (skills, `qa-framework.instructions.md`,
|
|
77
77
|
`QA-STRUCTURE-GUIDE.md`) and never touches your specs, test plans, automation code, or memory.
|
|
78
78
|
|
|
79
|
+
If upgrading from a version that wrote to `.github/copilot-instructions.md`, `upgrade`
|
|
80
|
+
automatically removes the QA section from that file (or deletes it if it contained only
|
|
81
|
+
framework content), and creates `.github/instructions/qa-framework.instructions.md` instead.
|
|
82
|
+
|
|
79
83
|
---
|
|
80
84
|
|
|
81
85
|
## Repository Structure (this package)
|
|
@@ -148,7 +152,7 @@ qa-framework/
|
|
|
148
152
|
│
|
|
149
153
|
└── scripts/ <- CLI commands
|
|
150
154
|
├── cli.js <- Entry point (qa-framework <command>)
|
|
151
|
-
├── init.js <- Scaffold qa/ tree + skills +
|
|
155
|
+
├── init.js <- Scaffold qa/ tree + skills + qa-framework.instructions.md
|
|
152
156
|
├── upgrade.js <- Refresh framework-owned files after npm update
|
|
153
157
|
├── generate.js <- Generate individual artifact from template
|
|
154
158
|
└── validate.js <- Check structure compliance
|
|
@@ -194,10 +198,15 @@ Additionally, `init` writes to the project root:
|
|
|
194
198
|
|
|
195
199
|
```
|
|
196
200
|
.github/
|
|
197
|
-
├──
|
|
198
|
-
└──
|
|
201
|
+
├── instructions/
|
|
202
|
+
│ └── qa-framework.instructions.md <- QA agent rules (framework-owned, applyTo: '**')
|
|
203
|
+
└── skills/ <- Copied from package skills/ (one folder per pipeline stage)
|
|
199
204
|
```
|
|
200
205
|
|
|
206
|
+
> `qa-framework.instructions.md` is a VS Code `*.instructions.md` file, separate from any
|
|
207
|
+
> custom `.github/copilot-instructions.md` the project may already have. The two files
|
|
208
|
+
> coexist without conflict and are independently upgradeable.
|
|
209
|
+
|
|
201
210
|
See [docs/folder-structure-guide.md](docs/folder-structure-guide.md) for a full explanation of every folder.
|
|
202
211
|
|
|
203
212
|
---
|
|
@@ -218,7 +227,7 @@ them via `read_file` without touching `node_modules`.
|
|
|
218
227
|
|
|
219
228
|
### QA Pipeline
|
|
220
229
|
|
|
221
|
-
Skills map to the pipeline stages enforced by `.github/
|
|
230
|
+
Skills map to the pipeline stages enforced by `.github/instructions/qa-framework.instructions.md`:
|
|
222
231
|
|
|
223
232
|
| Stage | Skill | Task | Track | Prerequisite |
|
|
224
233
|
|---|---|---|---|---|
|
|
@@ -254,7 +263,7 @@ Options:
|
|
|
254
263
|
--version, -v Show version
|
|
255
264
|
```
|
|
256
265
|
|
|
257
|
-
**upgrade** overwrites only framework-owned files (skills, `
|
|
266
|
+
**upgrade** overwrites only framework-owned files (skills, `qa-framework.instructions.md`,
|
|
258
267
|
`QA-STRUCTURE-GUIDE.md`) and never touches project-owned files. Run `--dry-run` to preview
|
|
259
268
|
without writing.
|
|
260
269
|
|
package/docs/installation.md
CHANGED
|
@@ -51,7 +51,7 @@ qa/
|
|
|
51
51
|
└── 08-azure-integration/README.md
|
|
52
52
|
└── memory/INDEX.md
|
|
53
53
|
|
|
54
|
-
.github/
|
|
54
|
+
.github/instructions/qa-framework.instructions.md <- QA agent rules (framework-owned)
|
|
55
55
|
.github/skills/ <- 8 agent skill sets (SKILL.md + references/)
|
|
56
56
|
```
|
|
57
57
|
|
|
@@ -93,7 +93,7 @@ After `npm update @keber/qa-framework`:
|
|
|
93
93
|
npx qa-framework upgrade
|
|
94
94
|
```
|
|
95
95
|
|
|
96
|
-
This overwrites only framework-owned files (`.github/skills/`,
|
|
96
|
+
This overwrites only framework-owned files (`.github/skills/`, `.github/instructions/qa-framework.instructions.md`,
|
|
97
97
|
`QA-STRUCTURE-GUIDE.md`) and migrates the `07-automation/` structure if upgrading from
|
|
98
98
|
v1.5.x. Your specs, tests, and memory are never touched.
|
|
99
99
|
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@keber/qa-framework",
|
|
3
|
-
"version": "1.
|
|
3
|
+
"version": "1.7.1",
|
|
4
4
|
"description": "Reusable spec-driven QA framework for IDE-agent-assisted automated testing. Installable as an npm package. Provides structure, templates, agent instructions, and optional integrations for Playwright and Azure DevOps.",
|
|
5
5
|
"keywords": [
|
|
6
6
|
"qa",
|
package/scripts/init.js
CHANGED
|
@@ -253,9 +253,9 @@ if (fs.existsSync(structureGuideSrc)) {
|
|
|
253
253
|
writeIfMissing(structureGuideDest, fs.readFileSync(structureGuideSrc, 'utf8'));
|
|
254
254
|
}
|
|
255
255
|
|
|
256
|
-
// --- .github/
|
|
256
|
+
// --- .github/instructions/qa-framework.instructions.md ---
|
|
257
257
|
const githubDir = path.join(cwd, '.github');
|
|
258
|
-
const copilotInstrPath = path.join(githubDir, '
|
|
258
|
+
const copilotInstrPath = path.join(githubDir, 'instructions', 'qa-framework.instructions.md');
|
|
259
259
|
|
|
260
260
|
// Detect whether the ado-qa skill is available (installed or already copied)
|
|
261
261
|
const adoQaInstalled = fs.existsSync(path.join(cwd, 'node_modules', '@keber', 'ado-qa'))
|
|
@@ -264,17 +264,16 @@ const adoSkillReady = fs.existsSync(path.join(cwd, '.github', 'skills', 'ado-qa
|
|
|
264
264
|
const adoAvailable = adoQaInstalled || adoSkillReady;
|
|
265
265
|
|
|
266
266
|
const adoSection = adoAvailable
|
|
267
|
-
?
|
|
268
|
-
## Azure DevOps integration (ado-qa)
|
|
267
|
+
? `## Azure DevOps integration (ado-qa)
|
|
269
268
|
|
|
270
269
|
The \`ado-qa\` skill is installed. Load it in PowerShell before any ADO operation:
|
|
271
270
|
\`\`\`powershell
|
|
272
271
|
. .github/skills/ado-qa/load.ps1
|
|
273
272
|
\`\`\`
|
|
274
273
|
Refer to \`qa/00-guides/AGENT-INSTRUCTIONS-ADO-INTEGRATION.md\` for the full ADO workflow.
|
|
274
|
+
|
|
275
275
|
`
|
|
276
|
-
:
|
|
277
|
-
## Azure DevOps integration (not installed)
|
|
276
|
+
: `## Azure DevOps integration (not installed)
|
|
278
277
|
|
|
279
278
|
> ⚠️ **Agent note:** \`ado-qa\` is not installed in this project.
|
|
280
279
|
> To enable Azure DevOps integration (query Work Items, Test Plans, create bugs, etc.) run:
|
|
@@ -282,50 +281,13 @@ Refer to \`qa/00-guides/AGENT-INSTRUCTIONS-ADO-INTEGRATION.md\` for the full ADO
|
|
|
282
281
|
> npm install github:keber/ado-qa
|
|
283
282
|
> \`\`\`
|
|
284
283
|
> This will automatically copy the ADO skills to \`.github/skills/\` via postinstall.
|
|
285
|
-
`;
|
|
286
284
|
|
|
287
|
-
const copilotInstrContent = `# QA Framework Instructions
|
|
288
|
-
|
|
289
|
-
This project uses \`@keber/qa-framework\` v${config.frameworkVersion ?? '1.0.0'} for spec-driven automated testing.
|
|
290
|
-
|
|
291
|
-
## Agent behavior rules
|
|
292
|
-
|
|
293
|
-
0. **On every conversation start:** check if \`qa/AGENT-NEXT-STEPS.md\` exists. If it does, read it and complete its steps before anything else.
|
|
294
|
-
1. Before performing any QA task, load the relevant skill from \`.github/skills/\`. For project context, read \`qa/memory/INDEX.md\` if it exists, then load only the memory files relevant to the current task — do not load all memory files unconditionally. Never execute a pipeline stage unless its prerequisite is satisfied.
|
|
295
|
-
2. Always save artifacts in the correct \`qa/\` subfolder - refer to \`qa/QA-STRUCTURE-GUIDE.md\`
|
|
296
|
-
3. Never hardcode credentials - always use env vars and \`<PLACEHOLDER>\` in documentation
|
|
297
|
-
4. Follow the naming conventions in \`qa/00-standards/naming-conventions.md\`
|
|
298
|
-
5. Read previous test suites code in \`qa/07-automation/e2e/tests/\` if available before writing new automation to maintain consistency in style and approach
|
|
299
|
-
6. Project QA config is at \`qa/qa-framework.config.json\`
|
|
300
|
-
7. Save learnings in \`qa/memory/\` with proper naming and metadata. Always update \`qa/memory/INDEX.md\` after adding or updating any memory file.
|
|
301
|
-
8. **Sprint/module completion criteria:** A sprint or module is complete when every TC in scope has one of these three valid states - and no other: (a) \`test()\` passing: the app meets the spec; (b) \`test.fail()\`: the app violates the spec **and** a file \`qa/06-defects/open/DEF-SIS-NNN_*.md\` exists documenting which business rule is violated and what the app actually does; (c) \`test.skip()\`: the TC is not yet executable, annotated with \`PENDING-CODE\` and a linked issue. A sprint is **not** complete if any TC is failing without \`test.fail()\`, or has \`test.fail()\` without a defect file, or has \`test.skip()\` without a \`PENDING-CODE\` reason.
|
|
302
|
-
9. **On sprint/module completion:** (1) update the module status row in \`qa/README.md\`; (2) move the completed sprint checklist from \`AGENT-NEXT-STEPS.md\` to the \`## Sprint History\` section of \`qa/README.md\`; (3) trim \`AGENT-NEXT-STEPS.md\` so it contains only the next sprint — never let it accumulate more than one active sprint.
|
|
303
|
-
10. **Assertion polarity rule (BLOCKING):** A test must assert what the **spec** requires, not what the app currently does. If the app fails to meet the spec, use \`test.fail()\` + correct assertion to document the defect — **never invert or weaken an assertion to make a test go green.** A failing test that exposes a real defect is always more valuable than a passing test that hides one.
|
|
304
|
-
11. **Character encoding safety (BLOCKING):** In ALL generated content (markdown, scripts, test titles, ADO fields, config files), never use characters at these Unicode code points: em-dash (U+2014), en-dash (U+2013), horizontal ellipsis (U+2026), smart/curly quotes (U+201C, U+201D, U+2018, U+2019), or directional arrows (U+2190-U+21FF). Use ASCII equivalents instead: \` - \` (hyphen with surrounding spaces) for dashes, \`...\` for ellipsis, \`"\` and \`'\` for quotes, \`->\` and \`<-\` for arrows. Exception: Latin Extended characters (U+00C0-U+024F) are always permitted - this covers Spanish vowels with accents (U+00E0-U+00FA), n with tilde (U+00F1/U+00D1), and u/o with umlaut (U+00FC/U+00DC, U+00F6/U+00D6).
|
|
305
|
-
|
|
306
|
-
## Formatting rules for generated content
|
|
307
|
-
1. Generate the output in \`UTF-8\` encoding without BOM (\`UTF-8\`, no signature).
|
|
308
|
-
2. Ensure the raw output is \`UTF-8\` encoded with no \`EF BB BF\` bytes at the beginning.
|
|
309
|
-
|
|
310
|
-
## QA Pipeline
|
|
311
|
-
|
|
312
|
-
Stages must run in order. Never start a stage unless its prerequisite is met.
|
|
313
|
-
|
|
314
|
-
| Stage | Task | Skill | Prerequisite |
|
|
315
|
-
|---|---|---|---|
|
|
316
|
-
| 1 | Analyze module | \`.github/skills/qa-module-analysis/SKILL.md\` | None |
|
|
317
|
-
| 2 | Generate specifications | \`.github/skills/qa-spec-generation/SKILL.md\` | 00-inventory.md exists |
|
|
318
|
-
| 3 | Generate test plan | \`.github/skills/qa-test-plan/SKILL.md\` | 05-test-scenarios.md exists |
|
|
319
|
-
| 4 | Generate test cases | \`.github/skills/qa-test-cases/SKILL.md\` | Test plan exists |
|
|
320
|
-
| 5 | Generate automation | \`.github/skills/qa-automation/SKILL.md\` | Specs approved, no PENDING-CODE |
|
|
321
|
-
| 5b | Stabilize failing tests | \`.github/skills/qa-test-stabilization/SKILL.md\` | Failing tests exist |
|
|
322
|
-
| 6 | Maintenance | \`.github/skills/qa-maintenance/SKILL.md\` | Application change delivered |
|
|
323
|
-
| — | ADO integration | \`.github/skills/qa-ado-integration/SKILL.md\` | ADO enabled in config |
|
|
324
|
-
|
|
325
|
-
## Project KB
|
|
326
|
-
1. \`qa/README.md\` — The Living Index: module status, sprint history, blockers, quick-start commands.
|
|
327
|
-
2. \`qa/memory/INDEX.md\` — Index of all memory files; load this before selecting which files to read.
|
|
328
285
|
`;
|
|
286
|
+
|
|
287
|
+
const instrTemplatePath = path.resolve(__dirname, '..', 'templates', 'qa-framework.instructions.md');
|
|
288
|
+
const copilotInstrContent = fs.readFileSync(instrTemplatePath, 'utf8')
|
|
289
|
+
.replace('{{VERSION}}', config.frameworkVersion ?? '1.0.0')
|
|
290
|
+
.replace('{{ADO_SECTION}}', adoSection);
|
|
329
291
|
writeIfMissing(copilotInstrPath, copilotInstrContent);
|
|
330
292
|
|
|
331
293
|
// --- AGENT-NEXT-STEPS.md — readable by the agent after install ---
|
|
@@ -337,7 +299,7 @@ const nextStepsContent = `# ✅ @keber/qa-framework installed successfully
|
|
|
337
299
|
## What was scaffolded
|
|
338
300
|
|
|
339
301
|
- \`qa/\` folder structure with spec templates and agent instructions
|
|
340
|
-
- \`.github/
|
|
302
|
+
- \`.github/instructions/qa-framework.instructions.md\` with QA agent behavior rules (framework-owned, safe to upgrade)
|
|
341
303
|
|
|
342
304
|
## Required next steps
|
|
343
305
|
|
package/scripts/upgrade.js
CHANGED
|
@@ -81,12 +81,51 @@ if (!fs.existsSync(skillsSrc)) {
|
|
|
81
81
|
}
|
|
82
82
|
|
|
83
83
|
// ---------------------------------------------------------------------------
|
|
84
|
-
// 2. .github/
|
|
84
|
+
// 2. .github/instructions/qa-framework.instructions.md — overwrite (framework-owned)
|
|
85
85
|
// ---------------------------------------------------------------------------
|
|
86
|
-
const copilotInstrPath = path.join(githubDir, '
|
|
87
|
-
const
|
|
86
|
+
const copilotInstrPath = path.join(githubDir, 'instructions', 'qa-framework.instructions.md');
|
|
87
|
+
const instrTemplatePath = path.resolve(__dirname, '..', 'templates', 'qa-framework.instructions.md');
|
|
88
|
+
const copilotContent = fs.readFileSync(instrTemplatePath, 'utf8')
|
|
89
|
+
.replace('{{VERSION}}', config.frameworkVersion ?? '1.0.0')
|
|
90
|
+
.replace('{{ADO_SECTION}}', '');
|
|
88
91
|
forceWrite(copilotInstrPath, copilotContent);
|
|
89
92
|
|
|
93
|
+
// ---------------------------------------------------------------------------
|
|
94
|
+
// 2b. Migration: strip QA Framework section from old copilot-instructions.md
|
|
95
|
+
//
|
|
96
|
+
// Handles all cases:
|
|
97
|
+
// A. File has custom instructions + QA section → keep custom, strip QA section
|
|
98
|
+
// B. File has ONLY QA section (any version) → delete the file
|
|
99
|
+
// C. File does not mention QA Framework → leave untouched
|
|
100
|
+
//
|
|
101
|
+
// The QA section always starts with "# QA Framework Instructions" across all
|
|
102
|
+
// previous versions, so that heading is the reliable split point.
|
|
103
|
+
// ---------------------------------------------------------------------------
|
|
104
|
+
const oldCopilotPath = path.join(githubDir, 'copilot-instructions.md');
|
|
105
|
+
if (fs.existsSync(oldCopilotPath)) {
|
|
106
|
+
const oldContent = fs.readFileSync(oldCopilotPath, 'utf8');
|
|
107
|
+
const qaMarker = '# QA Framework Instructions';
|
|
108
|
+
const qaIdx = oldContent.indexOf(qaMarker);
|
|
109
|
+
|
|
110
|
+
if (qaIdx !== -1) {
|
|
111
|
+
// Extract anything before the QA section, removing any trailing separator
|
|
112
|
+
const before = oldContent.slice(0, qaIdx).replace(/\s*\n---\s*$/, '').trim();
|
|
113
|
+
|
|
114
|
+
if (before.length === 0) {
|
|
115
|
+
// Case B: file contained only QA Framework content → delete it
|
|
116
|
+
if (!dryRun) fs.unlinkSync(oldCopilotPath);
|
|
117
|
+
updated.push(oldCopilotPath);
|
|
118
|
+
console.log(` [deleted] .github/copilot-instructions.md (contained only QA Framework rules — now in .github/instructions/qa-framework.instructions.md)`);
|
|
119
|
+
} else {
|
|
120
|
+
// Case A: file had custom content too → write back only the custom part
|
|
121
|
+
if (!dryRun) fs.writeFileSync(oldCopilotPath, before + '\n', 'utf8');
|
|
122
|
+
updated.push(oldCopilotPath);
|
|
123
|
+
console.log(` [cleaned] .github/copilot-instructions.md — removed QA Framework section, kept custom instructions`);
|
|
124
|
+
}
|
|
125
|
+
}
|
|
126
|
+
// Case C: no QA marker found → leave untouched (no log noise)
|
|
127
|
+
}
|
|
128
|
+
|
|
90
129
|
// ---------------------------------------------------------------------------
|
|
91
130
|
// 3. qa/QA-STRUCTURE-GUIDE.md — overwrite (framework doc)
|
|
92
131
|
// ---------------------------------------------------------------------------
|
|
@@ -276,40 +315,3 @@ function copyDirForce(srcDir, destDir) {
|
|
|
276
315
|
}
|
|
277
316
|
}
|
|
278
317
|
|
|
279
|
-
function buildCopilotInstructions(cfg) {
|
|
280
|
-
const version = cfg.frameworkVersion ?? '1.0.0';
|
|
281
|
-
return `# QA Framework Instructions
|
|
282
|
-
|
|
283
|
-
This project uses \`@keber/qa-framework\` v${version} for spec-driven automated testing.
|
|
284
|
-
|
|
285
|
-
## Agent behavior rules
|
|
286
|
-
|
|
287
|
-
0. **On every conversation start:** check if \`qa/AGENT-NEXT-STEPS.md\` exists. If it does, read it and complete its steps before anything else.
|
|
288
|
-
1. Before performing any QA task, load the relevant skill from \`.github/skills/\`. For project context, read \`qa/memory/INDEX.md\` if it exists, then load only the memory files relevant to the current task — do not load all memory files unconditionally. Never execute a pipeline stage unless its prerequisite is satisfied.
|
|
289
|
-
2. Always save artifacts in the correct \`qa/\` subfolder — refer to \`qa/QA-STRUCTURE-GUIDE.md\`
|
|
290
|
-
3. Never hardcode credentials — always use env vars and \`<PLACEHOLDER>\` in documentation
|
|
291
|
-
4. Follow the naming conventions in \`qa/00-standards/naming-conventions.md\`
|
|
292
|
-
5. Project QA config is at \`qa/qa-framework.config.json\`
|
|
293
|
-
6. Save learnings in \`qa/memory/\` with proper naming and metadata. Always update \`qa/memory/INDEX.md\` after adding or updating any memory file.
|
|
294
|
-
7. **On module/sprint completion** (all TCs passing): (1) update the module status row in \`qa/README.md\`; (2) move the completed sprint checklist from \`AGENT-NEXT-STEPS.md\` to the \`## Sprint History\` section of \`qa/README.md\`; (3) trim \`AGENT-NEXT-STEPS.md\` so it contains only the next sprint — never let it accumulate more than one active sprint.
|
|
295
|
-
|
|
296
|
-
## QA Pipeline
|
|
297
|
-
|
|
298
|
-
Stages must run in order. Never start a stage unless its prerequisite is met.
|
|
299
|
-
|
|
300
|
-
| Stage | Task | Skill | Prerequisite |
|
|
301
|
-
|---|---|---|---|
|
|
302
|
-
| 1 | Analyze module | \`.github/skills/qa-module-analysis/SKILL.md\` | None |
|
|
303
|
-
| 2 | Generate specifications | \`.github/skills/qa-spec-generation/SKILL.md\` | 00-inventory.md exists |
|
|
304
|
-
| 3 | Generate test plan | \`.github/skills/qa-test-plan/SKILL.md\` | 05-test-scenarios.md exists |
|
|
305
|
-
| 4 | Generate test cases | \`.github/skills/qa-test-cases/SKILL.md\` | Test plan exists |
|
|
306
|
-
| 5 | Generate automation | \`.github/skills/qa-automation/SKILL.md\` | Specs approved, no PENDING-CODE |
|
|
307
|
-
| 5b | Stabilize failing tests | \`.github/skills/qa-test-stabilization/SKILL.md\` | Failing tests exist |
|
|
308
|
-
| 6 | Maintenance | \`.github/skills/qa-maintenance/SKILL.md\` | Application change delivered |
|
|
309
|
-
| — | ADO integration | \`.github/skills/qa-ado-integration/SKILL.md\` | ADO enabled in config |
|
|
310
|
-
|
|
311
|
-
## Project KB
|
|
312
|
-
1. \`qa/README.md\` — The Living Index: module status, sprint history, blockers, quick-start commands.
|
|
313
|
-
2. \`qa/memory/INDEX.md\` — Index of all memory files; load this before selecting which files to read.
|
|
314
|
-
`;
|
|
315
|
-
}
|
|
@@ -0,0 +1,45 @@
|
|
|
1
|
+
---
|
|
2
|
+
applyTo: '**'
|
|
3
|
+
---
|
|
4
|
+
# QA Framework Instructions
|
|
5
|
+
|
|
6
|
+
This project uses `@keber/qa-framework` v{{VERSION}} for spec-driven automated testing.
|
|
7
|
+
|
|
8
|
+
## Agent behavior rules
|
|
9
|
+
|
|
10
|
+
0. **On every conversation start:** check if `qa/AGENT-NEXT-STEPS.md` exists. If it does, read it and complete its steps before anything else.
|
|
11
|
+
1. Before performing any QA task, load the relevant skill from `.github/skills/`. For project context, read `qa/memory/INDEX.md` if it exists, then load only the memory files relevant to the current task — do not load all memory files unconditionally. Never execute a pipeline stage unless its prerequisite is satisfied.
|
|
12
|
+
2. Always save artifacts in the correct `qa/` subfolder - refer to `qa/QA-STRUCTURE-GUIDE.md`
|
|
13
|
+
3. Never hardcode credentials - always use env vars and `<PLACEHOLDER>` in documentation
|
|
14
|
+
4. Follow the naming conventions in `qa/00-standards/naming-conventions.md`
|
|
15
|
+
5. Read previous test suites code in `qa/07-automation/e2e/tests/` if available before writing new automation to maintain consistency in style and approach
|
|
16
|
+
6. Project QA config is at `qa/qa-framework.config.json`
|
|
17
|
+
7. Save learnings in `qa/memory/` with proper naming and metadata. Always update `qa/memory/INDEX.md` after adding or updating any memory file.
|
|
18
|
+
8. **Sprint/module completion criteria:** A sprint or module is complete when every TC in scope has one of these three valid states - and no other: (a) `test()` passing: the app meets the spec; (b) `test.fail()`: the app violates the spec **and** a file `qa/06-defects/open/DEF-SIS-NNN_*.md` exists documenting which business rule is violated and what the app actually does; (c) `test.skip()`: the TC is not yet executable, annotated with `PENDING-CODE` and a linked issue. A sprint is **not** complete if any TC is failing without `test.fail()`, or has `test.fail()` without a defect file, or has `test.skip()` without a `PENDING-CODE` reason.
|
|
19
|
+
9. **On sprint/module completion:** (1) update the module status row in `qa/README.md`; (2) move the completed sprint checklist from `AGENT-NEXT-STEPS.md` to the `## Sprint History` section of `qa/README.md`; (3) trim `AGENT-NEXT-STEPS.md` so it contains only the next sprint — never let it accumulate more than one active sprint.
|
|
20
|
+
10. **Assertion polarity rule (BLOCKING):** A test must assert what the **spec** requires, not what the app currently does. If the app fails to meet the spec, use `test.fail()` + correct assertion to document the defect — **never invert or weaken an assertion to make a test go green.** A failing test that exposes a real defect is always more valuable than a passing test that hides one.
|
|
21
|
+
11. **Character encoding safety (BLOCKING):** In ALL generated content (markdown, scripts, test titles, ADO fields, config files), never use characters at these Unicode code points: em-dash (U+2014), en-dash (U+2013), horizontal ellipsis (U+2026), smart/curly quotes (U+201C, U+201D, U+2018, U+2019), or directional arrows (U+2190-U+21FF). Use ASCII equivalents instead: ` - ` (hyphen with surrounding spaces) for dashes, `...` for ellipsis, `"` and `'` for quotes, `->` and `<-` for arrows. Exception: Latin Extended characters (U+00C0-U+024F) are always permitted - this covers Spanish vowels with accents (U+00E0-U+00FA), n with tilde (U+00F1/U+00D1), and u/o with umlaut (U+00FC/U+00DC, U+00F6/U+00D6).
|
|
22
|
+
|
|
23
|
+
## Formatting rules for generated content
|
|
24
|
+
1. Generate the output in `UTF-8` encoding without BOM (`UTF-8`, no signature).
|
|
25
|
+
2. Ensure the raw output is `UTF-8` encoded with no `EF BB BF` bytes at the beginning.
|
|
26
|
+
|
|
27
|
+
{{ADO_SECTION}}
|
|
28
|
+
## QA Pipeline
|
|
29
|
+
|
|
30
|
+
Stages must run in order. Never start a stage unless its prerequisite is met.
|
|
31
|
+
|
|
32
|
+
| Stage | Task | Skill | Prerequisite |
|
|
33
|
+
|---|---|---|---|
|
|
34
|
+
| 1 | Analyze module | `.github/skills/qa-module-analysis/SKILL.md` | None |
|
|
35
|
+
| 2 | Generate specifications | `.github/skills/qa-spec-generation/SKILL.md` | 00-inventory.md exists |
|
|
36
|
+
| 3 | Generate test plan | `.github/skills/qa-test-plan/SKILL.md` | 05-test-scenarios.md exists |
|
|
37
|
+
| 4 | Generate test cases | `.github/skills/qa-test-cases/SKILL.md` | Test plan exists |
|
|
38
|
+
| 5 | Generate automation | `.github/skills/qa-automation/SKILL.md` | Specs approved, no PENDING-CODE |
|
|
39
|
+
| 5b | Stabilize failing tests | `.github/skills/qa-test-stabilization/SKILL.md` | Failing tests exist |
|
|
40
|
+
| 6 | Maintenance | `.github/skills/qa-maintenance/SKILL.md` | Application change delivered |
|
|
41
|
+
| — | ADO integration | `.github/skills/qa-ado-integration/SKILL.md` | ADO enabled in config |
|
|
42
|
+
|
|
43
|
+
## Project KB
|
|
44
|
+
1. `qa/README.md` — The Living Index: module status, sprint history, blockers, quick-start commands.
|
|
45
|
+
2. `qa/memory/INDEX.md` — Index of all memory files; load this before selecting which files to read.
|