@keber/qa-framework 1.6.1 → 1.6.2

package/CHANGELOG.md

@@ -33,6 +33,33 @@ has been removed or parameterized. See `docs/comparison-matrix.md` and `docs/gen
 
 ---
 
+## [1.6.2] - 2026-05-05
+
+### Added
+
+- **`tests/helpers/debug/`** — diagnostic scripts directory created by `init` and `upgrade` inside
+  `07-automation/e2e/tests/`. Scripts here are excluded from test runs via `testIgnore`.
+- **`tests/seeds/`** — data seeding specs directory created by `init` and `upgrade` inside
+  `07-automation/e2e/tests/`. Excluded from test runs via `testIgnore`.
+- **`testIgnore`** in `playwright.config.ts` template — excludes `**/helpers/debug/**` and
+  `**/seeds/**` from Playwright test discovery.
+- **`upgrade` step 5e** — patches existing `playwright.config.ts` in `e2e/`: replaces
+  `testDir: '.'` with `'./tests'`, injects `testIgnore` if missing, creates `helpers/debug/`
+  and `seeds/` if absent.
+- **Dual-Track Pipeline** section in `docs/folder-structure-guide.md` and `qa/QA-STRUCTURE-GUIDE.md`
+  — documents the automation track and manual track as parallel paths sharing `01-specifications/`
+  as source of truth. Includes entry point table (3 scenarios) and track selection guidance.
+  Clarifies that `03-test-cases/` is the manual track artifact and `04-test-data/` is optional.
+
+### Fixed
+
+- `qa/QA-STRUCTURE-GUIDE.md` — removed stale reference to deleted `docs/generalization-decisions.md`
+  from the `06-defects/` section.
+- `README.md` — pipeline table updated with Track column; `03-test-cases/` and `04-test-data/`
+  annotated as manual-track / optional in the directory tree.
+
+---
+
 ## [1.6.0] - 2026-04-29
 
 ### Changed (breaking - automated by `upgrade`)
@@ -69,11 +96,10 @@ has been removed or parameterized. See `docs/comparison-matrix.md` and `docs/gen
 - `07-automation/load/` — placeholder for load and performance tests; created on `init` and `upgrade`.
 - **`scripts/upgrade.js` migration section 5** — detects old v1.5.x structure and relocates all
   affected files non-destructively (moves only when destination does not exist; warns on conflict).
-- **Stage 3.5 - Test Stabilization** (`agent-instructions/04b-test-stabilization.md`) — agent
-  instruction file for exhaustively reviewing and stabilizing generated Playwright tests before
-  ADO integration. Covers baseline collection, failure classification (9 categories), false-positive
-  validation, confidence scoring (>=90% exit threshold), `STABILIZATION-REPORT.md` schema, and
-  upstream artifact update rules.
+- **Stage 5b - Test Stabilization** (`skills/qa-test-stabilization/`) — skill for exhaustively
+  reviewing and stabilizing generated Playwright tests before ADO integration. Covers baseline
+  collection, failure classification (9 categories), false-positive validation, confidence scoring
+  (>=90% exit threshold), `STABILIZATION-REPORT.md` schema, and upstream artifact update rules.
 
 ### Planned
 

package/README.md

@@ -1,7 +1,7 @@
 # qa-framework
 
 > Reusable, installable, agent-oriented QA framework for spec-driven automated testing.
-> Current version: **v1.6.0**
+> Current version: **v1.6.2**
 
 ---
 
@@ -170,8 +170,8 @@ qa/
 ├── 01-specifications/           <- Functional specs (used by generate command)
 ├── {module}/{submodule}/        <- 6-file spec sets created per module/submodule in config
 ├── 02-test-plans/               <- Test plans (automated + manual)
-├── 03-test-cases/               <- Explicit test case documents
-├── 04-test-data/                <- Test data, fixtures, factories
+├── 03-test-cases/               <- TC-*.md step-by-step docs (manual track; optional if fully automated)
+├── 04-test-data/                <- Shared test data factories/seeders (optional — per-module data lives in 01-specs)
 ├── 05-test-execution/           <- Execution reports and results
 ├── 06-defects/open|resolved/    <- Defect tracking
 ├── 07-automation/               <- Playwright automation code and config
@@ -220,16 +220,16 @@ them via `read_file` without touching `node_modules`.
 
 Skills map to the pipeline stages enforced by `.github/copilot-instructions.md`:
 
-| Stage | Skill | Task | Prerequisite |
-|---|---|---|---|
-| 1 | `qa-module-analysis` | Analyze module, produce 6-file spec set | None |
-| 2 | `qa-spec-generation` | Generate/update 6-file spec set | `00-inventory.md` exists |
-| 3 | `qa-test-plan` | Generate test plan | `05-test-scenarios.md` exists |
-| 4 | `qa-test-cases` | Generate TC-{ID}.md files | Test plan exists |
-| 5 | `qa-automation` | Implement Playwright tests | Specs approved, no PENDING-CODE |
-| 5b | `qa-test-stabilization` | Stabilize failing tests | Failing tests exist |
-| 6 | `qa-maintenance` | Update artifacts after app changes | Delivered change |
-| - | `qa-ado-integration` | Sync with Azure DevOps Test Plans | ADO enabled in config |
+| Stage | Skill | Task | Track | Prerequisite |
+|---|---|---|---|---|
+| 1 | `qa-module-analysis` | Analyze module, produce 6-file spec set | Both | None |
+| 2 | `qa-spec-generation` | Generate/update 6-file spec set | Both | `00-inventory.md` exists |
+| 3 | `qa-test-plan` | Generate test plan | Both | `05-test-scenarios.md` exists |
+| 4 | `qa-test-cases` | Generate TC-{ID}.md files | Manual | Test plan exists |
+| 5 | `qa-automation` | Implement Playwright tests | Automation | Specs approved, no PENDING-CODE |
+| 5b | `qa-test-stabilization` | Stabilize failing tests | Automation | Failing tests exist |
+| 6 | `qa-maintenance` | Update artifacts after app changes | Both | Delivered change |
+| - | `qa-ado-integration` | Sync with Azure DevOps Test Plans | Both | ADO enabled in config |
 
 See [docs/skills-architecture.md](docs/skills-architecture.md) for the full design rationale.
 

package/docs/folder-structure-guide.md

@@ -31,6 +31,59 @@ Agent skills live in `.github/skills/` (not inside `qa/`). See the `.github/skil
 
 ---
 
+## Dual-Track Pipeline
+
+All pipeline paths share the same input: `01-specifications/`. From there, work splits into
+two tracks that can run in parallel or independently depending on team maturity.
+
+```
+01-specifications/  (source of truth — what the app does and what must be tested)
+        │
+        ├─── Automation track ──────────────────────────────────────────────────
+        │         │
+        │         ▼
+        │    02-test-plans/automated/      ← scope + priority decisions
+        │         │
+        │         ▼
+        │    07-automation/e2e/tests/      ← Playwright .spec.ts files
+        │         │
+        │         ▼
+        │    05-test-execution/automated/  ← execution reports
+        │         │
+        │         ▼
+        │    08-azure-integration/         ← ADO sync via playwright-azure-reporter
+        │
+        └─── Manual track ──────────────────────────────────────────────────────
+                  │
+                  ▼
+             02-test-plans/manual/         ← scope + manual test approach
+                  │
+                  ▼
+             03-test-cases/manual/         ← TC-*.md with steps (for human testers)
+                  │
+                  ▼
+             05-test-execution/manual/     ← manual run evidence
+                  │
+                  ▼
+             08-azure-integration/         ← ADO sync via ado-integration skill
+```
+
+**Entry points:**
+
+| Scenario | Entry point | Tracks active |
+|---|---|---|
+| New system, no prior QA | Module analysis → spec generation | Automation (primary), Manual (as needed) |
+| Sprint with ADO stories | Stories → spec generation → test plan | Both, driven by sprint scope |
+| Maintenance/continuous improvement | Changed module → maintenance skill | Automation (update existing tests) |
+
+**Track selection guidance:**
+- Use **automation track** when the goal is regression coverage and CI integration.
+- Use **manual track** when: (a) team has manual testers who need step-by-step documents; (b) auditability or traceability to external standards is required; (c) features are not yet automatable.
+- Both tracks use the same TC-IDs from `01-specifications/*/05-test-scenarios.md` — traceability is preserved regardless of which track executes a given TC.
+- `04-test-data/` is optional in both tracks. Use it when test data factories or seeders are shared across multiple modules. Per-submodule data belongs in `01-specifications/*/04-test-data.md`.
+
+---
+
 ## .github/skills/
 
 **Purpose**: 3-layer QA pipeline skills installed by the framework. Loaded by the agent on demand — only the relevant skill for the current task is loaded, not all skills at once.

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@keber/qa-framework",
-  "version": "1.6.1",
+  "version": "1.6.2",
   "description": "Reusable spec-driven QA framework for IDE-agent-assisted automated testing. Installable as an npm package. Provides structure, templates, agent instructions, and optional integrations for Playwright and Azure DevOps.",
   "keywords": [
     "qa",

package/scripts/init.js

@@ -86,6 +86,8 @@ const topLevelFolders = [
   '06-defects/open',
   '06-defects/resolved',
   '07-automation/e2e/tests',
+  '07-automation/e2e/tests/helpers/debug',
+  '07-automation/e2e/tests/seeds',
   '08-azure-integration',
   'memory',
 ];

package/scripts/upgrade.js

@@ -19,8 +19,8 @@
  *     - qa/07-automation/.env.example          -> qa/07-automation/e2e/
  *     - qa/07-automation/fixtures/             -> qa/07-automation/e2e/fixtures/
  *     - qa/07-automation/e2e/{module}/         -> qa/07-automation/e2e/tests/{module}/
- *     - Creates integration/ and load/ placeholders
- *
+ *     - Creates integration/ and load/ placeholders *     - Patches playwright.config.ts: testDir '.' -> './tests', adds testIgnore
+ *     - Creates tests/helpers/debug/ and tests/seeds/ if absent *
  *   NEVER touches (project-owned):
  *     - qa/01-specifications/       <- Your specs
  *     - qa/02-test-plans/           <- Your test plans
@@ -187,6 +187,45 @@ if (!fs.existsSync(loadReadme)) {
   console.log(`  [created] ${path.relative(cwd, loadReadme)}`);
 }
 
+// 5e. Patch playwright.config.ts in e2e/:
+const playwrightConfigPath = path.join(e2eDir, 'playwright.config.ts');
+if (fs.existsSync(playwrightConfigPath)) {
+  let cfg = fs.readFileSync(playwrightConfigPath, 'utf8');
+  let cfgChanged = false;
+
+  // Fix testDir: '.' -> './tests'
+  if (/testDir\s*:\s*['"]\.['"]/.test(cfg)) {
+    cfg = cfg.replace(/testDir\s*:\s*['"]\.['"]/g, "testDir:  './tests'");
+    cfgChanged = true;
+    console.log(`  [patched] e2e/playwright.config.ts — testDir: '.' -> './tests'`);
+  }
+
+  // Inject testIgnore after testDir line if not present
+  if (!cfg.includes('testIgnore')) {
+    cfg = cfg.replace(
+      /(testDir\s*:.*\n)/,
+      "$1  testIgnore: ['**/helpers/debug/**', '**/seeds/**'],\n"
+    );
+    cfgChanged = true;
+    console.log(`  [patched] e2e/playwright.config.ts — added testIgnore`);
+  }
+
+  if (cfgChanged) {
+    if (!dryRun) fs.writeFileSync(playwrightConfigPath, cfg, 'utf8');
+    updated.push(playwrightConfigPath);
+  }
+}
+
+// Create tests/helpers/debug and tests/seeds if missing
+for (const subdir of ['tests/helpers/debug', 'tests/seeds']) {
+  const subdirPath = path.join(e2eDir, subdir);
+  if (!fs.existsSync(subdirPath)) {
+    if (!dryRun) fs.mkdirSync(subdirPath, { recursive: true });
+    updated.push(subdirPath);
+    console.log(`  [created] ${path.relative(cwd, subdirPath)}/`);
+  }
+}
+
 // ---------------------------------------------------------------------------
 // Summary
 // ---------------------------------------------------------------------------

package/templates/automation-scaffold/playwright.config.ts

@@ -39,6 +39,8 @@ for (const key of required) {
 export default defineConfig({
   // ------ Test discovery ------
   testDir:  './tests',
+  // Exclude non-suite directories from test runs
+  testIgnore: ['**/helpers/debug/**', '**/seeds/**'],
   // Use any subdir pattern your project standardizes on, e.g.:
   // testMatch: ['**/*.spec.ts'],