@keber/qa-framework 1.0.4

package/agent-instructions/05-ado-integration.md

@@ -0,0 +1,244 @@
+# Agent Instructions: Azure DevOps Integration
+
+**File**: `agent-instructions/05-ado-integration.md`  
+**Purpose**: Instructions for an agent to create, configure, and maintain Azure DevOps Test Plans integration with Playwright automation results.
+
+> **This section is OPTIONAL**. Only use it when `integrations.azureDevOps.enabled = true` in `qa-framework.config.json`.
+
+---
+
+## Prerequisites
+
+1. Azure DevOps PAT stored in `ADO_PAT` environment variable  
+   Required scopes: `Work Items (Read, Write)`, `Test Management (Read, Write)`
+2. `ADO_ORG` and `ADO_PROJECT` environment variables set
+3. PowerShell 5.1+ available
+4. `@alex_neo/playwright-azure-reporter` installed in the E2E package
+5. `qa/08-azure-integration/module-registry.json` exists with module paths and plan IDs
+
+---
+
+## Step 1: Create ADO Test Plan
+
+If no Test Plan exists yet:
+
+```powershell
+# qa/08-azure-integration/scripts/create-testplan-from-mapping.ps1
+# Reads ado-ids-mapping-{project}.json and creates the plan + suites + TCs in ADO
+# Parameters: -Organization $env:ADO_ORG -Project $env:ADO_PROJECT -Token $env:ADO_PAT
+.\qa\08-azure-integration\scripts\create-testplan-from-mapping.ps1 `
+  -Organization $env:ADO_ORG `
+  -Project $env:ADO_PROJECT `
+  -Token $env:ADO_PAT `
+  -MappingFile .\qa\08-azure-integration\ado-ids-mapping-{project}.json
+```
+
+After running, the script outputs a `TestPlanId` and `SuiteIds`.  
+Record these in `qa/08-azure-integration/module-registry.json`.
+
+---
+
+## Step 2: Inject ADO IDs into Spec Files
+
+Once Test Cases are created in ADO and have Work Item IDs:
+
+```powershell
+# Dry run first to preview changes
+.\qa\08-azure-integration\scripts\inject-ado-ids.ps1 `
+  -MappingFile .\qa\08-azure-integration\ado-ids-mapping-{project}.json `
+  -TestDir .\qa\07-automation\e2e\tests `
+  -DryRun
+
+# Apply changes
+.\qa\08-azure-integration\scripts\inject-ado-ids.ps1 `
+  -MappingFile .\qa\08-azure-integration\ado-ids-mapping-{project}.json `
+  -TestDir .\qa\07-automation\e2e\tests
+```
+
+This converts test titles from:
+```
+'[TC-OPER-CAT-001] Access catalog list @P0'
+```
+to:
+```
+'[22957] Access catalog list @P0'
+```
+
+Plus adds annotation:
+```typescript
+test.info().annotations.push({ type: 'TestCase', description: '22957' });
+```
+
+---
+
+## Step 3: Configure playwright-azure-reporter
+
+In `playwright.config.ts`:
+
+```typescript
+import type { PlaywrightTestConfig } from '@playwright/test';
+
+const config: PlaywrightTestConfig = {
+  reporter: [
+    ['html'],
+    ['@alex_neo/playwright-azure-reporter', {
+      orgUrl: `https://dev.azure.com/${process.env.ADO_ORG}`,
+      token: process.env.ADO_PAT,
+      planId: Number(process.env.ADO_PLAN_ID),
+      projectName: process.env.ADO_PROJECT,
+      isDisabled: !process.env.CI,                    // only publish in CI
+      publishTestResultsMode: 'testRun',
+      testRunTitle: `QA Run - ${new Date().toISOString()}`,
+      uploadAttachments: true,
+      attachmentsType: ['screenshot', 'video', 'trace'],
+    }],
+  ],
+};
+export default config;
+```
+
+**Note**: Set `isDisabled: !process.env.CI` so the reporter only publishes from CI, not from local runs.
+
+---
+
+## Step 4: Module Registry
+
+Maintain `qa/08-azure-integration/module-registry.json`:
+
+```json
+{
+  "modules": [
+    {
+      "name": "operacion",
+      "specsPath": "qa/07-automation/e2e/tests/operacion",
+      "planId": 0,
+      "suiteId": 0,
+      "description": "Módulo Operación — 7 submodules"
+    }
+  ]
+}
+```
+
+The CI pipeline uses this file to detect which module changed (via `git diff`) and run only the relevant spec.
+
+---
+
+## Step 5: ADO IDs Mapping File
+
+Maintain `qa/08-azure-integration/ado-ids-mapping-{project}.json`:
+
+```json
+{
+  "planId": 22304,
+  "module": "operacion",
+  "specsPath": "qa/07-automation/e2e/tests/operacion",
+  "suites": [
+    {
+      "suiteId": 22956,
+      "suiteName": "Suite 1.1 - Catálogos de Trabajo",
+      "testCases": [
+        { "id": 22957, "title": "Access catalog list @P0", "tags": ["automatable", "playwright"] },
+        { "id": 22958, "title": "Create labor type @P1", "tags": ["automatable", "playwright"] }
+      ]
+    }
+  ]
+}
+```
+
+---
+
+## Step 6: Configure CI Pipeline
+
+Template in `qa/08-azure-integration/pipelines/azure-pipeline-qa.yml`:
+
+```yaml
+trigger:
+  branches:
+    include:
+      - {{CI_TRIGGER_BRANCH}}
+
+pool:
+  vmImage: 'ubuntu-latest'
+
+variables:
+  - group: qa-secrets        # Must contain: QA_USER_EMAIL, QA_USER_PASSWORD, QA_BASE_URL, ADO_PAT
+
+steps:
+  - checkout: self
+    fetchDepth: 2
+
+  - task: NodeTool@0
+    inputs:
+      versionSpec: '20.x'
+    displayName: 'Install Node.js 20'
+
+  - script: |
+      cd qa/07-automation/e2e
+      npm install
+      npx playwright install chromium --with-deps
+    displayName: 'Install Playwright'
+
+  - script: |
+      cd qa/07-automation/e2e
+      npx playwright test
+    displayName: 'Run Playwright tests'
+    env:
+      QA_BASE_URL: $(QA_BASE_URL)
+      QA_USER_EMAIL: $(QA_USER_EMAIL)
+      QA_USER_PASSWORD: $(QA_USER_PASSWORD)
+      ADO_PAT: $(ADO_PAT)
+      ADO_PLAN_ID: $(ADO_PLAN_ID)
+      CI: 'true'
+
+  - task: PublishTestResults@2
+    inputs:
+      testResultsFormat: 'JUnit'
+      testResultsFiles: '**/test-results/*.xml'
+    displayName: 'Publish JUnit results'
+    condition: always()
+
+  - task: PublishPipelineArtifact@1
+    inputs:
+      targetPath: 'qa/07-automation/e2e/playwright-report'
+      artifact: 'playwright-html-report'
+    displayName: 'Publish HTML report'
+    condition: always()
+```
+
+---
+
+## Syncing ADO TC Titles with Spec Titles
+
+If TC titles drift between ADO and the spec files:
+
+```powershell
+.\qa\08-azure-integration\scripts\sync-ado-titles.ps1 `
+  -Organization $env:ADO_ORG `
+  -Project $env:ADO_PROJECT `
+  -Token $env:ADO_PAT `
+  -MappingFile .\qa\08-azure-integration\ado-ids-mapping-{project}.json `
+  -DryRun
+```
+
+This reads ADO TC titles and compares them against the spec. Outputs a diff report. Apply with `-DryRun:$false`.
+
+---
+
+## Credential Security Rules for ADO
+
+1. `ADO_PAT` is **never** committed to any file — always comes from env var or CI variable group
+2. `ADO_PAT` is **never** logged, printed, or written to any markdown file
+3. If a PAT appears in any log or output, revoke it immediately in ADO Portal and generate a new one
+4. Variable group `qa-secrets` in Azure DevOps must be scoped to the pipeline only
+5. PAT minimum scopes: `Work Items (Read, Write)`, `Test Management (Read, Write)` — do not use full access PATs
+
+---
+
+## Troubleshooting
+
+| Symptom | Likely Cause | Fix |
+|---------|-------------|-----|
+| Reporter silently does nothing in CI | `isDisabled: true` or `CI` env not set | Verify `process.env.CI` is `'true'` in pipeline |
+| Tests report as "Not Run" in ADO | ADO WI ID not in test title or annotation | Re-run `inject-ado-ids.ps1` |
+| TestRun created but no results attached | `publishTestResultsMode` wrong | Use `'testRun'` not `'testCase'` |
+| PAT error 401 | PAT expired or wrong scopes | Generate new PAT with correct scopes |

package/agent-instructions/06-maintenance.md

@@ -0,0 +1,125 @@
+# Agent Instructions: Maintenance and Updates
+
+**File**: `agent-instructions/06-maintenance.md`  
+**Purpose**: Instructions for updating QA artifacts after application code changes.
+
+---
+
+## When to use
+
+- A developer reports a feature change or new feature in a module
+- A sprint delivers changes to an already-analyzed module
+- A defect is resolved and related `test.skip()` calls need to be reactivated
+- The QA environment is updated and existing specs need to be refreshed
+
+---
+
+## Step 1: Identify What Changed
+
+Gather from the developer or ticket:
+- Which module/submodule was changed?
+- Was it a new feature, a modification, or a bug fix?
+- Are there new routes, fields, or API endpoints?
+- Were any existing behaviors removed or changed?
+
+If the change scope is unclear, navigate to the affected module in the QA environment and compare against the existing `00-inventory.md`.
+
+---
+
+## Step 2: Spec Update Rules
+
+### A new field was added to a form
+
+1. Add the field to `00-inventory.md` under UI Elements table
+2. If the field has validation: add a business rule to `01-business-rules.md` (`RN-{MODULE}-{next-number}`)
+3. Add TCs to `05-test-scenarios.md`:
+   - TC for happy path with the new field
+   - TC for required validation (if applicable)
+   - TC for any new business rule
+
+### A field was removed from the UI
+
+1. Mark the field in `00-inventory.md` as `[REMOVED - {YYYY-MM-DD}]` — do not delete the row
+2. If automation tests reference the field selector: update or skip them
+3. If a business rule only applied to that field: mark as `[DEPRECATED - {YYYY-MM-DD}]`
+4. Leave TCs in `05-test-scenarios.md` marked as `[DEPRECATED]` for traceability
+
+### A workflow was changed
+
+1. Update `02-workflows.md` with the new flow
+2. Mark the old version as `[PREVIOUS - {YYYY-MM-DD}]`
+3. Review all TCs that test the affected steps
+4. Update expected results in `05-test-scenarios.md`
+
+### A new submodule was added
+
+Follow the full module analysis process in `00-module-analysis.md`.
+
+### A defect was fixed
+
+1. Find all `test.skip()` calls that reference the DEF-ID or ADO WI
+2. Remove the `test.skip()` wrapper
+3. Run the test at least 2 times with different EXEC_IDX values to confirm stability
+4. Update `06-defects/DEF-{NNN}.md` (or ADO WI) status to Resolved
+5. If the spec assertion was adapted to document the bug (e.g., `toBeFalsy()` for a wrong default), restore the correct assertion
+
+---
+
+## Step 3: Automation Update Rules
+
+### Selectors changed (redesign, framework update)
+
+1. Locate the POM file or spec file that contains the old selector
+2. Check `CORRECTIONS` block at top of spec file (convention for documenting selector audits)
+3. Update the selector
+4. Run the affected test 2+ times to confirm the new selector is stable
+5. Add an entry to the `CORRECTIONS` block noting the change and date
+
+### New test suite needed
+
+Follow `04-automation-generation.md` for the new suite.  
+Update `COVERAGE-MAPPING.md` with the new TCs.
+
+### Test became flaky after app update
+
+Use the 5-layer debugging methodology:
+1. Environment: is QA env stable?
+2. Timeout: did an operation become slower?
+3. Data: did a seed or precondition change?
+4. Selector: did the element structure change?
+5. System: is this a real defect?
+
+---
+
+## Step 4: Version the Spec
+
+At the bottom of each updated spec file, add to the changelog:
+
+```markdown
+## Changelog
+
+| Version | Date | Description |
+|---------|------|-------------|
+| 1.0 | YYYY-MM-DD | Initial creation |
+| 1.1 | YYYY-MM-DD | Added RN-{MODULE}-{NNN}: {rule title}. TC count: N → M |
+| 1.2 | YYYY-MM-DD | Removed field {name} from UI inventory (deprecated) |
+```
+
+---
+
+## Step 5: Update qa/README.md
+
+After every maintenance session, update the master index:
+- TC count (if changed)
+- Automation status (if changed)
+- Last updated date for the module
+
+---
+
+## Step 6: Write a Session Summary
+
+Create `qa/SESSION-SUMMARY-{YYYY-MM-DD}-maintenance.md` documenting:
+- What changed
+- Which files were updated
+- Any new TCs or deprecated TCs
+- Any tests that were reactivated or newly skipped

package/docs/architecture.md

@@ -0,0 +1,227 @@
+# docs/architecture.md
+
+## Framework Architecture
+
+**Version**: 1.0.0  
+**Date**: 2026-03-04
+
+---
+
+## Design Goals
+
+1. **Decoupled core** — the framework works without Playwright, without Azure DevOps, without any specific CI/CD system
+2. **Layered optionality** — features are added as explicit opt-in integrations, not baked into the core
+3. **Agent-first design** — every convention exists so that an IDE agent can navigate and produce artifacts predictably
+4. **Spec-before-automation** — the specification layer is always the source of truth; automation references specs, never the reverse
+5. **Parameterization over hardcoding** — project-specific values live in `qa-framework.config.json`, not in framework files
+
+---
+
+## Component Map
+
+```
+┌─────────────────────────────────────────────────────────────────────────────┐
+│                          qa-framework (npm package)                         │
+│                                                                             │
+│  ┌──────────────────────────────────────────────────────────────────────┐   │
+│  │                        FRAMEWORK CORE                               │   │
+│  │                                                                      │   │
+│  │  ┌────────────────┐  ┌──────────────────┐  ┌─────────────────────┐  │   │
+│  │  │  qa/ structure │  │  Agent           │  │  Standards &        │  │   │
+│  │  │  (9 folders)   │  │  instructions    │  │  Naming conventions │  │   │
+│  │  │                │  │  (7 files)       │  │                     │  │   │
+│  │  └────────────────┘  └──────────────────┘  └─────────────────────┘  │   │
+│  │                                                                      │   │
+│  │  ┌────────────────┐  ┌──────────────────┐  ┌─────────────────────┐  │   │
+│  │  │  6-file        │  │  CLI commands    │  │  Config schema      │  │   │
+│  │  │  submodule     │  │  (init/generate/ │  │  (qa-framework.     │  │   │
+│  │  │  templates     │  │   validate)      │  │   config.json)      │  │   │
+│  │  └────────────────┘  └──────────────────┘  └─────────────────────┘  │   │
+│  └──────────────────────────────────────────────────────────────────────┘   │
+│                                                                             │
+│  ┌──────────────────────────────────────────────────────────────────────┐   │
+│  │                      OPTIONAL INTEGRATIONS                          │   │
+│  │                                                                      │   │
+│  │  ┌─────────────────────────┐    ┌──────────────────────────────┐    │   │
+│  │  │  playwright/            │    │  playwright-azure-reporter/  │    │   │
+│  │  │  - playwright.config.ts │    │  - reporter config template  │    │   │
+│  │  │  - global-setup.ts      │    │  - ado-ids-mapping.json      │    │   │
+│  │  │  - fixtures/auth.ts     │    │  - inject-ado-ids.ps1        │    │   │
+│  │  │  - .env.example         │    │  - module-registry.json      │    │   │
+│  │  └─────────────────────────┘    └──────────────────────────────┘    │   │
+│  │                                                                      │   │
+│  │  ┌─────────────────────────┐    ┌──────────────────────────────┐    │   │
+│  │  │  ado-powershell/        │    │  blazor-radzen/ (stub)       │    │   │
+│  │  │  - create-testplan.ps1  │    │  - openDropdownAndSelect()   │    │   │
+│  │  │  - sync-ado-titles.ps1  │    │  - waitForBlazorRender()     │    │   │
+│  │  │  - azure-pipeline.yml   │    │  - toBeAttached() guidance   │    │   │
+│  │  └─────────────────────────┘    └──────────────────────────────┘    │   │
+│  └──────────────────────────────────────────────────────────────────────┘   │
+└─────────────────────────────────────────────────────────────────────────────┘
+
+                         installs into project as:
+
+┌─────────────────────────────────────────────────────────────────────────────┐
+│                         TARGET PROJECT                                      │
+│                                                                             │
+│  qa/                                                                        │
+│  ├── README.md                   ← living index (project-specific)          │
+│  ├── QA-STRUCTURE-GUIDE.md       ← copy from framework                      │
+│  ├── qa-framework.config.json    ← project config (parameterized)           │
+│  │                                                                          │
+│  ├── 00-guides/                  ← agent instructions (copied from pkg)     │
+│  ├── 00-standards/               ← naming + templates (copied from pkg)     │
+│  ├── 01-specifications/          ← generated per module (project work)      │
+│  ├── 02-test-plans/              ← generated (project work)                 │
+│  ├── 03-test-cases/              ← generated (project work)                 │
+│  ├── 04-test-data/               ← project data (project work)              │
+│  ├── 05-test-execution/          ← execution reports (project work)         │
+│  ├── 06-defects/                 ← optional defect cache                    │
+│  ├── 07-automation/              ← Playwright code (project work)           │
+│  └── 08-azure-integration/       ← optional ADO integration                 │
+└─────────────────────────────────────────────────────────────────────────────┘
+```
+
+---
+
+## Data Flow: Spec-Driven Automation
+
+```
+Module analysis (agent)
+        │
+        ▼
+01-specifications/module-X/submodule-Y/
+  ├── 00-inventory.md       ← what exists in the UI
+  ├── 01-business-rules.md  ← RN-* identifiers
+  ├── 02-workflows.md       ← FL-* flowcharts
+  ├── 03-roles-permissions.md
+  ├── 04-test-data.md
+  └── 05-test-scenarios.md  ← TC-* identifiers  ──────────────────────┐
+                                                                       │
+        │                                                              │
+        ▼                                                              ▼
+02-test-plans/{module}.md               03-test-cases/{TC-ID}.md
+(selects TCs, assigns priority)         (detailed per-test docs)
+        │                                              │
+        └─────────────────────┬────────────────────────┘
+                              │
+                              ▼
+                  07-automation/e2e/tests/{module}/
+                    *.spec.ts  ←── test title contains TC-ID
+                                                   │
+                              ┌────────────────────┘
+                              │
+                              ▼
+               [ADO enabled?]
+                  YES → playwright-azure-reporter syncs results to ADO Test Plan
+                  NO  → 05-test-execution/automated/{date}.md   (local report)
+                              │
+                              ▼
+                    06-defects/ (if test.skip for known bug)
+                      DEF-*.md or ADO WI reference
+```
+
+---
+
+## Layer Definitions
+
+### Layer 1 — Framework Core (mandatory)
+
+Installed always. Contains:
+
+- `qa/` directory skeleton (9 folders)
+- `00-guides/` — agent instructions
+- `00-standards/` — naming conventions, templates
+- `QA-STRUCTURE-GUIDE.md`
+- `qa-framework.config.json` schema
+
+### Layer 2 — Playwright Integration (opt-in)
+
+Installed when `integrations.playwright.enabled = true`:
+
+- `qa/07-automation/e2e/playwright.config.ts`
+- `qa/07-automation/e2e/global-setup.ts`
+- `qa/07-automation/e2e/.env.example`
+- `qa/07-automation/e2e/fixtures/auth.ts`
+- `qa/07-automation/e2e/package.json`
+
+### Layer 3 — Azure DevOps Integration (opt-in)
+
+Installed when `integrations.azureDevOps.enabled = true`:
+
+- `qa/08-azure-integration/` structure
+- `module-registry.json` template
+- `ado-ids-mapping.json` template
+- PowerShell scripts (inject, create-plan, sync)
+- Azure Pipeline YAML template
+
+### Layer 4 — Framework Adapters (opt-in per adapter)
+
+Technology-specific extensions. Currently available:
+
+- `blazor-radzen` — Playwright helpers for Radzen Blazor apps (STUB - v1.1 planned)
+- `aspnet-mvc` — Select2/jQuery datatable helpers (STUB - v1.1 planned)
+
+---
+
+## Configuration Architecture
+
+```
+qa-framework.config.json  (project-level, committed to repo)
+        │
+        ├── project.*           → Display values, URLs (non-secret)
+        ├── modules[]           → Module codes, paths, ADO IDs
+        ├── conventions.*       → Naming patterns, TC ID format
+        ├── testUsers[]         → Role→envVar mapping (NOT credentials)
+        └── integrations.*      → Feature flags + integration config
+                │
+                └── credentials come from:
+                      .env  (local, gitignored)
+                      CI variable group (qa-secrets)  [ADO only]
+```
+
+---
+
+## Versioning Strategy
+
+The framework uses **Semantic Versioning**:
+
+| Change type | Version bump |
+|---|---|
+| New folder added to `qa/` tree | MAJOR (breaks existing structure) |
+| New template file | MINOR |
+| New integration adapter | MINOR |
+| Agent instruction update | PATCH |
+| Documentation correction | PATCH |
+
+**Upgrade path**: When a new MAJOR version changes the `qa/` structure, the `MIGRATION-NOTES.md` in the package provides a step-by-step migration script. Projects should pin their framework version in `package.json` and upgrade deliberately.
+
+---
+
+## Naming Convention Rules (summary)
+
+Full rules: [docs/folder-structure-guide.md](folder-structure-guide.md)
+
+| Artifact | Pattern | Example |
+|---|---|---|
+| Folders | kebab-case | `module-operacion/` |
+| Numbered QA folders | `NN-name/` | `01-specifications/` |
+| Numbered spec files | `NN-name.md` | `00-inventory.md` |
+| Test case IDs | `TC-{MODULE}-{SUBMODULE}-{NUM}` | `TC-OPER-CAT-001` |
+| Business rule IDs | `RN-{MODULE}-{NUM}` | `RN-OPER-001` |
+| Workflow IDs | `FL-{MODULE}-{NUM}` | `FL-OPER-001` |
+| Defect IDs | `DEF-{NUM}` | `DEF-001` |
+| Execution folders | `YYYY-MM-DD_HH-MM-SS_desc` | `2026-03-04_14-00-00_sprint40-p0` |
+| Test title (no ADO) | `[TC-ID] Title @Pp` | `[TC-OPER-CAT-001] Access catalog @P0` |
+| Test title (with ADO) | `[ADO_WI_ID] Title @Pp` | `[22957] Access catalog @P0` |
+| Screenshot | `NN-description.png` in `diagnosis/` | `01-login-failure.png` |
+
+---
+
+## Assumptions
+
+1. The primary test runner is Playwright. Other runners (Jest, Cypress) are not excluded but are not provided adapters in v1.0.
+2. The target application runs in a browser. Back-end API-only testing is not the primary use case of this framework (though API testing can be added to `07-automation/` as needed).
+3. The IDE agent is GitHub Copilot or equivalent. The instructions are written in Markdown and are IDE-agnostic.
+4. The project uses Git. The `qa/` directory lives inside the same repository as the application code (monorepo-friendly).
+5. Credentials are always managed via environment variables. There is no fallback to hardcoded credentials in any framework file.