@keber/qa-framework 1.0.4

package/docs/installation.md

@@ -0,0 +1,239 @@
+# docs/installation.md
+
+## Installing `qa-framework`
+
+---
+
+## Prerequisites
+
+- Node.js >= 18
+- npm >= 8
+- Git (the `qa/` directory should live inside a Git repository)
+- PowerShell >= 5.1 (only required for Azure DevOps integration scripts)
+
+---
+
+## Option A — npm install (recommended)
+
+```bash
+# Install as a dev dependency
+npm install --save-dev keber/qa-framework
+
+# Run the init command
+npx qa-framework init
+```
+
+The `init` command will prompt you for:
+
+1. **Project name** (used in config and directory labels)
+2. **QA base URL** (e.g., `https://myproject-qa.example.com`)
+3. **Login path** (e.g., `/login` or `/Seguridad/Login`)
+4. **Number of test user roles** (1 = single role, 2+ = multi-role)
+5. **Enable Playwright integration?** (y/n)
+6. **Enable Azure DevOps integration?** (y/n)
+
+After answering these, `init` creates:
+
+```
+qa/
+├── README.md                       ← Prefilled with your project name
+├── QA-STRUCTURE-GUIDE.md           ← Full structure guide
+├── qa-framework.config.json        ← Your project config
+├── 00-guides/                      ← Agent instructions (copied)
+├── 00-standards/                   ← Templates and naming conventions
+├── 01-specifications/README.md
+├── 02-test-plans/README.md
+├── 03-test-cases/README.md
+├── 04-test-data/README.md
+├── 05-test-execution/README.md
+├── 06-defects/README.md
+├── 07-automation/README.md
+└── 08-azure-integration/README.md  ← Only if ADO enabled
+```
+
+If Playwright is enabled, it also creates:
+
+```
+qa/07-automation/e2e/
+├── package.json
+├── playwright.config.ts
+├── global-setup.ts
+├── .env.example
+└── fixtures/
+    └── auth.ts
+```
+
+---
+
+## Option B — Clone or copy (early adoption / no npm registry)
+
+```bash
+# From your project root
+git clone https://github.com/your-org/qa-framework.git tools/qa-framework
+
+# Run init
+node tools/qa-framework/scripts/cli.js init
+```
+
+---
+
+## Option C — Manual scaffold (advanced)
+
+If you prefer to control exactly what gets created:
+
+```bash
+# Clone the package
+git clone https://github.com/your-org/qa-framework.git tools/qa-framework
+
+# Copy only the templates you need
+cp -r tools/qa-framework/templates/specification qa/01-specifications/shared/
+cp -r tools/qa-framework/agent-instructions qa/00-guides/
+cp -r tools/qa-framework/templates/automation-scaffold qa/07-automation/e2e/
+
+# Then manually fill in qa-framework.config.json
+```
+
+---
+
+## Post-Install Configuration
+
+Edit `qa/qa-framework.config.json` (or `qa-framework.config.json` at project root):
+
+### Required fields
+
+```json
+{
+  "project": {
+    "name": "my-project",
+    "displayName": "My Project QA",
+    "qaBaseUrl": "https://my-qa-env.example.com",
+    "loginPath": "/login"
+  }
+}
+```
+
+### Optional: Playwright
+
+```json
+{
+  "integrations": {
+    "playwright": {
+      "enabled": true,
+      "automationRoot": "qa/07-automation/e2e",
+      "workers": { "local": 2, "ci": 1 },
+      "timeout": 30000,
+      "actionTimeout": 10000
+    }
+  }
+}
+```
+
+### Optional: Azure DevOps
+
+```json
+{
+  "integrations": {
+    "azureDevOps": {
+      "enabled": true,
+      "organization": "my-ado-org",
+      "project": "my-ado-project",
+      "variableGroup": "qa-secrets"
+    }
+  }
+}
+```
+
+**Never** put `testPlanId`, `suiteId`, or `ADO_PAT` in the config file directly.  
+Use environment variables:
+
+- `ADO_PAT` — Personal Access Token
+- `ADO_PLAN_ID` — Test Plan ID (can also go in `module-registry.json`)
+- `ADO_SUITE_ID` — Suite ID
+
+---
+
+## Environment Variables
+
+Create `qa/07-automation/e2e/.env` (local, **never commit**):
+
+```bash
+# Required
+QA_BASE_URL=https://my-qa-env.example.com
+QA_USER_EMAIL=qa-user@example.com
+QA_USER_PASSWORD=
+
+# Multi-role (if using multi-role auth)
+QA_USER_ADMIN_EMAIL=qa-admin@example.com
+QA_USER_ADMIN_PASSWORD=
+
+# ADO (only if integration enabled)
+ADO_PAT=<your-personal-access-token>
+ADO_ORG=my-ado-org
+ADO_PROJECT=my-ado-project
+```
+
+Add `.env` to `.gitignore`:
+
+```bash
+echo "qa/07-automation/e2e/.env" >> .gitignore
+echo ".auth/" >> .gitignore
+```
+
+---
+
+## Running the First Test
+
+After installing and configuring:
+
+```bash
+# Enter the automation directory
+cd qa/07-automation/e2e
+
+# Install Playwright
+npm install
+npx playwright install chromium
+
+# Run all tests
+npx playwright test
+
+# Run a specific module
+npx playwright test tests/my-module/
+
+# Open the HTML report
+npx playwright show-report
+```
+
+---
+
+## Verifying Installation
+
+```bash
+npx qa-framework validate
+```
+
+This checks:
+
+- Required folders exist
+- `qa-framework.config.json` is present and valid
+- `.env.example` exists
+- `.env` is gitignored
+- Agent instruction files are in place
+- No credentials are hardcoded in any Markdown file
+
+---
+
+## Upgrading
+
+```bash
+npm update keber/qa-framework
+npx qa-framework upgrade
+```
+
+The `upgrade` command:
+
+1. Checks the current framework version in your project
+2. Shows a diff of changed template and instruction files
+3. Prompts before overwriting any file that has local modifications
+4. Never touches `01-specifications/`, `02-test-plans/`, `03-test-cases/`, `04-test-data/`, `05-test-execution/`, `06-defects/` — only framework-owned files are updated
+
+See `MIGRATION-NOTES.md` for version-specific migration instructions.

package/docs/spec-driven-philosophy.md

@@ -0,0 +1,170 @@
+# docs/spec-driven-philosophy.md
+
+## Spec-Driven Automated Testing — Core Philosophy
+
+This document explains the methodology that underpins the entire `qa-framework`.
+
+---
+
+## The Core Principle: UI is the Source of Truth
+
+> A test case is only valid if it describes behavior that is **visible**, **accessible**, and **observable** in the running QA environment.
+
+Never write test cases from:
+- Source code (a route may be implemented but hidden in the UI)
+- Requirements documents (a requirement may not yet be deployed)
+- Database schemas (business logic in the DB does not equal a user-facing feature)
+
+**Always** write test cases after observing the actual QA environment behavior.
+
+### TC Origin Classification
+
+Every test case must declare its origin:
+
+| Origin tag | Meaning |
+|---|---|
+| `UI-OBSERVED` | The behavior was observed directly in the running QA environment |
+| `PENDING-CODE` | The feature is planned but not yet deployed |
+| `BLOCKED-PERMISSIONS` | The feature exists but the QA user cannot access it |
+
+A test case with origin `PENDING-CODE` or `BLOCKED-PERMISSIONS` must be in `test.skip()` until its status changes.
+
+---
+
+## The Spec-Before-Automation Rule
+
+```
+Specification exists FIRST → Automation comes second
+```
+
+Automation artifacts (spec files, page objects) always reference a spec TC-ID. It is never acceptable to write a Playwright test without a corresponding TC in `05-test-scenarios.md`.
+
+This provides:
+- **Traceability**: every test can be traced back to a business requirement
+- **Coverage visibility**: the spec is the source of truth for what is and isn't covered
+- **Prioritization**: P0/P1/P2/P3 priorities are set in the spec, not in the test file
+
+---
+
+## The 6-File Submodule Pattern
+
+The basic unit of QA work is a **submodule**. Every submodule produces exactly 6 files:
+
+```
+submodule-{name}/
+├── 00-inventory.md         What exists (UI elements, APIs, routes)
+├── 01-business-rules.md    Rules the system enforces (RN-* IDs)
+├── 02-workflows.md         User flows (FL-* flowcharts)
+├── 03-roles-permissions.md Who can do what
+├── 04-test-data.md         What data is needed to test
+└── 05-test-scenarios.md    Test cases (TC-* IDs, priority, steps)
+```
+
+This pattern ensures every submodule is analyzed with the same depth, regardless of complexity.
+
+---
+
+## Test Pyramid Targets
+
+| Layer | Target |
+|---|---|
+| Unit tests (back-end) | ~70% of all tests |
+| Integration tests | ~20% of all tests |
+| E2E browser tests | ~10% of all tests |
+
+The framework focuses on the **E2E layer** (10%), because this is where spec-driven automation adds the most value and where agents can contribute most directly.
+
+E2E tests should cover:
+- Critical happy paths (P0)
+- Key negative scenarios (P1)
+- Cross-module integration flows (P1)
+
+E2E tests should NOT try to cover:
+- Every edge case (that's the unit test layer)
+- Every data validation (that's the integration test layer)
+- Pure UI cosmetics
+
+---
+
+## Priority Levels
+
+| Priority | Meaning | Target in automation |
+|---|---|---|
+| **P0** | Critical — system unusable without this | Must be automated |
+| **P1** | High — significant impact if broken | Should be automated |
+| **P2** | Medium — moderate impact | Automate if feasible |
+| **P3** | Low — minor impact | Manual testing acceptable |
+
+In the automation suite, P0 tests form the **smoke suite** that runs on every CI build.
+P1 tests run on scheduled runs or before each release.
+P2/P3 are candidates for manual exploratory testing.
+
+---
+
+## Defect Handling in Automation
+
+When a test fails because of a **known bug** (not a test error):
+
+1. File a defect record (`06-defects/DEF-NNN.md` or ADO Work Item)
+2. Add `test.skip(true, 'DEF-NNN: description. Reactivate when ADO #{WI_ID} is resolved.')` to the test
+3. Make the test assertion document the buggy behavior: do not make it pass incorrectly
+4. When the bug is fixed: remove the skip, verify it passes ≥2 times with different `EXEC_IDX`, then close the defect
+
+**Never** make a test "green" by removing the assertion that catches the bug. The test should accurately describe expected behavior.
+
+---
+
+## Correctness Criteria for Automation
+
+A test is considered **stable** only when:
+
+1. It passes ≥ 2 consecutive runs with **different `EXEC_IDX` values** (i.e., different time slots)
+2. It does not rely on pre-existing data that could disappear
+3. It can be run independently (not dependent on another test's side effects)
+
+The `EXEC_IDX` pattern:
+
+```typescript
+// Changes every 60 seconds, wraps at 100,000
+const EXEC_IDX = Math.floor(Date.now() / 60_000) % 100_000;
+
+// Use in test data to generate unique values
+const testDate = `${2120 + (EXEC_IDX % 10)}-01-${String((EXEC_IDX % 28) + 1).padStart(2, '0')}`;
+```
+
+This avoids:
+- Database unique constraint violations (same data in back-to-back runs)
+- Need for test teardown (different dates don't collide)
+- Test data that expires or becomes invalid
+
+---
+
+## Credential Security Rules
+
+1. **Never** put real credentials in any Markdown file
+2. **Never** put real credentials in test code directly; always read from `process.env`
+3. **Always** use `page.evaluate()` (not `page.fill()`) to input passwords, so they are excluded from Playwright traces
+4. When writing execution reports or session summaries, replace any accidentally captured credentials with `<PLACEHOLDER>`
+5. `.env` files are always gitignored; use `.env.example` as the committed template
+
+---
+
+## Agent-Assisted vs Human QA Work
+
+This framework supports both:
+
+| Task | Primary actor |
+|---|---|
+| Module analysis (discover UI, map routes) | Agent (using Playwright CLI) |
+| Writing 6-file submodule specs | Agent (using module-analysis instructions) |
+| Test case generation | Agent (guided by TC templates) |
+| Automation code | Agent (guided by automation instructions) |
+| Exploratory testing | Human |
+| Bug filing and root cause | Human (agent can assist) |
+| ADO sync | Agent (guided by ADO integration instructions) |
+| Maintenance after code change | Agent + Human review |
+
+The handoff between agent and human should always happen at a known checkpoint:
+- After the spec set is produced → human reviews before automation is written
+- After automation is written → human reviews coverage mapping
+- After a defect is filed → human decides priority and fix approach

package/docs/usage-with-agent.md

@@ -0,0 +1,203 @@
+# docs/usage-with-agent.md
+
+## Using `qa-framework` with an IDE Agent (GitHub Copilot)
+
+---
+
+## Overview
+
+This framework is designed so that an IDE agent (such as GitHub Copilot in VS Code) can:
+
+1. Understand the QA structure by reading the agent instruction files
+2. Navigate the `qa/` directory predictably
+3. Generate spec, plan, test case, and automation artifacts
+4. Maintain existing artifacts after code changes
+5. Optionally synchronize results with Azure DevOps
+
+---
+
+## Quick Reference: What to Tell the Agent
+
+```
+# Install the framework
+"Run the qa-framework init command for this project"
+
+# Read the framework instructions
+"Read the file qa/00-guides/AGENT-INSTRUCTIONS-MODULE-ANALYSIS.md"
+
+# Generate the QA structure
+"Generate the full qa/ directory structure using the framework config"
+
+# Analyze a module
+"Analyze the module at [URL/route] following qa/00-guides/AGENT-INSTRUCTIONS-MODULE-ANALYSIS.md"
+
+# Generate spec, plan, cases, and automation
+"For submodule {name}, generate all 6 spec files following the templates in qa/00-standards/"
+"Generate a test plan for module {name}"
+"Write Playwright E2E tests for TC-{ID} through TC-{ID}"
+
+# Integrate results with Azure DevOps
+"Follow qa/08-azure-integration/AGENT-ADO-INTEGRATION.md to sync results with ADO"
+
+# Maintenance
+"Module {name} was updated. Review and update the spec files following qa/00-guides/AGENT-INSTRUCTIONS-MAINTENANCE.md"
+```
+
+---
+
+## Setting Up Agent Instructions in VS Code
+
+### Option A — Reference files in the agent conversation
+
+Simply paste the relevant instruction file path into the Copilot chat:
+
+```
+Read qa/00-guides/AGENT-INSTRUCTIONS-MODULE-ANALYSIS.md and then analyze the module at {URL}
+```
+
+### Option B — Workspace instructions file (recommended)
+
+Create `.github/copilot-instructions.md` in your project root:
+
+```markdown
+# QA Framework Instructions
+
+This project uses qa-framework v1.0.0 for spec-driven automated testing.
+
+## Agent behavior rules
+
+1. Before performing any QA task, read the relevant instruction file from `qa/00-guides/`
+2. Always save artifacts in the correct `qa/` subfolder per `qa/QA-STRUCTURE-GUIDE.md`
+3. Never hardcode credentials — always use env vars and `<PLACEHOLDER>` in documentation
+4. follow the naming conventions in `qa/00-standards/naming-conventions.md`
+
+## Available instructions
+
+- Module analysis: `qa/00-guides/AGENT-INSTRUCTIONS-MODULE-ANALYSIS.md`
+- Spec generation: `qa/00-guides/AGENT-INSTRUCTIONS-SPEC-GENERATION.md`
+- Test cases: `qa/00-guides/AGENT-INSTRUCTIONS-TEST-CASES.md`
+- Automation: `qa/00-guides/AGENT-INSTRUCTIONS-AUTOMATION.md`
+- ADO integration: `qa/00-guides/AGENT-INSTRUCTIONS-ADO-INTEGRATION.md`
+```
+
+### Option C — Individual task prompts
+
+For each major QA task, use the prompt from the relevant `agent-instructions/*.md` file
+as the starting prompt. These are designed to be copied and pasted directly.
+
+---
+
+## Typical Agent Session Workflow
+
+### Session 1: Initial Module Analysis
+
+```
+1. Agent reads qa/00-guides/AGENT-INSTRUCTIONS-MODULE-ANALYSIS.md
+2. Agent navigates to QA_BASE_URL using Playwright CLI
+3. Agent explores the module: screenshots, routes, form fields, APIs
+4. Agent produces qa/01-specifications/module-{name}/submodule-{x}/ (6 files each)
+5. Agent writes SESSION-SUMMARY-{date}.md at qa/ root
+6. Human reviews specs and approves before proceeding
+```
+
+### Session 2: Test Plan + Test Cases
+
+```
+1. Agent reads approved specs from 01-specifications/
+2. Agent reads qa/00-guides/AGENT-INSTRUCTIONS-TEST-CASES.md
+3. Agent selects TCs for automation (P0/P1) and manual (P2/P3)
+4. Agent writes qa/02-test-plans/{module}-automated-test-plan.md
+5. Human reviews priorities
+```
+
+### Session 3: Automation
+
+```
+1. Agent reads approved test plan
+2. Agent reads qa/00-guides/AGENT-INSTRUCTIONS-AUTOMATION.md
+3. Agent writes Playwright spec files in qa/07-automation/e2e/tests/{module}/
+4. Agent runs tests, iterates until stable
+5. Agent writes qa/05-test-execution/automated/{date}.md
+6. Human reviews automation coverage
+```
+
+### Session 4: ADO Sync (if enabled)
+
+```
+1. Agent reads qa/08-azure-integration/AGENT-ADO-INTEGRATION.md
+2. Agent runs inject-ado-ids.ps1 to prefix test titles
+3. Agent verifies runner output appears in ADO Test Plan
+4. Agent updates qa/08-azure-integration/ado-ids-mapping-{project}.json
+```
+
+---
+
+## Agent Do's and Don'ts
+
+### DO
+
+- ✅ Read the instruction file before starting any QA task
+- ✅ Save all artifacts in the exact path specified by the structure guide
+- ✅ Use TC-ID, RN-ID, FL-ID naming consistently
+- ✅ Replace credentials with `<PLACEHOLDER>` in any markdown output
+- ✅ Write a brief session summary after each working session
+- ✅ Reference the DEF-ID or ADO WI ID in every `test.skip()` call
+- ✅ Validate test stability with ≥2 passes using different `EXEC_IDX` values
+
+### DON'T
+
+- ❌ Invent test cases without first observing the QA environment
+- ❌ Hardcode credentials, DNIs, or personal data in any file
+- ❌ Save screenshots or debug output to the project root
+- ❌ Run Playwright directly from inside `qa/07-automation/e2e/` (always from project root)
+- ❌ Modify files in `00-guides/` or `00-standards/` (those are framework-owned)
+- ❌ Create test cases for features marked `BLOCKED-PERMISSIONS` without noting the blocker
+- ❌ Remove a `test.skip()` without verifying the underlying defect is resolved
+
+---
+
+## File Naming Quick Reference
+
+```
+Module spec folder:   qa/01-specifications/module-{kebab-name}/
+Submodule folder:     qa/01-specifications/module-{x}/submodule-{kebab-name}/
+TC ID format:         TC-{MODULE}-{SUBMODULE}-{NNN}     e.g. TC-OPER-CAT-001
+RN ID format:         RN-{MODULE}-{NNN}                 e.g. RN-OPER-001
+FL ID format:         FL-{MODULE}-{NNN}                 e.g. FL-OPER-001
+Defect file:          DEF-{NNN}-{slug}.md               e.g. DEF-001-switch-default-off.md
+Execution report:     YYYY-MM-DD_HH-MM-SS_desc.md
+Test title (no ADO):  [TC-OPER-CAT-001] Title @P0
+Test title (ADO):     [22957] Title @P0
+Screenshot:           NNN-description.png in qa/07-automation/e2e/diagnosis/
+```
+
+---
+
+## Checking Framework Compliance
+
+At any time, the agent can be asked to validate the QA structure:
+
+```
+npx qa-framework validate
+```
+
+This performs:
+- Structure check (all 9 folders exist)
+- Config check (required fields populated)
+- Credential scan (no hardcoded passwords in markdown)
+- Naming check (IC IDs follow `TC-{MODULE}-{NNN}` pattern)
+- Coverage check (all spec TCs have matching test file references)
+
+---
+
+## Continuing from a Previous Session
+
+Ask the agent:
+
+```
+"Read qa/SESSION-SUMMARY-{last-date}.md and qa/README.md, then tell me where we left off 
+and what the next steps are."
+```
+
+The session summary + README combination gives the agent sufficient context to resume
+without re-reading the entire qa/ directory.

package/examples/module-example/README.md

@@ -0,0 +1,34 @@
+# Example Module: Suppliers
+
+This example shows a **complete, filled-in** application of the `keber/qa-framework`
+to a fictional **Suppliers** module. Use it as a reference when creating your own modules.
+
+## What this example demonstrates
+
+| Element | File |
+|---------|------|
+| Module inventory (UI + API) | `suppliers/00-inventory.md` |
+| Business rules | `suppliers/01-business-rules.md` |
+| Create supplier workflow | `suppliers/02-workflows.md` |
+| Role × feature matrix | `suppliers/03-roles-permissions.md` |
+| Test data shapes + EXEC_IDX | `suppliers/04-test-data.md` |
+| 10 baseline test scenarios | `suppliers/05-test-scenarios.md` |
+| Playwright E2E spec file | `suppliers/suppliers-create.spec.ts` |
+
+## How to use this example
+
+1. Read each file in sequence to understand how the 6-file pattern comes together.
+2. Copy the `suppliers/` folder as a starting point for your own submodule.
+3. Replace all fictional data with your project's real selectors, URLs, and rules.
+4. Run the spec file locally after configuring `.env`:
+   ```bash
+   npx playwright test examples/module-example/suppliers/suppliers-create.spec.ts
+   ```
+
+## Fictional context
+
+- **Application**: GarcesFruit ERP (fictional)
+- **Module**: Suppliers (Proveedores)
+- **Stack**: Blazor WebAssembly + .NET API + ADO
+- **Auth**: Email-based login (`#email-input` / `#password-input`)
+- **Roles**: Admin, Purchasing, Viewer

package/examples/module-example/suppliers/00-inventory.md

@@ -0,0 +1,56 @@
+# Suppliers — Inventory
+
+**Module**: Suppliers (SUP)
+**Submodule**: Create / Edit / List
+**Last updated**: 2025-01-15
+**Author**: QA Framework Example
+
+---
+
+## UI Elements
+
+| Element | Type | Selector | Notes |
+|---------|------|----------|-------|
+| "New Supplier" button | Button | `button:has-text("New Supplier")` | Visible to Admin, Purchasing |
+| Supplier name field | Input (text) | `#supplier-name` | Required; max 200 chars |
+| RUT/Tax ID field | Input (text) | `#supplier-rut` | Format: XXXXXXXX-X |
+| Email field | Input (email) | `#supplier-email` | Validated by regex |
+| Supplier type dropdown | Select | `#supplier-type` | Options: Nacional, Internacional |
+| Save button | Button | `button:has-text("Save")` | Disabled until all required fields filled |
+| Cancel button | Button | `button:has-text("Cancel")` | Returns to list without saving |
+| Supplier grid | Table | `.supplier-grid` | Sortable columns: Name, RUT, Type, Status |
+| Search input | Input (text) | `#supplier-search` | Filters grid client-side |
+| Delete button (row) | Button | `.row-actions button[aria-label="Delete"]` | Admin only; shows confirmation |
+| Edit button (row) | Button | `.row-actions button[aria-label="Edit"]` | Admin, Purchasing |
+| Export button | Button | `button:has-text("Export")` | Downloads .xlsx |
+| Status badge | Badge | `.supplier-status-badge` | Values: Active, Inactive, Pending |
+| Toast notification | Alert | `.toast-container .toast` | Appears on Save/Delete success/error |
+
+---
+
+## API Endpoints
+
+| Method | Path | Auth | Description |
+|--------|------|------|-------------|
+| GET | `/api/suppliers` | Required | List all suppliers (paginated) |
+| GET | `/api/suppliers/{id}` | Required | Get supplier by ID |
+| POST | `/api/suppliers` | Admin, Purchasing | Create new supplier |
+| PUT | `/api/suppliers/{id}` | Admin, Purchasing | Update supplier |
+| DELETE | `/api/suppliers/{id}` | Admin only | Soft-delete supplier |
+| GET | `/api/suppliers/export` | Required | Export XLSX |
+
+---
+
+## Key Identifiers
+
+- **Route**: `/suppliers`
+- **Modal selector**: `.supplier-modal` (appears on "New Supplier" click)
+- **Success notification**: `.toast.toast-success:has-text("Supplier saved")`
+- **Error notification**: `.toast.toast-error`
+
+---
+
+## Related Submodules
+
+- Purchase Orders (references supplier by ID)
+- Supplier Contacts (child of supplier; CRUD separate)