@keber/qa-framework 1.5.2 → 1.6.2

package/agent-instructions/01-spec-generation.md

@@ -1,281 +0,0 @@
-# Agent Instructions: Specification Generation
-
-> ⚠️ **DEPRECATED**: This file is superseded by `.github/skills/qa-spec-generation/SKILL.md`.
-> Kept as reference during the transition period. Do not update this file.
-
-**File**: `agent-instructions/01-spec-generation.md`  
-**Purpose**: Detailed instructions for generating the 6-file submodule specification set from a completed module analysis or from existing source material (UI observation, requirements docs, developer input).
-
----
-
-## When to use
-
-- After Phase 2 (Exploration) of module analysis is complete
-- When updating specs after a code change
-- When a spec file is missing or incomplete
-
----
-
-## Inputs Required
-
-Before generating specs, gather:
-
-1. **Module exploration notes** — from the analysis session
-2. **Module code** — from `qa/00-standards/naming-conventions.md` or allocate a new one
-3. **Submodule name** and its kebab-case directory name
-4. **QA environment observations** — what you actually saw in the UI
-
----
-
-## File-by-File Generation Rules
-
-### `00-inventory.md` — UI and API Inventory
-
-**Header block** (always required):
-
-```markdown
-# MODULE: {Module Display Name} — Submodule: {Submodule Display Name}
-
-| Field | Value |
-|-------|-------|
-| Module code | {MODULE_CODE} |
-| Submodule code | {SUBMODULE_CODE} |
-| Primary URL | {{QA_BASE_URL}}/path/to/submodule |
-| Status | Active / Deprecated / In Development |
-| Last updated | YYYY-MM-DD |
-```
-
-**UI element table**:
-
-```markdown
-## UI Elements
-
-| Element | Type | Required | Notes |
-|---------|------|----------|-------|
-| {field-name} | text / select / checkbox / date | Yes/No | {validation rule} |
-```
-
-**API endpoints table**:
-
-```markdown
-## API Endpoints
-
-| Method | Path | Purpose |
-|--------|------|---------|
-| GET | /api/{endpoint} | {description} |
-| POST | /api/{endpoint} | {description} |
-```
-
-**Do NOT include**: credentials, personal data, real user IDs.
-
----
-
-### `01-business-rules.md` — Business Rules
-
-**Format for each rule**:
-
-```markdown
-### RN-{MODULE}-{NNN}: {Rule Title}
-
-- **Type**: Validation / Access Control / State Machine / Calculation / Integration
-- **Trigger**: {What user action triggers this rule}
-- **Behavior**: {What the system does when the rule is evaluated}
-- **Error message** (if validation): "{exact text as shown in UI}"
-- **Notes**: {any special circumstances}
-```
-
-**Rule identification tips**:
-- Every required field → at least 1 RN (field is required)
-- Every unique constraint → 1 RN (duplicate not allowed)
-- Every state transition → 1 RN (can only approve if status=pending)
-- Every role restriction → 1 RN (only role X can see this)
-- Every auto-calculated field → 1 RN (total = quantity × price)
-
-**Minimum**: 5 rules per submodule. **Typical**: 8–15.
-
----
-
-### `02-workflows.md` — User Workflows
-
-**Format for each workflow**:
-
-```markdown
-### FL-{MODULE}-{NNN}: {Workflow Title}
-
-**Actor**: {Role who performs this workflow}
-**Trigger**: {What initiates the workflow}
-**Precondition**: {What must be true before this starts}
-
-**Flow**:
-```
-[Start]
-    │
-    ▼
-[Step 1: action]
-    │
-    ├─ [Alternative: condition] ──► [Alternative path]
-    │
-    ▼
-[Step 2: action]
-    │
-    ▼
-[End: outcome]
-```
-
-**Postcondition**: {State of the system after the workflow completes}
-```
-
-Cover at minimum:
-- Primary happy path (end-to-end main scenario)
-- Rejection / cancellation path (if applicable)
-- Error path (validation failure, system error)
-
----
-
-### `03-roles-permissions.md` — Role Matrix
-
-**Format**:
-
-```markdown
-# Roles and Permissions — {Submodule}
-
-## Access Matrix
-
-| Feature / Action | {Role 1} | {Role 2} | {Role 3} |
-|-----------------|---------|---------|---------|
-| View list | ✅ | ✅ | ❌ |
-| Create new | ✅ | ❌ | ❌ |
-| Edit | ✅ | ❌ | ❌ |
-| Delete | ✅ | ❌ | ❌ |
-| Approve | ❌ | ✅ | ❌ |
-| Export | ✅ | ✅ | ❌ |
-
-## Test User Reference
-
-| Role | Env var (email) | Env var (password) |
-|------|----------------|-------------------|
-| {Role 1} | `QA_USER_{ROLE}_EMAIL` | `QA_USER_{ROLE}_PASSWORD` |
-
-**Notes**: {any conditional access rules, e.g., "Admin can only edit own records"}
-```
-
-**Do NOT include**: actual email addresses, passwords, or personal identifiers.
-
----
-
-### `04-test-data.md` — Test Data Requirements
-
-**Format**:
-
-```markdown
-# Test Data — {Submodule}
-
-## Prerequisites
-
-The following data must exist in the QA environment before tests run:
-
-| Item | Description | Source |
-|------|-------------|--------|
-| {entity} | {description} | QA seed / pre-existing |
-
-## Data shapes for key scenarios
-
-### Scenario: {happy path}
-- Field A: {value type, e.g., "any valid text, max 100 chars"}
-- Field B: {value type}
-- User: role `{ROLE_NAME}`, credentials from `QA_USER_{ROLE}_EMAIL` env var
-
-### Scenario: {negative — duplicate}
-- Precondition: {entity} with the same key already exists
-- Field A: {same value as existing record}
-- Expected: {system rejection behavior}
-
-## Dynamic data generation
-
-For tests that create new records, use the EXEC_IDX pattern to avoid collisions:
-
-```typescript
-const EXEC_IDX = Math.floor(Date.now() / 60_000) % 100_000;
-const uniqueTitle = `Test-${EXEC_IDX}`;
-```
-
-## Data isolation rules
-
-1. Each test must operate on its own data (no shared mutable state between tests)
-2. Tests that require pre-existing data must provision it in `beforeAll`
-3. Provisioned data does not need to be cleaned up if EXEC_IDX-based names are used
-```
-
----
-
-### `05-test-scenarios.md` — Test Cases
-
-**Header block** (always required):
-
-```markdown
-# Test Scenarios — {Module}: {Submodule}
-
-**Module code**: {MODULE_CODE}  
-**Submodule code**: {SUBMODULE_CODE}  
-**Last updated**: YYYY-MM-DD
-
-## Summary
-
-| Priority | Count | Automated | Manual |
-|----------|-------|-----------|--------|
-| P0 | {N} | {N} | {N} |
-| P1 | {N} | {N} | {N} |
-| P2 | {N} | {N} | {N} |
-| P3 | {N} | {N} | {N} |
-| **Total** | **{N}** | **{N}** | **{N}** |
-```
-
-**Format for each test case**:
-
-```markdown
-### TC-{MODULE}-{SUBMODULE}-{NNN}: {TC Title}
-
-| Field | Value |
-|-------|-------|
-| Priority | P0 / P1 / P2 / P3 |
-| Type | Functional / Negative / Regression / Security / Integration |
-| Origin | UI-OBSERVED / PENDING-CODE / BLOCKED-PERMISSIONS |
-| Automation | Yes / Partial / No |
-| Playwright | (leave blank until automation is written; then: `tests/{module}/file.spec.ts`) |
-
-**Preconditions**:
-- {precondition 1}
-
-**Steps**:
-1. {step 1}
-2. {step 2}
-3. {step 3}
-
-**Expected result**: {what should happen}
-
-**Notes**: {any special considerations, DEF references, skip conditions}
-```
-
-**Mandatory coverage categories** (must have at least 1 TC per category where applicable):
-
-- [ ] Access: unauthenticated user redirected to login
-- [ ] Access: role without permission receives error / empty page
-- [ ] Happy path: primary workflow with valid data succeeds
-- [ ] Negative: required field missing → validation error shown
-- [ ] Negative: duplicate record → system rejects with error message
-- [ ] Negative: invalid format → system rejects with format error
-- [ ] State transition: state changes correctly on action
-- [ ] Export / download: if feature exists, file is generated
-- [ ] Pagination / search: if feature exists, filtering works correctly
-
----
-
-## Common Anti-Patterns to Avoid
-
-| Anti-pattern | Why it's wrong | Correct approach |
-|---|---|---|
-| TC for a feature not in QA env | Creates invalid test expectations | Mark as `PENDING-CODE`, don't write test steps |
-| TC with a real user's DNI in steps | Security issue | Use env var reference |
-| TC with "click the X button" without knowing the element exists | Assumes UI without observation | Verify element exists first |
-| 100+ TCs per submodule by copy-pasting variations | Bloats the spec without value | Merge near-identical cases, use parameterization |
-| TC with "verify system works correctly" as expected result | Untestable | Write observable, measurable expected result |

package/agent-instructions/02-test-plan-generation.md

@@ -1,205 +0,0 @@
-# Agent Instructions: Test Plan Generation
-
-> ⚠️ **DEPRECATED**: This file is superseded by `.github/skills/qa-test-plan/SKILL.md`.
-> Kept as reference during the transition period. Do not update this file.
-
-**File**: `agent-instructions/02-test-plan-generation.md`  
-**Purpose**: Instructions for generating a test plan document for a module, after its spec set is complete.
-
----
-
-## When to use
-
-- After all submodule specs (`05-test-scenarios.md`) are written and reviewed
-- When preparing for a sprint or release
-- When deciding which TCs to automate vs test manually
-
----
-
-## Inputs Required
-
-1. `qa/01-specifications/module-{name}/` — all submodule specs
-2. Scope definition: which submodules are in this test plan
-3. Priority guidance (from the team or implied by sprint scope)
-
----
-
-## Test Plan Document Structure
-
-Create: `qa/02-test-plans/automated/{module}-test-plan.md`
-
-```markdown
-# Test Plan: {Module Display Name}
-
-| Field | Value |
-|-------|-------|
-| Version | 1.0 |
-| Date | YYYY-MM-DD |
-| Module | {module-code} |
-| Test plan type | Automated / Manual / Combined |
-| Author | {agent or human name} |
-| Status | Draft / Approved / Active |
-
----
-
-## 1. Objective
-
-{1-2 sentences: what this test plan covers and why}
-
----
-
-## 2. Scope
-
-### In scope
-- {submodule 1}: {what is being tested}
-- {submodule 2}: {what is being tested}
-
-### Out of scope
-- {what is explicitly excluded and why}
-
----
-
-## 3. Automation Feasibility Analysis
-
-| Submodule | Total TCs | Fully automatable | Partially | Not automatable | Reason for exclusions |
-|-----------|-----------|------------------|-----------|-----------------|----------------------|
-| {name} | {N} | {N} ({%}) | {N} ({%}) | {N} ({%}) | {brief reason} |
-
-**Definition**:
-- **Fully automatable**: deterministic, observable, no external dependencies
-- **Partially automatable**: some steps can be automated, others require manual verification
-- **Not automatable**: requires human judgment, external system access, or is too risky to automate in QA
-
----
-
-## 4. Priority Distribution
-
-| Priority | TCs selected | Automated | Manual |
-|----------|-------------|-----------|--------|
-| P0 (Critical — must run every build) | {N} | {N} | {N} |
-| P1 (High — must run before release) | {N} | {N} | {N} |
-| P2 (Medium — periodic regression) | {N} | {N} | {N} |
-| P3 (Low — manual only) | {N} | 0 | {N} |
-
----
-
-## 5. TC Selection and Automation Plan
-
-### P0 Suite — {suite name}
-
-| TC ID | Title | Type | Automation status |
-|-------|-------|------|------------------|
-| TC-{M}-{S}-001 | {title} | Functional | Queued |
-| TC-{M}-{S}-002 | {title} | Negative | Queued |
-
-### P1 Suite — {suite name}
-
-| TC ID | Title | Type | Automation status |
-|-------|-------|------|------------------|
-| TC-{M}-{S}-010 | {title} | Functional | Queued |
-
----
-
-## 6. Azure DevOps Test Plan (if enabled)
-
-| Item | Value |
-|------|-------|
-| ADO Plan ID | (populate after creation) |
-| ADO Suite IDs | (populate after creation) |
-| Reporter | @alex_neo/playwright-azure-reporter |
-| Sync script | qa/08-azure-integration/scripts/inject-ado-ids.ps1 |
-
----
-
-## 7. Test Environment
-
-| Item | Value |
-|------|-------|
-| QA URL | {{QA_BASE_URL}} |
-| Browser | Chromium (Playwright) |
-| Workers (local) | {N from config} |
-| Workers (CI) | {N from config} |
-| Parallel sessions supported | Yes / No |
-
----
-
-## 8. Risks and Assumptions
-
-| Risk | Likelihood | Impact | Mitigation |
-|------|-----------|--------|------------|
-| Unstable QA data | Medium | High | Use EXEC_IDX-generated data |
-| External service (email, etc.) not available | High | Medium | Use 3-layer validation strategy |
-| QA env down-time | Low | High | Retry with 2h buffer in CI |
-
----
-
-## 9. Excluded TCs
-
-| TC ID | Reason for exclusion |
-|-------|---------------------|
-| TC-{M}-{S}-NNN | `BLOCKED-PERMISSIONS`: QA user cannot access |
-| TC-{M}-{S}-NNN | `PENDING-CODE`: feature not yet deployed |
-| TC-{M}-{S}-NNN | Not automatable: requires visual inspection / external access |
-
----
-
-## 10. Execution Schedule
-
-| Suite | Trigger | Frequency |
-|-------|---------|-----------|
-| P0 smoke | Every CI build | On push to main / PR |
-| P1 regression | Scheduled | Nightly |
-| P2 full regression | Manual trigger | Before each release |
-
----
-
-## Changelog
-
-| Version | Date | Description |
-|---------|------|-------------|
-| 1.0 | YYYY-MM-DD | Initial creation |
-```
-
----
-
-## Rules for Priority Assignment
-
-Use this decision tree for each TC:
-
-```
-Is this the primary happy path of the feature?
-├── YES → P0
-
-Is a bug here a show-stopper or critical data corruption risk?
-├── YES → P0
-├── NO →
-   Is this a common negative scenario (empty required field, duplicate)?
-   ├── YES → P1
-   Is this a cross-module dependency or state machine transition?
-   ├── YES → P1
-   Is this a secondary feature (export, pagination, search)?
-   ├── YES → P2
-   Is this an edge case, cosmetic, or low-frequency scenario?
-   ├── YES → P3
-```
-
----
-
-## Automation Feasibility Decision Criteria
-
-**Mark as fully automatable** when:
-- All interactions are observable via browser automation
-- The result is deterministic (same input → same output)
-- No external systems are involved (live email server, payment gateway)
-- Data can be set up programmatically
-
-**Mark as partially automatable** when:
-- Some result requires human visual inspection
-- An external system is involved but can be mocked (email intercept, API stub)
-- Data setup requires a manual step first
-
-**Mark as not automatable** when:
-- Requires physical access or real hardware
-- Result is inherently non-deterministic (performance, visual design)
-- Test would create irreversible side effects in QA environment (live invoice, real payment)
-- The feature is blocked by permissions that cannot be granted to QA users

package/agent-instructions/03-test-case-generation.md

@@ -1,150 +0,0 @@
-# Agent Instructions: Test Case Generation
-
-> ⚠️ **DEPRECATED**: This file is superseded by `.github/skills/qa-test-cases/SKILL.md`.
-> Kept as reference during the transition period. Do not update this file.
-
-**File**: `agent-instructions/03-test-case-generation.md`  
-**Purpose**: Instructions for writing detailed, standalone test case documents from approved spec scenarios.
-
----
-
-## When to use
-
-- When a TC from `05-test-scenarios.md` needs more detail than the table row provides
-- When writing complex multi-step TCs that require dedicated documentation
-- When preparing TCs for manual testers who need step-by-step instructions
-
----
-
-## Test Case Template
-
-```markdown
-# TC-{MODULE}-{SUBMODULE}-{NNN}: {TC Title}
-
-| Field | Value |
-|-------|-------|
-| TC ID | TC-{MODULE}-{SUBMODULE}-{NNN} |
-| Module | {Module display name} |
-| Submodule | {Submodule display name} |
-| Priority | P0 / P1 / P2 / P3 |
-| Type | Functional / Negative / Regression / Security / Integration |
-| Origin | UI-OBSERVED / PENDING-CODE / BLOCKED-PERMISSIONS |
-| Observation date | YYYY-MM-DD |
-| Automatable | Yes / Partial / No |
-| Playwright file | (fill when automation is written) |
-| Status | Draft / Approved / Automated / Deprecated |
-
----
-
-## Objective
-
-{1-2 sentences: what this test case verifies and why it matters}
-
----
-
-## Preconditions
-
-1. User is logged in as **{role}**
-2. The following data exists in the QA environment:
-   - {item 1}: {description}
-   - {item 2}: {description}
-3. {any other system state requirement}
-
----
-
-## Test Data
-
-| Field | Value |
-|-------|-------|
-| {field name} | {value or generation rule} |
-| {field name} | Use `EXEC_IDX`-generated value for uniqueness |
-| User | Credentials from `QA_USER_{ROLE}_EMAIL` env var |
-
----
-
-## Steps
-
-| Step | Action | Expected result |
-|------|--------|----------------|
-| 1 | Navigate to {URL} | {page loads, specific element visible} |
-| 2 | Click {element description} | {modal opens / form appears / etc.} |
-| 3 | Fill {field} with {value} | {field accepts input} |
-| 4 | Click {submit/save/confirm button} | {success message / redirect / state change} |
-| 5 | Verify {observable outcome} | {specific text, element, state} |
-
----
-
-## Expected Results
-
-**Final state**: {what the system should look like after all steps are complete}
-
-**Verification points**:
-- [ ] {verification 1}: {element} should {state}
-- [ ] {verification 2}: {element} should contain {text}
-- [ ] {verification 3}: page/modal should transition to {expected state}
-
----
-
-## Known Issues / Related Defects
-
-| Issue | DEF-ID / ADO WI | Status | Impact on this TC |
-|-------|----------------|--------|------------------|
-| {description} | DEF-NNN | Open | TC skipped until resolved |
-
----
-
-## Evidence
-
-{Link to screenshot or video from execution, if available}
-
----
-
-## Automation Notes
-
-```typescript
-// Playwright selector for key element (discovered during implementation)
-// page.locator('{selector}')
-
-// Key assertion
-// await expect(page.locator('{selector}')).toHaveText('{expected text}');
-```
-
----
-
-## Changelog
-
-| Version | Date | Author | Description |
-|---------|------|--------|-------------|
-| 1.0 | YYYY-MM-DD | {author} | Initial creation |
-```
-
----
-
-## Rules for TC Content Quality
-
-### Write measurable expected results
-
-❌ BAD: "The system should work correctly"  
-✅ GOOD: "A success toast message 'Record saved' appears and the datatable row count increases by 1"
-
-❌ BAD: "The form validates the input"  
-✅ GOOD: "A red validation message 'This field is required' appears below the Name field"
-
-### Write observable steps
-
-❌ BAD: "Add a new record with valid data"  
-✅ GOOD: "Click the 'New' button. Fill 'Name' with 'Test-{EXEC_IDX}'. Click 'Save'."
-
-### Link to defects explicitly
-
-If a TC is skipped due to a bug:  
-✅ GOOD: "Step 4 — SKIPPED: DEF-001 (ADO #21944). The switch defaults to OFF when it should be ON. Reactivate after fix."
-
-### Priority guidelines recap
-
-| Priority | Use when |
-|---|---|
-| P0 | Core feature is unusable without this working |
-| P1 | Major functionality gap if this fails |
-| P2 | Minor functionality gap, workaround exists |
-| P3 | Edge case, cosmetic, or rare scenario |