@keber/qa-framework 1.0.4

package/agent-instructions/01-spec-generation.md

@@ -0,0 +1,278 @@
+# Agent Instructions: Specification Generation
+
+**File**: `agent-instructions/01-spec-generation.md`  
+**Purpose**: Detailed instructions for generating the 6-file submodule specification set from a completed module analysis or from existing source material (UI observation, requirements docs, developer input).
+
+---
+
+## When to use
+
+- After Phase 2 (Exploration) of module analysis is complete
+- When updating specs after a code change
+- When a spec file is missing or incomplete
+
+---
+
+## Inputs Required
+
+Before generating specs, gather:
+
+1. **Module exploration notes** — from the analysis session
+2. **Module code** — from `qa/00-standards/naming-conventions.md` or allocate a new one
+3. **Submodule name** and its kebab-case directory name
+4. **QA environment observations** — what you actually saw in the UI
+
+---
+
+## File-by-File Generation Rules
+
+### `00-inventory.md` — UI and API Inventory
+
+**Header block** (always required):
+
+```markdown
+# MODULE: {Module Display Name} — Submodule: {Submodule Display Name}
+
+| Field | Value |
+|-------|-------|
+| Module code | {MODULE_CODE} |
+| Submodule code | {SUBMODULE_CODE} |
+| Primary URL | {{QA_BASE_URL}}/path/to/submodule |
+| Status | Active / Deprecated / In Development |
+| Last updated | YYYY-MM-DD |
+```
+
+**UI element table**:
+
+```markdown
+## UI Elements
+
+| Element | Type | Required | Notes |
+|---------|------|----------|-------|
+| {field-name} | text / select / checkbox / date | Yes/No | {validation rule} |
+```
+
+**API endpoints table**:
+
+```markdown
+## API Endpoints
+
+| Method | Path | Purpose |
+|--------|------|---------|
+| GET | /api/{endpoint} | {description} |
+| POST | /api/{endpoint} | {description} |
+```
+
+**Do NOT include**: credentials, personal data, real user IDs.
+
+---
+
+### `01-business-rules.md` — Business Rules
+
+**Format for each rule**:
+
+```markdown
+### RN-{MODULE}-{NNN}: {Rule Title}
+
+- **Type**: Validation / Access Control / State Machine / Calculation / Integration
+- **Trigger**: {What user action triggers this rule}
+- **Behavior**: {What the system does when the rule is evaluated}
+- **Error message** (if validation): "{exact text as shown in UI}"
+- **Notes**: {any special circumstances}
+```
+
+**Rule identification tips**:
+- Every required field → at least 1 RN (field is required)
+- Every unique constraint → 1 RN (duplicate not allowed)
+- Every state transition → 1 RN (can only approve if status=pending)
+- Every role restriction → 1 RN (only role X can see this)
+- Every auto-calculated field → 1 RN (total = quantity × price)
+
+**Minimum**: 5 rules per submodule. **Typical**: 8–15.
+
+---
+
+### `02-workflows.md` — User Workflows
+
+**Format for each workflow**:
+
+```markdown
+### FL-{MODULE}-{NNN}: {Workflow Title}
+
+**Actor**: {Role who performs this workflow}
+**Trigger**: {What initiates the workflow}
+**Precondition**: {What must be true before this starts}
+
+**Flow**:
+```
+[Start]
+    │
+    ▼
+[Step 1: action]
+    │
+    ├─ [Alternative: condition] ──► [Alternative path]
+    │
+    ▼
+[Step 2: action]
+    │
+    ▼
+[End: outcome]
+```
+
+**Postcondition**: {State of the system after the workflow completes}
+```
+
+Cover at minimum:
+- Primary happy path (end-to-end main scenario)
+- Rejection / cancellation path (if applicable)
+- Error path (validation failure, system error)
+
+---
+
+### `03-roles-permissions.md` — Role Matrix
+
+**Format**:
+
+```markdown
+# Roles and Permissions — {Submodule}
+
+## Access Matrix
+
+| Feature / Action | {Role 1} | {Role 2} | {Role 3} |
+|-----------------|---------|---------|---------|
+| View list | ✅ | ✅ | ❌ |
+| Create new | ✅ | ❌ | ❌ |
+| Edit | ✅ | ❌ | ❌ |
+| Delete | ✅ | ❌ | ❌ |
+| Approve | ❌ | ✅ | ❌ |
+| Export | ✅ | ✅ | ❌ |
+
+## Test User Reference
+
+| Role | Env var (email) | Env var (password) |
+|------|----------------|-------------------|
+| {Role 1} | `QA_USER_{ROLE}_EMAIL` | `QA_USER_{ROLE}_PASSWORD` |
+
+**Notes**: {any conditional access rules, e.g., "Admin can only edit own records"}
+```
+
+**Do NOT include**: actual email addresses, passwords, or personal identifiers.
+
+---
+
+### `04-test-data.md` — Test Data Requirements
+
+**Format**:
+
+```markdown
+# Test Data — {Submodule}
+
+## Prerequisites
+
+The following data must exist in the QA environment before tests run:
+
+| Item | Description | Source |
+|------|-------------|--------|
+| {entity} | {description} | QA seed / pre-existing |
+
+## Data shapes for key scenarios
+
+### Scenario: {happy path}
+- Field A: {value type, e.g., "any valid text, max 100 chars"}
+- Field B: {value type}
+- User: role `{ROLE_NAME}`, credentials from `QA_USER_{ROLE}_EMAIL` env var
+
+### Scenario: {negative — duplicate}
+- Precondition: {entity} with the same key already exists
+- Field A: {same value as existing record}
+- Expected: {system rejection behavior}
+
+## Dynamic data generation
+
+For tests that create new records, use the EXEC_IDX pattern to avoid collisions:
+
+```typescript
+const EXEC_IDX = Math.floor(Date.now() / 60_000) % 100_000;
+const uniqueTitle = `Test-${EXEC_IDX}`;
+```
+
+## Data isolation rules
+
+1. Each test must operate on its own data (no shared mutable state between tests)
+2. Tests that require pre-existing data must provision it in `beforeAll`
+3. Provisioned data does not need to be cleaned up if EXEC_IDX-based names are used
+```
+
+---
+
+### `05-test-scenarios.md` — Test Cases
+
+**Header block** (always required):
+
+```markdown
+# Test Scenarios — {Module}: {Submodule}
+
+**Module code**: {MODULE_CODE}  
+**Submodule code**: {SUBMODULE_CODE}  
+**Last updated**: YYYY-MM-DD
+
+## Summary
+
+| Priority | Count | Automated | Manual |
+|----------|-------|-----------|--------|
+| P0 | {N} | {N} | {N} |
+| P1 | {N} | {N} | {N} |
+| P2 | {N} | {N} | {N} |
+| P3 | {N} | {N} | {N} |
+| **Total** | **{N}** | **{N}** | **{N}** |
+```
+
+**Format for each test case**:
+
+```markdown
+### TC-{MODULE}-{SUBMODULE}-{NNN}: {TC Title}
+
+| Field | Value |
+|-------|-------|
+| Priority | P0 / P1 / P2 / P3 |
+| Type | Functional / Negative / Regression / Security / Integration |
+| Origin | UI-OBSERVED / PENDING-CODE / BLOCKED-PERMISSIONS |
+| Automation | Yes / Partial / No |
+| Playwright | (leave blank until automation is written; then: `tests/{module}/file.spec.ts`) |
+
+**Preconditions**:
+- {precondition 1}
+
+**Steps**:
+1. {step 1}
+2. {step 2}
+3. {step 3}
+
+**Expected result**: {what should happen}
+
+**Notes**: {any special considerations, DEF references, skip conditions}
+```
+
+**Mandatory coverage categories** (must have at least 1 TC per category where applicable):
+
+- [ ] Access: unauthenticated user redirected to login
+- [ ] Access: role without permission receives error / empty page
+- [ ] Happy path: primary workflow with valid data succeeds
+- [ ] Negative: required field missing → validation error shown
+- [ ] Negative: duplicate record → system rejects with error message
+- [ ] Negative: invalid format → system rejects with format error
+- [ ] State transition: state changes correctly on action
+- [ ] Export / download: if feature exists, file is generated
+- [ ] Pagination / search: if feature exists, filtering works correctly
+
+---
+
+## Common Anti-Patterns to Avoid
+
+| Anti-pattern | Why it's wrong | Correct approach |
+|---|---|---|
+| TC for a feature not in QA env | Creates invalid test expectations | Mark as `PENDING-CODE`, don't write test steps |
+| TC with a real user's DNI in steps | Security issue | Use env var reference |
+| TC with "click the X button" without knowing the element exists | Assumes UI without observation | Verify element exists first |
+| 100+ TCs per submodule by copy-pasting variations | Bloats the spec without value | Merge near-identical cases, use parameterization |
+| TC with "verify system works correctly" as expected result | Untestable | Write observable, measurable expected result |

package/agent-instructions/02-test-plan-generation.md

@@ -0,0 +1,202 @@
+# Agent Instructions: Test Plan Generation
+
+**File**: `agent-instructions/02-test-plan-generation.md`  
+**Purpose**: Instructions for generating a test plan document for a module, after its spec set is complete.
+
+---
+
+## When to use
+
+- After all submodule specs (`05-test-scenarios.md`) are written and reviewed
+- When preparing for a sprint or release
+- When deciding which TCs to automate vs test manually
+
+---
+
+## Inputs Required
+
+1. `qa/01-specifications/module-{name}/` — all submodule specs
+2. Scope definition: which submodules are in this test plan
+3. Priority guidance (from the team or implied by sprint scope)
+
+---
+
+## Test Plan Document Structure
+
+Create: `qa/02-test-plans/automated/{module}-test-plan.md`
+
+```markdown
+# Test Plan: {Module Display Name}
+
+| Field | Value |
+|-------|-------|
+| Version | 1.0 |
+| Date | YYYY-MM-DD |
+| Module | {module-code} |
+| Test plan type | Automated / Manual / Combined |
+| Author | {agent or human name} |
+| Status | Draft / Approved / Active |
+
+---
+
+## 1. Objective
+
+{1-2 sentences: what this test plan covers and why}
+
+---
+
+## 2. Scope
+
+### In scope
+- {submodule 1}: {what is being tested}
+- {submodule 2}: {what is being tested}
+
+### Out of scope
+- {what is explicitly excluded and why}
+
+---
+
+## 3. Automation Feasibility Analysis
+
+| Submodule | Total TCs | Fully automatable | Partially | Not automatable | Reason for exclusions |
+|-----------|-----------|------------------|-----------|-----------------|----------------------|
+| {name} | {N} | {N} ({%}) | {N} ({%}) | {N} ({%}) | {brief reason} |
+
+**Definition**:
+- **Fully automatable**: deterministic, observable, no external dependencies
+- **Partially automatable**: some steps can be automated, others require manual verification
+- **Not automatable**: requires human judgment, external system access, or is too risky to automate in QA
+
+---
+
+## 4. Priority Distribution
+
+| Priority | TCs selected | Automated | Manual |
+|----------|-------------|-----------|--------|
+| P0 (Critical — must run every build) | {N} | {N} | {N} |
+| P1 (High — must run before release) | {N} | {N} | {N} |
+| P2 (Medium — periodic regression) | {N} | {N} | {N} |
+| P3 (Low — manual only) | {N} | 0 | {N} |
+
+---
+
+## 5. TC Selection and Automation Plan
+
+### P0 Suite — {suite name}
+
+| TC ID | Title | Type | Automation status |
+|-------|-------|------|------------------|
+| TC-{M}-{S}-001 | {title} | Functional | Queued |
+| TC-{M}-{S}-002 | {title} | Negative | Queued |
+
+### P1 Suite — {suite name}
+
+| TC ID | Title | Type | Automation status |
+|-------|-------|------|------------------|
+| TC-{M}-{S}-010 | {title} | Functional | Queued |
+
+---
+
+## 6. Azure DevOps Test Plan (if enabled)
+
+| Item | Value |
+|------|-------|
+| ADO Plan ID | (populate after creation) |
+| ADO Suite IDs | (populate after creation) |
+| Reporter | @alex_neo/playwright-azure-reporter |
+| Sync script | qa/08-azure-integration/scripts/inject-ado-ids.ps1 |
+
+---
+
+## 7. Test Environment
+
+| Item | Value |
+|------|-------|
+| QA URL | {{QA_BASE_URL}} |
+| Browser | Chromium (Playwright) |
+| Workers (local) | {N from config} |
+| Workers (CI) | {N from config} |
+| Parallel sessions supported | Yes / No |
+
+---
+
+## 8. Risks and Assumptions
+
+| Risk | Likelihood | Impact | Mitigation |
+|------|-----------|--------|------------|
+| Unstable QA data | Medium | High | Use EXEC_IDX-generated data |
+| External service (email, etc.) not available | High | Medium | Use 3-layer validation strategy |
+| QA env down-time | Low | High | Retry with 2h buffer in CI |
+
+---
+
+## 9. Excluded TCs
+
+| TC ID | Reason for exclusion |
+|-------|---------------------|
+| TC-{M}-{S}-NNN | `BLOCKED-PERMISSIONS`: QA user cannot access |
+| TC-{M}-{S}-NNN | `PENDING-CODE`: feature not yet deployed |
+| TC-{M}-{S}-NNN | Not automatable: requires visual inspection / external access |
+
+---
+
+## 10. Execution Schedule
+
+| Suite | Trigger | Frequency |
+|-------|---------|-----------|
+| P0 smoke | Every CI build | On push to main / PR |
+| P1 regression | Scheduled | Nightly |
+| P2 full regression | Manual trigger | Before each release |
+
+---
+
+## Changelog
+
+| Version | Date | Description |
+|---------|------|-------------|
+| 1.0 | YYYY-MM-DD | Initial creation |
+```
+
+---
+
+## Rules for Priority Assignment
+
+Use this decision tree for each TC:
+
+```
+Is this the primary happy path of the feature?
+├── YES → P0
+
+Is a bug here a show-stopper or critical data corruption risk?
+├── YES → P0
+├── NO →
+   Is this a common negative scenario (empty required field, duplicate)?
+   ├── YES → P1
+   Is this a cross-module dependency or state machine transition?
+   ├── YES → P1
+   Is this a secondary feature (export, pagination, search)?
+   ├── YES → P2
+   Is this an edge case, cosmetic, or low-frequency scenario?
+   ├── YES → P3
+```
+
+---
+
+## Automation Feasibility Decision Criteria
+
+**Mark as fully automatable** when:
+- All interactions are observable via browser automation
+- The result is deterministic (same input → same output)
+- No external systems are involved (live email server, payment gateway)
+- Data can be set up programmatically
+
+**Mark as partially automatable** when:
+- Some result requires human visual inspection
+- An external system is involved but can be mocked (email intercept, API stub)
+- Data setup requires a manual step first
+
+**Mark as not automatable** when:
+- Requires physical access or real hardware
+- Result is inherently non-deterministic (performance, visual design)
+- Test would create irreversible side effects in QA environment (live invoice, real payment)
+- The feature is blocked by permissions that cannot be granted to QA users

package/agent-instructions/03-test-case-generation.md

@@ -0,0 +1,147 @@
+# Agent Instructions: Test Case Generation
+
+**File**: `agent-instructions/03-test-case-generation.md`  
+**Purpose**: Instructions for writing detailed, standalone test case documents from approved spec scenarios.
+
+---
+
+## When to use
+
+- When a TC from `05-test-scenarios.md` needs more detail than the table row provides
+- When writing complex multi-step TCs that require dedicated documentation
+- When preparing TCs for manual testers who need step-by-step instructions
+
+---
+
+## Test Case Template
+
+```markdown
+# TC-{MODULE}-{SUBMODULE}-{NNN}: {TC Title}
+
+| Field | Value |
+|-------|-------|
+| TC ID | TC-{MODULE}-{SUBMODULE}-{NNN} |
+| Module | {Module display name} |
+| Submodule | {Submodule display name} |
+| Priority | P0 / P1 / P2 / P3 |
+| Type | Functional / Negative / Regression / Security / Integration |
+| Origin | UI-OBSERVED / PENDING-CODE / BLOCKED-PERMISSIONS |
+| Observation date | YYYY-MM-DD |
+| Automatable | Yes / Partial / No |
+| Playwright file | (fill when automation is written) |
+| Status | Draft / Approved / Automated / Deprecated |
+
+---
+
+## Objective
+
+{1-2 sentences: what this test case verifies and why it matters}
+
+---
+
+## Preconditions
+
+1. User is logged in as **{role}**
+2. The following data exists in the QA environment:
+   - {item 1}: {description}
+   - {item 2}: {description}
+3. {any other system state requirement}
+
+---
+
+## Test Data
+
+| Field | Value |
+|-------|-------|
+| {field name} | {value or generation rule} |
+| {field name} | Use `EXEC_IDX`-generated value for uniqueness |
+| User | Credentials from `QA_USER_{ROLE}_EMAIL` env var |
+
+---
+
+## Steps
+
+| Step | Action | Expected result |
+|------|--------|----------------|
+| 1 | Navigate to {URL} | {page loads, specific element visible} |
+| 2 | Click {element description} | {modal opens / form appears / etc.} |
+| 3 | Fill {field} with {value} | {field accepts input} |
+| 4 | Click {submit/save/confirm button} | {success message / redirect / state change} |
+| 5 | Verify {observable outcome} | {specific text, element, state} |
+
+---
+
+## Expected Results
+
+**Final state**: {what the system should look like after all steps are complete}
+
+**Verification points**:
+- [ ] {verification 1}: {element} should {state}
+- [ ] {verification 2}: {element} should contain {text}
+- [ ] {verification 3}: page/modal should transition to {expected state}
+
+---
+
+## Known Issues / Related Defects
+
+| Issue | DEF-ID / ADO WI | Status | Impact on this TC |
+|-------|----------------|--------|------------------|
+| {description} | DEF-NNN | Open | TC skipped until resolved |
+
+---
+
+## Evidence
+
+{Link to screenshot or video from execution, if available}
+
+---
+
+## Automation Notes
+
+```typescript
+// Playwright selector for key element (discovered during implementation)
+// page.locator('{selector}')
+
+// Key assertion
+// await expect(page.locator('{selector}')).toHaveText('{expected text}');
+```
+
+---
+
+## Changelog
+
+| Version | Date | Author | Description |
+|---------|------|--------|-------------|
+| 1.0 | YYYY-MM-DD | {author} | Initial creation |
+```
+
+---
+
+## Rules for TC Content Quality
+
+### Write measurable expected results
+
+❌ BAD: "The system should work correctly"  
+✅ GOOD: "A success toast message 'Record saved' appears and the datatable row count increases by 1"
+
+❌ BAD: "The form validates the input"  
+✅ GOOD: "A red validation message 'This field is required' appears below the Name field"
+
+### Write observable steps
+
+❌ BAD: "Add a new record with valid data"  
+✅ GOOD: "Click the 'New' button. Fill 'Name' with 'Test-{EXEC_IDX}'. Click 'Save'."
+
+### Link to defects explicitly
+
+If a TC is skipped due to a bug:  
+✅ GOOD: "Step 4 — SKIPPED: DEF-001 (ADO #21944). The switch defaults to OFF when it should be ON. Reactivate after fix."
+
+### Priority guidelines recap
+
+| Priority | Use when |
+|---|---|
+| P0 | Core feature is unusable without this working |
+| P1 | Major functionality gap if this fails |
+| P2 | Minor functionality gap, workaround exists |
+| P3 | Edge case, cosmetic, or rare scenario |