@keber/qa-framework 1.2.0-0 → 1.4.0-0

package/skills/qa-maintenance/SKILL.md

@@ -0,0 +1,110 @@
+---
+name: qa-maintenance
+description: >
+  Stage 6 of the QA pipeline. Updates QA artifacts after application changes.
+  Keeps specs, test cases, and automation in sync with the product.
+  Use when asked to update specs after a change, maintain tests after a feature
+  change, sync QA with a new release, update a spec after a bug fix, or
+  perform QA maintenance after a sprint.
+  Requires a delivered application change to be the trigger.
+---
+
+# QA Skill: Maintenance (Stage 6 of 6)
+
+**Stage**: 6 — Maintenance  
+**Prerequisite**: An application change (bug fix, feature change, new requirement) has been delivered  
+**Output**: Updated specs, TCs, and automation aligned with the new application behavior  
+**Cycle**: Returns to Stage 2 (spec update) → Stage 4 (TC update) → Stage 5 (automation update) as needed
+
+> **Rule**: Never update specs to match an incorrect application behavior. Specs define correct behavior; automation reflects specs. If the app is wrong, file a defect, don't update the spec.
+
+---
+
+## Inputs Required
+
+1. Description of the change — which submodule, what changed, and why
+2. `qa/01-specifications/{module}/` — specs affected by the change
+3. `qa/07-automation/e2e/{module}/` — automation affected by the change
+4. Optionally: ADO work item or defect report describing the change
+
+---
+
+## Process
+
+### Step 1 — Identify affected artifacts
+
+For the delivered change, identify:
+- Which spec files reference the changed behavior (search for affected term/field/flow)
+- Which TCs exercise that behavior
+- Which automation files implement those TCs
+- Whether any test data in `04-test-data.md` is affected
+
+### Step 2 — Update specs first
+
+Update spec files before touching automation. Spec update rules:
+
+| Situation | Action |
+|---|---|
+| UI label changed | Update `02-workflows.md` step text |
+| Business rule changed | Update `01-business-rules.md` rule + affected scenarios in `05-test-scenarios.md` |
+| New required field added | Update `02-workflows.md` + `04-test-data.md` + `05-test-scenarios.md` |
+| Field removed | Mark TC as `OBSOLETE-SCENARIO` in `05-test-scenarios.md`, delete corresponding TC file |
+| Permission change | Update `03-roles-permissions.md` |
+| Flow reordered | Update `02-workflows.md` with new step sequence |
+
+Detailed rules: `references/update-rules.md`
+
+### Step 3 — Update TCs
+
+After specs are updated:
+- Update step text in affected `TC-*.md` files
+- If a TC no longer applies: mark as `[OBSOLETE]` in the index, do not delete the file
+- If a new scenario added to specs: generate new TC following Stage 4
+
+### Step 4 — Update automation
+
+After TCs are updated:
+- Update locators for changed UI elements
+- Update assertions for changed business rules
+- Remove or skip automation for `OBSOLETE-SCENARIO` TCs
+- Run updated spec file locally to confirm passing before committing
+
+Automation update rules: `references/update-rules.md`
+
+### Step 5 — Bump spec version and log change
+
+In the spec file's YAML frontmatter (or header block):
+```
+version: {incremented}
+last-updated: {date}
+change-summary: {one-line description of what changed}
+```
+
+### Step 6 — Update README and close session
+
+- Update `qa/README.md` module row (spec version, TC count if changed, automation status)
+- Update `AGENT-NEXT-STEPS.md`:
+  - Remove completed items from the active sprint checklist
+  - If all TCs passing: trigger Module Completion Ritual (move sprint to history, trim file)
+- Generate session summary in `qa/memory/` if significant changes were made
+
+---
+
+## Key Rules
+
+| Rule | Correct | Wrong |
+|---|---|---|
+| Spec is ground truth | Update spec first, then tests | Fix test to match wrong behavior |
+| Obsolete TCs | Mark as OBSOLETE, keep file | Delete TC files |
+| Scope discipline | Only touch artifacts linked to the change | Rewrite unrelated specs "while you're there" |
+| Version tracking | Bump version + log change summary | Silently update spec content |
+
+---
+
+## Outputs
+
+- Updated `qa/01-specifications/{module}/*.md` files
+- Updated `qa/03-test-cases/automated/TC-*.md` files
+- Updated `qa/07-automation/e2e/{module}/*.spec.ts`
+- Updated `qa/README.md`
+- Updated `AGENT-NEXT-STEPS.md` (or sprint moved to history if complete)

package/skills/qa-maintenance/references/update-rules.md

@@ -0,0 +1,118 @@
+# Reference: Spec and Automation Update Rules
+
+> Loaded by `skills/qa-maintenance/` when performing post-change spec and automation updates.
+
+---
+
+## Spec Update Rules by Change Type
+
+### New field added to a form
+
+1. Add field row to `00-inventory.md` UI Elements table
+2. Add business rule to `01-business-rules.md` if the field has validation: `RN-{MODULE}-{next-number}`
+3. Add TCs to `05-test-scenarios.md`:
+   - Happy path TC with new field populated
+   - Required validation TC (if the field is required)
+   - Any new business rule violation TC
+
+### Field removed from UI
+
+1. Mark row in `00-inventory.md` as `[REMOVED - {YYYY-MM-DD}]` — do NOT delete the row
+2. Mark the related business rule in `01-business-rules.md` as `[DEPRECATED - {YYYY-MM-DD}]`
+3. Mark related TCs in `05-test-scenarios.md` as `[DEPRECATED]` for traceability
+4. Update or skip automation tests that reference the removed element's selector
+
+### Workflow changed
+
+1. Update `02-workflows.md` with the new flow diagram
+2. Mark the old version as `[PREVIOUS - {YYYY-MM-DD}]` inside the same file (keep for reference)
+3. Review all TCs that test the affected steps — update expected results in `05-test-scenarios.md`
+
+### New submodule added
+
+Follow the full module analysis process: `skills/qa-module-analysis/`.
+
+### Defect resolved
+
+1. Find all `test.skip()` calls that reference the DEF-ID or ADO WI number
+2. Remove the `test.skip()` wrapper
+3. If the spec assertion was adapted to document the bug (e.g., `toBeFalsy()` for a wrong default), restore the correct assertion
+4. Run the test at least 2 times with different `EXEC_IDX` values to confirm stability
+5. Update `qa/06-defects/open/DEF-{NNN}.md` status to Resolved; move file to `06-defects/resolved/`
+
+---
+
+## Automation Update Rules
+
+### Selectors changed (redesign or framework update)
+
+1. Locate the spec file or page object containing the old selector
+2. Update the selector — prefer `getByRole`, `getByLabel`, `getByTestId`
+3. Run the affected test 2+ times to confirm stability
+4. Add an entry to the correction log at top of the spec file:
+
+```typescript
+/* CORRECTIONS
+ * YYYY-MM-DD: Updated selector for {element} from '#old-id' to getByRole('button', { name: '...' })
+ */
+```
+
+### New test suite needed
+
+Follow `skills/qa-automation/` for the new suite.  
+Update `COVERAGE-MAPPING.md` with new TC rows.
+
+### Test became flaky after app update
+
+5-layer debugging methodology:
+1. **Environment**: is QA env stable? Can you log in manually?
+2. **Timeout**: did an operation become slower? Increase `actionTimeout` if justified
+3. **Data**: did a seed or precondition change? Review `beforeAll` provisioning
+4. **Selector**: did the element structure change? Inspect with Playwright UI mode
+5. **System**: is this a real defect? File with DEF reference and skip the test
+
+---
+
+## Spec Version Changelog Block
+
+Add to the bottom of each updated spec file:
+
+```markdown
+## Changelog
+
+| Version | Date | Description |
+|---------|------|-------------|
+| 1.0 | YYYY-MM-DD | Initial creation |
+| 1.1 | YYYY-MM-DD | Added RN-{MODULE}-{NNN}: {rule title}. TC count: N → M |
+| 1.2 | YYYY-MM-DD | Removed field {name} (deprecated) |
+```
+
+---
+
+## Session Summary Template
+
+Create at: `qa/SESSION-SUMMARY-{YYYY-MM-DD}-maintenance.md`
+
+```markdown
+# Maintenance Session — {YYYY-MM-DD}
+
+## Change trigger
+{Brief description of the application change that triggered this session}
+
+## Artifacts updated
+| File | Type of change |
+|------|---------------|
+| qa/01-specifications/{module}/{submodule}/00-inventory.md | Added field {name} |
+| qa/01-specifications/{module}/{submodule}/05-test-scenarios.md | Added TCs {IDs} |
+
+## TC changes
+- New TCs: {IDs} (+N total)
+- Deprecated TCs: {IDs} (-N total)
+- Net TC count: {module} → {N}
+
+## Tests reactivated
+- {TC-ID}: removed test.skip() — DEF-{NNN} resolved
+
+## Tests newly skipped
+- {TC-ID}: test.skip() added — DEF-{NNN} filed for {bug description}
+```

package/skills/qa-module-analysis/SKILL.md

@@ -0,0 +1,95 @@
+---
+name: qa-module-analysis
+description: >
+  Stage 1 of the QA pipeline. Analyzes an application module using browser automation
+  to produce the 6-file specification set (inventory, business rules, workflows,
+  roles, test data, test scenarios). Use when asked to analyze a module, explore a
+  feature, document an application area, or start QA coverage for a new module.
+  Prerequisite for all subsequent QA stages.
+---
+
+# QA Skill: Module Analysis (Stage 1 of 6)
+
+**Stage**: 1 — Module Analysis  
+**Prerequisite**: None — this is the first stage  
+**Output**: `qa/01-specifications/module-{name}/submodule-{name}/` with 6 spec files  
+**Next stage**: Stage 2 — Specification Generation (`qa-spec-generation`)
+
+> **Pipeline rule**: This stage must be completed before any spec generation, test planning,
+> or automation can begin. Do not skip to a later stage.
+
+---
+
+## Prerequisites
+
+Before starting:
+
+1. `qa/qa-framework.config.json` exists and contains `project.qaBaseUrl`
+2. `qa/00-standards/naming-conventions.md` is available to check existing module codes
+3. Browser automation (Playwright MCP or equivalent) is available
+4. Ask the user for: module name, URL/route prefix, submodules in scope (or discover from UI)
+
+**Security**: Never write credentials to any file. Use `<PLACEHOLDER>` for any credential value.
+
+---
+
+## Phase 1 — Preparation
+
+1. Check if `qa/01-specifications/module-{name}/` already exists
+   - **Exists**: read all existing files; only re-analyze what changed
+   - **New**: create the directory and plan submodules
+2. Allocate a unique 2–6 letter uppercase module code (check naming-conventions.md for conflicts)
+3. Create `qa/01-specifications/module-{name}/README.md` listing all planned submodules before analysis begins
+
+---
+
+## Phase 2 — Exploration (one pass per submodule)
+
+For each submodule, use browser automation to collect:
+
+1. **Navigation** — URLs, page titles, breadcrumbs
+2. **UI inventory** — all inputs (type, required, validation), dropdowns (options), buttons, tables, modals, badges
+3. **API endpoints** — observe network requests during create/read/update/delete; record method, path, payload shape
+4. **Role access** — log in as each configured role; record `can access / cannot access / permission error`
+5. **Workflow** — execute the primary happy path; record steps, branch points, state transitions
+6. **Business rules** — attempt invalid inputs and boundary conditions; record validation messages exactly
+
+Detailed exploration checklist: `references/exploration-checklist.md`
+
+---
+
+## Phase 3 — Documentation (6-file output)
+
+Produce 6 files per submodule. File formats and templates: `references/spec-file-formats.md`
+
+| File | Contents |
+|---|---|
+| `00-inventory.md` | Module/submodule header, UI element table, API endpoint table |
+| `01-business-rules.md` | `RN-{MODULE}-{NNN}` rules with type, trigger, error message |
+| `02-workflows.md` | `FL-{MODULE}-{NNN}` flows with ASCII/Mermaid diagrams |
+| `03-roles-permissions.md` | Role × feature access matrix; env var references for test users |
+| `04-test-data.md` | Prerequisites, data shapes per scenario, EXEC_IDX pattern |
+| `05-test-scenarios.md` | TC table: 50–85 TCs per submodule, all mandatory coverage categories |
+
+TC target: **50–85 per submodule**. Mark origin as `UI-OBSERVED`, `PENDING-CODE`, or `BLOCKED-PERMISSIONS`.
+
+---
+
+## Phase 4 — Review
+
+Completeness checklist before closing the stage:
+
+- [ ] All menu items in the module have a submodule folder
+- [ ] All `UI-OBSERVED` TCs were actually observed in this session
+- [ ] No credentials appear in any file
+- [ ] All TC IDs are unique across the module
+- [ ] Module README updated with submodule table and TC counts
+- [ ] `qa/README.md` module status row updated
+
+---
+
+## Outputs
+
+- `qa/01-specifications/module-{name}/` — fully populated spec tree
+- `qa/01-specifications/module-{name}/README.md` — updated with submodule table
+- `qa/README.md` — module status row updated

package/skills/qa-module-analysis/references/exploration-checklist.md

@@ -0,0 +1,121 @@
+# Reference: Module Exploration Checklist
+
+> Loaded by `skills/qa-module-analysis/` when performing Phase 2 exploration.
+
+---
+
+## 2.1 Navigation and Route Mapping
+
+```
+Navigate to {{QA_BASE_URL}} and log in with the appropriate role
+Navigate to the submodule URL
+Record:
+  - Full URL of each page/view
+  - Page title
+  - Navigation breadcrumb
+```
+
+---
+
+## 2.2 UI Inventory
+
+For each screen in the submodule:
+
+```
+Record all:
+  - Input fields (name, type, required/optional, validation rules visible in UI)
+  - Dropdowns (static or dynamic/AJAX; enumerate options if static)
+  - Buttons (action label + what they trigger)
+  - Tables/grids (columns, default sorting, pagination behavior)
+  - Modals (trigger condition, fields, confirmation button behavior)
+  - Status indicators / badges / alerts
+  - File upload / download controls
+```
+
+---
+
+## 2.3 API Endpoint Discovery
+
+```
+Observe network requests during key interactions (create, read, update, delete)
+Record:
+  - Endpoint path (e.g., /api/GetUsers, /api/CreateUser)
+  - HTTP method (GET, POST, PUT, DELETE)
+  - Request payload shape (field names)
+  - Response shape (key fields returned)
+```
+
+---
+
+## 2.4 Role-Based Access Testing
+
+```
+For each configured test user role:
+  Log in as that role
+  Navigate to the submodule
+  Record: can access? (YES / NO / PERMISSION_ERROR)
+  Record: which actions are available vs hidden/disabled
+```
+
+---
+
+## 2.5 Workflow Discovery
+
+```
+Execute the primary happy path with appropriate test data
+Record each step (clickable element → navigation result → system state change)
+Identify branch points (e.g., approve vs reject)
+Record state transitions (pending → approved → rejected)
+```
+
+---
+
+## 2.6 Business Rule Discovery
+
+```
+For each input field and business action:
+  Attempt invalid inputs: empty, too long, wrong format, duplicate, out of range
+  Record: validation message exact text
+  Record: which field shows the error
+  Record: system behavior (block submission / inline error / toast / redirect)
+```
+
+---
+
+## Phase 4 Completeness Checklist
+
+After completing all submodule files:
+
+- [ ] All menu items in the module have a corresponding submodule folder
+- [ ] All TCs marked `UI-OBSERVED` were actually observed in this session
+- [ ] `01-business-rules.md` covers all visible validation behaviors
+- [ ] `03-roles-permissions.md` reflects the access observed for each role
+- [ ] No credentials appear in any file
+- [ ] All TC IDs are unique across the module
+- [ ] Module README lists all submodules with their codes and TC counts
+- [ ] Session summary created at `qa/SESSION-SUMMARY-{date}.md`
+
+---
+
+## Session Summary Template
+
+```markdown
+# Session Summary — {YYYY-MM-DD}
+
+## Module analyzed
+{module-name} ({module-code})
+
+## Submodules documented
+| Submodule | Code | TC count | Notes |
+|-----------|------|----------|-------|
+| {name}    | {code} | {N}    | {any notable blocking issues} |
+
+## Files created
+- {path/to/file.md}
+
+## Blockers
+- {any features marked PENDING-CODE or BLOCKED-PERMISSIONS}
+
+## Next steps
+- {what remains before automation can begin}
+```

package/skills/qa-module-analysis/references/spec-file-formats.md

@@ -0,0 +1,219 @@
+# Reference: Specification File Formats
+
+> Shared by `skills/qa-module-analysis/` and `skills/qa-spec-generation/`.  
+> Contains the exact format templates for all 6 spec files.
+
+---
+
+## `00-inventory.md`
+
+```markdown
+# MODULE: {Module Display Name} — Submodule: {Submodule Display Name}
+
+| Field | Value |
+|-------|-------|
+| Module code | {MODULE_CODE} |
+| Submodule code | {SUBMODULE_CODE} |
+| Primary URL | {{QA_BASE_URL}}/path/to/submodule |
+| Status | Active / Deprecated / In Development |
+| Last updated | YYYY-MM-DD |
+
+## UI Elements
+
+| Element | Type | Required | Notes |
+|---------|------|----------|-------|
+| {field-name} | text / select / checkbox / date | Yes/No | {validation rule} |
+
+## API Endpoints
+
+| Method | Path | Purpose |
+|--------|------|---------|
+| GET | /api/{endpoint} | {description} |
+| POST | /api/{endpoint} | {description} |
+```
+
+---
+
+## `01-business-rules.md`
+
+```markdown
+### RN-{MODULE}-{NNN}: {Rule Title}
+
+- **Type**: Validation / Access Control / State Machine / Calculation / Integration
+- **Trigger**: {What user action triggers this rule}
+- **Behavior**: {What the system does when the rule is evaluated}
+- **Error message** (if validation): "{exact text as shown in UI}"
+- **Notes**: {any special circumstances}
+```
+
+**Rule identification tips**:
+- Every required field → at least 1 RN
+- Every unique constraint → 1 RN (duplicate not allowed)
+- Every state transition → 1 RN (can only approve if status=pending)
+- Every role restriction → 1 RN
+- Every auto-calculated field → 1 RN
+
+---
+
+## `02-workflows.md`
+
+```markdown
+### FL-{MODULE}-{NNN}: {Workflow Title}
+
+**Actor**: {Role who performs this workflow}
+**Trigger**: {What initiates the workflow}
+**Precondition**: {What must be true before this starts}
+
+**Flow**:
+```
+[Start]
+    │
+    ▼
+[Step 1: action]
+    │
+    ├─ [Alternative: condition] ──► [Alternative path]
+    │
+    ▼
+[Step 2: action]
+    │
+    ▼
+[End: outcome]
+```
+
+**Postcondition**: {State of the system after the workflow completes}
+```
+
+---
+
+## `03-roles-permissions.md`
+
+```markdown
+# Roles and Permissions — {Submodule}
+
+## Access Matrix
+
+| Feature / Action | {Role 1} | {Role 2} | {Role 3} |
+|-----------------|---------|---------|---------|
+| View list | ✅ | ✅ | ❌ |
+| Create new | ✅ | ❌ | ❌ |
+| Edit | ✅ | ❌ | ❌ |
+| Delete | ✅ | ❌ | ❌ |
+| Approve | ❌ | ✅ | ❌ |
+| Export | ✅ | ✅ | ❌ |
+
+## Test User Reference
+
+| Role | Env var (email) | Env var (password) |
+|------|----------------|-------------------|
+| {Role 1} | `QA_USER_{ROLE}_EMAIL` | `QA_USER_{ROLE}_PASSWORD` |
+
+**Notes**: {conditional access rules}
+```
+
+---
+
+## `04-test-data.md`
+
+```markdown
+# Test Data — {Submodule}
+
+## Prerequisites
+
+| Item | Description | Source |
+|------|-------------|--------|
+| {entity} | {description} | QA seed / pre-existing |
+
+## Data shapes for key scenarios
+
+### Scenario: {happy path}
+- Field A: {value type}
+- User: role `{ROLE_NAME}`, credentials from `QA_USER_{ROLE}_EMAIL` env var
+
+### Scenario: {negative — duplicate}
+- Precondition: {entity} with the same key already exists
+- Field A: {same value as existing record}
+- Expected: {system rejection behavior}
+
+## Dynamic data generation (EXEC_IDX pattern)
+
+```typescript
+const EXEC_IDX = process.env.EXEC_IDX ?? String(Math.floor(Date.now() / 60_000) % 100_000);
+const uniqueTitle = `Test-${EXEC_IDX}`;
+```
+
+## Data isolation rules
+
+1. Each test operates on its own data (no shared mutable state between TCs)
+2. Tests requiring pre-existing data provision it in `beforeAll`
+3. EXEC_IDX-named data does not require cleanup
+```
+
+---
+
+## `05-test-scenarios.md`
+
+```markdown
+# Test Scenarios — {Module}: {Submodule}
+
+**Module code**: {MODULE_CODE}  
+**Submodule code**: {SUBMODULE_CODE}  
+**Last updated**: YYYY-MM-DD
+
+## Summary
+
+| Priority | Count | Automated | Manual |
+|----------|-------|-----------|--------|
+| P0 | {N} | {N} | {N} |
+| P1 | {N} | {N} | {N} |
+| P2 | {N} | {N} | {N} |
+| P3 | {N} | {N} | {N} |
+| **Total** | **{N}** | **{N}** | **{N}** |
+
+---
+
+### TC-{MODULE}-{SUBMODULE}-{NNN}: {TC Title}
+
+| Field | Value |
+|-------|-------|
+| Priority | P0 / P1 / P2 / P3 |
+| Type | Functional / Negative / Regression / Security / Integration |
+| Origin | UI-OBSERVED / PENDING-CODE / BLOCKED-PERMISSIONS |
+| Automation | Yes / Partial / No |
+| Playwright | (fill after automation is written) |
+
+**Preconditions**:
+- {precondition 1}
+
+**Steps**:
+1. {step 1}
+2. {step 2}
+3. {step 3}
+
+**Expected result**: {observable, measurable outcome}
+
+**Notes**: {DEF references, skip conditions, edge cases}
+```
+
+### Mandatory coverage checklist
+
+- [ ] Access: unauthenticated user redirected to login
+- [ ] Access: role without permission receives error or empty page
+- [ ] Happy path: primary workflow with valid data succeeds
+- [ ] Negative: required field missing → validation error shown
+- [ ] Negative: duplicate record → system rejects with error message
+- [ ] Negative: invalid format → format error shown
+- [ ] State transition: state changes correctly on action
+- [ ] Export/download: file is generated (if feature exists)
+- [ ] Pagination/search: filtering works correctly (if feature exists)
+
+---
+
+## Common Anti-Patterns
+
+| Anti-pattern | Why it's wrong | Correct approach |
+|---|---|---|
+| TC for a feature not in QA env | Invalid test expectations | Mark as `PENDING-CODE` |
+| Real user data in steps | Security issue | Use env var reference |
+| "click the X button" without observed element | Assumes UI | Verify element exists first |
+| 100+ TCs by copy-pasting variations | Bloats spec | Merge near-identical cases |
+| "verify system works correctly" | Untestable | Write observable expected result |