qa-workflow-cc 1.0.0

package/skills/qa/templates/qa-report-template.md

@@ -0,0 +1,271 @@
+# QA Cycle Report Template (Generic)
+
+Category B template for QA bootstrapping. Contains `{{variable}}` placeholders that the bootstrap skill populates with project-specific content.
+
+## Variables
+
+| Variable | Description | Example |
+|----------|-------------|---------|
+| `{{appCategories}}` | Markdown sections for each app/module under "Results by Category" | Portal Business, Client Portal, API, Worker |
+| `{{exitCriteria}}` | Customizable exit criteria rows for the gate table | App-specific Lighthouse targets, isolation fields, auth types |
+| `{{projectName}}` | Name of the project being tested | TruLine, Acme CRM |
+| `{{scopeOptions}}` | Valid scope arguments for this project's QA runs | full, portal-business, portal, api, security, ux |
+| `{{testIdPrefixes}}` | Project-specific test ID prefix conventions | PB-*, CP-*, API-*, WK-* |
+
+---
+
+## Template
+
+Use this template when generating QA cycle reports at `docs/qa-reports/cycle-{N}-{YYYY-MM-DD}.md`.
+
+````markdown
+# QA Cycle {N} Report -- {YYYY-MM-DD}
+
+**Project:** {{projectName}}
+**Scope:** {scope from {{scopeOptions}}}
+**Duration:** {start time} -- {end time}
+**Previous Cycle:** {N-1 or "First cycle"}
+
+---
+
+## Executive Summary
+
+| Metric | Value | Delta |
+|--------|-------|-------|
+| Total Tests | {N} | {+/-N from last cycle} |
+| Passing | {N} ({%}) | {+/-N} |
+| Failing | {N} ({%}) | {+/-N} |
+| Skipped | {N} | {+/-N} |
+| Critical Defects | {N} | {+/-N} |
+| Major Defects | {N} | {+/-N} |
+| Minor Defects | {N} | {+/-N} |
+| Cosmetic Defects | {N} | {+/-N} |
+| UX Score (Nielsen avg) | {X.X}/5.0 | {+/-X.X} |
+
+### Health Assessment
+
+{One paragraph summary: Is the platform ready for users? What are the blockers? What improved since the last cycle?}
+
+---
+
+## Results by Category
+
+{{appCategories}}
+
+<!-- BOOTSTRAP INSTRUCTIONS:
+     Replace {{appCategories}} with one section per app/module in your project.
+     Each section follows this format:
+
+### {App/Module Name} -- Functional ({pass}/{total})
+
+| Test ID | Description | Status | Severity | Notes |
+|---------|------------|--------|----------|-------|
+| {PREFIX}-{AREA}-{NNN} | ... | PASS/FAIL | -- | -- |
+
+     Common section types to include:
+     - One section per frontend app (e.g., "Web App", "Mobile App", "Admin Panel")
+     - One section per backend service (e.g., "API Routers", "Background Workers")
+     - A "Security" section for isolation and auth boundary tests
+     - A "UX Evaluation" section for heuristic scoring
+     - A "Performance" section for benchmarks and vitals
+
+     Example for a 3-app project:
+
+### Web Dashboard -- Functional ({pass}/{total})
+
+| Test ID | Description | Status | Severity | Notes |
+|---------|------------|--------|----------|-------|
+| WEB-AUTH-001 | ... | PASS | -- | -- |
+
+### Mobile App -- Functional ({pass}/{total})
+
+| Test ID | Description | Status | Severity | Notes |
+|---------|------------|--------|----------|-------|
+| MOB-NAV-001 | ... | PASS | -- | -- |
+
+### API Routers ({pass}/{total})
+
+| Test ID | Description | Status | Severity | Notes |
+|---------|------------|--------|----------|-------|
+| API-USER-001 | ... | PASS | -- | -- |
+-->
+
+### Security ({pass}/{total})
+
+| Test ID | Description | Status | Severity | Notes |
+|---------|------------|--------|----------|-------|
+| SEC-ISO-001 | ... | PASS/FAIL | -- | -- |
+
+### UX Evaluation
+
+{Nielsen heuristic scores table -- use the nielsen-heuristics.md rubric}
+{WCAG findings}
+{UI/UX rule compliance check}
+
+### Performance ({pass}/{total})
+
+| Test ID | Description | Status | Severity | Notes |
+|---------|------------|--------|----------|-------|
+| PERF-CWV-001 | ... | PASS/FAIL | -- | -- |
+
+---
+
+## Defect Catalog
+
+### Critical Defects ({N})
+
+#### DEF-{NNN}: {Title}
+- **Test ID:** {test ID that caught this defect}
+- **Feature/PRD Section:** {section reference or feature name}
+- **Severity:** Critical
+- **Description:** {what is broken}
+- **Evidence:** {file path, line number, error message, screenshot reference}
+- **Impact:** {who is affected and how -- user-facing? data integrity? security?}
+- **Suggested Fix:** {brief technical approach to resolution}
+
+### Major Defects ({N})
+
+#### DEF-{NNN}: {Title}
+- **Test ID:** {test ID}
+- **Feature/PRD Section:** {section reference}
+- **Severity:** Major
+- **Description:** {what is broken}
+- **Evidence:** {file path, line number, error}
+- **Impact:** {who is affected and how}
+- **Suggested Fix:** {brief approach}
+
+### Minor Defects ({N})
+
+#### DEF-{NNN}: {Title}
+- **Test ID:** {test ID}
+- **Severity:** Minor
+- **Description:** {what is broken}
+- **Evidence:** {evidence}
+- **Suggested Fix:** {brief approach}
+
+### Cosmetic Defects ({N})
+
+#### DEF-{NNN}: {Title}
+- **Test ID:** {test ID}
+- **Severity:** Cosmetic
+- **Description:** {visual issue}
+- **Evidence:** {screenshot or file reference}
+
+---
+
+## Exit Criteria Status
+
+{{exitCriteria}}
+
+<!-- BOOTSTRAP INSTRUCTIONS:
+     Replace {{exitCriteria}} with a gate table customized to the project.
+     The table below shows the standard rows. Add, remove, or adjust thresholds
+     based on project requirements.
+
+     Standard exit criteria table:
+
+| Gate | Threshold | Current | Status |
+|------|-----------|---------|--------|
+| P0 features pass | 100% | {%} | PASS/FAIL |
+| P1 features pass | 100% | {%} | PASS/FAIL |
+| Critical defects open | 0 | {N} | PASS/FAIL |
+| Major defects open | 0 | {N} | PASS/FAIL |
+| Minor defects open | <10 | {N} | PASS/FAIL |
+| UX Score (Nielsen avg) | >= 3.5 | {X.X} | PASS/FAIL |
+| WCAG 2.1 AA critical violations | 0 | {N} | PASS/FAIL |
+| Lighthouse Performance | >= 80 | {N} | PASS/FAIL |
+| Lighthouse Accessibility | >= 85 | {N} | PASS/FAIL |
+| Tenant isolation verified | 100% | {%} | PASS/FAIL |
+| Auth boundary tests | 100% | {%} | PASS/FAIL |
+
+     Project-specific rows to consider adding:
+     - API uptime percentage (for production monitoring)
+     - E2E test suite pass rate
+     - Security scan (SAST/DAST) pass
+     - Load test throughput threshold
+     - Mobile-specific gates (app store compliance, offline mode)
+-->
+
+**Overall Verdict:** {PASS -- Ready for release | FAIL -- Requires Cycle N+1}
+
+---
+
+## Recommendations
+
+### Priority Fixes for Next Cycle
+
+1. {Highest-impact fix with defect reference}
+2. {Second-highest fix}
+3. {Third fix}
+
+### Areas Needing More Test Coverage
+
+1. {Feature or module with low coverage and why}
+2. {Feature or module with known gaps}
+
+### Architecture Concerns
+
+1. {Systemic issue observed during testing that warrants design attention}
+2. {Performance or scalability concern}
+
+### Regression Watch
+
+1. {Previously fixed defect that should be monitored in future cycles}
+
+---
+
+*Generated by QA Orchestrator -- Cycle {N}*
+````
+
+---
+
+## Defect Severity Definitions (Universal)
+
+These definitions are the same for every project. They determine fix priority and release blocking.
+
+| Severity | Definition | Examples | Release Impact |
+|----------|-----------|---------|----------------|
+| **Critical** | Feature completely broken, data loss risk, security vulnerability, crash | Cross-tenant data leak, auth bypass, core feature crash, data corruption | Blocks release. Fix immediately. |
+| **Major** | Feature partially broken with no workaround, significant user impact | File upload fails, real-time messaging broken, form submission error | Blocks release. Fix before ship. |
+| **Minor** | Feature works but with friction, workaround exists | Filter state lost on back nav, sort not persisting, slow page transition | Does not block release. Fix within next sprint. |
+| **Cosmetic** | Visual-only issue, no functional impact | Icon misalignment, wrong font weight, inconsistent spacing, color mismatch | Does not block release. Fix when convenient. |
+
+---
+
+## Report Naming Convention
+
+```
+docs/qa-reports/
+  cycle-1-2026-02-09.md        # QA cycle reports
+  cycle-2-2026-02-12.md
+  fix-plan-cycle-1.md           # Fix plans (generated after defect analysis)
+  fix-plan-cycle-2.md
+  cycle-state.json              # Machine-readable cycle state for resume
+  final-certification.md        # Written when all gates pass
+```
+
+---
+
+## Cycle State JSON Schema
+
+```json
+{
+  "currentCycle": 1,
+  "startedAt": "2026-02-09T10:00:00Z",
+  "completedAt": "2026-02-09T12:30:00Z",
+  "scope": "full",
+  "results": {
+    "total": 0,
+    "pass": 0,
+    "fail": 0,
+    "skip": 0,
+    "critical": 0,
+    "major": 0,
+    "minor": 0,
+    "cosmetic": 0
+  },
+  "failedTests": [],
+  "openDefects": [],
+  "exitCriteriaMet": false
+}
+```