qa-skills 3.0.0

package/skills/qa-test-healer/SKILL.md

@@ -0,0 +1,101 @@
+---
+name: qa-test-healer
+description: Self-healing test system that runs failing tests, debugs with MCP tools, auto-fixes broken selectors/assertions/waits, and marks unfixable tests as test.fixme() with explanation.
+---
+
+# QA Test Healer
+
+## Purpose
+
+Automatically fix broken tests after code changes. Inspired by fugazi/test-automation-skills-agents, this skill runs failing tests in headed/debug mode, analyzes failures, applies targeted fixes, and either restores passing tests or marks unfixable cases as `test.fixme()` with a clear explanation.
+
+## Trigger Phrases
+
+- "Heal my failing tests"
+- "Fix broken Playwright/Cypress tests"
+- "Auto-fix test failures"
+- "Tests are failing after my changes"
+- "Debug and fix selector/assertion failures"
+- "Run failing test and fix it"
+
+## Healing Workflow
+
+1. **Receive failing test(s)** — User provides failing test path(s), error output, or CI failure logs
+2. **Run test in headed mode / with debug output** — Execute the test with visible browser and verbose logging to capture failure context
+3. **Analyze failure** — Determine root cause: selector broken? assertion changed? timeout? missing element? network change?
+4. **Apply fix strategy** — Use the appropriate strategy from `references/fix-strategies.md`
+5. **Re-run to verify fix** — Execute the test again; confirm it passes
+6. **If unfixable** — Mark as `test.fixme('explanation')` with a clear reason for manual review
+
+## Fix Strategies
+
+| Failure Type | Strategy | Action |
+|--------------|----------|--------|
+| **Broken selectors** | Find new locator | Use accessibility snapshot (Playwright MCP) to inspect DOM; update POM/test with stable locator (getByRole, getByTestId preferred) |
+| **Changed assertions** | Compare expected vs actual | If intentional product change, update expected values; otherwise flag for review |
+| **Timeout issues** | Add explicit waits | Increase timeout thresholds, add `expect` with timeout, or use `waitFor` for async content |
+| **Missing elements** | Check feature status | Determine if feature was removed/redesigned; flag for manual review or mark fixme |
+| **Network changes** | Update mocks/routes | Update intercepted routes, mocks, or add retry logic for flaky endpoints |
+
+See `references/fix-strategies.md` for detailed before/after examples.
+
+## Diagnosis Patterns
+
+Use `references/diagnosis-patterns.md` for:
+
+- **Error message parsing** — Extract selector, assertion, timeout from stack traces and failure output
+- **DOM comparison** — Use browser snapshot to compare expected vs actual structure
+- **Network analysis** — Inspect requests/responses when API or mock changes cause failures
+
+## MCP Integration
+
+- **Playwright MCP (cursor-ide-browser)** — Run tests, navigate to failure state, take accessibility snapshots, inspect DOM, verify fixes
+- **Lock/unlock workflow** — Lock browser before interactions; unlock when done
+
+## Scope
+
+**Can do (autonomous):**
+- Fix broken selectors by finding new locators via DOM inspection
+- Update assertions when expected values have legitimately changed
+- Add or adjust waits and timeouts for flaky/timeout failures
+- Update network mocks, routes, or retry logic
+- Mark unfixable tests as `test.fixme()` with explanation
+- Run tests in headed mode to reproduce and verify fixes
+
+**Cannot do (requires confirmation):**
+- Rewrite test logic or change test intent
+- Add new test cases or modify test structure significantly
+- Change production code
+
+**Will not do (out of scope):**
+- Modify production/application code
+- Execute tests in CI without user request
+- Approve or merge code changes
+
+## References
+
+| Topic | File |
+|-------|------|
+| Detailed fix strategies with before/after examples | `references/fix-strategies.md` |
+| Error parsing, DOM comparison, network analysis | `references/diagnosis-patterns.md` |
+
+## Quality Checklist
+
+- [ ] Root cause correctly identified before applying fix
+- [ ] Fix strategy matches failure type per `references/fix-strategies.md`
+- [ ] Test passes after fix (re-run verified)
+- [ ] Unfixable tests marked with `test.fixme('clear explanation')`
+- [ ] No production code modified
+- [ ] POM updated if selector changed in shared locator
+- [ ] Traceability preserved (test case IDs, comments)
+
+## Troubleshooting
+
+| Symptom | Likely Cause | Fix |
+|---------|--------------|-----|
+| Can't reproduce failure locally | Environment difference, flaky test | Run with `--retries=0`; check CI env vars; use same baseURL |
+| Snapshot shows different structure | Dynamic content, A/B test | Wait for stable state; use more resilient locators |
+| Fix works once then fails | Race condition, shared state | Add proper wait; ensure test isolation |
+| Multiple failures in one file | Cascading failure, setup issue | Fix setup/fixture first; run tests individually |
+| test.fixme() not recognized | Framework difference | Use `test.skip()` or equivalent for Cypress/pytest |
+| MCP browser not available | Playwright MCP not configured | Use terminal to run tests; analyze error output only |

package/skills/qa-test-healer/references/diagnosis-patterns.md

@@ -0,0 +1,142 @@
+# Diagnosis Patterns for Test Failures
+
+Patterns for diagnosing test failures: error message parsing, DOM comparison, and network analysis.
+
+## 1. Error Message Parsing
+
+### Playwright Error Structure
+
+Playwright failures typically include:
+- **Error type** — `TimeoutError`, `Error`, `AssertionError`
+- **Locator** — The selector that failed
+- **Action** — click, fill, toBeVisible, etc.
+- **Expected vs actual** — For assertions
+
+### Extraction Patterns
+
+| Pattern | Regex/Approach | Use |
+|---------|----------------|-----|
+| Selector in error | `locator\('([^']+)'\)` or `getByRole\('([^']+)'` | Identify which locator failed |
+| Timeout value | `Timeout (\d+)ms` | Check if timeout is too short |
+| Expected text | `Expected: (.*)` | Compare with actual |
+| Actual text | `Received: (.*)` | Compare with expected |
+| Stack trace file:line | `at.*\(([^:]+):(\d+):\d+\)` | Locate exact line in test file |
+
+### Example Parse
+
+```
+Error: expect(locator).toHaveText(expected)
+Expected: "Welcome"
+Received: "Welcome back!"
+```
+
+**Extracted:** Assertion `toHaveText`; expected `"Welcome"`; actual `"Welcome back!"` → Likely intentional product change.
+
+### Cypress Error Structure
+
+- `cy.get(...).should(...)` — Selector and assertion in chain
+- `Timed out retrying` — Timeout/visibility issue
+- `Expected X to equal Y` — Assertion mismatch
+
+### pytest/Playwright Python
+
+- `TimeoutError` with locator repr
+- `AssertionError: assert X == Y`
+
+---
+
+## 2. DOM Comparison
+
+### Using Accessibility Snapshot
+
+1. **Navigate to failure state** — Run test in headed mode; stop at failure or use `page.pause()`
+2. **Take snapshot** — `browser_snapshot` (Playwright MCP) returns accessibility tree
+3. **Compare expected vs actual** — Look for:
+   - Missing elements (role/name not in tree)
+   - Renamed elements (text changed)
+   - Structure change (nested differently)
+   - New overlays (modal, loading) blocking interaction
+
+### Snapshot Analysis Checklist
+
+- [ ] Is the target element present? (search by role, name, or ref)
+- [ ] Is it visible/enabled? (check `aria-hidden`, `disabled`)
+- [ ] Is it obscured? (modal, overlay, another element on top)
+- [ ] Has the hierarchy changed? (element moved under different parent)
+- [ ] Are there multiple matches? (strict mode violation)
+
+### Interpreting Snapshot Output
+
+```
+button "Submit" [ref=abc123]
+  - enabled, visible
+```
+
+If test expects `button "Save"` but snapshot shows `button "Submit"` → Update locator or mark as product change.
+
+### Scoped Snapshots
+
+Use `selector` parameter to snapshot a subtree (e.g., a form or modal) when the full page is large.
+
+---
+
+## 3. Network Analysis
+
+### When to Use
+
+- Test fails after API call
+- Mock not applied
+- Request timeout or 404
+- Response body/status changed
+
+### Inspection Steps
+
+1. **List requests** — `browser_network_requests` (Playwright MCP) or test trace
+2. **Filter by URL** — Find the request that failed or returned unexpected data
+3. **Check method/URL** — Did endpoint path change? (e.g., `/api/users` → `/api/v2/users`)
+4. **Check response** — Status code, body shape
+5. **Check timing** — Request took too long?
+
+### Common Network Failure Causes
+
+| Cause | Symptom | Fix |
+|-------|---------|-----|
+| Endpoint renamed | 404 | Update `page.route` URL pattern |
+| Response shape changed | Assertion on body fails | Update mock response or assertion |
+| Request not sent | No matching request | Ensure route registered before `goto` |
+| Slow API | Timeout | Increase timeout or add retry |
+| CORS/auth | Request blocked | Update route handler or headers |
+
+### Route Registration Order
+
+```typescript
+// Correct: route before navigation
+await page.route('**/api/data', handler);
+await page.goto('/page');
+```
+
+```typescript
+// Wrong: navigation may complete before route is set
+await page.goto('/page');
+await page.route('**/api/data', handler);
+```
+
+---
+
+## 4. Combining Diagnosis Methods
+
+For complex failures, combine:
+
+1. **Parse error** → Get selector, assertion, expected/actual
+2. **Snapshot DOM** → Verify element exists and state
+3. **Check network** → If failure involves API, verify requests
+
+### Decision Flow
+
+```
+Parse error
+  → Selector timeout? → Snapshot: is element present? → Yes: increase timeout / add wait
+  → Selector timeout? → Snapshot: element missing? → Fixme or update flow
+  → Assertion mismatch? → Compare expected vs actual → Update or fixme
+  → Network error? → Inspect requests → Update route/mock
+```

package/skills/qa-test-healer/references/fix-strategies.md

@@ -0,0 +1,177 @@
+# Fix Strategies for Test Healing
+
+Detailed strategies for each failure type with before/after examples.
+
+## 1. Broken Selectors
+
+### Symptoms
+- `TimeoutError: locator resolved to X elements` or `strict mode violation`
+- `Error: No element found for selector`
+- `Locator.click: Target closed`
+
+### Strategy
+1. Run test in headed mode; pause at failure point
+2. Take accessibility snapshot via Playwright MCP (`browser_snapshot`)
+3. Identify the target element in the snapshot (role, name, testid)
+4. Choose stable locator: `getByRole` > `getByTestId` > `getByLabel` > `getByText` > CSS
+5. Update POM or test file with new locator
+
+### Before
+```typescript
+await page.click('button.submit-btn');
+```
+
+### After
+```typescript
+await page.getByRole('button', { name: 'Submit' }).click();
+```
+
+### POM Update
+If the selector lives in a Page Object, update the locator there so all tests using it benefit.
+
+```typescript
+// Before (LoginPage.ts)
+get submitButton() { return this.page.locator('button.submit-btn'); }
+
+// After
+get submitButton() { return this.page.getByRole('button', { name: 'Submit' }); }
+```
+
+---
+
+## 2. Changed Assertions
+
+### Symptoms
+- `expect(locator).toHaveText(expected)` — actual text differs
+- `expect(locator).toHaveValue(expected)` — value mismatch
+- Snapshot/visual diff mismatch
+
+### Strategy
+1. Compare expected vs actual from error output
+2. Determine if change is intentional (product update) or a bug
+3. If intentional: update expected value in test
+4. If unintentional: mark as fixme with "Product regression: expected X, got Y"
+
+### Before
+```typescript
+await expect(page.getByRole('heading')).toHaveText('Welcome, User');
+```
+
+### After (intentional product change)
+```typescript
+await expect(page.getByRole('heading')).toHaveText('Welcome back!');
+```
+
+### After (unintentional — mark fixme)
+```typescript
+test.fixme('Product regression: heading shows "Error" instead of "Welcome, User"');
+```
+
+---
+
+## 3. Timeout Issues
+
+### Symptoms
+- `Timeout 5000ms exceeded`
+- `Waiting for locator`
+- Element appears but after a long delay
+
+### Strategy
+1. Identify what is slow: network, animation, lazy load?
+2. Add explicit wait for the condition (prefer `expect` with timeout over `waitForTimeout`)
+3. Increase timeout only when necessary (e.g., known slow API)
+
+### Before
+```typescript
+await page.goto('/dashboard');
+await page.getByText('Data loaded').click(); // Fails: element appears after 3s
+```
+
+### After
+```typescript
+await page.goto('/dashboard');
+await expect(page.getByText('Data loaded')).toBeVisible({ timeout: 10000 });
+await page.getByText('Data loaded').click();
+```
+
+### For slow API
+```typescript
+await expect(page.getByRole('status', { name: 'Loaded' })).toBeVisible({ timeout: 15000 });
+```
+
+---
+
+## 4. Missing Elements
+
+### Symptoms
+- `Locator not found`
+- Element was removed from DOM
+- Feature redesigned
+
+### Strategy
+1. Take snapshot; confirm element is absent
+2. Check if feature was removed or moved (e.g., new modal, different page)
+3. If moved: update navigation/flow and locator
+4. If removed: mark test as fixme with "Feature removed: [description]"
+5. If uncertain: mark fixme with "Manual review: element not found"
+
+### Before
+```typescript
+await page.getByRole('button', { name: 'Old CTA' }).click();
+```
+
+### After (feature removed)
+```typescript
+test.fixme('Feature removed: "Old CTA" button no longer exists in UI');
+```
+
+### After (element moved)
+```typescript
+await page.getByRole('link', { name: 'Settings' }).click();
+await page.getByRole('button', { name: 'New CTA' }).click();
+```
+
+---
+
+## 5. Network Changes
+
+### Symptoms
+- API returns different status/body
+- Mock route no longer matches
+- Request timeout or 404
+
+### Strategy
+1. Inspect network requests via `browser_network_requests` or test logs
+2. Update `page.route` URL pattern if endpoint changed
+3. Update mock response body/status if API contract changed
+4. Add retry logic for flaky endpoints (use sparingly)
+
+### Before
+```typescript
+await page.route('**/api/users', route => route.fulfill({ status: 200, body: '[]' }));
+```
+
+### After (endpoint changed)
+```typescript
+await page.route('**/api/v2/users', route => route.fulfill({ status: 200, body: '[]' }));
+```
+
+### After (response shape changed)
+```typescript
+await page.route('**/api/users', route => route.fulfill({
+  status: 200,
+  body: JSON.stringify({ data: [] }),
+}));
+```
+
+---
+
+## Locator Priority (Stability)
+
+When replacing selectors, prefer in this order:
+
+1. **getByRole** — Most resilient to DOM structure changes
+2. **getByTestId** — Explicit, requires `data-testid` in app
+3. **getByLabel** — Good for form fields
+4. **getByText** — Use for unique visible text
+5. **CSS/XPath** — Last resort; brittle

package/skills/qa-test-reporter/SKILL.md

@@ -0,0 +1,130 @@
+---
+name: qa-test-reporter
+description: Aggregate test results from all frameworks, parse JUnit XML/JSON/Allure reports, generate HTML/markdown dashboards with trends, and produce ISO 29119-3 compliant test status and completion reports.
+output_dir: reports/test-runs
+---
+
+# QA Test Reporter
+
+## Purpose
+
+Aggregate, analyze, and report test results across all testing types and frameworks. Parse outputs from Jest, Vitest, Playwright, pytest, Cypress, k6, JMeter, and other tools; produce unified dashboards, trend analysis, and ISO 29119-3 compliant reports with go/no-go recommendations.
+
+## Input Formats
+
+| Format | Source | Key Fields |
+| ------ | ------ | ---------- |
+| **JUnit XML** | Jest, pytest, Maven, Gradle, most CI runners | `testsuites`, `testsuite`, `testcase`, `failure`, `skipped`, `time` |
+| **JSON reports** | Custom runners, Vitest, Playwright JSON reporter | Varies; see `references/report-formats.md` |
+| **Allure results** | Allure adapter (any framework) | `result.json`, `container.json`, attachments |
+| **CI/CD output** | GitHub Actions, GitLab CI, Jenkins | Job logs, artifact paths, workflow status |
+| **Coverage reports** | Istanbul, c8, pytest-cov, JaCoCo | Line/branch/function coverage JSON |
+
+See `references/report-formats.md` for schemas and parsing rules.
+
+## Output Formats
+
+| Output | Description |
+| ------ | ----------- |
+| **HTML report** | Interactive dashboard with pass/fail/skip charts, drill-down by category/priority/type |
+| **Markdown summary** | Concise summary for PR comments, Slack, or documentation |
+| **ISO 29119-3 Test Status Report** | Periodic progress report during execution |
+| **ISO 29119-3 Test Completion Report** | Final report with exit criteria and recommendations |
+| **Executive summary** | Go/no-go recommendation with risk areas and open defects |
+
+## Report Sections
+
+Every report includes (where data is available):
+
+1. **Summary metrics** — Total pass/fail/skip, duration, pass rate %
+2. **Test results by category** — By suite, module, priority, or test type
+3. **Failure analysis** — Top failures by frequency, flaky test detection
+4. **Coverage summary** — Line/branch/function coverage if provided
+5. **Risk areas** — Low coverage, high failure density, blocked tests
+6. **Open defects** — Linked incidents, severity distribution
+7. **Go/no-go recommendation** — Clear recommendation with rationale
+
+## Trend Analysis
+
+Track pass/fail rates over time using **Memory MCP** for history persistence:
+
+- Store run metadata: date, build ID, pass rate, fail count, duration
+- Compare current run vs. previous runs (e.g., last 7 days, last 10 builds)
+- Surface regressions: pass rate drop, new flaky tests, duration increase
+- Use stored trends in executive summary and status reports
+
+## Integrations
+
+| Integration | Use |
+| ----------- | --- |
+| **GitHub MCP** | Fetch Actions workflow runs, job logs, artifact URLs for test results |
+| **Memory MCP** | Persist and retrieve trend data for historical comparison |
+| **qa-diagram-generator** | Generate trend charts (line, bar) for HTML dashboards |
+
+## Trigger Phrases
+
+- "Aggregate test results from [paths/artifacts]"
+- "Generate test report from JUnit XML / Allure / JSON"
+- "Create HTML test dashboard"
+- "ISO 29119-3 test status report" / "test completion report"
+- "Go/no-go recommendation for release"
+- "Test trend analysis" / "pass rate over time"
+- "Parse GitHub Actions test results"
+
+## Workflow
+
+1. **Collect** — Parse one or more report files (JUnit XML, JSON, Allure) or fetch from GitHub Actions
+2. **Normalize** — Map to common model: test name, suite, result, duration, failure message
+3. **Analyze** — Compute metrics, detect flaky tests, identify top failures
+4. **Trend** — Query Memory MCP for history; compare with current run
+5. **Generate** — Produce requested outputs (HTML, Markdown, ISO reports, executive summary)
+
+## Scope
+
+**Can do (autonomous):**
+- Parse JUnit XML, JSON, Allure results from provided paths or URLs
+- Aggregate results from multiple frameworks into unified view
+- Generate HTML dashboard, Markdown summary, ISO 29119-3 reports
+- Produce go/no-go recommendation from metrics and risk areas
+- Use Memory MCP to store/retrieve trend data
+- Use GitHub MCP to fetch Actions results when configured
+- Call qa-diagram-generator for trend charts
+
+**Cannot do (requires confirmation):**
+- Fetch from private repos or artifacts without access
+- Override stakeholder-defined go/no-go criteria
+- Modify test execution or CI configuration
+
+**Will not do (out of scope):**
+- Execute tests or automation
+- Modify production code or environments
+- Approve release (approval is stakeholder responsibility)
+
+## Quality Checklist
+
+- [ ] All input formats correctly parsed per `references/report-formats.md`
+- [ ] Summary metrics accurate (pass/fail/skip counts match source)
+- [ ] ISO 29119-3 reports include mandatory sections per `references/iso-29119-reports.md`
+- [ ] Go/no-go recommendation supported by metrics and risk analysis
+- [ ] Trend data stored/retrieved correctly via Memory MCP
+- [ ] No hardcoded secrets; artifact URLs from env or user input
+- [ ] HTML dashboard accessible (semantic structure, readable charts)
+
+## Troubleshooting
+
+| Symptom | Likely Cause | Fix |
+| ------- | ------------ | --- |
+| Parse errors on JUnit XML | Non-standard or malformed XML | Validate against schema in `references/report-formats.md`; handle missing attributes |
+| Missing trend data | Memory MCP not configured or empty | Verify MCP config; run without trends or prompt for manual history |
+| GitHub Actions fetch fails | Token missing, private repo | Check GitHub MCP auth; use local artifact paths instead |
+| Flaky detection inaccurate | Insufficient run history | Require minimum runs (e.g., 5+) before flagging flaky |
+| ISO report incomplete | Mandatory sections missing | Map available data to template; flag gaps for user input |
+| Go/no-go unclear | Conflicting metrics | Document rationale; present risk areas for stakeholder decision |
+
+## Reference Files
+
+| Topic | File |
+| ----- | ---- |
+| Report format schemas (JUnit, Allure, coverage) | `references/report-formats.md` |
+| ISO 29119-3 status & completion report templates | `references/iso-29119-reports.md` |
+| Reporting best practices | `references/best-practices.md` |