qa-skills 3.0.0 → 3.1.0

package/skills/qa-test-healer/SKILL.md

@@ -1,101 +0,0 @@
----
-name: qa-test-healer
-description: Self-healing test system that runs failing tests, debugs with MCP tools, auto-fixes broken selectors/assertions/waits, and marks unfixable tests as test.fixme() with explanation.
----
-
-# QA Test Healer
-
-## Purpose
-
-Automatically fix broken tests after code changes. Inspired by fugazi/test-automation-skills-agents, this skill runs failing tests in headed/debug mode, analyzes failures, applies targeted fixes, and either restores passing tests or marks unfixable cases as `test.fixme()` with a clear explanation.
-
-## Trigger Phrases
-
-- "Heal my failing tests"
-- "Fix broken Playwright/Cypress tests"
-- "Auto-fix test failures"
-- "Tests are failing after my changes"
-- "Debug and fix selector/assertion failures"
-- "Run failing test and fix it"
-
-## Healing Workflow
-
-1. **Receive failing test(s)** — User provides failing test path(s), error output, or CI failure logs
-2. **Run test in headed mode / with debug output** — Execute the test with visible browser and verbose logging to capture failure context
-3. **Analyze failure** — Determine root cause: selector broken? assertion changed? timeout? missing element? network change?
-4. **Apply fix strategy** — Use the appropriate strategy from `references/fix-strategies.md`
-5. **Re-run to verify fix** — Execute the test again; confirm it passes
-6. **If unfixable** — Mark as `test.fixme('explanation')` with a clear reason for manual review
-
-## Fix Strategies
-
-| Failure Type | Strategy | Action |
-|--------------|----------|--------|
-| **Broken selectors** | Find new locator | Use accessibility snapshot (Playwright MCP) to inspect DOM; update POM/test with stable locator (getByRole, getByTestId preferred) |
-| **Changed assertions** | Compare expected vs actual | If intentional product change, update expected values; otherwise flag for review |
-| **Timeout issues** | Add explicit waits | Increase timeout thresholds, add `expect` with timeout, or use `waitFor` for async content |
-| **Missing elements** | Check feature status | Determine if feature was removed/redesigned; flag for manual review or mark fixme |
-| **Network changes** | Update mocks/routes | Update intercepted routes, mocks, or add retry logic for flaky endpoints |
-
-See `references/fix-strategies.md` for detailed before/after examples.
-
-## Diagnosis Patterns
-
-Use `references/diagnosis-patterns.md` for:
-
-- **Error message parsing** — Extract selector, assertion, timeout from stack traces and failure output
-- **DOM comparison** — Use browser snapshot to compare expected vs actual structure
-- **Network analysis** — Inspect requests/responses when API or mock changes cause failures
-
-## MCP Integration
-
-- **Playwright MCP (cursor-ide-browser)** — Run tests, navigate to failure state, take accessibility snapshots, inspect DOM, verify fixes
-- **Lock/unlock workflow** — Lock browser before interactions; unlock when done
-
-## Scope
-
-**Can do (autonomous):**
-- Fix broken selectors by finding new locators via DOM inspection
-- Update assertions when expected values have legitimately changed
-- Add or adjust waits and timeouts for flaky/timeout failures
-- Update network mocks, routes, or retry logic
-- Mark unfixable tests as `test.fixme()` with explanation
-- Run tests in headed mode to reproduce and verify fixes
-
-**Cannot do (requires confirmation):**
-- Rewrite test logic or change test intent
-- Add new test cases or modify test structure significantly
-- Change production code
-
-**Will not do (out of scope):**
-- Modify production/application code
-- Execute tests in CI without user request
-- Approve or merge code changes
-
-## References
-
-| Topic | File |
-|-------|------|
-| Detailed fix strategies with before/after examples | `references/fix-strategies.md` |
-| Error parsing, DOM comparison, network analysis | `references/diagnosis-patterns.md` |
-
-## Quality Checklist
-
-- [ ] Root cause correctly identified before applying fix
-- [ ] Fix strategy matches failure type per `references/fix-strategies.md`
-- [ ] Test passes after fix (re-run verified)
-- [ ] Unfixable tests marked with `test.fixme('clear explanation')`
-- [ ] No production code modified
-- [ ] POM updated if selector changed in shared locator
-- [ ] Traceability preserved (test case IDs, comments)
-
-## Troubleshooting
-
-| Symptom | Likely Cause | Fix |
-|---------|--------------|-----|
-| Can't reproduce failure locally | Environment difference, flaky test | Run with `--retries=0`; check CI env vars; use same baseURL |
-| Snapshot shows different structure | Dynamic content, A/B test | Wait for stable state; use more resilient locators |
-| Fix works once then fails | Race condition, shared state | Add proper wait; ensure test isolation |
-| Multiple failures in one file | Cascading failure, setup issue | Fix setup/fixture first; run tests individually |
-| test.fixme() not recognized | Framework difference | Use `test.skip()` or equivalent for Cypress/pytest |
-| MCP browser not available | Playwright MCP not configured | Use terminal to run tests; analyze error output only |

package/skills/qa-test-healer/references/diagnosis-patterns.md

@@ -1,142 +0,0 @@
-# Diagnosis Patterns for Test Failures
-
-Patterns for diagnosing test failures: error message parsing, DOM comparison, and network analysis.
-
-## 1. Error Message Parsing
-
-### Playwright Error Structure
-
-Playwright failures typically include:
-- **Error type** — `TimeoutError`, `Error`, `AssertionError`
-- **Locator** — The selector that failed
-- **Action** — click, fill, toBeVisible, etc.
-- **Expected vs actual** — For assertions
-
-### Extraction Patterns
-
-| Pattern | Regex/Approach | Use |
-|---------|----------------|-----|
-| Selector in error | `locator\('([^']+)'\)` or `getByRole\('([^']+)'` | Identify which locator failed |
-| Timeout value | `Timeout (\d+)ms` | Check if timeout is too short |
-| Expected text | `Expected: (.*)` | Compare with actual |
-| Actual text | `Received: (.*)` | Compare with expected |
-| Stack trace file:line | `at.*\(([^:]+):(\d+):\d+\)` | Locate exact line in test file |
-
-### Example Parse
-
-```
-Error: expect(locator).toHaveText(expected)
-Expected: "Welcome"
-Received: "Welcome back!"
-```
-
-**Extracted:** Assertion `toHaveText`; expected `"Welcome"`; actual `"Welcome back!"` → Likely intentional product change.
-
-### Cypress Error Structure
-
-- `cy.get(...).should(...)` — Selector and assertion in chain
-- `Timed out retrying` — Timeout/visibility issue
-- `Expected X to equal Y` — Assertion mismatch
-
-### pytest/Playwright Python
-
-- `TimeoutError` with locator repr
-- `AssertionError: assert X == Y`
-
----
-
-## 2. DOM Comparison
-
-### Using Accessibility Snapshot
-
-1. **Navigate to failure state** — Run test in headed mode; stop at failure or use `page.pause()`
-2. **Take snapshot** — `browser_snapshot` (Playwright MCP) returns accessibility tree
-3. **Compare expected vs actual** — Look for:
-   - Missing elements (role/name not in tree)
-   - Renamed elements (text changed)
-   - Structure change (nested differently)
-   - New overlays (modal, loading) blocking interaction
-
-### Snapshot Analysis Checklist
-
-- [ ] Is the target element present? (search by role, name, or ref)
-- [ ] Is it visible/enabled? (check `aria-hidden`, `disabled`)
-- [ ] Is it obscured? (modal, overlay, another element on top)
-- [ ] Has the hierarchy changed? (element moved under different parent)
-- [ ] Are there multiple matches? (strict mode violation)
-
-### Interpreting Snapshot Output
-
-```
-button "Submit" [ref=abc123]
-  - enabled, visible
-```
-
-If test expects `button "Save"` but snapshot shows `button "Submit"` → Update locator or mark as product change.
-
-### Scoped Snapshots
-
-Use `selector` parameter to snapshot a subtree (e.g., a form or modal) when the full page is large.
-
----
-
-## 3. Network Analysis
-
-### When to Use
-
-- Test fails after API call
-- Mock not applied
-- Request timeout or 404
-- Response body/status changed
-
-### Inspection Steps
-
-1. **List requests** — `browser_network_requests` (Playwright MCP) or test trace
-2. **Filter by URL** — Find the request that failed or returned unexpected data
-3. **Check method/URL** — Did endpoint path change? (e.g., `/api/users` → `/api/v2/users`)
-4. **Check response** — Status code, body shape
-5. **Check timing** — Request took too long?
-
-### Common Network Failure Causes
-
-| Cause | Symptom | Fix |
-|-------|---------|-----|
-| Endpoint renamed | 404 | Update `page.route` URL pattern |
-| Response shape changed | Assertion on body fails | Update mock response or assertion |
-| Request not sent | No matching request | Ensure route registered before `goto` |
-| Slow API | Timeout | Increase timeout or add retry |
-| CORS/auth | Request blocked | Update route handler or headers |
-
-### Route Registration Order
-
-```typescript
-// Correct: route before navigation
-await page.route('**/api/data', handler);
-await page.goto('/page');
-```
-
-```typescript
-// Wrong: navigation may complete before route is set
-await page.goto('/page');
-await page.route('**/api/data', handler);
-```
-
----
-
-## 4. Combining Diagnosis Methods
-
-For complex failures, combine:
-
-1. **Parse error** → Get selector, assertion, expected/actual
-2. **Snapshot DOM** → Verify element exists and state
-3. **Check network** → If failure involves API, verify requests
-
-### Decision Flow
-
-```
-Parse error
-  → Selector timeout? → Snapshot: is element present? → Yes: increase timeout / add wait
-  → Selector timeout? → Snapshot: element missing? → Fixme or update flow
-  → Assertion mismatch? → Compare expected vs actual → Update or fixme
-  → Network error? → Inspect requests → Update route/mock
-```

package/skills/qa-test-healer/references/fix-strategies.md

@@ -1,177 +0,0 @@
-# Fix Strategies for Test Healing
-
-Detailed strategies for each failure type with before/after examples.
-
-## 1. Broken Selectors
-
-### Symptoms
-- `TimeoutError: locator resolved to X elements` or `strict mode violation`
-- `Error: No element found for selector`
-- `Locator.click: Target closed`
-
-### Strategy
-1. Run test in headed mode; pause at failure point
-2. Take accessibility snapshot via Playwright MCP (`browser_snapshot`)
-3. Identify the target element in the snapshot (role, name, testid)
-4. Choose stable locator: `getByRole` > `getByTestId` > `getByLabel` > `getByText` > CSS
-5. Update POM or test file with new locator
-
-### Before
-```typescript
-await page.click('button.submit-btn');
-```
-
-### After
-```typescript
-await page.getByRole('button', { name: 'Submit' }).click();
-```
-
-### POM Update
-If the selector lives in a Page Object, update the locator there so all tests using it benefit.
-
-```typescript
-// Before (LoginPage.ts)
-get submitButton() { return this.page.locator('button.submit-btn'); }
-
-// After
-get submitButton() { return this.page.getByRole('button', { name: 'Submit' }); }
-```
-
----
-
-## 2. Changed Assertions
-
-### Symptoms
-- `expect(locator).toHaveText(expected)` — actual text differs
-- `expect(locator).toHaveValue(expected)` — value mismatch
-- Snapshot/visual diff mismatch
-
-### Strategy
-1. Compare expected vs actual from error output
-2. Determine if change is intentional (product update) or a bug
-3. If intentional: update expected value in test
-4. If unintentional: mark as fixme with "Product regression: expected X, got Y"
-
-### Before
-```typescript
-await expect(page.getByRole('heading')).toHaveText('Welcome, User');
-```
-
-### After (intentional product change)
-```typescript
-await expect(page.getByRole('heading')).toHaveText('Welcome back!');
-```
-
-### After (unintentional — mark fixme)
-```typescript
-test.fixme('Product regression: heading shows "Error" instead of "Welcome, User"');
-```
-
----
-
-## 3. Timeout Issues
-
-### Symptoms
-- `Timeout 5000ms exceeded`
-- `Waiting for locator`
-- Element appears but after a long delay
-
-### Strategy
-1. Identify what is slow: network, animation, lazy load?
-2. Add explicit wait for the condition (prefer `expect` with timeout over `waitForTimeout`)
-3. Increase timeout only when necessary (e.g., known slow API)
-
-### Before
-```typescript
-await page.goto('/dashboard');
-await page.getByText('Data loaded').click(); // Fails: element appears after 3s
-```
-
-### After
-```typescript
-await page.goto('/dashboard');
-await expect(page.getByText('Data loaded')).toBeVisible({ timeout: 10000 });
-await page.getByText('Data loaded').click();
-```
-
-### For slow API
-```typescript
-await expect(page.getByRole('status', { name: 'Loaded' })).toBeVisible({ timeout: 15000 });
-```
-
----
-
-## 4. Missing Elements
-
-### Symptoms
-- `Locator not found`
-- Element was removed from DOM
-- Feature redesigned
-
-### Strategy
-1. Take snapshot; confirm element is absent
-2. Check if feature was removed or moved (e.g., new modal, different page)
-3. If moved: update navigation/flow and locator
-4. If removed: mark test as fixme with "Feature removed: [description]"
-5. If uncertain: mark fixme with "Manual review: element not found"
-
-### Before
-```typescript
-await page.getByRole('button', { name: 'Old CTA' }).click();
-```
-
-### After (feature removed)
-```typescript
-test.fixme('Feature removed: "Old CTA" button no longer exists in UI');
-```
-
-### After (element moved)
-```typescript
-await page.getByRole('link', { name: 'Settings' }).click();
-await page.getByRole('button', { name: 'New CTA' }).click();
-```
-
----
-
-## 5. Network Changes
-
-### Symptoms
-- API returns different status/body
-- Mock route no longer matches
-- Request timeout or 404
-
-### Strategy
-1. Inspect network requests via `browser_network_requests` or test logs
-2. Update `page.route` URL pattern if endpoint changed
-3. Update mock response body/status if API contract changed
-4. Add retry logic for flaky endpoints (use sparingly)
-
-### Before
-```typescript
-await page.route('**/api/users', route => route.fulfill({ status: 200, body: '[]' }));
-```
-
-### After (endpoint changed)
-```typescript
-await page.route('**/api/v2/users', route => route.fulfill({ status: 200, body: '[]' }));
-```
-
-### After (response shape changed)
-```typescript
-await page.route('**/api/users', route => route.fulfill({
-  status: 200,
-  body: JSON.stringify({ data: [] }),
-}));
-```
-
----
-
-## Locator Priority (Stability)
-
-When replacing selectors, prefer in this order:
-
-1. **getByRole** — Most resilient to DOM structure changes
-2. **getByTestId** — Explicit, requires `data-testid` in app
-3. **getByLabel** — Good for form fields
-4. **getByText** — Use for unique visible text
-5. **CSS/XPath** — Last resort; brittle

package/skills/qa-test-reporter/SKILL.md

@@ -1,130 +0,0 @@
----
-name: qa-test-reporter
-description: Aggregate test results from all frameworks, parse JUnit XML/JSON/Allure reports, generate HTML/markdown dashboards with trends, and produce ISO 29119-3 compliant test status and completion reports.
-output_dir: reports/test-runs
----
-
-# QA Test Reporter
-
-## Purpose
-
-Aggregate, analyze, and report test results across all testing types and frameworks. Parse outputs from Jest, Vitest, Playwright, pytest, Cypress, k6, JMeter, and other tools; produce unified dashboards, trend analysis, and ISO 29119-3 compliant reports with go/no-go recommendations.
-
-## Input Formats
-
-| Format | Source | Key Fields |
-| ------ | ------ | ---------- |
-| **JUnit XML** | Jest, pytest, Maven, Gradle, most CI runners | `testsuites`, `testsuite`, `testcase`, `failure`, `skipped`, `time` |
-| **JSON reports** | Custom runners, Vitest, Playwright JSON reporter | Varies; see `references/report-formats.md` |
-| **Allure results** | Allure adapter (any framework) | `result.json`, `container.json`, attachments |
-| **CI/CD output** | GitHub Actions, GitLab CI, Jenkins | Job logs, artifact paths, workflow status |
-| **Coverage reports** | Istanbul, c8, pytest-cov, JaCoCo | Line/branch/function coverage JSON |
-
-See `references/report-formats.md` for schemas and parsing rules.
-
-## Output Formats
-
-| Output | Description |
-| ------ | ----------- |
-| **HTML report** | Interactive dashboard with pass/fail/skip charts, drill-down by category/priority/type |
-| **Markdown summary** | Concise summary for PR comments, Slack, or documentation |
-| **ISO 29119-3 Test Status Report** | Periodic progress report during execution |
-| **ISO 29119-3 Test Completion Report** | Final report with exit criteria and recommendations |
-| **Executive summary** | Go/no-go recommendation with risk areas and open defects |
-
-## Report Sections
-
-Every report includes (where data is available):
-
-1. **Summary metrics** — Total pass/fail/skip, duration, pass rate %
-2. **Test results by category** — By suite, module, priority, or test type
-3. **Failure analysis** — Top failures by frequency, flaky test detection
-4. **Coverage summary** — Line/branch/function coverage if provided
-5. **Risk areas** — Low coverage, high failure density, blocked tests
-6. **Open defects** — Linked incidents, severity distribution
-7. **Go/no-go recommendation** — Clear recommendation with rationale
-
-## Trend Analysis
-
-Track pass/fail rates over time using **Memory MCP** for history persistence:
-
-- Store run metadata: date, build ID, pass rate, fail count, duration
-- Compare current run vs. previous runs (e.g., last 7 days, last 10 builds)
-- Surface regressions: pass rate drop, new flaky tests, duration increase
-- Use stored trends in executive summary and status reports
-
-## Integrations
-
-| Integration | Use |
-| ----------- | --- |
-| **GitHub MCP** | Fetch Actions workflow runs, job logs, artifact URLs for test results |
-| **Memory MCP** | Persist and retrieve trend data for historical comparison |
-| **qa-diagram-generator** | Generate trend charts (line, bar) for HTML dashboards |
-
-## Trigger Phrases
-
-- "Aggregate test results from [paths/artifacts]"
-- "Generate test report from JUnit XML / Allure / JSON"
-- "Create HTML test dashboard"
-- "ISO 29119-3 test status report" / "test completion report"
-- "Go/no-go recommendation for release"
-- "Test trend analysis" / "pass rate over time"
-- "Parse GitHub Actions test results"
-
-## Workflow
-
-1. **Collect** — Parse one or more report files (JUnit XML, JSON, Allure) or fetch from GitHub Actions
-2. **Normalize** — Map to common model: test name, suite, result, duration, failure message
-3. **Analyze** — Compute metrics, detect flaky tests, identify top failures
-4. **Trend** — Query Memory MCP for history; compare with current run
-5. **Generate** — Produce requested outputs (HTML, Markdown, ISO reports, executive summary)
-
-## Scope
-
-**Can do (autonomous):**
-- Parse JUnit XML, JSON, Allure results from provided paths or URLs
-- Aggregate results from multiple frameworks into unified view
-- Generate HTML dashboard, Markdown summary, ISO 29119-3 reports
-- Produce go/no-go recommendation from metrics and risk areas
-- Use Memory MCP to store/retrieve trend data
-- Use GitHub MCP to fetch Actions results when configured
-- Call qa-diagram-generator for trend charts
-
-**Cannot do (requires confirmation):**
-- Fetch from private repos or artifacts without access
-- Override stakeholder-defined go/no-go criteria
-- Modify test execution or CI configuration
-
-**Will not do (out of scope):**
-- Execute tests or automation
-- Modify production code or environments
-- Approve release (approval is stakeholder responsibility)
-
-## Quality Checklist
-
-- [ ] All input formats correctly parsed per `references/report-formats.md`
-- [ ] Summary metrics accurate (pass/fail/skip counts match source)
-- [ ] ISO 29119-3 reports include mandatory sections per `references/iso-29119-reports.md`
-- [ ] Go/no-go recommendation supported by metrics and risk analysis
-- [ ] Trend data stored/retrieved correctly via Memory MCP
-- [ ] No hardcoded secrets; artifact URLs from env or user input
-- [ ] HTML dashboard accessible (semantic structure, readable charts)
-
-## Troubleshooting
-
-| Symptom | Likely Cause | Fix |
-| ------- | ------------ | --- |
-| Parse errors on JUnit XML | Non-standard or malformed XML | Validate against schema in `references/report-formats.md`; handle missing attributes |
-| Missing trend data | Memory MCP not configured or empty | Verify MCP config; run without trends or prompt for manual history |
-| GitHub Actions fetch fails | Token missing, private repo | Check GitHub MCP auth; use local artifact paths instead |
-| Flaky detection inaccurate | Insufficient run history | Require minimum runs (e.g., 5+) before flagging flaky |
-| ISO report incomplete | Mandatory sections missing | Map available data to template; flag gaps for user input |
-| Go/no-go unclear | Conflicting metrics | Document rationale; present risk areas for stakeholder decision |
-
-## Reference Files
-
-| Topic | File |
-| ----- | ---- |
-| Report format schemas (JUnit, Allure, coverage) | `references/report-formats.md` |
-| ISO 29119-3 status & completion report templates | `references/iso-29119-reports.md` |
-| Reporting best practices | `references/best-practices.md` |