qa-skills 3.0.0

package/skills/qa-codeceptjs-writer/references/patterns.md

@@ -0,0 +1,285 @@
+# CodeceptJS Test Patterns
+
+Common patterns for scenario-driven E2E testing with CodeceptJS.
+
+## Scenario Syntax
+
+### Feature and Scenario Blocks
+
+```typescript
+Feature('Login');
+
+Scenario('user can log in with valid credentials', ({ I }) => {
+  I.amOnPage('/login');
+  I.fillField('Username', 'john');
+  I.fillField('Password', 'secret');
+  I.click('Sign In');
+  I.see('Welcome');
+});
+```
+
+### Common I Actions
+
+| Action | Example |
+|--------|---------|
+| Navigate | `I.amOnPage('/')` or `I.amOnPage('https://example.com')` |
+| Click | `I.click('Submit')` or `I.click('.btn-primary')` |
+| Fill field | `I.fillField('Username', 'john')` or `I.fillField('#email', 'a@b.com')` |
+| See text | `I.see('Welcome')` or `I.see('Dashboard', 'h1')` |
+| See element | `I.seeElement('.user-menu')` |
+| Grab text | `const text = await I.grabTextFrom('.title')` |
+| Select option | `I.selectOption('Country', 'US')` |
+| Check checkbox | `I.checkOption('Subscribe')` |
+| Wait | `I.wait(2)` or `I.waitForVisible('.loaded')` |
+
+### Locator Strategies
+
+CodeceptJS resolves locators in order: ID, CSS, XPath, name attribute, label text, link/button text.
+
+```typescript
+// By text (link/button)
+I.click('Login');
+I.click('Submit');
+
+// By label (for fillField)
+I.fillField('Email', 'user@example.com');
+
+// By CSS
+I.click('.submit-btn');
+I.seeElement('#header');
+
+// By XPath
+I.click('//button[contains(., "Save")]');
+
+// Strict locators (explicit type)
+I.seeElement({ css: 'div.user' });
+I.click({ xpath: '//button[@type="submit"]' });
+```
+
+## within() for Scoped Locators
+
+Scope actions to a specific container:
+
+```typescript
+Scenario('interact within modal', ({ I }) => {
+  I.amOnPage('/');
+  I.click('Open modal');
+  within('.modal', () => {
+    I.fillField('Name', 'Test');
+    I.click('Save');
+  });
+  I.see('Saved');
+});
+```
+
+## BDD with Gherkin
+
+### Feature File (.feature)
+
+```gherkin
+Feature: Checkout
+  In order to buy products
+  As a customer
+  I need to be able to checkout the selected products
+
+  Scenario: order several products
+    Given I have product with $600 price in my cart
+    And I have product with $1000 price in my cart
+    When I go to checkout process
+    Then I should see that total number of products is 2
+    And my order amount is $1600
+```
+
+### Step Definitions
+
+```typescript
+// step_definitions/checkout_steps.ts
+const { I, productPage } = inject();
+
+Given(/I have product with \$(\d+) price in my cart/, async (price) => {
+  I.amOnPage('/products');
+  await productPage.addProduct({ price: parseInt(price) });
+  I.click('Add to cart');
+});
+
+When('I go to checkout process', () => {
+  I.click('Checkout');
+});
+
+Then('I should see that total number of products is {int}', (num) => {
+  I.see(num.toString(), '.cart-count');
+});
+
+Then('my order amount is {int}', (sum) => {
+  I.see(`$${sum}`, '.order-total');
+});
+```
+
+### Background and Scenario Outline
+
+```gherkin
+Feature: Dashboard
+
+  Background:
+    Given I am logged in as administrator
+    And I open dashboard page
+
+  Scenario Outline: view report for role
+    When I select role "<role>"
+    Then I should see "<report>" report
+    Examples:
+      | role   | report    |
+      | admin  | Full      |
+      | user   | Limited   |
+```
+
+## Page Objects
+
+### Define Page Object
+
+```typescript
+// pages/LoginPage.ts
+export default class LoginPage {
+  constructor(private I: CodeceptJS.I) {}
+
+  async login(email: string, password: string) {
+    this.I.fillField('Email', email);
+    this.I.fillField('Password', password);
+    this.I.click('Sign In');
+  }
+
+  async open() {
+    this.I.amOnPage('/login');
+  }
+
+  get errorMessage() {
+    return '.alert-error';
+  }
+}
+```
+
+### Use in Tests (inject)
+
+```typescript
+// login_test.ts
+const { I, loginPage } = inject();
+
+Scenario('login flow', () => {
+  loginPage.open();
+  loginPage.login('user@example.com', 'secret');
+  I.see('Dashboard');
+});
+```
+
+### Use in Step Definitions
+
+```typescript
+const { I, loginPage } = inject();
+
+Given('I am logged in', () => {
+  loginPage.open();
+  loginPage.login('admin@example.com', 'admin123');
+});
+```
+
+## Data-Driven Testing
+
+### Data().Scenario
+
+```typescript
+const accounts = new DataTable(['login', 'password']);
+accounts.add(['davert', '123456']);
+accounts.add(['admin', 'admin123']);
+
+Data(accounts).Scenario('Test Login', ({ I, current }) => {
+  I.amOnPage('/login');
+  I.fillField('Username', current.login);
+  I.fillField('Password', current.password);
+  I.click('Sign In');
+  I.see('Welcome ' + current.login);
+});
+```
+
+### Data from Array or Generator
+
+```typescript
+Data(['chrome', 'firefox']).Scenario('cross-browser', ({ I, current }) => {
+  I.amOnPage('/');
+  I.see('Home');
+});
+```
+
+## Custom Helpers
+
+### Define Custom Helper
+
+```typescript
+// helpers/MyHelper.ts
+import { Helper } from 'codeceptjs';
+
+class MyHelper extends Helper {
+  async loginAs(user: string) {
+    const { I } = this.helpers;
+    await I.amOnPage('/login');
+    await I.fillField('Username', user);
+    await I.fillField('Password', 'secret');
+    await I.click('Sign In');
+  }
+}
+
+export = MyHelper;
+```
+
+### Use Custom Helper
+
+```typescript
+// In codecept.conf.ts
+helpers: {
+  Playwright: { ... },
+  MyHelper: {}
+}
+
+// In test
+const { I, MyHelper } = inject();
+Scenario('test', async () => {
+  await MyHelper.loginAs('admin');
+  I.see('Dashboard');
+});
+```
+
+## pause() for Debugging
+
+```typescript
+Scenario('debug flow', ({ I }) => {
+  I.amOnPage('/');
+  I.fillField('Search', 'test');
+  pause(); // Enters interactive console; inspect state, run I.* commands
+});
+```
+
+## Forms and Validation
+
+```typescript
+Scenario('submit registration form', ({ I }) => {
+  I.amOnPage('/register');
+  I.fillField('Email', 'new@example.com');
+  I.fillField('Password', 'SecurePass123');
+  I.checkOption('Terms');
+  I.selectOption('Country', 'United States');
+  I.click('Create account');
+  I.see('Welcome');
+});
+```
+
+## Async and Promises
+
+```typescript
+Scenario('grab and assert', async ({ I }) => {
+  I.amOnPage('/dashboard');
+  const count = await I.grabTextFrom('.item-count');
+  const num = parseInt(count, 10);
+  if (num > 0) {
+    I.seeElement('.items-list');
+  }
+});
+```

package/skills/qa-coverage-analyzer/SKILL.md

@@ -0,0 +1,166 @@
+---
+name: qa-coverage-analyzer
+description: Analyze test coverage across three dimensions -- requirements/model coverage via RTM, technique coverage per ISO 29119-4, and code coverage via Istanbul/V8/JaCoCo -- with risk-based gap recommendations.
+output_dir: reports/coverage
+---
+
+# QA Coverage Analyzer
+
+## Purpose
+
+Perform multi-dimensional test coverage analysis across requirements traceability, test design techniques, and code coverage. Identify gaps, prioritize by risk, and produce actionable recommendations for improving test effectiveness.
+
+## Trigger Phrases
+
+- "Analyze test coverage" / "Coverage analysis for [project/module]"
+- "Requirements traceability matrix" / "RTM coverage"
+- "Technique coverage" / "ISO 29119-4 coverage"
+- "Code coverage report" / "Line/branch coverage analysis"
+- "Coverage gaps" / "Uncovered requirements"
+- "Coverage dashboard" / "Coverage heatmap"
+- "Risk-based coverage recommendations"
+
+## Three Coverage Dimensions
+
+| Dimension | What It Measures | Key Artifacts |
+|-----------|------------------|----------------|
+| **1. Requirements/Model** | Which requirements have tests; RTM completeness | RTM (Req → Model → Test → Execution) |
+| **2. Technique** | Which ISO 29119-4 techniques applied | EP, BVA, decision tables, state transitions |
+| **3. Code** | Line, branch, condition, function coverage | Istanbul, V8, JaCoCo, coverage.py, SonarQube |
+
+See `references/coverage-dimensions.md` for detailed measurement methods.
+
+## Dimension 1: Requirements/Model Coverage
+
+**Model:** Requirement → Test Model → Test Case → Execution
+
+- **Track:** Which requirements have linked test models and test cases
+- **Identify gaps:** Requirements with no tests, tests with no execution, orphan tests
+- **Metrics:** % requirements covered, % executed, traceability completeness
+
+**Inputs:** Requirements doc, RTM, test case inventory, execution results (Zephyr, TestRail, etc.)
+
+## Dimension 2: Technique Coverage (ISO 29119-4)
+
+**Techniques:** Equivalence Partitioning (EP), Boundary Value Analysis (BVA), Decision Tables, State Transition, Use Case, Classification Trees
+
+- **Track:** Which techniques applied per requirement/module
+- **Identify gaps:** Requirements tested with only one technique; modules with technique imbalance
+- **Metrics:** Technique distribution, technique-per-requirement ratio
+
+**Inputs:** Test case metadata (technique tags), requirements, test design docs
+
+## Dimension 3: Code Coverage
+
+**Metrics:** Line, branch, condition, function coverage
+
+| Language/Stack | Tool | Output Format |
+|----------------|------|---------------|
+| JavaScript/TypeScript | Istanbul (c8, nyc), V8 | lcov, json-summary |
+| Java | JaCoCo | xml, html |
+| Python | coverage.py | xml, html, json |
+| SonarQube | SonarQube | Unified dashboard |
+
+See `references/tools.md` for setup and integration.
+
+## Outputs
+
+1. **Coverage Dashboard** — Summary of all three dimensions with percentages and trends
+2. **Coverage Gaps List** — Uncovered requirements, low-technique areas, low-code-coverage modules
+3. **Risk-Based Recommendations** — High-risk uncovered areas prioritized; suggested actions
+4. **Heatmap by Module** — Visual (via qa-diagram-generator) showing coverage intensity per module
+
+### Dashboard Template
+
+```markdown
+# Coverage Dashboard
+
+## Summary
+| Dimension | Coverage | Status |
+|-----------|----------|--------|
+| Requirements | X% | ✅/⚠️/❌ |
+| Technique | X techniques avg | ✅/⚠️/❌ |
+| Code | Line X%, Branch Y% | ✅/⚠️/❌ |
+
+## Gaps
+- [Req ID] Uncovered
+- [Module] Low technique diversity
+- [File/Module] Low code coverage
+
+## Risk-Based Recommendations
+1. [High] ...
+2. [Medium] ...
+3. [Low] ...
+```
+
+## Workflow
+
+1. **Gather inputs:** RTM, test cases, execution results, code coverage reports
+2. **Analyze each dimension:** Compute coverage per dimension (see `references/coverage-dimensions.md`)
+3. **Identify gaps:** Uncovered requirements, technique-poor areas, low-code-coverage modules
+4. **Prioritize by risk:** Use risk matrix (see qa-test-strategy `references/risk-matrix.md`); high-risk + low coverage = top priority
+5. **Generate outputs:** Dashboard, gaps list, recommendations, heatmap
+6. **Historical tracking:** Use Memory MCP to store trends for comparison over time
+
+## MCP Tools Used
+
+- **Sequential Thinking MCP:** For decomposition of complex coverage analysis; reconciling conflicting signals across dimensions; prioritizing gaps.
+- **Memory MCP:** For historical coverage tracking; trend comparison (sprint-over-sprint, release-over-release).
+
+## Integration with Other Skills
+
+| Need | Skill | Usage |
+|------|-------|-------|
+| RTM, test cases from requirements | qa-testcase-from-docs | Source for requirements/model coverage |
+| Test design techniques | qa-testcase-from-docs | `references/test-design-techniques.md` |
+| Risk matrix | qa-test-strategy | `references/risk-matrix.md` |
+| Heatmap, charts | qa-diagram-generator | Coverage heatmap, quadrant charts |
+| NFR thresholds | qa-nfr-analyst | Coverage targets as NFRs |
+
+## Scope
+
+**Can do (autonomous):**
+- Analyze coverage across all three dimensions from provided artifacts
+- Produce coverage dashboard, gaps list, risk-based recommendations
+- Call qa-diagram-generator for heatmaps and charts
+- Use Sequential Thinking for complex analysis; Memory for historical tracking
+- Reference coverage-dimensions, tools, best-practices
+
+**Cannot do (requires confirmation):**
+- Change coverage thresholds set by stakeholders
+- Override organizational coverage policy
+- Modify test code or production code
+
+**Will not do (out of scope):**
+- Execute tests or generate coverage reports (consume existing reports)
+- Implement fixes for coverage gaps
+- Deploy or modify production systems
+
+## Quality Checklist
+
+- [ ] All three dimensions analyzed (requirements, technique, code)
+- [ ] Gaps list includes requirement IDs, modules, and file paths where applicable
+- [ ] Recommendations prioritized by risk (high/medium/low)
+- [ ] Dashboard includes percentages and status indicators
+- [ ] Heatmap generated for module-level view when requested
+- [ ] References to coverage-dimensions, tools, best-practices correct
+- [ ] No hardcoded secrets
+
+## Troubleshooting
+
+| Symptom | Likely Cause | Fix |
+|---------|--------------|-----|
+| Missing RTM | No traceability in place | Run qa-testcase-from-docs to build RTM; ask for requirement IDs |
+| Technique tags absent | Test cases not tagged | Add technique metadata; use test-design-techniques reference |
+| Code coverage format unknown | Tool output not recognized | Check tools.md for supported formats; request lcov/xml |
+| All dimensions green but quality issues | Coverage ≠ quality | Emphasize technique diversity; recommend exploratory testing |
+| Recommendations too generic | Insufficient risk context | Request risk matrix; use risk-matrix reference |
+| Historical comparison fails | No prior data in Memory | Store current run in Memory for future comparison |
+
+## Reference Files
+
+| Topic | Reference |
+|-------|-----------|
+| Coverage dimensions | `references/coverage-dimensions.md` |
+| Coverage tools | `references/tools.md` |
+| Best practices | `references/best-practices.md` |

package/skills/qa-coverage-analyzer/references/best-practices.md

@@ -0,0 +1,142 @@
+# Coverage Analysis Best Practices
+
+Meaningful thresholds, risk-based prioritization, and avoiding common pitfalls when analyzing test coverage.
+
+---
+
+## Meaningful Thresholds
+
+### Avoid Arbitrary Targets
+
+| Bad | Better |
+|-----|--------|
+| "We need 100% coverage" | Set targets by risk: critical paths 90%, utilities 60% |
+| Same threshold for all modules | Different thresholds per module type (core vs. glue) |
+| Line-only focus | Include branch/condition where logic is complex |
+
+### Suggested Ranges (Adjust per Project)
+
+| Dimension | Conservative | Typical | Aggressive |
+|-----------|--------------|---------|------------|
+| Requirements | 95% | 90% | 85% |
+| Technique (≥2 per req) | 80% | 70% | 60% |
+| Code (line) | 85% | 75% | 65% |
+| Code (branch) | 75% | 65% | 55% |
+
+### Per-Module Thresholds
+
+- **Critical (auth, payment, core business):** 85%+ line, 75%+ branch
+- **Important (API, integrations):** 75%+ line, 65%+ branch
+- **Supporting (utils, helpers):** 60%+ line
+- **Generated/boilerplate:** Exclude from coverage or set low target
+
+---
+
+## Risk-Based Prioritization
+
+### Apply Risk to Gaps
+
+1. **Identify risk** per requirement/module (use qa-test-strategy `references/risk-matrix.md`)
+2. **Score gaps:** Uncovered high-risk = highest priority
+3. **Order recommendations:** High → Medium → Low
+
+### Priority Matrix
+
+| Risk | Coverage Gap | Action |
+|------|--------------|--------|
+| High | Uncovered | Immediate: add tests |
+| High | Low technique | Add EP, BVA, DT, or ST |
+| High | Low code | Increase automation |
+| Medium | Uncovered | Plan for next sprint |
+| Medium | Low technique | Schedule technique review |
+| Low | Any | Backlog or accept |
+
+### Focus Areas First
+
+1. Authentication and authorization
+2. Payment and financial logic
+3. Data validation and sanitization
+4. Critical user journeys
+5. Integration points with external systems
+
+---
+
+## Technique Coverage Best Practices
+
+### Avoid Single-Technique Bias
+
+- **EP only:** May miss boundaries and invalid combinations
+- **BVA only:** May miss equivalence classes
+- **Happy path only:** Miss negative and edge cases
+
+### Technique Selection by Domain
+
+| Domain | Recommended Techniques |
+|--------|------------------------|
+| Input validation | EP, BVA |
+| Business rules | Decision table |
+| Workflows | State transition, use case |
+| APIs | EP, BVA, decision table (for status codes) |
+| UI flows | Use case, state transition |
+
+### Tagging Discipline
+
+- Tag every test case with at least one technique
+- Use consistent abbreviations (EP, BVA, DT, ST, UC, CT)
+- Review quarterly: are techniques balanced?
+
+---
+
+## Code Coverage Pitfalls
+
+### Don't Chase 100%
+
+- Diminishing returns above ~85% line coverage
+- Some code is defensive, error paths, or hard to trigger
+- Focus on high-risk and frequently changed code
+
+### Exclude Appropriately
+
+- Generated code (e.g., OpenAPI clients)
+- Configuration files
+- Test files themselves
+- Third-party code
+
+### Branch vs. Line
+
+- Branch coverage reveals untested decision paths
+- Prioritize branch coverage for complex conditionals (`if (a && b || c)`)
+
+---
+
+## Dashboard and Reporting
+
+### Keep Dashboards Actionable
+
+- Show gaps, not just percentages
+- Link to specific requirements, test cases, files
+- Include trend (improving vs. declining)
+
+### Heatmap Best Practices
+
+- Use color scale: red (low) → yellow → green (high)
+- Group by module or feature
+- Highlight modules below threshold
+
+### Frequency
+
+- **Code coverage:** Every CI run
+- **Requirements/technique:** Per sprint or release
+- **Full three-dimension analysis:** Per major release or quarterly
+
+---
+
+## Integration with QA Workflow
+
+| Phase | Coverage Activity |
+|-------|-------------------|
+| Requirements | Define traceability; assign IDs |
+| Test design | Apply techniques; tag test cases |
+| Implementation | Run coverage in CI; set gates |
+| Release | Full three-dimension analysis; gap report |
+| Retrospective | Review trends; adjust thresholds |