@alanwang-merit/meritdata-platform-skill-package 1.0.9 → 1.0.10

package/skills/test-driven-development/writing-good-tests.md

@@ -0,0 +1,198 @@
+# Writing Good Tests
+
+**Load this reference when:** writing or changing tests, adding mocks, or
+adding cleanup/helper methods for tests.
+
+## Overview
+
+A test exists to catch a specific break. Two principles govern everything
+here:
+
+```
+1. Every test names the break it catches
+2. Every test exercises the real thing
+```
+
+Strict TDD produces both naturally: a test written first and watched
+failing against real code has already proven it can fail, and only earns
+a mock when the real dependency proves slow or external.
+
+## Principle 1: Name the Break
+
+Before writing the test body, answer: **what production change should
+make this test fail — and is that change a bug or a decision?** A test
+earns its place by catching a wrong branch, missing side effect, wrong
+argument, boundary case, or broken contract.
+
+**Derive expectations independently.** Use literals and hand-checked
+fixtures; table-driven tests with literal `want` values are the preferred
+shape. An expectation computed by the code under test — or its helpers —
+passes no matter what that code does:
+
+```typescript
+// ❌ Mirror assertion: the same builder computes both sides — always true
+const expected = buildSearchQuery({ tag: 'urgent' });
+expect(buildSearchQuery({ tag: 'urgent' })).toBe(expected);
+
+// ✅ Hand-derived literal
+expect(buildSearchQuery({ tag: 'urgent' })).toBe('tag:"urgent"');
+```
+
+**No change detectors.** If only intentional decisions can fail a test —
+a constant's value, exact message wording, private structure — it fires
+on redesign and sleeps through bugs. Test the behavior that depends on
+the decision: not `expect(MAX_RETRIES).toBe(5)` but "a failing call is
+retried 5 times and the 6th attempt never happens."
+
+**Behavior, not text.** Asserting that a script, skill, or config
+contains an exact line proves only that the source is the source. Run
+scripts against controlled inputs and assert outputs, side effects, or
+exit codes. Documents that instruct agents are tested by the consuming
+agent's behavior (superpowers:writing-skills); prose for humans earns no
+test at all.
+
+**Your code, not the framework.** Test the contract your code makes at
+its boundaries — the route you register, the query you emit, the payload
+you produce. Upstream mechanics are their maintainers' tests to write
+(the classic: asserting your router invokes a registered handler — that
+is the framework's test, not yours). When upstream behavior genuinely
+surprised you, write one narrow characterization test naming the
+assumption. The same boundary applies inside your code: constructors,
+getters, constants, and trivial forwarding earn tests only when they
+validate, normalize, default, derive, enforce, or cause side effects —
+otherwise assert the first consumer-visible result that depends on them.
+
+### Gate Function
+
+```
+BEFORE writing the test body:
+  Name the production change that would make this test fail.
+
+  Cannot name one            → redesign around an observable behavior
+  "The source text changed"  → run the artifact and assert its effects
+  Only intentional decisions → change detector; test the behavior
+                               that depends on the decision
+
+  Confirm the expected value is derived without the code under test.
+  IF it reuses the code's logic or helpers:
+    Replace it with a literal or hand-checked fixture
+```
+
+## Principle 2: Exercise the Real Thing
+
+**The mock earns no assertions.** A mock assertion passes when the mock
+is present and fails when it is absent — it says nothing about the
+component. Assert the real component's behavior; if the mock is what you
+are checking, unmock it or delete the assertion.
+
+```typescript
+// ✅ Real behavior
+expect(screen.getByRole('navigation')).toBeInTheDocument();
+
+// ❌ Mock existence
+expect(screen.getByTestId('sidebar-mock')).toBeInTheDocument();
+```
+
+**your human partner's correction:** "Are we testing the behavior of a
+mock?"
+
+**Mock at the right level.** Learn every side effect of the real method
+before replacing it; mock the slow or external operation and keep what
+the test depends on real. When unsure, run the test against the real
+implementation first and observe what actually needs to happen.
+
+```typescript
+// ❌ The mock swallows the config write that duplicate detection reads
+vi.mock('ToolCatalog', () => ({
+  discoverAndCacheTools: vi.fn().mockResolvedValue(undefined)
+}));
+
+// ✅ Mock only the slow server startup; the config write stays real
+vi.mock('MCPServerManager');
+```
+
+**Make doubles specific.** When arguments, call counts, or ordering are
+part of the contract, assert them — a fake that accepts anything verifies
+nothing. Give each branch (success, error, malformed) its own fixture or
+spy, so the wrong branch cannot satisfy the expectation.
+
+**Mirror real data completely.** Mock the complete structure as it exists
+in reality — all documented fields — not just the ones your test reads.
+Partial mocks fail silently when downstream code reads an omitted field:
+the test passes while integration breaks.
+
+**Production classes carry production methods only.** Cleanup that only
+tests need lives in test utilities, never as a `destroy()` on the
+production class. Ask: is this method called only from tests? Does this
+class own this resource's lifecycle? Wrong answers → test utility.
+
+**Prefer real components over complex mocks.** When mock setup outgrows
+the test logic, mocks miss methods the real components have, or tests
+break when the mock changes, switch to an integration test with real
+components. **your human partner's question:** "Do we need to be using a
+mock here?"
+
+### Gate Function
+
+```
+BEFORE adding a mock or test helper:
+  List the real method's side effects; keep the ones the test
+  depends on real — mock the slow/external level below them.
+
+  Mock responses mirror the complete real structure.
+
+  A method only tests call lives in test utilities, not production.
+
+  About to assert on the mock itself?
+    Unmock it or delete the assertion.
+```
+
+## Tests Ship With the Implementation
+
+The TDD cycle — failing test, minimal implementation, refactor — is what
+"complete" means. Ship the tests the behavior needs and only those:
+trivial code and human prose earn none, and a test written to satisfy
+process costs maintenance forever.
+
+## The Mutation Check
+
+Before finishing, mentally mutate the production code; at least one test
+should fail for each realistic mutation:
+
+- Wrong constant or argument
+- Wrong branch handler
+- Missing state change or side effect
+- Empty or default return
+- Missing validation for zero, empty, nil, unauthorized, or malformed input
+
+A mutation nothing catches marks the behavior as unprotected — or the
+test as tautological.
+
+## Quick Reference
+
+| When you... | Do |
+|-------------|-----|
+| Write any test | Name the break it catches — a bug, not a decision |
+| Build an expected value | Derive it by hand; never with the code under test |
+| Test a script or document | Run it / pressure-test its consumer; never grep its text |
+| Reach for a dependency test | Test your boundary contract, not their documented mechanics |
+| Want to assert on a mocked element | Test the real component, or unmock it |
+| Are about to mock a method | Learn its side effects; mock the slow/external level |
+| Build a mock response | Mirror the real structure completely |
+| Need cleanup only tests use | Put it in test utilities |
+| Watch mock setup balloon | Switch to an integration test with real components |
+| Finish a test file | Run the mutation check |
+
+## Warning Signs
+
+- Setup and assertion share the same object, guaranteeing equality
+- The test can fail only through a panic, crash, or missing selector
+- The test fails on every intentional change, never on accidental breakage
+- Expected values are hidden behind loops, builders, or helpers
+- The test greps source text, or asserts a removed symbol stays removed
+- The test would still matter if only the framework remained
+- The test exists for coverage, checking no side effect or outcome
+- An assertion checks a `*-mock` test ID, or fails if you remove the mock
+- A method is called only from test files
+- Mock setup is more than half the test, or you can't explain why the mock is needed
+- Mocking "just to be safe"

package/skills/test-fixing/SKILL.md

@@ -0,0 +1,121 @@
+---
+name: test-fixing
+description: "Systematically identify and fix all failing tests using smart grouping strategies. Use when explicitly asks to fix tests (\"fix these tests\", \"make tests pass\"), reports test failures (\"tests are failing\", \"test suite is broken\"), or completes implementation and wants tests passing."
+risk: safe
+source: community
+date_added: "2026-02-27"
+---
+
+# Test Fixing
+
+Systematically identify and fix all failing tests using smart grouping strategies.
+
+## When to Use
+- Explicitly asks to fix tests ("fix these tests", "make tests pass")
+- Reports test failures ("tests are failing", "test suite is broken")
+- Completes implementation and wants tests passing
+- Mentions CI/CD failures due to tests
+
+## Systematic Approach
+
+### 1. Initial Test Run
+
+Run `make test` to identify all failing tests.
+
+Analyze output for:
+
+- Total number of failures
+- Error types and patterns
+- Affected modules/files
+
+### 2. Smart Error Grouping
+
+Group similar failures by:
+
+- **Error type**: ImportError, AttributeError, AssertionError, etc.
+- **Module/file**: Same file causing multiple test failure
+- **Root cause**: Missing dependencies, API changes, refactoring impacts
+
+Prioritize groups by:
+
+- Number of affected tests (highest impact first)
+- Dependency order (fix infrastructure before functionality)
+
+### 3. Systematic Fixing Process
+
+For each group (starting with highest impact):
+
+1. **Identify root cause**
+
+   - Read relevant code
+   - Check recent changes with `git diff`
+   - Understand the error pattern
+
+2. **Implement fix**
+
+   - Use Edit tool for code changes
+   - Follow project conventions (see CLAUDE.md)
+   - Make minimal, focused changes
+
+3. **Verify fix**
+
+   - Run subset of tests for this group
+   - Use pytest markers or file patterns:
+     ```bash
+     uv run pytest tests/path/to/test_file.py -v
+     uv run pytest -k "pattern" -v
+     ```
+   - Ensure group passes before moving on
+
+4. **Move to next group**
+
+### 4. Fix Order Strategy
+
+**Infrastructure first:**
+
+- Import errors
+- Missing dependencies
+- Configuration issues
+
+**Then API changes:**
+
+- Function signature changes
+- Module reorganization
+- Renamed variables/functions
+
+**Finally, logic issues:**
+
+- Assertion failures
+- Business logic bugs
+- Edge case handling
+
+### 5. Final Verification
+
+After all groups fixed:
+
+- Run complete test suite: `make test`
+- Verify no regressions
+- Check test coverage remains intact
+
+## Best Practices
+
+- Fix one group at a time
+- Run focused tests after each fix
+- Use `git diff` to understand recent changes
+- Look for patterns in failures
+- Don't move to next group until current passes
+- Keep changes minimal and focused
+
+## Example Workflow
+
+User: "The tests are failing after my refactor"
+
+1. Run `make test` → 15 failures identified
+2. Group errors:
+   - 8 ImportErrors (module renamed)
+   - 5 AttributeErrors (function signature changed)
+   - 2 AssertionErrors (logic bugs)
+3. Fix ImportErrors first → Run subset → Verify
+4. Fix AttributeErrors → Run subset → Verify
+5. Fix AssertionErrors → Run subset → Verify
+6. Run full suite → All pass ✓

package/skills/test-master/SKILL.md

@@ -0,0 +1,94 @@
+---
+name: test-master
+description: Generates test files, creates mocking strategies, analyzes code coverage, designs test architectures, and produces test plans and defect reports across functional, performance, and security testing disciplines. Use when writing unit tests, integration tests, or E2E tests; creating test strategies or automation frameworks; analyzing coverage gaps; performance testing with k6 or Artillery; security testing with OWASP methods; debugging flaky tests; or working on QA, regression, test automation, quality gates, shift-left testing, or test maintenance.
+license: MIT
+metadata:
+  author: https://github.com/Jeffallan
+  version: "1.1.1"
+  domain: quality
+  triggers: test, testing, QA, unit test, integration test, E2E, coverage, performance test, security test, regression, test strategy, test automation, test framework, quality metrics, defect, exploratory, usability, accessibility, localization, manual testing, shift-left, quality gate, flaky test, test maintenance
+  role: specialist
+  scope: testing
+  output-format: report
+  related-skills: fullstack-guardian, playwright-expert, devops-engineer, debugging-wizard, code-reviewer, feature-forge
+---
+
+# Test Master
+
+Comprehensive testing specialist ensuring software quality through functional, performance, and security testing.
+
+## Core Workflow
+
+1. **Define scope** — Identify what to test and which testing types apply
+2. **Create strategy** — Plan the test approach across functional, performance, and security perspectives
+3. **Write tests** — Implement tests with proper assertions (see example below)
+4. **Execute** — Run tests and collect results
+   - If tests fail: classify the failure (assertion error vs. environment/flakiness), fix root cause, re-run
+   - If tests are flaky: isolate ordering dependencies, check async handling, add retry or stabilization logic
+5. **Report** — Document findings with severity ratings and actionable fix recommendations
+   - Verify coverage targets are met before closing; flag gaps explicitly
+
+## Quick-Start Example
+
+A minimal Jest unit test illustrating the key patterns this skill enforces:
+
+```js
+// ✅ Good: meaningful description, specific assertion, isolated dependency
+describe('calculateDiscount', () => {
+  it('applies 10% discount for premium users', () => {
+    const result = calculateDiscount({ price: 100, userTier: 'premium' });
+    expect(result).toBe(90); // specific outcome, not just truthy
+  });
+
+  it('throws on negative price', () => {
+    expect(() => calculateDiscount({ price: -1, userTier: 'standard' }))
+      .toThrow('Price must be non-negative');
+  });
+});
+```
+
+Apply the same structure for pytest (`def test_…`, `assert result == expected`) and other frameworks.
+
+## Reference Guide
+
+Load detailed guidance based on context:
+
+<!-- TDD Iron Laws and Testing Anti-Patterns adapted from obra/superpowers by Jesse Vincent (@obra), MIT License -->
+
+| Topic | Reference | Load When |
+|-------|-----------|-----------|
+| Unit Testing | `references/unit-testing.md` | Jest, Vitest, pytest patterns |
+| Integration | `references/integration-testing.md` | API testing, Supertest |
+| E2E | `references/e2e-testing.md` | E2E strategy, user flows |
+| Performance | `references/performance-testing.md` | k6, load testing |
+| Security | `references/security-testing.md` | Security test checklist |
+| Reports | `references/test-reports.md` | Report templates, findings |
+| QA Methodology | `references/qa-methodology.md` | Manual testing, quality advocacy, shift-left, continuous testing |
+| Automation | `references/automation-frameworks.md` | Framework patterns, scaling, maintenance, team enablement |
+| TDD Iron Laws | `references/tdd-iron-laws.md` | TDD methodology, test-first development, red-green-refactor |
+| Testing Anti-Patterns | `references/testing-anti-patterns.md` | Test review, mock issues, test quality problems |
+
+## Constraints
+
+**MUST DO**
+- Test happy paths AND error/edge cases (e.g., empty input, null, boundary values)
+- Mock external dependencies — never call real APIs or databases in unit tests
+- Use meaningful `it('…')` descriptions that read as plain-English specifications
+- Assert specific outcomes (`expect(result).toBe(90)`), not just truthiness
+- Run tests in CI/CD; document and remediate coverage gaps
+
+**MUST NOT**
+- Skip error-path testing (e.g., don't test only the success branch of a try/catch)
+- Use production data in tests — use fixtures or factories instead
+- Create order-dependent tests — each test must be independently runnable
+- Ignore flaky tests — quarantine and fix them; don't just re-run until green
+- Test implementation details (internal method calls) — test observable behaviour
+
+## Output Templates
+
+When creating test plans, provide:
+1. Test scope and approach
+2. Test cases with expected outcomes
+3. Coverage analysis
+4. Findings with severity (Critical/High/Medium/Low)
+5. Specific fix recommendations

package/skills/test-master/references/automation-frameworks.md

@@ -0,0 +1,294 @@
+# Automation Frameworks
+
+## Advanced Framework Patterns
+
+### Screenplay Pattern
+```typescript
+// Better separation of concerns than POM
+export class Actor {
+  constructor(private page: Page) {}
+  attemptsTo(...tasks: Task[]) {
+    return Promise.all(tasks.map(t => t.performAs(this)));
+  }
+}
+
+class Login implements Task {
+  constructor(private email: string, private password: string) {}
+  async performAs(actor: Actor) {
+    await actor.page.getByLabel('Email').fill(this.email);
+    await actor.page.getByLabel('Password').fill(this.password);
+    await actor.page.getByRole('button', { name: 'Login' }).click();
+  }
+}
+
+// Clear, maintainable test code
+await new Actor(page).attemptsTo(new Login('user@test.com', 'pass'));
+```
+
+### Keyword-Driven Testing
+```typescript
+const keywords = {
+  NAVIGATE: (page, url) => page.goto(url),
+  CLICK: (page, selector) => page.click(selector),
+  TYPE: (page, selector, text) => page.fill(selector, text),
+  VERIFY: (page, selector) => expect(page.locator(selector)).toBeVisible(),
+};
+
+// Data drives execution - ideal for non-technical authors
+const steps = [
+  { keyword: 'NAVIGATE', args: ['/login'] },
+  { keyword: 'TYPE', args: ['#email', 'user@test.com'] },
+  { keyword: 'CLICK', args: ['#submit'] },
+];
+
+for (const step of steps) await keywords[step.keyword](page, ...step.args);
+```
+
+### Model-Based Testing
+```typescript
+// State machine defines valid transitions
+const cartModel = {
+  empty: { addItem: 'hasItems' },
+  hasItems: { addItem: 'hasItems', removeItem: 'hasItems|empty', checkout: 'checkingOut' },
+  checkingOut: { confirm: 'complete', cancel: 'hasItems' },
+};
+
+// Generate comprehensive test paths automatically
+const testPaths = generatePathsFromModel(cartModel);
+```
+
+## Maintenance Strategies
+
+### Self-Healing Locators
+```typescript
+// Multi-strategy finder with automatic fallback
+async function findElement(page: Page, strategies: string[]): Promise<Locator> {
+  for (const selector of strategies) {
+    const el = page.locator(selector);
+    if (await el.count() > 0) return el;
+  }
+  throw new Error(`Not found: ${strategies.join(', ')}`);
+}
+
+// Usage: tries best -> good -> fallback
+const submit = await findElement(page, [
+  '[data-testid="submit"]',     // Best: stable test ID
+  'button:has-text("Submit")',  // Good: semantic
+  'button.primary',             // Fallback: CSS
+]);
+```
+
+### Error Recovery & Smart Retry
+```typescript
+// Auto-retry with recovery actions
+async function clickWithRecovery(page: Page, selector: string, retries = 3) {
+  for (let i = 0; i < retries; i++) {
+    try {
+      await page.click(selector, { timeout: 5000 });
+      return;
+    } catch (e) {
+      if (i === retries - 1) throw e;
+      await page.reload();
+      await page.waitForLoadState('networkidle');
+    }
+  }
+}
+
+// Exponential backoff for flaky operations
+async function retryWithBackoff<T>(fn: () => Promise<T>, retries = 3): Promise<T> {
+  for (let i = 0; i < retries; i++) {
+    try {
+      return await fn();
+    } catch (e) {
+      if (i === retries - 1) throw e;
+      await new Promise(r => setTimeout(r, 1000 * Math.pow(2, i)));
+    }
+  }
+}
+```
+
+## Scaling Strategies
+
+### Parallel & Distributed Execution
+```typescript
+// playwright.config.ts
+export default defineConfig({
+  workers: process.env.CI ? 8 : 4,
+  fullyParallel: true,
+  retries: process.env.CI ? 2 : 0,
+  
+  // Shard tests across multiple machines
+  shard: process.env.SHARD ? {
+    current: parseInt(process.env.SHARD_INDEX),
+    total: parseInt(process.env.SHARD_TOTAL),
+  } : undefined,
+});
+```
+
+```yaml
+# GitHub Actions: distribute across 5 workers
+strategy:
+  matrix:
+    shard: [1, 2, 3, 4, 5]
+steps:
+  - run: npx playwright test --shard=${{ matrix.shard }}/5
+```
+
+### Resource Optimization
+```typescript
+// Reuse browser contexts for faster execution
+let browser: Browser;
+let context: BrowserContext;
+
+test.beforeAll(async () => {
+  browser = await chromium.launch();
+  context = await browser.newContext();
+});
+
+test('test 1', async () => {
+  const page = await context.newPage();
+  // Test logic
+  await page.close();
+});
+
+test.afterAll(async () => {
+  await context.close();
+  await browser.close();
+});
+```
+
+## CI/CD Integration
+
+### Complete Pipeline
+```yaml
+name: E2E Tests
+on: [push, pull_request]
+
+jobs:
+  test:
+    runs-on: ubuntu-latest
+    strategy:
+      matrix:
+        shard: [1, 2, 3, 4]
+    
+    steps:
+      - uses: actions/checkout@v3
+      - uses: actions/setup-node@v3
+      - run: npm ci
+      - run: npx playwright install --with-deps
+      
+      - run: npx playwright test --shard=${{ matrix.shard }}/4
+        env:
+          CI: true
+      
+      - uses: actions/upload-artifact@v3
+        if: always()
+        with:
+          name: report-${{ matrix.shard }}
+          path: playwright-report/
+```
+
+### Test Data Factories
+```typescript
+export class UserFactory {
+  static create(overrides?: Partial<User>): User {
+    return {
+      id: faker.string.uuid(),
+      email: faker.internet.email(),
+      name: faker.person.fullName(),
+      role: 'user',
+      ...overrides,
+    };
+  }
+
+  static createMany(count: number) {
+    return Array.from({ length: count }, () => this.create());
+  }
+}
+
+// Seed test data
+test.beforeEach(async ({ page }) => {
+  await page.request.post('/api/test/seed', {
+    data: { users: UserFactory.createMany(10) },
+  });
+});
+```
+
+## Team Enablement
+
+### Training Program
+```markdown
+**Week 1-2**: Framework basics, page objects, first test
+**Week 3-4**: Data-driven, API integration, CI/CD
+**Week 5-6**: Performance, error handling, scaling
+**Ongoing**: Code reviews, knowledge sharing
+```
+
+### Code Review Checklist
+```markdown
+- [ ] Independent tests (no order dependency)
+- [ ] Semantic locators (getByRole, getByLabel)
+- [ ] Proper waits (no arbitrary timeouts)
+- [ ] Error cases tested
+- [ ] Test data cleanup
+- [ ] Meaningful test names
+- [ ] Page objects updated
+```
+
+## Automation Strategy
+
+### ROI Calculation
+```typescript
+const manual = { timePerRun: 30, runsPerSprint: 10 };
+const automation = { development: 120, maintenance: 5 };
+
+const timeSaved = (manual.timePerRun * manual.runsPerSprint) - automation.maintenance;
+const breakEven = Math.ceil(automation.development / timeSaved);
+const annualSavings = (timeSaved * 26 - automation.development) / 60; // hours
+
+// Example: Break-even in 1 sprint, save 110 hours/year
+```
+
+### Selection Criteria
+```markdown
+**Automate**: Repetitive, stable UI, critical paths, data-driven, positive ROI
+**Don't Automate**: Exploratory, changing UI, one-time, usability, negative ROI
+```
+
+## Reporting & Metrics
+
+### Custom Reporter
+```typescript
+class MetricsReporter implements Reporter {
+  onTestEnd(test: TestCase, result: TestResult) {
+    this.sendMetrics({
+      name: test.title,
+      duration: result.duration,
+      status: result.status,
+      retries: result.retry,
+    });
+  }
+}
+```
+
+## Quick Reference
+
+| Pattern | Best For | Complexity |
+|---------|----------|-----------|
+| Page Object | Reusable components | Medium |
+| Screenplay | Complex workflows | High |
+| Keyword-Driven | Non-tech testers | Low |
+| Model-Based | State machines | High |
+
+| Scaling | Use Case |
+|---------|----------|
+| Parallel | Reduce time |
+| Distributed | Large suites |
+| Cloud | Cross-browser |
+| Resource Reuse | Speed |
+
+| Tool | Category |
+|------|----------|
+| Playwright, Cypress | Web E2E |
+| Appium, Detox | Mobile |
+| k6, Gatling | Performance |