devflow-kit 0.8.1 → 1.0.0

package/plugins/devflow-core-skills/skills/test-patterns/SKILL.md

@@ -0,0 +1,183 @@
+---
+name: test-patterns
+description: This skill should be used when the user asks to "write tests", "fix failing tests", "improve test coverage", "add integration tests", "debug a flaky test", or reviews test quality. Provides behavior-focused testing patterns, coverage analysis, and detection of brittle test anti-patterns like implementation coupling and non-deterministic assertions.
+user-invocable: false
+allowed-tools: Read, Grep, Glob, AskUserQuestion
+activation:
+  file-patterns:
+    - "**/*.test.*"
+    - "**/*.spec.*"
+    - "**/test/**"
+    - "**/tests/**"
+    - "**/__tests__/**"
+  exclude:
+    - "node_modules/**"
+---
+
+# Test Patterns
+
+## Iron Law
+
+> **TESTS VALIDATE BEHAVIOR, NOT IMPLEMENTATION**
+>
+> A test should fail when behavior breaks, not when implementation changes. If refactoring
+> breaks tests without changing behavior, the tests are wrong. Mock boundaries, not internals.
+> Test the contract, not the code. If tests are hard to write, the design is wrong — fix the
+> architecture, not the tests.
+
+---
+
+## Test Design Red Flags
+
+### 1. Complex Setup
+
+**RED FLAG**: Test setup >10 lines means the design is wrong.
+
+```typescript
+// VIOLATION: Too many dependencies
+beforeEach(async () => {
+  mockDb = new MockDatabase();
+  await mockDb.connect();
+  mockCache = new MockCache();
+  // ... 10+ more lines
+  service = new UserService(mockDb, mockCache, mockLogger, mockConfig);
+});
+
+// CORRECT: Simple setup
+it('should return Ok with valid data', () => {
+  const result = createUser({ name: 'test', email: 'test@example.com' });
+  expect(result.ok).toBe(true);
+});
+```
+
+**Detection**: `beforeEach` >10 lines, multiple mocks, async setup, database seeding
+
+### 2. Repetitive Boilerplate
+
+**RED FLAG**: Same pattern repeated >3 times means the API is wrong.
+
+```typescript
+// VIOLATION: Try/catch everywhere
+try { await api.createUser(data); fail(); } catch (e) { expect(e.status).toBe(400); }
+
+// CORRECT: Result types eliminate repetition
+const result = createUser(invalidData);
+expect(result.ok).toBe(false);
+expect(result.error.type).toBe('ValidationError');
+```
+
+### 3. Difficult Mocking
+
+**RED FLAG**: Mock setup >20 lines means dependencies are wrong.
+
+```typescript
+// VIOLATION: Nested mock structures
+mockDb = { transaction: jest.fn(), orders: { create: jest.fn(), update: jest.fn() } };
+
+// CORRECT: Pure functions need no mocking
+const result = processOrder(order);
+expect(result.ok).toBe(true);
+```
+
+### 4. Implementation Testing
+
+**RED FLAG**: Testing internals means tests are fragile.
+
+```typescript
+// VIOLATION: Spying on private methods
+const spy = jest.spyOn(cart as any, 'updateTotal');
+expect(spy).toHaveBeenCalled();
+
+// CORRECT: Test observable behavior
+expect(cart.getTotal()).toBe(10);
+```
+
+---
+
+## Coverage & Review
+
+### Coverage Issues
+
+- **Untested new code**: New functions/branches without corresponding tests
+- **Missing edge cases**: Only happy path tested, no error paths
+- **Missing error paths**: `throw`/`reject` in source without matching test assertions
+
+### Test Quality Issues
+
+- **Brittle tests**: Testing HOW (mock call verification) not WHAT (outcome)
+- **Unclear test names**: `it('test1')` instead of `it('validates email format on creation')`
+- **Missing AAA structure**: Mixed arrange/act/assert without clear separation
+
+### Mocking Issues
+
+- **Over-mocking**: Everything mocked, nothing actually tested
+- **Mocking third-party internals**: Mock at your own interface boundary instead
+
+---
+
+## Severity Guidelines
+
+| Severity | Criteria |
+|----------|----------|
+| **CRITICAL** | Tests pass but don't verify behavior; critical paths untested; tests mock everything |
+| **HIGH** | Missing error path coverage; flaky tests; extremely slow (>10s); >10 line setup |
+| **MEDIUM** | Some edge cases missing; weak assertions; unclear structure |
+| **LOW** | Organization could improve; naming could be clearer |
+
+---
+
+## Test Suite Safety
+
+```typescript
+// vitest.config.ts / jest.config.js
+{ fileParallelism: false, maxWorkers: 1, testTimeout: 10000 }
+```
+
+```bash
+NODE_OPTIONS="--max-old-space-size=512" npm test
+```
+
+---
+
+## Extended References
+
+For comprehensive examples and detection patterns:
+
+| Reference | Contents |
+|-----------|----------|
+| `references/violations.md` | Full violation examples for all categories |
+| `references/patterns.md` | Correct test patterns and organization |
+| `references/detection.md` | Bash commands for automated detection |
+| `references/report-template.md` | Full report format for documenting issues |
+
+---
+
+## Quality Gates
+
+Tests pass design review when:
+- [ ] Setup code <10 lines per test file
+- [ ] No repetitive try/catch or error handling patterns
+- [ ] Mocking requires <5 lines of setup
+- [ ] No spying on private methods or internal state
+- [ ] Tests verify behavior, not implementation details
+- [ ] Pure business logic testable without mocks
+- [ ] New code has corresponding tests
+- [ ] All branches covered (happy path + errors + edge cases)
+- [ ] Test names describe expected behavior
+- [ ] Tests follow Arrange-Act-Assert structure
+- [ ] No real delays (use mocked timers)
+- [ ] No flaky patterns (race conditions, timing dependencies)
+
+---
+
+## Review Checklist
+
+- [ ] New code has corresponding tests
+- [ ] All branches covered (happy path + errors + edge cases)
+- [ ] Tests verify behavior, not implementation
+- [ ] Test names describe expected behavior
+- [ ] Tests follow Arrange-Act-Assert structure
+- [ ] No real delays (use mocked timers)
+- [ ] Assertions are specific and meaningful
+- [ ] Mocking limited to boundaries (not internals)
+- [ ] No flaky patterns (race conditions, timing dependencies)

package/plugins/devflow-core-skills/skills/test-patterns/references/detection.md

@@ -0,0 +1,149 @@
+# Test Issue Detection
+
+Commands and patterns for detecting test quality issues.
+
+---
+
+## Coverage Detection
+
+### Find Untested Functions
+
+```bash
+# List exported functions in source
+grep -rn "export function\|export async function" --include="*.ts" src/ | cut -d: -f1,3 | sort
+
+# List tested functions
+grep -rn "describe\|it\(" --include="*.test.ts" | grep -oE "'[^']+'" | sort -u
+
+# Compare to find gaps (manual comparison needed)
+```
+
+### Find Missing Error Tests
+
+```bash
+# Count error-throwing code in source
+grep -rn "throw\|reject\|Error" --include="*.ts" src/ | grep -v test | wc -l
+
+# Count error test assertions
+grep -rn "rejects.toThrow\|toThrow\|toThrowError" --include="*.test.ts" | wc -l
+
+# Large discrepancy indicates missing error tests
+```
+
+---
+
+## Quality Detection
+
+### Tests Without Assertions
+
+```bash
+# Find test blocks that may lack assertions
+grep -rn "it\(.*=>" --include="*.test.ts" -A20 | grep -v "expect" | head -50
+
+# Find empty test blocks
+grep -rn "it\(.*{\s*}\)" --include="*.test.ts"
+```
+
+### Weak Assertions
+
+```bash
+# Find overly permissive assertions
+grep -rn "toBeDefined\|toBeTruthy\|not.toBeNull\|not.toBeUndefined" --include="*.test.ts"
+
+# Count for comparison with strong assertions
+grep -rn "toEqual\|toMatchObject\|toHaveLength\|toBe(" --include="*.test.ts" | wc -l
+```
+
+### Implementation Testing
+
+```bash
+# Find tests that verify mock calls (may indicate implementation testing)
+grep -rn "toHaveBeenCalledWith\|toHaveBeenCalled\|toHaveBeenCalledTimes" --include="*.test.ts"
+```
+
+---
+
+## Design Detection
+
+### Slow Tests
+
+```bash
+# Find tests with long timeouts (>5000ms)
+grep -rn "}, [0-9][0-9][0-9][0-9][0-9])" --include="*.test.ts"
+
+# Find real delays in tests
+grep -rn "setTimeout\|sleep\|delay" --include="*.test.ts"
+```
+
+### Complex Setup
+
+```bash
+# Find tests with many mock objects
+grep -rn "jest.fn\|sinon.stub\|mock" --include="*.test.ts" | cut -d: -f1 | uniq -c | sort -rn | head -10
+
+# Find long beforeEach blocks
+grep -rn "beforeEach" --include="*.test.ts" -A30 | head -100
+```
+
+---
+
+## Mocking Detection
+
+### Over-Mocking
+
+```bash
+# Count mocks per test file
+for f in $(find . -name "*.test.ts" -type f); do
+  count=$(grep -c "jest.fn\|mock" "$f" 2>/dev/null || echo 0)
+  echo "$count $f"
+done | sort -rn | head -20
+
+# Files with >20 mocks may be over-mocked
+```
+
+### Third-Party Library Mocking
+
+```bash
+# Find jest.mock of node_modules
+grep -rn "jest.mock\(['\"]" --include="*.test.ts" | grep -v "\./" | grep -v "\.\./"
+
+# These should be wrapped in interfaces instead
+```
+
+---
+
+## Summary Report Script
+
+```bash
+#!/bin/bash
+# test-health-check.sh - Quick test quality assessment
+
+echo "=== Test Health Check ==="
+echo ""
+
+echo "Coverage Indicators:"
+echo "  Source functions: $(grep -rn 'export function' --include='*.ts' src/ 2>/dev/null | wc -l)"
+echo "  Test blocks: $(grep -rn 'it\(' --include='*.test.ts' 2>/dev/null | wc -l)"
+echo ""
+
+echo "Quality Indicators:"
+echo "  Strong assertions: $(grep -rn 'toEqual\|toMatchObject' --include='*.test.ts' 2>/dev/null | wc -l)"
+echo "  Weak assertions: $(grep -rn 'toBeDefined\|toBeTruthy' --include='*.test.ts' 2>/dev/null | wc -l)"
+echo ""
+
+echo "Design Indicators:"
+echo "  Long timeouts: $(grep -rn '}, [0-9]\{5,\})' --include='*.test.ts' 2>/dev/null | wc -l)"
+echo "  Mock count: $(grep -rn 'jest.fn\|mock' --include='*.test.ts' 2>/dev/null | wc -l)"
+```
+
+---
+
+## Test Coverage Guidelines
+
+| Code Type | Required Coverage | Test Type |
+|-----------|-------------------|-----------|
+| Business logic | 90%+ | Unit tests |
+| API endpoints | 80%+ | Integration tests |
+| UI components | 70%+ | Component tests |
+| Utilities | 100% | Unit tests |
+| Error paths | 100% | Unit tests |

package/plugins/devflow-core-skills/skills/test-patterns/references/patterns.md

@@ -0,0 +1,220 @@
+# Test Patterns - Correct Examples
+
+Reference for proper test patterns and approaches.
+
+---
+
+## Simple Setup Patterns
+
+### Pure Function Testing
+
+```typescript
+// CORRECT: Simple setup indicates good design
+describe('createUser', () => {
+  it('should return Ok with valid data', () => {
+    const result = createUser({ name: 'test', email: 'test@example.com' });
+    expect(result.ok).toBe(true);
+    expect(result.value.name).toBe('test');
+  });
+
+  it('should return Err with invalid email', () => {
+    const result = createUser({ name: 'test', email: 'invalid' });
+    expect(result.ok).toBe(false);
+    expect(result.error.type).toBe('ValidationError');
+  });
+});
+```
+
+---
+
+## Minimal Mocking Patterns
+
+### Single Dependency Injection
+
+```typescript
+// CORRECT: Single mock for focused test
+describe('OrderCalculator', () => {
+  it('should calculate total with tax', () => {
+    const taxService = { getRate: () => 0.1 }; // Simple mock
+    const calculator = new OrderCalculator(taxService);
+    const result = calculator.calculateTotal([
+      { price: 100, quantity: 2 },
+      { price: 50, quantity: 1 },
+    ]);
+    expect(result).toBe(275);
+  });
+});
+```
+
+---
+
+## Behavior-Focused Testing
+
+```typescript
+// CORRECT: Testing behavior, not implementation
+describe('ShoppingCart', () => {
+  it('should calculate correct total after adding items', () => {
+    const cart = new ShoppingCart();
+    cart.addItem({ id: '1', price: 10 });
+    cart.addItem({ id: '2', price: 20 });
+    expect(cart.getTotal()).toBe(30);
+  });
+
+  it('should be empty after clearing', () => {
+    const cart = new ShoppingCart();
+    cart.addItem({ id: '1', price: 10 });
+    cart.clear();
+    expect(cart.getItems()).toEqual([]);
+    expect(cart.getTotal()).toBe(0);
+  });
+});
+```
+
+---
+
+## Coverage Patterns
+
+### Branch Coverage
+
+```typescript
+describe('calculateDiscount', () => {
+  it('returns 10% for regular customers', () => {
+    expect(calculateDiscount(100, 'regular')).toBe(10);
+  });
+  it('returns 20% for premium customers', () => {
+    expect(calculateDiscount(100, 'premium')).toBe(20);
+  });
+  it('returns 30% for VIP customers', () => {
+    expect(calculateDiscount(100, 'vip')).toBe(30);
+  });
+});
+```
+
+### Error Path Coverage
+
+```typescript
+describe('fetchUser', () => {
+  it('returns user data on success', async () => { /* ... */ });
+  it('throws ApiError on 404', async () => {
+    api.get.mockResolvedValue({ ok: false, status: 404 });
+    await expect(fetchUser('123')).rejects.toThrow(ApiError);
+  });
+  it('handles network errors', async () => {
+    api.get.mockRejectedValue(new Error('Network error'));
+    await expect(fetchUser('123')).rejects.toThrow('Network error');
+  });
+});
+```
+
+---
+
+## Separation of Concerns Testing
+
+```typescript
+// CORRECT: Business logic tested without I/O
+describe('OrderPricing (pure logic)', () => {
+  it('should calculate subtotal', () => {
+    const items = [{ price: 100, quantity: 2 }, { price: 50, quantity: 1 }];
+    expect(calculateSubtotal(items)).toBe(250);
+  });
+  it('should apply percentage discount', () => {
+    expect(applyDiscount(100, { type: 'percent', value: 20 })).toBe(80);
+  });
+});
+
+// Integration test for I/O wrapper
+describe('OrderService (integration)', () => {
+  it('should persist order and return result', async () => {
+    const mockRepo = { save: jest.fn().mockResolvedValue({ id: '1' }) };
+    const service = new OrderService(mockRepo);
+    const result = await service.createOrder({ items: [{ price: 100, quantity: 1 }] });
+    expect(result.ok).toBe(true);
+  });
+});
+```
+
+---
+
+## Parameterized Testing
+
+```typescript
+describe('validateEmail', () => {
+  const validEmails = ['test@example.com', 'user.name@domain.org', 'user+tag@example.co.uk'];
+  const invalidEmails = ['', 'invalid', '@nodomain.com', 'spaces in@email.com'];
+
+  test.each(validEmails)('should accept valid email: %s', (email) => {
+    expect(validateEmail(email).ok).toBe(true);
+  });
+
+  test.each(invalidEmails)('should reject invalid email: %s', (email) => {
+    expect(validateEmail(email).ok).toBe(false);
+  });
+});
+```
+
+---
+
+## Boundary Mocking
+
+```typescript
+// CORRECT: Wrap third-party libraries in your own interface
+interface HttpClient {
+  get<T>(url: string): Promise<T>;
+}
+
+class MockHttpClient implements HttpClient {
+  private responses: Map<string, any> = new Map();
+  mockResponse(url: string, data: any) { this.responses.set(url, data); }
+  async get<T>(url: string): Promise<T> { return this.responses.get(url); }
+}
+
+// Tests use MockHttpClient, production uses AxiosHttpClient
+```
+
+---
+
+## Test Organization
+
+```typescript
+describe('UserService', () => {
+  let service: UserService;
+  let repo: InMemoryUserRepo;
+
+  beforeEach(() => {
+    repo = new InMemoryUserRepo();
+    service = new UserService(repo);
+  });
+
+  describe('create', () => {
+    it('creates user with valid data', async () => { /* ... */ });
+    it('rejects invalid email', async () => { /* ... */ });
+  });
+
+  describe('update', () => {
+    it('updates existing user', async () => { /* ... */ });
+    it('returns error for non-existent user', async () => { /* ... */ });
+  });
+});
+```
+
+---
+
+## Test Data Factories
+
+```typescript
+// CORRECT: Factories for test data, not test setup
+const createTestUser = (overrides = {}): User => ({
+  id: 'test-id',
+  name: 'Test User',
+  email: 'test@example.com',
+  role: 'user',
+  ...overrides,
+});
+
+describe('UserPermissions', () => {
+  it('should allow admin to delete users', () => {
+    const admin = createTestUser({ role: 'admin' });
+    expect(canDeleteUser(admin)).toBe(true);
+  });
+});
+```

package/plugins/devflow-core-skills/skills/test-patterns/references/report-template.md

@@ -0,0 +1,108 @@
+# Test Quality Report Template
+
+Use this template when reporting test design issues.
+
+---
+
+## Report Format
+
+```markdown
+## Test Design Issues Detected
+
+## [SEVERITY] - [Category] ([Root Cause Type])
+**File**: path/to/file.test.ts:line-range
+**Issue**: Brief description of the problem
+**Root Cause**: What architectural issue causes this
+**Symptom**:
+```code
+// The problematic code snippet
+```
+**Correct Design**:
+```code
+// What good design looks like
+```
+**Action Required**: Specific action to fix
+
+## Summary
+- **Critical**: N issues (block implementation)
+- **High**: N issues (refactor needed)
+- **Files affected**: N test files
+- **Root cause**: Brief architectural diagnosis
+
+## STOP - Design Issues Detected
+
+[List fundamental design flaws discovered]
+
+**DO NOT work around these issues in tests.**
+**DO NOT add more complex test helpers.**
+**DO NOT mock more things to make tests pass.**
+
+## Next Steps
+
+1. **STOP writing tests** - Current design cannot be tested simply
+2. **ANALYZE root cause** - Identify architectural issue
+3. **PROPOSE redesign** - Show correct pattern
+4. **GET APPROVAL** - User confirms design changes
+5. **IMPLEMENT redesign** - Fix architecture first
+6. **WRITE SIMPLE TESTS** - Tests should be trivial after redesign
+```
+
+---
+
+## Severity Levels
+
+| Severity | Use For | Examples |
+|----------|---------|---------|
+| **CRITICAL** | Fundamental design flaws | Complex setup >10 lines, repetitive boilerplate >3 times, mock objects >5 methods, testing private methods |
+| **HIGH** | Maintenance complications | Difficult mocking >20 lines, implementation testing, environment manipulation, database seeding |
+| **MEDIUM** | Reduced clarity | Test helper abuse, inconsistent assertion patterns, missing edge case coverage |
+| **LOW** | Minor improvements | Organization, naming clarity |
+
+---
+
+## Category Labels
+
+| Category | Root Cause Type | Indicates |
+|----------|-----------------|-----------|
+| Complex Setup | Design Problem | Too many dependencies |
+| Repetitive Boilerplate | API Problem | Inconsistent error handling |
+| Difficult Mocking | Coupling Problem | Tight coupling to externals |
+| Implementation Testing | Fragile Tests | Tests coupled to internals |
+| Environment Manipulation | DI Problem | Direct environment access |
+| Database Seeding | Separation Problem | Logic mixed with data access |
+
+---
+
+## Test Suite Safety Configuration
+
+### Full Configuration Examples
+
+```typescript
+// vitest.config.ts
+export default defineConfig({
+  test: {
+    fileParallelism: false,
+    maxWorkers: 1,
+    pool: 'forks',
+    testTimeout: 10000,
+  }
+});
+
+// jest.config.js
+module.exports = {
+  maxWorkers: 1,
+  runInBand: true,
+  testTimeout: 10000,
+};
+```
+
+```bash
+# Memory limits
+NODE_OPTIONS="--max-old-space-size=512" npm test
+
+# Go
+go test -p 1 ./...
+
+# Rust
+cargo test -- --test-threads=1
+```