swe-workflow-skills 0.1.0

package/skills/technical-debt-review/templates/debt-audit.md

@@ -0,0 +1,85 @@
+# Technical Debt Audit
+
+**Codebase / Service**: [Name]
+**Date**: [DATE]
+**Reviewed by**: [Name(s)]
+**Review scope**: [Whole codebase / specific module / specific time period]
+
+---
+
+## Executive Summary
+
+[2-3 sentences: overall health assessment and top priority]
+
+**Overall health**: [Green / Yellow / Red]
+- Green: Debt is manageable, team moves at full speed
+- Yellow: Debt is noticeable, slowing sprint velocity
+- Red: Debt is critical, actively blocking features or causing incidents
+
+---
+
+## Hotspots
+
+| File / Module | Debt types | Severity | Churn (last 90d) | Notes |
+|---------------|-----------|---------|-----------------|-------|
+| [path/to/file] | Complexity, Test | High | 23 changes | God class with 800 lines |
+| | | | | |
+
+---
+
+## Debt Inventory
+
+### Critical — Address Immediately
+
+| Item | Type | Location | Description | Effort |
+|------|------|---------|-------------|--------|
+| | | | | |
+
+### High — Schedule This Quarter
+
+| Item | Type | Location | Description | Effort | Prerequisite |
+|------|------|---------|-------------|--------|-------------|
+| | | | | | |
+
+### Medium — Address Opportunistically
+
+| Item | Type | Location | Description |
+|------|------|---------|-------------|
+| | | | |
+
+### Accepted Debt — Won't Address
+
+| Item | Reason for accepting |
+|------|---------------------|
+| | |
+
+---
+
+## Remediation Roadmap
+
+### This Sprint (Quick Wins)
+- [ ] [Item]: [Owner] — est. [X hours]
+- [ ] [Item]: [Owner] — est. [X hours]
+
+### Next Quarter
+- [ ] [Item]: requires [prerequisite]. est. [X weeks]. Owner: [Name]
+- [ ] [Item]: requires [prerequisite]. est. [X weeks]. Owner: [Name]
+
+### Strategic Investment (requires planning)
+- [Item]: est. [X months]. Needs: [team buy-in / dedicated sprint / architecture design]
+
+---
+
+## Metrics to Track Progress
+
+| Metric | Current | Target | Measurement |
+|--------|---------|--------|-------------|
+| Test coverage (critical paths) | [X%] | [Y%] | [How measured] |
+| Average function length (hotspot files) | [X lines] | [Y lines] | [How measured] |
+| Bug rate from [module] | [X/month] | [Y/month] | [How measured] |
+
+---
+
+## Next Review Date
+
+[Date — typically 1 quarter from now]

package/skills/test-data-strategy/SKILL.md

@@ -0,0 +1,129 @@
+---
+name: test-data-strategy
+description: "Design and generate test data — factories, synthetic data, property-based testing, boundary analysis, contract testing, load scenarios, GDPR-safe data. Triggers: test data, synthetic data, fake data, factory, seed data, fixtures, property-based testing, fuzz testing, boundary values, contract testing, Pact, load test data, anonymize, GDPR test data."
+model: sonnet
+allowed-tools: Read, Grep, Glob, Write, Edit
+---
+
+# Test Data Strategy
+
+Design and generate the data that powers tests. Bad test data causes false confidence (tests pass but bugs hide) and false alarms (tests fail on irrelevant data quirks). Good test data is realistic, covers edge cases, and respects privacy.
+
+## Scope Boundary
+
+This skill designs **what data** to create and **how** to create it. For **how to structure tests** (naming, pyramid, framework setup), use `test-suite-design`. For **test-first implementation**, use `tdd-workflow`.
+
+## Workflow
+
+### Step 1: Assess Data Needs
+
+For the feature or system under test, identify:
+
+- **What entities are involved?** (Users, orders, products, transactions)
+- **What relationships matter?** (User has many orders, order has many items)
+- **What data states exist?** (Active user, suspended user, deleted user)
+- **What are the boundary values?** (Empty, one item, max items, zero price, max price)
+- **What constitutes invalid data?** (Missing required fields, wrong types, out-of-range values)
+- **Is production data available for reference?** (Understand real distributions)
+- **Are there privacy constraints?** (GDPR, HIPAA, PCI — cannot use real customer data)
+
+### Step 2: Choose the Right Strategy
+
+Different testing needs require different data strategies:
+
+| Strategy | Best For | When to Use |
+|----------|----------|-------------|
+| **Factories** | Unit/integration tests | Most tests — deterministic, focused, fast |
+| **Property-based testing** | Algorithmic code, parsers, serialization | When exhaustive example testing is impractical |
+| **Boundary value analysis** | Validation, calculations, range checks | Any function with numeric or string limits |
+| **Synthetic data generation** | Load testing, staging environments | When you need volume or realistic distributions |
+| **Contract testing** | Service-to-service boundaries | When multiple services agree on a data format |
+| **Snapshot testing** | Complex outputs (API responses, renders) | When output is large and changes infrequently |
+
+See [references/data-generation-patterns.md](references/data-generation-patterns.md) for implementation of each strategy.
+
+### Step 3: Design Factory Architecture
+
+For most applications, factories are the backbone of test data. Design them in layers:
+
+**Layer 1 — Build functions** (create plain objects in memory):
+```
+buildUser({ role: 'admin' }) → { id: '...', name: '...', role: 'admin', ... }
+```
+
+**Layer 2 — Create functions** (insert into database and return):
+```
+createUser({ role: 'admin' }) → User record in DB
+```
+
+**Layer 3 — Scenario builders** (create a complete test scenario):
+```
+createCheckoutScenario() → { user, cart with 3 items, valid payment method, applied discount }
+```
+
+Layer 3 is the key insight most teams miss. Complex tests need complete scenarios, not individual entities assembled ad-hoc in every test.
+
+### Step 4: Define Boundary Values
+
+For every input that has limits, test:
+
+```
+          ┌─────────────────────────────────┐
+Below min │  Min  │  Nominal  │  Max  │ Above max
+          └─────────────────────────────────┘
+  Invalid   Edge    Valid       Edge    Invalid
+```
+
+For each input, create test data for:
+- **Below minimum** (invalid: -1 for quantity, empty string for name)
+- **At minimum** (edge: 0 or 1, single character)
+- **Nominal** (typical valid value)
+- **At maximum** (edge: 255 characters, max integer)
+- **Above maximum** (invalid: 256 characters, overflow value)
+- **Special values** (null, undefined, NaN, Infinity, empty array, whitespace-only string)
+
+### Step 5: Implement Property-Based Tests
+
+For functions where example-based tests can't cover the input space, use property-based testing:
+
+Instead of `test("sorts [3,1,2] to [1,2,3]")`, assert the property: "for ANY array, sort(arr) produces a result where every element is ≤ the next."
+
+Properties to look for:
+- **Roundtrip**: `deserialize(serialize(x)) === x`
+- **Idempotence**: `f(f(x)) === f(x)` (e.g., normalization, formatting)
+- **Invariants**: `sort(arr).length === arr.length` (no elements lost or added)
+- **Commutativity**: `f(a, b) === f(b, a)` (order independence)
+- **Oracle**: `myFunction(x) === referenceImplementation(x)` (compare against known-good)
+
+See [references/data-generation-patterns.md](references/data-generation-patterns.md) for framework-specific implementation.
+
+### Step 6: Handle Privacy and Compliance
+
+When test data must resemble production data but cannot contain real PII:
+
+**Never copy production data to test environments unmasked.** Instead:
+
+1. **Use factories for most tests** — no real data involved, no compliance risk
+2. **Generate synthetic data for staging** — realistic distributions without real identities
+3. **Mask production data only when necessary** — anonymize PII before copying to lower environments
+4. **Version control your data generation seeds** — for reproducible test runs
+5. **Audit test environments** — ensure no real PII leaked into test databases
+
+See [references/data-generation-patterns.md](references/data-generation-patterns.md) for privacy-safe generation patterns.
+
+### Step 7: Design Load Test Data
+
+Load testing requires different data characteristics:
+
+- **Volume**: Thousands or millions of records, not tens
+- **Variety**: Realistic distribution of data states (not all active users)
+- **Uniqueness**: Unique identifiers to prevent constraint violations during concurrent loads
+- **Pre-seeded**: Data must exist before the test runs (not created during the test)
+- **Cleanable**: Easy to identify and remove after the test (`test_` prefix, specific date ranges)
+
+## Principles Applied
+
+- **KISS**: Start with factories. Add property-based testing for algorithmic code. Add synthetic generation for load tests. Don't build everything on day one.
+- **DRY**: Centralize factories in a shared test helpers directory. Every test file should import from the same factory, not define its own.
+- **YAGNI**: Don't build a synthetic data pipeline for unit tests. Factories are sufficient for 90% of test data needs.
+- **Functional Independence**: Test data setup should be self-contained per test. Never depend on data created by another test.

package/skills/test-data-strategy/evals/evals.json

@@ -0,0 +1,49 @@
+{
+  "skill_name": "test-data-strategy",
+  "evals": [
+    {
+      "id": 1,
+      "prompt": "I'm building an e-commerce API with users, products, orders, and payments. I need to set up test data for our test suite — unit tests, integration tests, and we'll eventually need load testing data. We're using TypeScript, Prisma, and PostgreSQL. Design the test data strategy.",
+      "expected_output": "Should design a 3-layer factory architecture (build/create/scenario), define boundary values for key fields, set up Faker for realistic data, and outline the load test data generation approach.",
+      "assertions": [
+        "Designs layered factories (build → create → scenario)",
+        "Creates factories for all key entities (user, product, order, payment)",
+        "Defines scenario builders for complex test setups (checkout scenario)",
+        "Identifies boundary values for price (0, negative, max), quantity (0, 1, max), and user roles",
+        "Uses Faker with a fixed seed for reproducibility",
+        "Separates unit test data (in-memory) from integration test data (database)",
+        "Outlines load test data generation as a separate concern",
+        "Addresses referential integrity across related factories"
+      ]
+    },
+    {
+      "id": 2,
+      "prompt": "I have a function that validates and normalizes email addresses. It handles international domains, plus addressing (user+tag@), case normalization, and various edge cases. How do I test this thoroughly? Example-based tests feel insufficient.",
+      "expected_output": "Should recommend property-based testing for this kind of input-space-heavy function, define key properties (roundtrip, normalization idempotence), and supplement with explicit boundary values.",
+      "assertions": [
+        "Recommends property-based testing (fast-check or Hypothesis)",
+        "Defines the idempotence property: normalize(normalize(email)) === normalize(email)",
+        "Defines the preservation property: normalizing a valid email doesn't invalidate it",
+        "Supplements with explicit boundary examples for known edge cases",
+        "Tests special characters, unicode domains, maximum-length emails",
+        "Does NOT rely solely on hand-picked examples",
+        "Provides working property-based test code"
+      ]
+    },
+    {
+      "id": 3,
+      "prompt": "Our QA team is using a copy of our production database for testing. This has real customer data including names, emails, and credit card tokens. We're a fintech startup and our compliance officer is concerned. How do we fix this without losing test quality?",
+      "expected_output": "Should recommend transitioning to factories for most tests, synthetic generation for staging, and masked production data only for specific data-shape-dependent scenarios. Should be specific about what data to mask and how.",
+      "assertions": [
+        "Identifies this as a compliance violation (PII in test environments)",
+        "Recommends factories for unit and integration tests (no real data needed)",
+        "Recommends synthetic data generation for staging",
+        "If production data is needed, recommends masking with specific techniques per field type",
+        "Provides concrete masking examples (emails, names, card tokens)",
+        "Warns against just copying and 'finding/replacing' — proper masking tools needed",
+        "Suggests separating the transition into phases (don't try to replace everything at once)",
+        "Does NOT just say 'don't use production data' without providing alternatives"
+      ]
+    }
+  ]
+}

package/skills/test-data-strategy/references/data-generation-patterns.md

@@ -0,0 +1,341 @@
+# Data Generation Patterns
+
+## Contents
+- Factory patterns by language
+- Property-based testing frameworks
+- Synthetic data generation
+- Contract testing patterns
+- Privacy-safe data patterns
+- Load test data patterns
+
+## Factory Patterns by Language
+
+### JavaScript/TypeScript (with Faker.js)
+
+```typescript
+import { faker } from '@faker-js/faker';
+
+// Layer 1: Build (in-memory objects)
+export function buildUser(overrides: Partial<User> = {}): User {
+  return {
+    id: faker.string.uuid(),
+    name: faker.person.fullName(),
+    email: faker.internet.email(),
+    role: 'member',
+    active: true,
+    createdAt: faker.date.past(),
+    ...overrides,
+  };
+}
+
+export function buildOrder(overrides: Partial<Order> = {}): Order {
+  return {
+    id: faker.string.uuid(),
+    userId: faker.string.uuid(),
+    items: [buildOrderItem(), buildOrderItem()],
+    total: 0, // calculated below
+    status: 'pending',
+    createdAt: new Date(),
+    ...overrides,
+  };
+}
+
+// Layer 2: Create (persisted to database)
+export async function createUser(overrides: Partial<User> = {}): Promise<User> {
+  const data = buildUser(overrides);
+  return db.users.create({ data });
+}
+
+// Layer 3: Scenario builders
+export async function createCheckoutScenario() {
+  const user = await createUser({ active: true });
+  const products = await Promise.all([
+    createProduct({ price: 1999 }),
+    createProduct({ price: 3499 }),
+  ]);
+  const cart = await createCart({ userId: user.id, items: products });
+  const paymentMethod = await createPaymentMethod({ userId: user.id });
+
+  return { user, products, cart, paymentMethod };
+}
+```
+
+### Python (with Faker + factory_boy)
+
+```python
+import factory
+from faker import Faker
+
+fake = Faker()
+
+# Layer 1: Build
+def build_user(**overrides):
+    defaults = {
+        "id": fake.uuid4(),
+        "name": fake.name(),
+        "email": fake.email(),
+        "role": "member",
+        "active": True,
+        "created_at": fake.date_time_this_year(),
+    }
+    return {**defaults, **overrides}
+
+# Layer 2: Create (with factory_boy + SQLAlchemy)
+class UserFactory(factory.alchemy.SQLAlchemyModelFactory):
+    class Meta:
+        model = User
+        sqlalchemy_session = session
+
+    name = factory.LazyFunction(fake.name)
+    email = factory.LazyFunction(fake.email)
+    role = "member"
+    active = True
+
+# Usage:
+admin = UserFactory(role="admin")
+inactive = UserFactory(active=False)
+
+# Layer 3: Scenario
+def create_checkout_scenario(session):
+    user = UserFactory(active=True)
+    products = [ProductFactory(price=1999), ProductFactory(price=3499)]
+    cart = CartFactory(user=user, items=products)
+    return {"user": user, "products": products, "cart": cart}
+```
+
+## Property-Based Testing Frameworks
+
+### JavaScript: fast-check
+
+```typescript
+import fc from 'fast-check';
+
+// Roundtrip property: JSON parse/stringify
+test('JSON roundtrip preserves data', () => {
+  fc.assert(
+    fc.property(fc.jsonValue(), (value) => {
+      expect(JSON.parse(JSON.stringify(value))).toEqual(value);
+    })
+  );
+});
+
+// Invariant: sort preserves length
+test('sort preserves array length', () => {
+  fc.assert(
+    fc.property(fc.array(fc.integer()), (arr) => {
+      expect([...arr].sort((a, b) => a - b).length).toBe(arr.length);
+    })
+  );
+});
+
+// Custom arbitrary for domain objects
+const userArbitrary = fc.record({
+  name: fc.string({ minLength: 1, maxLength: 100 }),
+  email: fc.emailAddress(),
+  age: fc.integer({ min: 0, max: 150 }),
+});
+
+test('user validation accepts all valid users', () => {
+  fc.assert(
+    fc.property(userArbitrary, (user) => {
+      expect(validateUser(user).isValid).toBe(true);
+    })
+  );
+});
+```
+
+### Python: Hypothesis
+
+```python
+from hypothesis import given, strategies as st
+
+# Roundtrip
+@given(st.text())
+def test_encode_decode_roundtrip(s):
+    assert decode(encode(s)) == s
+
+# Invariant
+@given(st.lists(st.integers()))
+def test_sort_preserves_length(lst):
+    assert len(sorted(lst)) == len(lst)
+
+# Custom strategy
+user_strategy = st.fixed_dictionaries({
+    "name": st.text(min_size=1, max_size=100),
+    "email": st.emails(),
+    "age": st.integers(min_value=0, max_value=150),
+})
+
+@given(user_strategy)
+def test_user_validation(user):
+    result = validate_user(user)
+    assert result.is_valid
+```
+
+## Synthetic Data Generation
+
+### For staging environments (Faker at scale)
+
+```python
+import pandas as pd
+from faker import Faker
+
+fake = Faker()
+Faker.seed(42)  # Reproducible!
+
+def generate_users(n=10000):
+    return pd.DataFrame([{
+        "id": f"usr_{i:06d}",
+        "name": fake.name(),
+        "email": fake.email(),
+        "country": fake.country_code(),
+        "signup_date": fake.date_between("-2y", "today"),
+        "plan": fake.random_element(["free", "pro", "enterprise"]),
+        "active": fake.boolean(chance_of_getting_true=85),
+    } for i in range(n)])
+
+def generate_orders(users_df, avg_orders_per_user=5):
+    orders = []
+    for _, user in users_df.iterrows():
+        n_orders = max(0, int(fake.random.gauss(avg_orders_per_user, 2)))
+        for _ in range(n_orders):
+            orders.append({
+                "id": fake.uuid4(),
+                "user_id": user["id"],
+                "amount": round(fake.random.uniform(5, 500), 2),
+                "status": fake.random_element(["completed", "pending", "refunded"]),
+                "created_at": fake.date_between(user["signup_date"], "today"),
+            })
+    return pd.DataFrame(orders)
+```
+
+### Statistical distributions for realistic data
+
+Don't use uniform random for everything. Real data has distributions:
+- User activity → Power law (few super-active users, long tail of inactive)
+- Transaction amounts → Log-normal (many small, few large)
+- Time of day → Bimodal (peaks in morning and evening)
+- Geographic → Clustered (real users aren't uniformly distributed)
+
+Use numpy or scipy for realistic distributions:
+```python
+import numpy as np
+
+# Log-normal transaction amounts (mean ~$50, long tail to $10K)
+amounts = np.random.lognormal(mean=3.9, sigma=1.2, size=10000)
+amounts = np.clip(amounts, 1, 10000).round(2)
+```
+
+## Contract Testing Patterns
+
+### Pact (consumer-driven contracts)
+
+Contract testing verifies that two services agree on the data format at their boundary, without requiring both to be running.
+
+**Consumer side** (the service that CALLS the API):
+```javascript
+const { PactV3 } = require('@pact-foundation/pact');
+
+const provider = new PactV3({ consumer: 'OrderService', provider: 'UserService' });
+
+test('get user by ID', async () => {
+  provider
+    .given('user 42 exists')
+    .uponReceiving('a request for user 42')
+    .withRequest({ method: 'GET', path: '/users/42' })
+    .willRespondWith({
+      status: 200,
+      body: { id: 42, name: string('Jane'), email: email() },
+    });
+
+  await provider.executeTest(async (mockServer) => {
+    const user = await userClient.getUser(42, mockServer.url);
+    expect(user.name).toBe('Jane');
+  });
+});
+```
+
+**Provider side** (the service that SERVES the API):
+```javascript
+const { Verifier } = require('@pact-foundation/pact');
+
+test('verify contracts', async () => {
+  await new Verifier({
+    providerBaseUrl: 'http://localhost:3000',
+    pactUrls: ['./pacts/orderservice-userservice.json'],
+    stateHandlers: {
+      'user 42 exists': async () => { await createUser({ id: 42, name: 'Jane' }); },
+    },
+  }).verifyProvider();
+});
+```
+
+## Privacy-Safe Data Patterns
+
+### Strategy selection
+
+| Scenario | Recommended Approach |
+|----------|---------------------|
+| Unit tests | Factories (no real data involved) |
+| Integration tests | Factories + test database (created fresh per suite) |
+| Staging environment | Synthetic generation at scale |
+| Performance testing | Synthetic with realistic distributions |
+| Bug reproduction | Masked production subset (anonymize PII first) |
+
+### Masking techniques
+
+| Data Type | Masking Method |
+|-----------|---------------|
+| Email | `john.doe@company.com` → `user_a1b2@test.example.com` |
+| Name | `John Doe` → `Person_82947` or Faker-generated name |
+| Phone | `555-123-4567` → `555-000-XXXX` |
+| SSN | `123-45-6789` → `XXX-XX-6789` (keep last 4 if needed) |
+| Address | Replace with Faker address in same country/region |
+| Credit card | `4111...1234` → `4111...0000` (keep BIN, zero out rest) |
+| Date of birth | Shift by random offset (preserve age distribution) |
+
+### Deterministic seeding for reproducibility
+
+Always seed random generators in test data scripts:
+```python
+Faker.seed(42)
+np.random.seed(42)
+random.seed(42)
+```
+
+Commit the seed value. Tests using seeded factories produce the same data every run, making failures reproducible.
+
+## Load Test Data Patterns
+
+### Pre-seeding for load tests (k6 example)
+
+```javascript
+// data/users.json — pre-generated, committed or generated in CI
+// Generate with: node scripts/generate-load-test-data.js > data/users.json
+
+import { SharedArray } from 'k6/data';
+
+const users = new SharedArray('users', function () {
+  return JSON.parse(open('./data/users.json'));
+});
+
+export default function () {
+  const user = users[__VU % users.length]; // Each VU gets a different user
+  const res = http.post('https://api.example.com/login', JSON.stringify({
+    email: user.email,
+    password: user.password,
+  }));
+  check(res, { 'login succeeded': (r) => r.status === 200 });
+}
+```
+
+### Data volume guidelines
+
+| Test Type | Typical Data Volume |
+|-----------|-------------------|
+| Smoke test | 10-50 records |
+| Load test | 10K-100K records |
+| Stress test | 100K-1M records |
+| Soak test | 100K records, sustained over hours |
+
+Generate data before the test run, not during. Creating data during a load test contaminates the results (you're measuring data creation speed, not application performance).