@dzhechkov/keysarium 1.4.0 → 1.5.0

package/templates/.claude/skills/feature-adr/references/agentic-qe/enterprise-integration-testing.md

@@ -0,0 +1,111 @@
+# Enterprise Integration Testing (Condensed for Feature ADR)
+
+> Source: agentic-qe `enterprise-integration-testing` skill (Tier 3)
+> Condensed from 735 lines — SAP-specific sections removed, universal patterns retained.
+
+## Purpose
+
+Verify that independently correct modules/systems work correctly together.
+The biggest risks are in the seams between components, not within them.
+
+## Integration Testing Protocol
+
+### 1. Map E2E Flow
+
+Before writing tests, map the complete flow:
+```
+User Action → API → Service A → Service B → Database → Response
+```
+
+Identify every integration boundary (API calls, DB queries, message queues, file I/O).
+
+### 2. Test Each Boundary
+
+For each integration point:
+
+| Boundary Type | What to Test |
+|---|---|
+| API (REST/GraphQL) | Contract compliance, schema validation, error codes |
+| Database | Query correctness, transaction isolation, migration compatibility |
+| Message Queue | Message format, ordering, DLQ handling, retry logic |
+| File I/O | Format validation, encoding, path handling |
+| External Service | Mock fidelity, timeout handling, circuit breaker |
+
+### 3. Cross-Module Data Consistency
+
+After each integration event, verify:
+- Data written by module A is correctly read by module B
+- No data loss or transformation errors at boundaries
+- Eventual consistency windows are respected (async flows)
+
+### 4. Contract Testing
+
+```
+Consumer defines expected contract
+  → Provider validates against contract
+    → Break = blocked merge
+```
+
+Key checks:
+- Request/response schemas match between caller and callee
+- Error response formats are consistent
+- API versioning is respected
+- No breaking changes in shared interfaces
+
+### 5. Error Recovery
+
+Test failure scenarios at every boundary:
+- What happens when Service B is down?
+- Are partial operations compensated/rolled back?
+- Do correlation IDs propagate through the chain?
+- Are DLQ messages reprocessable?
+
+## Integration Quality Gates
+
+| Gate | Threshold | Action |
+|---|---|---|
+| Cross-module data consistency | 100% for critical paths | BLOCK on failure |
+| API contract compliance | 100% | BLOCK on failure |
+| Error recovery | All compensation paths tested | WARN if partial |
+| Performance | E2E flow < defined SLA | WARN if exceeded |
+
+## Hierarchical Testing Strategy
+
+Execute integration tests in phases:
+1. **Contract validation** — verify interfaces match
+2. **Integration testing** — verify boundary behavior
+3. **E2E validation** — verify complete flows
+4. **Authorization check** — verify access control across modules
+
+## QCSD Flags for Integration
+
+```
+HAS_CROSS_MODULE_DATA: true   → Data replicated/shared across modules
+HAS_ASYNC_PROCESSING: true    → Message queues, event-driven flows
+HAS_EXTERNAL_SERVICES: true   → Third-party API integrations
+HAS_DATABASE_SHARING: true    → Multiple modules access same DB
+```
+
+## Best Practices
+
+### Do
+- Map the full E2E flow before writing any tests
+- Test each integration boundary separately AND end-to-end
+- Validate data consistency across all modules after each event
+- Use service virtualization for hard-to-provision dependencies
+- Include correlation ID tracing in test assertions
+
+### Avoid
+- Testing only through the UI without API-level tests
+- Ignoring async processing errors (DLQ, failed events)
+- Assuming cross-module data is immediately consistent
+- Testing integrations only in isolation (mock everything)
+- Deploying without verifying no breaking API changes
+
+## Agent Coordination
+
+When used as Fleet QE Agent 3 (Integration Integrity Checker):
+- Focus scope: cross-module interactions within the feature
+- Input: `{CODE_CHANGES}` file list, `{ARCHITECTURE}` component diagram
+- Output: integration issues found, contract violations, data consistency risks
+- Verdict per issue: BLOCKER / WARNING / INFO

package/templates/.claude/skills/feature-adr/references/agentic-qe/mutation-testing.md

@@ -0,0 +1,236 @@
+---
+name: mutation-testing
+description: "Test quality validation through mutation testing, assessing test suite effectiveness by introducing code mutations and measuring kill rate. Use when evaluating test quality, identifying weak tests, or proving tests actually catch bugs."
+category: specialized-testing
+priority: high
+tokenEstimate: 900
+agents: [qe-test-generator, qe-coverage-analyzer, qe-quality-analyzer, qe-mutation-tester]
+implementation_status: optimized
+optimization_version: 1.0
+last_optimized: 2025-12-02
+dependencies: []
+quick_reference_card: true
+tags: [mutation, stryker, test-quality, kill-rate, assertions, effectiveness]
+
+# ADR-056 Trust Tier 3 Validation
+trust_tier: 3
+validation:
+  schema_path: schemas/output.json
+  validator_path: scripts/validate-config.json
+  eval_path: evals/mutation-testing.yaml
+---
+
+# Mutation Testing
+
+<default_to_action>
+When validating test quality or improving test effectiveness:
+1. MUTATE code (change + to -, >= to >, remove statements)
+2. RUN tests against each mutant
+3. VERIFY tests catch mutations (kill mutants)
+4. IDENTIFY surviving mutants (tests need improvement)
+5. STRENGTHEN tests to kill surviving mutants
+
+**Quick Mutation Metrics:**
+- Mutation Score = Killed / (Killed + Survived)
+- Target: > 80% mutation score
+- Surviving mutants = weak tests
+
+**Critical Success Factors:**
+- High coverage ≠ good tests (100% coverage, 0% assertions)
+- Mutation testing proves tests actually catch bugs
+- Focus on critical code paths first
+</default_to_action>
+
+## Quick Reference Card
+
+### When to Use
+- Evaluating test suite quality
+- Finding gaps in test assertions
+- Proving tests catch bugs
+- Before critical releases
+
+### Mutation Score Interpretation
+| Score | Interpretation |
+|-------|----------------|
+| **90%+** | Excellent test quality |
+| **80-90%** | Good, minor improvements |
+| **60-80%** | Needs attention |
+| **< 60%** | Significant gaps |
+
+### Common Mutation Operators
+| Category | Original | Mutant |
+|----------|----------|--------|
+| **Arithmetic** | `a + b` | `a - b` |
+| **Relational** | `x >= 18` | `x > 18` |
+| **Logical** | `a && b` | `a \|\| b` |
+| **Conditional** | `if (x)` | `if (true)` |
+| **Statement** | `return x` | *(removed)* |
+
+---
+
+## How Mutation Testing Works
+
+```javascript
+// Original code
+function isAdult(age) {
+  return age >= 18; // ← Mutant: change >= to >
+}
+
+// Strong test (catches mutation)
+test('18 is adult', () => {
+  expect(isAdult(18)).toBe(true); // Kills mutant!
+});
+
+// Weak test (mutation survives)
+test('19 is adult', () => {
+  expect(isAdult(19)).toBe(true); // Doesn't catch >= vs >
+});
+// Surviving mutant → Test needs boundary value
+```
+
+---
+
+## Using Stryker
+
+```bash
+# Install
+npm install --save-dev @stryker-mutator/core @stryker-mutator/jest-runner
+
+# Initialize
+npx stryker init
+```
+
+**Configuration:**
+```json
+{
+  "packageManager": "npm",
+  "reporters": ["html", "clear-text", "progress"],
+  "testRunner": "jest",
+  "coverageAnalysis": "perTest",
+  "mutate": [
+    "src/**/*.ts",
+    "!src/**/*.spec.ts"
+  ],
+  "thresholds": {
+    "high": 90,
+    "low": 70,
+    "break": 60
+  }
+}
+```
+
+**Run:**
+```bash
+npx stryker run
+```
+
+**Output:**
+```
+Mutation Score: 87.3%
+Killed: 124
+Survived: 18
+No Coverage: 3
+Timeout: 1
+```
+
+---
+
+## Fixing Surviving Mutants
+
+```javascript
+// Surviving mutant: >= changed to >
+function calculateDiscount(quantity) {
+  if (quantity >= 10) { // Mutant survives!
+    return 0.1;
+  }
+  return 0;
+}
+
+// Original weak test
+test('large order gets discount', () => {
+  expect(calculateDiscount(15)).toBe(0.1); // Doesn't test boundary
+});
+
+// Fixed: Add boundary test
+test('exactly 10 gets discount', () => {
+  expect(calculateDiscount(10)).toBe(0.1); // Kills mutant!
+});
+
+test('9 does not get discount', () => {
+  expect(calculateDiscount(9)).toBe(0); // Tests below boundary
+});
+```
+
+---
+
+## Agent-Driven Mutation Testing
+
+```typescript
+// Analyze mutation score and generate fixes
+await Task("Mutation Analysis", {
+  targetFile: 'src/payment.ts',
+  generateMissingTests: true,
+  minScore: 80
+}, "qe-test-generator");
+
+// Returns:
+// {
+//   mutationScore: 0.65,
+//   survivedMutations: [
+//     { line: 45, operator: '>=', mutant: '>', killedBy: null }
+//   ],
+//   generatedTests: [
+//     'test for boundary at line 45'
+//   ]
+// }
+
+// Coverage + mutation correlation
+await Task("Coverage Quality Analysis", {
+  coverageData: coverageReport,
+  mutationData: mutationReport,
+  identifyWeakCoverage: true
+}, "qe-coverage-analyzer");
+```
+
+---
+
+## Agent Coordination Hints
+
+### Memory Namespace
+```
+aqe/mutation-testing/
+├── mutation-results/*   - Stryker reports
+├── surviving/*          - Surviving mutants
+├── generated-tests/*    - Tests to kill mutants
+└── trends/*             - Mutation score over time
+```
+
+### Fleet Coordination
+```typescript
+const mutationFleet = await FleetManager.coordinate({
+  strategy: 'mutation-testing',
+  agents: [
+    'qe-test-generator',     // Generate tests for survivors
+    'qe-coverage-analyzer',  // Coverage correlation
+    'qe-quality-analyzer'    // Quality assessment
+  ],
+  topology: 'sequential'
+});
+```
+
+---
+
+## Related Skills
+- [tdd-london-chicago](../tdd-london-chicago/) - Write effective tests first
+- [test-design-techniques](../test-design-techniques/) - Boundary value analysis
+- [quality-metrics](../quality-metrics/) - Measure test effectiveness
+
+---
+
+## Remember
+
+**High code coverage ≠ good tests.** 100% coverage but weak assertions = useless. Mutation testing proves tests actually catch bugs.
+
+**Focus on critical paths first.** Don't mutation test everything - prioritize payment, authentication, data integrity code.
+
+**With Agents:** Agents run mutation analysis, identify surviving mutants, and generate missing test cases to kill them. Automated improvement of test quality.

package/templates/.claude/skills/feature-adr/references/agentic-qe/performance-testing.md

@@ -0,0 +1,315 @@
+---
+name: performance-testing
+description: "Test application performance, scalability, and resilience. Use when planning load testing, stress testing, or optimizing system performance."
+category: specialized-testing
+priority: high
+tokenEstimate: 1100
+agents: [qe-performance-tester, qe-quality-analyzer, qe-production-intelligence]
+implementation_status: optimized
+optimization_version: 1.0
+last_optimized: 2025-12-02
+dependencies: []
+quick_reference_card: true
+tags: [performance, load-testing, stress-testing, scalability, k6, bottlenecks]
+trust_tier: 3
+validation:
+  schema_path: schemas/output.json
+  validator_path: scripts/validate-config.json
+  eval_path: evals/performance-testing.yaml
+---
+
+# Performance Testing
+
+<default_to_action>
+When testing performance or planning load tests:
+1. DEFINE SLOs: p95 response time, throughput, error rate targets
+2. IDENTIFY critical paths: revenue flows, high-traffic pages, key APIs
+3. CREATE realistic scenarios: user journeys, think time, varied data
+4. EXECUTE with monitoring: CPU, memory, DB queries, network
+5. ANALYZE bottlenecks and fix before production
+
+**Quick Test Type Selection:**
+- Expected load validation → Load testing
+- Find breaking point → Stress testing
+- Sudden traffic spike → Spike testing
+- Memory leaks, resource exhaustion → Endurance/soak testing
+- Horizontal/vertical scaling → Scalability testing
+
+**Critical Success Factors:**
+- Performance is a feature, not an afterthought
+- Test early and often, not just before release
+- Focus on user-impacting bottlenecks
+</default_to_action>
+
+## Quick Reference Card
+
+### When to Use
+- Before major releases
+- After infrastructure changes
+- Before scaling events (Black Friday)
+- When setting SLAs/SLOs
+
+### Test Types
+| Type | Purpose | When |
+|------|---------|------|
+| **Load** | Expected traffic | Every release |
+| **Stress** | Beyond capacity | Quarterly |
+| **Spike** | Sudden surge | Before events |
+| **Endurance** | Memory leaks | After code changes |
+| **Scalability** | Scaling validation | Infrastructure changes |
+
+### Key Metrics
+| Metric | Target | Why |
+|--------|--------|-----|
+| p95 response | < 200ms | User experience |
+| Throughput | 10k req/min | Capacity |
+| Error rate | < 0.1% | Reliability |
+| CPU | < 70% | Headroom |
+| Memory | < 80% | Stability |
+
+### Tools
+- **k6**: Modern, JS-based, CI/CD friendly
+- **JMeter**: Enterprise, feature-rich
+- **Artillery**: Simple YAML configs
+- **Gatling**: Scala, great reporting
+
+### Agent Coordination
+- `qe-performance-tester`: Load test orchestration
+- `qe-quality-analyzer`: Results analysis
+- `qe-production-intelligence`: Production comparison
+
+---
+
+## Defining SLOs
+
+**Bad:** "The system should be fast"
+**Good:** "p95 response time < 200ms under 1,000 concurrent users"
+
+```javascript
+export const options = {
+  thresholds: {
+    http_req_duration: ['p(95)<200'],  // 95% < 200ms
+    http_req_failed: ['rate<0.01'],     // < 1% failures
+  },
+};
+```
+
+---
+
+## Realistic Scenarios
+
+**Bad:** Every user hits homepage repeatedly
+**Good:** Model actual user behavior
+
+```javascript
+// Realistic distribution
+// 40% browse, 30% search, 20% details, 10% checkout
+export default function () {
+  const action = Math.random();
+  if (action < 0.4) browse();
+  else if (action < 0.7) search();
+  else if (action < 0.9) viewProduct();
+  else checkout();
+
+  sleep(randomInt(1, 5)); // Think time
+}
+```
+
+---
+
+## Common Bottlenecks
+
+### Database
+**Symptoms:** Slow queries under load, connection pool exhaustion
+**Fixes:** Add indexes, optimize N+1 queries, increase pool size, read replicas
+
+### N+1 Queries
+```javascript
+// BAD: 100 orders = 101 queries
+const orders = await Order.findAll();
+for (const order of orders) {
+  const customer = await Customer.findById(order.customerId);
+}
+
+// GOOD: 1 query
+const orders = await Order.findAll({ include: [Customer] });
+```
+
+### Synchronous Processing
+**Problem:** Blocking operations in request path (sending email during checkout)
+**Fix:** Use message queues, process async, return immediately
+
+### Memory Leaks
+**Detection:** Endurance testing, memory profiling
+**Common causes:** Event listeners not cleaned, caches without eviction
+
+### External Dependencies
+**Solutions:** Aggressive timeouts, circuit breakers, caching, graceful degradation
+
+---
+
+## k6 CI/CD Example
+
+```javascript
+// performance-test.js
+import http from 'k6/http';
+import { check, sleep } from 'k6';
+
+export const options = {
+  stages: [
+    { duration: '1m', target: 50 },   // Ramp up
+    { duration: '3m', target: 50 },   // Steady
+    { duration: '1m', target: 0 },    // Ramp down
+  ],
+  thresholds: {
+    http_req_duration: ['p(95)<200'],
+    http_req_failed: ['rate<0.01'],
+  },
+};
+
+export default function () {
+  const res = http.get('https://api.example.com/products');
+  check(res, {
+    'status is 200': (r) => r.status === 200,
+    'response time < 200ms': (r) => r.timings.duration < 200,
+  });
+  sleep(1);
+}
+```
+
+```yaml
+# GitHub Actions
+- name: Run k6 test
+  uses: grafana/k6-action@v0.3.0
+  with:
+    filename: performance-test.js
+```
+
+---
+
+## Analyzing Results
+
+### Good Results
+```
+Load: 1,000 users | p95: 180ms | Throughput: 5,000 req/s
+Error rate: 0.05% | CPU: 65% | Memory: 70%
+```
+
+### Problems
+```
+Load: 1,000 users | p95: 3,500ms ❌ | Throughput: 500 req/s ❌
+Error rate: 5% ❌ | CPU: 95% ❌ | Memory: 90% ❌
+```
+
+### Root Cause Analysis
+1. Correlate metrics: When response time spikes, what changes?
+2. Check logs: Errors, warnings, slow queries
+3. Profile code: Where is time spent?
+4. Monitor resources: CPU, memory, disk
+5. Trace requests: End-to-end flow
+
+---
+
+## Anti-Patterns
+
+| ❌ Anti-Pattern | ✅ Better |
+|----------------|-----------|
+| Testing too late | Test early and often |
+| Unrealistic scenarios | Model real user behavior |
+| 0 to 1000 users instantly | Ramp up gradually |
+| No monitoring during tests | Monitor everything |
+| No baseline | Establish and track trends |
+| One-time testing | Continuous performance testing |
+
+---
+
+## Agent-Assisted Performance Testing
+
+```typescript
+// Comprehensive load test
+await Task("Load Test", {
+  target: 'https://api.example.com',
+  scenarios: {
+    checkout: { vus: 100, duration: '5m' },
+    search: { vus: 200, duration: '5m' },
+    browse: { vus: 500, duration: '5m' }
+  },
+  thresholds: {
+    'http_req_duration': ['p(95)<200'],
+    'http_req_failed': ['rate<0.01']
+  }
+}, "qe-performance-tester");
+
+// Bottleneck analysis
+await Task("Analyze Bottlenecks", {
+  testResults: perfTest,
+  metrics: ['cpu', 'memory', 'db_queries', 'network']
+}, "qe-performance-tester");
+
+// CI integration
+await Task("CI Performance Gate", {
+  mode: 'smoke',
+  duration: '1m',
+  vus: 10,
+  failOn: { 'p95_response_time': 300, 'error_rate': 0.01 }
+}, "qe-performance-tester");
+```
+
+---
+
+## Agent Coordination Hints
+
+### Memory Namespace
+```
+aqe/performance/
+├── results/*       - Test execution results
+├── baselines/*     - Performance baselines
+├── bottlenecks/*   - Identified bottlenecks
+└── trends/*        - Historical trends
+```
+
+### Fleet Coordination
+```typescript
+const perfFleet = await FleetManager.coordinate({
+  strategy: 'performance-testing',
+  agents: [
+    'qe-performance-tester',
+    'qe-quality-analyzer',
+    'qe-production-intelligence',
+    'qe-deployment-readiness'
+  ],
+  topology: 'sequential'
+});
+```
+
+---
+
+## Pre-Production Checklist
+
+- [ ] Load test passed (expected traffic)
+- [ ] Stress test passed (2-3x expected)
+- [ ] Spike test passed (sudden surge)
+- [ ] Endurance test passed (24+ hours)
+- [ ] Database indexes in place
+- [ ] Caching configured
+- [ ] Monitoring and alerting set up
+- [ ] Performance baseline established
+
+---
+
+## Related Skills
+- [agentic-quality-engineering](../agentic-quality-engineering/) - Agent coordination
+- [api-testing-patterns](../api-testing-patterns/) - API performance
+- [chaos-engineering-resilience](../chaos-engineering-resilience/) - Resilience testing
+
+---
+
+## Remember
+
+**Performance is a feature:** Test it like functionality
+**Test continuously:** Not just before launch
+**Monitor production:** Synthetic + real user monitoring
+**Fix what matters:** Focus on user-impacting bottlenecks
+**Trend over time:** Catch degradation early
+
+**With Agents:** Agents automate load testing, analyze bottlenecks, and compare with production. Use agents to maintain performance at scale.