@agentic-qe/v3 3.0.0-alpha.5 → 3.0.0-alpha.7

package/assets/skills/agentic-quality-engineering/SKILL.md

@@ -0,0 +1,335 @@
+---
+name: agentic-quality-engineering
+description: "AI agents as force multipliers for quality work. Core skill for all 19 QE agents using PACT principles."
+category: qe-core
+priority: critical
+tokenEstimate: 1400
+agents: [qe-test-generator, qe-test-executor, qe-coverage-analyzer, qe-quality-gate, qe-quality-analyzer, qe-performance-tester, qe-security-scanner, qe-requirements-validator, qe-production-intelligence, qe-fleet-commander, qe-deployment-readiness, qe-regression-risk-analyzer, qe-test-data-architect, qe-api-contract-validator, qe-flaky-test-hunter, qe-visual-tester, qe-chaos-engineer, qe-code-complexity, qx-partner]
+implementation_status: optimized
+optimization_version: 1.0
+last_optimized: 2025-12-02
+dependencies: []
+quick_reference_card: true
+tags: [pact, agents, fleet, coordination, autonomous, foundational]
+---
+
+# Agentic Quality Engineering
+
+<default_to_action>
+When implementing agentic QE or coordinating agents:
+1. SPAWN appropriate agent(s) for the task using `Task` tool with agent type
+2. CONFIGURE agent coordination (hierarchical/mesh/sequential)
+3. EXECUTE with PACT principles: Proactive analysis, Autonomous operation, Collaborative feedback, Targeted risk focus
+4. VALIDATE results through quality gates before deployment
+5. LEARN from outcomes - store patterns in `aqe/learning/*` namespace
+
+**Quick Agent Selection:**
+- Test generation needed → `qe-test-generator`
+- Coverage gaps → `qe-coverage-analyzer`
+- Quality decision → `qe-quality-gate`
+- Security scan → `qe-security-scanner`
+- Performance test → `qe-performance-tester`
+- Full pipeline → `qe-fleet-commander`
+
+**Critical Success Factors:**
+- Agents amplify human expertise, not replace it
+- Human-in-the-loop for critical decisions
+- Measure: bugs caught, time saved, coverage improved
+</default_to_action>
+
+## Quick Reference Card
+
+### When to Use
+- Designing autonomous testing systems
+- Scaling QE with intelligent agents
+- Implementing multi-agent coordination
+- Building CI/CD quality pipelines
+
+### PACT Principles
+| Principle | Agent Behavior | Human Role |
+|-----------|---------------|------------|
+| **P**roactive | Analyze pre-merge, predict risk | Set guardrails |
+| **A**utonomous | Execute tests, fix flaky tests | Review critical |
+| **C**ollaborative | Multi-agent coordination | Provide context |
+| **T**argeted | Risk-based prioritization | Define risk areas |
+
+### 19-Agent Fleet
+| Category | Agents | Primary Use |
+|----------|--------|-------------|
+| Core Testing (5) | test-generator, test-executor, coverage-analyzer, quality-gate, quality-analyzer | Daily testing |
+| Performance/Security (2) | performance-tester, security-scanner | Non-functional |
+| Strategic (3) | requirements-validator, production-intelligence, fleet-commander | Planning |
+| Advanced (4) | regression-risk-analyzer, test-data-architect, api-contract-validator, flaky-test-hunter | Specialized |
+| Visual/Chaos (2) | visual-tester, chaos-engineer | Edge cases |
+| Deployment (1) | deployment-readiness | Release |
+| Analysis (1) | code-complexity | Maintainability |
+
+### Coordination Patterns
+```
+Hierarchical: fleet-commander → [generators] → [executors] → quality-gate
+Mesh: test-gen ↔ coverage ↔ quality (peer decisions)
+Sequential: risk-analyzer → test-gen → executor → coverage → gate
+```
+
+### Success Criteria
+✅ 10x deployment frequency with same/better quality
+✅ Coverage gaps detected in real-time
+✅ Bugs caught pre-production
+❌ Agents acting without human oversight on critical decisions
+❌ Deploying all 19 agents at once (start with 1-2)
+
+---
+
+## Core Concepts
+
+### QE Evolution
+| Stage | Approach | Limitation |
+|-------|----------|------------|
+| Traditional | Manual everything | Human bottleneck |
+| Automation | Scripts + fixed scenarios | Needs orchestration |
+| **Agentic** | AI agents + human judgment | Requires trust-building |
+
+**Core Premise:** Agents amplify human expertise for 10x scale.
+
+### Key Capabilities
+
+**1. Intelligent Test Generation**
+```typescript
+// Agent analyzes code change, generates targeted tests
+const tests = await qeTestGenerator.generate(prDiff);
+// → Happy path, edge cases, error handling tests
+```
+
+**2. Pattern Detection** - Scan logs, find anomalies, correlate errors
+
+**3. Adaptive Strategy** - Adjust test focus based on risk signals
+
+**4. Root Cause Analysis** - Link failures to code changes, suggest fixes
+
+---
+
+## Agent Coordination
+
+### Memory Namespaces
+```
+aqe/test-plan/*     - Test planning decisions
+aqe/coverage/*      - Coverage analysis results
+aqe/quality/*       - Quality metrics and gates
+aqe/learning/*      - Patterns and Q-values
+aqe/coordination/*  - Cross-agent state
+```
+
+### Memory Operations (MCP Tools)
+
+**CRITICAL**: Always use `mcp__agentic-qe__memory_store` with `persist: true` for learnings.
+
+**1. Store data to persistent memory:**
+```javascript
+// Store test plan decisions (persisted to .agentic-qe/memory.db)
+mcp__agentic_qe__memory_store({
+  key: "aqe/test-plan/pr-123",
+  namespace: "aqe/test-plan",
+  value: {
+    prNumber: 123,
+    riskLevel: "medium",
+    requiredCoverage: 85,
+    testTypes: ["unit", "integration"],
+    estimatedTime: 1800
+  },
+  persist: true,  // ⚠️ REQUIRED for cross-session persistence
+  ttl: 604800     // 7 days (0 = permanent)
+})
+```
+
+**2. Retrieve prior learnings before task:**
+```javascript
+// Query patterns before starting test generation
+const priorData = await mcp__agentic_qe__memory_retrieve({
+  key: "aqe/learning/patterns/test-generation/*",
+  namespace: "aqe/learning",
+  includeMetadata: true
+})
+
+// Use patterns to guide current task
+if (priorData.success) {
+  console.log(`Loaded ${priorData.patterns.length} prior patterns`);
+}
+```
+
+**3. Store coverage analysis results:**
+```javascript
+mcp__agentic_qe__memory_store({
+  key: "aqe/coverage/auth-module",
+  namespace: "aqe/coverage",
+  value: {
+    moduleId: "auth-module",
+    currentCoverage: 78,
+    gaps: ["error-handling", "edge-cases"],
+    suggestedTests: 12,
+    priority: "high"
+  },
+  persist: true,
+  ttl: 1209600  // 14 days
+})
+```
+
+### Three-Phase Memory Protocol
+
+For coordinated multi-agent tasks, use the STATUS → PROGRESS → COMPLETE pattern:
+
+```javascript
+// PHASE 1: STATUS - Task starting
+mcp__agentic_qe__memory_store({
+  key: "aqe/coordination/task-123/status",
+  namespace: "aqe/coordination",
+  value: { status: "running", agent: "qe-test-generator", startTime: Date.now() },
+  persist: true
+})
+
+// PHASE 2: PROGRESS - Intermediate updates
+mcp__agentic_qe__memory_store({
+  key: "aqe/coordination/task-123/progress",
+  namespace: "aqe/coordination",
+  value: { progress: 50, action: "generating-unit-tests", testsGenerated: 25 },
+  persist: true
+})
+
+// PHASE 3: COMPLETE - Task finished
+mcp__agentic_qe__memory_store({
+  key: "aqe/coordination/task-123/complete",
+  namespace: "aqe/coordination",
+  value: {
+    status: "complete",
+    result: "success",
+    testsGenerated: 47,
+    coverageAchieved: 92.3,
+    duration: 15000
+  },
+  persist: true
+})
+```
+
+### Blackboard Events
+| Event | Trigger | Subscribers |
+|-------|---------|-------------|
+| `test:generated` | New tests created | executor, coverage |
+| `coverage:gap` | Gap detected | test-generator |
+| `quality:decision` | Gate evaluated | fleet-commander |
+| `security:finding` | Vulnerability found | quality-gate |
+
+### Example: PR Quality Pipeline
+```typescript
+// 1. Risk analysis
+const risks = await Task("Analyze PR", prDiff, "qe-regression-risk-analyzer");
+
+// 2. Generate tests for risks
+const tests = await Task("Generate tests", risks, "qe-test-generator");
+
+// 3. Execute + analyze
+const results = await Task("Run tests", tests, "qe-test-executor");
+const coverage = await Task("Check coverage", results, "qe-coverage-analyzer");
+
+// 4. Quality decision
+const decision = await Task("Evaluate", {results, coverage}, "qe-quality-gate");
+// → GO/NO-GO with rationale
+```
+
+---
+
+## Implementation Phases
+
+| Phase | Duration | Goal | Agent(s) |
+|-------|----------|------|----------|
+| Experiment | Weeks 1-4 | Validate one use case | 1 agent |
+| Integrate | Months 2-3 | CI/CD pipeline | 3-4 agents |
+| Scale | Months 4-6 | Multiple use cases | 8+ agents |
+| Evolve | Ongoing | Continuous learning | Full fleet |
+
+### Phase 1 Example
+```bash
+# Week 1: Deploy single agent
+aqe agent spawn qe-test-generator
+
+# Weeks 2-3: Generate tests for 10 PRs
+# Track: bugs found, test quality, review time
+
+# Week 4: Measure impact
+aqe agent metrics qe-test-generator
+# → Tests: 150, Bugs: 12, Time saved: 8h
+```
+
+---
+
+## Limitations & Strengths
+
+### Agents Excel At
+- **Volume**: Scan thousands of logs in seconds
+- **Patterns**: Find correlations humans miss
+- **Tireless**: 24/7 testing and monitoring
+- **Speed**: Instant code change analysis
+
+### Agents Need Humans For
+- Business context and priorities
+- Ethical judgment and trade-offs
+- Creative exploration ("what if" scenarios)
+- Domain expertise (healthcare, finance, legal)
+
+---
+
+## Best Practices
+
+| Do | Don't |
+|----|-------|
+| Start with one agent, one use case | Deploy all 18 at once |
+| Build feedback loops early | Deploy and forget |
+| Human reviews agent output | Auto-merge without review |
+| Measure bugs caught, time saved | Track vanity metrics (test count) |
+| Build trust gradually | Give full autonomy immediately |
+
+### Trust Progression
+```
+Month 1: Agent suggests → Human decides
+Month 2: Agent acts → Human reviews after
+Month 3: Agent autonomous on low-risk
+Month 4: Agent handles critical with oversight
+```
+
+---
+
+## Agent Coordination Hints
+
+```yaml
+coordination:
+  topology: hierarchical
+  commander: qe-fleet-commander
+  memory_namespace: aqe/coordination
+  blackboard_topic: qe-fleet
+
+preload_skills:
+  - agentic-quality-engineering  # Always (this skill)
+  - risk-based-testing           # For prioritization
+  - quality-metrics              # For measurement
+
+agent_assignments:
+  qe-test-generator: [api-testing-patterns, tdd-london-chicago]
+  qe-coverage-analyzer: [quality-metrics, risk-based-testing]
+  qe-security-scanner: [security-testing, risk-based-testing]
+  qe-performance-tester: [performance-testing]
+```
+
+---
+
+## Related Skills
+- `holistic-testing-pact` - PACT principles deep dive
+- `risk-based-testing` - Prioritize agent focus
+- `quality-metrics` - Measure agent effectiveness
+- `api-testing-patterns`, `security-testing`, `performance-testing` - Specialized testing
+
+## Resources
+- Agent definitions: `.claude/agents/`
+- CLI: `aqe agent --help`
+- Fleet status: `aqe fleet status`
+
+---
+
+**Success Metric:** Deploy 10x more frequently with same or better quality through intelligent agent collaboration.

package/assets/skills/api-testing-patterns/SKILL.md

@@ -0,0 +1,294 @@
+---
+name: api-testing-patterns
+description: "Comprehensive API testing patterns including contract testing, REST/GraphQL testing, and integration testing. Use when testing APIs or designing API test strategies."
+category: testing-methodologies
+priority: high
+tokenEstimate: 1200
+agents: [qe-api-contract-validator, qe-test-generator, qe-performance-tester, qe-security-scanner]
+implementation_status: optimized
+optimization_version: 1.0
+last_optimized: 2025-12-02
+dependencies: []
+quick_reference_card: true
+tags: [api, rest, graphql, contract-testing, pact, integration, microservices]
+---
+
+# API Testing Patterns
+
+<default_to_action>
+When testing APIs or designing API test strategy:
+1. IDENTIFY testing level: contract, integration, or component
+2. TEST the contract, not implementation (consumer perspective)
+3. VALIDATE auth, input, errors, idempotency, concurrency
+4. AUTOMATE in CI/CD with schema validation
+5. MONITOR production APIs for contract drift
+
+**Quick Pattern Selection:**
+- Microservices → Consumer-driven contracts (Pact)
+- REST APIs → CRUD + pagination + filtering tests
+- GraphQL → Query validation + complexity limits
+- External deps → Mock with component testing
+- Performance → Load test critical endpoints
+
+**Critical Success Factors:**
+- APIs are contracts - test from consumer perspective
+- Always test error scenarios, not just happy paths
+- Version your API tests to prevent breaking changes
+</default_to_action>
+
+## Quick Reference Card
+
+### When to Use
+- Testing REST or GraphQL APIs
+- Validating microservice contracts
+- Designing API test strategies
+- Preventing breaking API changes
+
+### Testing Levels
+| Level | Purpose | Dependencies | Speed |
+|-------|---------|--------------|-------|
+| Contract | Provider-consumer agreement | None | Fast |
+| Component | API in isolation | Mocked | Fast |
+| Integration | Real dependencies | Database, services | Slower |
+
+### Critical Test Scenarios
+| Scenario | Must Test | Example |
+|----------|----------|---------|
+| Auth | 401/403 handling | Expired token, wrong user |
+| Input | 400 validation | Missing fields, wrong types |
+| Errors | 500 graceful handling | DB down, timeout |
+| Idempotency | Duplicate prevention | Same idempotency key |
+| Concurrency | Race conditions | Parallel checkout |
+
+### Tools
+- **Contract**: Pact, Spring Cloud Contract
+- **REST**: Supertest, REST-assured, Playwright
+- **Load**: k6, Artillery, JMeter
+
+### Agent Coordination
+- `qe-api-contract-validator`: Validate contracts, detect breaking changes
+- `qe-test-generator`: Generate tests from OpenAPI spec
+- `qe-performance-tester`: Load test endpoints
+- `qe-security-scanner`: API security testing
+
+---
+
+## Contract Testing
+
+**Pattern: Consumer-Driven Contracts**
+```javascript
+// Consumer defines expectations
+const contract = {
+  request: { method: 'POST', path: '/orders', body: { productId: 'abc', quantity: 2 } },
+  response: { status: 201, body: { orderId: 'string', total: 'number' } }
+};
+
+// Provider must fulfill
+test('order API meets contract', async () => {
+  const response = await api.post('/orders', { productId: 'abc', quantity: 2 });
+
+  expect(response.status).toBe(201);
+  expect(response.body).toMatchSchema({
+    orderId: expect.any(String),
+    total: expect.any(Number)
+  });
+});
+```
+
+**When:** Microservices, distributed systems, third-party integrations
+
+---
+
+## Critical Test Patterns
+
+### Authentication & Authorization
+```javascript
+describe('Auth', () => {
+  it('rejects without token', async () => {
+    expect((await api.get('/orders')).status).toBe(401);
+  });
+
+  it('rejects expired token', async () => {
+    const expired = generateExpiredToken();
+    expect((await api.get('/orders', { headers: { Authorization: `Bearer ${expired}` } })).status).toBe(401);
+  });
+
+  it('blocks cross-user access', async () => {
+    const userAToken = generateToken({ userId: 'A' });
+    expect((await api.get('/orders/user-B-order', { headers: { Authorization: `Bearer ${userAToken}` } })).status).toBe(403);
+  });
+});
+```
+
+### Input Validation
+```javascript
+describe('Validation', () => {
+  it('validates required fields', async () => {
+    const response = await api.post('/orders', { quantity: 2 }); // Missing productId
+    expect(response.status).toBe(400);
+    expect(response.body.errors).toContain('productId is required');
+  });
+
+  it('validates types', async () => {
+    expect((await api.post('/orders', { productId: 'abc', quantity: 'two' })).status).toBe(400);
+  });
+
+  it('validates ranges', async () => {
+    expect((await api.post('/orders', { productId: 'abc', quantity: -5 })).status).toBe(400);
+  });
+});
+```
+
+### Idempotency
+```javascript
+it('prevents duplicates with idempotency key', async () => {
+  const key = 'unique-123';
+  const data = { productId: 'abc', quantity: 2 };
+
+  const r1 = await api.post('/orders', data, { headers: { 'Idempotency-Key': key } });
+  const r2 = await api.post('/orders', data, { headers: { 'Idempotency-Key': key } });
+
+  expect(r1.body.orderId).toBe(r2.body.orderId); // Same order
+});
+```
+
+### Concurrency
+```javascript
+it('handles race condition on inventory', async () => {
+  const promises = Array(10).fill().map(() =>
+    api.post('/orders', { productId: 'abc', quantity: 1 })
+  );
+  const responses = await Promise.all(promises);
+  const successful = responses.filter(r => r.status === 201);
+
+  const inventory = await db.inventory.findById('abc');
+  expect(inventory.quantity).toBe(initialQuantity - successful.length);
+});
+```
+
+---
+
+## REST CRUD Pattern
+```javascript
+describe('Product CRUD', () => {
+  let productId;
+
+  it('CREATE', async () => {
+    const r = await api.post('/products', { name: 'Widget', price: 10 });
+    expect(r.status).toBe(201);
+    productId = r.body.id;
+  });
+
+  it('READ', async () => {
+    const r = await api.get(`/products/${productId}`);
+    expect(r.body.name).toBe('Widget');
+  });
+
+  it('UPDATE', async () => {
+    const r = await api.put(`/products/${productId}`, { price: 12 });
+    expect(r.body.price).toBe(12);
+  });
+
+  it('DELETE', async () => {
+    expect((await api.delete(`/products/${productId}`)).status).toBe(204);
+    expect((await api.get(`/products/${productId}`)).status).toBe(404);
+  });
+});
+```
+
+---
+
+## Best Practices
+
+### ✅ Do This
+- Test from consumer perspective
+- Use schema validation (not exact values)
+- Test error scenarios extensively
+- Version API tests
+- Automate in CI/CD
+
+### ❌ Avoid This
+- Testing implementation, not contract
+- Ignoring HTTP semantics (status codes)
+- No negative testing
+- Asserting on field order or extra fields
+- Slow tests (mock external services)
+
+---
+
+## Agent-Assisted API Testing
+
+```typescript
+// Validate contracts
+await Task("Contract Validation", {
+  spec: 'openapi.yaml',
+  endpoint: '/orders',
+  checkBreakingChanges: true
+}, "qe-api-contract-validator");
+
+// Generate tests from spec
+await Task("Generate API Tests", {
+  spec: 'openapi.yaml',
+  coverage: 'comprehensive',
+  include: ['happy-paths', 'input-validation', 'auth-scenarios', 'error-handling']
+}, "qe-test-generator");
+
+// Load test
+await Task("API Load Test", {
+  endpoint: '/orders',
+  rps: 1000,
+  duration: '5min'
+}, "qe-performance-tester");
+
+// Security scan
+await Task("API Security Scan", {
+  spec: 'openapi.yaml',
+  checks: ['sql-injection', 'xss', 'broken-auth', 'rate-limiting']
+}, "qe-security-scanner");
+```
+
+---
+
+## Agent Coordination Hints
+
+### Memory Namespace
+```
+aqe/api-testing/
+├── contracts/*        - API contract definitions
+├── generated-tests/*  - Generated test suites
+├── validation/*       - Contract validation results
+└── performance/*      - Load test results
+```
+
+### Fleet Coordination
+```typescript
+const apiFleet = await FleetManager.coordinate({
+  strategy: 'contract-testing',
+  agents: ['qe-api-contract-validator', 'qe-test-generator', 'qe-test-executor'],
+  topology: 'mesh'
+});
+
+await apiFleet.execute({
+  services: [
+    { name: 'orders-api', consumers: ['checkout-ui', 'admin-api'] },
+    { name: 'payment-api', consumers: ['orders-api'] }
+  ]
+});
+```
+
+---
+
+## Related Skills
+- [agentic-quality-engineering](../agentic-quality-engineering/) - API testing with agents
+- [tdd-london-chicago](../tdd-london-chicago/) - London school for API testing
+- [performance-testing](../performance-testing/) - API load testing
+- [security-testing](../security-testing/) - API security validation
+- [contract-testing](../contract-testing/) - Consumer-driven contracts deep dive
+
+---
+
+## Remember
+
+API testing = verifying contracts and behavior, not implementation. Focus on what matters to consumers: correct responses, proper error handling, acceptable performance.
+
+**With Agents:** Agents automate contract validation, generate comprehensive test suites from specs, and monitor production APIs for drift. Use agents to maintain API quality at scale.