agentic-qe 2.0.0 → 2.1.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.claude/agents/qx-partner.md +17 -4
- package/.claude/skills/accessibility-testing/SKILL.md +144 -692
- package/.claude/skills/agentic-quality-engineering/SKILL.md +176 -529
- package/.claude/skills/api-testing-patterns/SKILL.md +180 -560
- package/.claude/skills/brutal-honesty-review/SKILL.md +113 -603
- package/.claude/skills/bug-reporting-excellence/SKILL.md +116 -517
- package/.claude/skills/chaos-engineering-resilience/SKILL.md +127 -72
- package/.claude/skills/cicd-pipeline-qe-orchestrator/SKILL.md +209 -404
- package/.claude/skills/code-review-quality/SKILL.md +158 -608
- package/.claude/skills/compatibility-testing/SKILL.md +148 -38
- package/.claude/skills/compliance-testing/SKILL.md +132 -63
- package/.claude/skills/consultancy-practices/SKILL.md +114 -446
- package/.claude/skills/context-driven-testing/SKILL.md +117 -381
- package/.claude/skills/contract-testing/SKILL.md +176 -141
- package/.claude/skills/database-testing/SKILL.md +137 -130
- package/.claude/skills/exploratory-testing-advanced/SKILL.md +160 -629
- package/.claude/skills/holistic-testing-pact/SKILL.md +140 -188
- package/.claude/skills/localization-testing/SKILL.md +145 -33
- package/.claude/skills/mobile-testing/SKILL.md +132 -448
- package/.claude/skills/mutation-testing/SKILL.md +147 -41
- package/.claude/skills/performance-testing/SKILL.md +200 -546
- package/.claude/skills/quality-metrics/SKILL.md +164 -519
- package/.claude/skills/refactoring-patterns/SKILL.md +132 -699
- package/.claude/skills/regression-testing/SKILL.md +120 -926
- package/.claude/skills/risk-based-testing/SKILL.md +157 -660
- package/.claude/skills/security-testing/SKILL.md +199 -538
- package/.claude/skills/sherlock-review/SKILL.md +163 -699
- package/.claude/skills/shift-left-testing/SKILL.md +161 -465
- package/.claude/skills/shift-right-testing/SKILL.md +161 -519
- package/.claude/skills/six-thinking-hats/SKILL.md +175 -1110
- package/.claude/skills/skills-manifest.json +71 -20
- package/.claude/skills/tdd-london-chicago/SKILL.md +131 -448
- package/.claude/skills/technical-writing/SKILL.md +103 -154
- package/.claude/skills/test-automation-strategy/SKILL.md +166 -772
- package/.claude/skills/test-data-management/SKILL.md +126 -910
- package/.claude/skills/test-design-techniques/SKILL.md +179 -89
- package/.claude/skills/test-environment-management/SKILL.md +136 -91
- package/.claude/skills/test-reporting-analytics/SKILL.md +169 -92
- package/.claude/skills/testability-scoring/SKILL.md +172 -538
- package/.claude/skills/testability-scoring/scripts/generate-html-report.js +0 -0
- package/.claude/skills/visual-testing-advanced/SKILL.md +155 -78
- package/.claude/skills/xp-practices/SKILL.md +151 -587
- package/CHANGELOG.md +86 -0
- package/README.md +23 -16
- package/dist/agents/QXPartnerAgent.d.ts +47 -1
- package/dist/agents/QXPartnerAgent.d.ts.map +1 -1
- package/dist/agents/QXPartnerAgent.js +2086 -125
- package/dist/agents/QXPartnerAgent.js.map +1 -1
- package/dist/agents/lifecycle/AgentLifecycleManager.d.ts.map +1 -1
- package/dist/agents/lifecycle/AgentLifecycleManager.js +34 -31
- package/dist/agents/lifecycle/AgentLifecycleManager.js.map +1 -1
- package/dist/cli/commands/init-claude-md-template.d.ts.map +1 -1
- package/dist/cli/commands/init-claude-md-template.js +14 -0
- package/dist/cli/commands/init-claude-md-template.js.map +1 -1
- package/dist/core/SwarmCoordinator.d.ts +180 -0
- package/dist/core/SwarmCoordinator.d.ts.map +1 -0
- package/dist/core/SwarmCoordinator.js +473 -0
- package/dist/core/SwarmCoordinator.js.map +1 -0
- package/dist/core/memory/ReflexionMemoryAdapter.d.ts +109 -0
- package/dist/core/memory/ReflexionMemoryAdapter.d.ts.map +1 -0
- package/dist/core/memory/ReflexionMemoryAdapter.js +306 -0
- package/dist/core/memory/ReflexionMemoryAdapter.js.map +1 -0
- package/dist/core/memory/RuVectorPatternStore.d.ts +28 -0
- package/dist/core/memory/RuVectorPatternStore.d.ts.map +1 -1
- package/dist/core/memory/RuVectorPatternStore.js +70 -0
- package/dist/core/memory/RuVectorPatternStore.js.map +1 -1
- package/dist/core/memory/SparseVectorSearch.d.ts +55 -0
- package/dist/core/memory/SparseVectorSearch.d.ts.map +1 -0
- package/dist/core/memory/SparseVectorSearch.js +130 -0
- package/dist/core/memory/SparseVectorSearch.js.map +1 -0
- package/dist/core/memory/TieredCompression.d.ts +81 -0
- package/dist/core/memory/TieredCompression.d.ts.map +1 -0
- package/dist/core/memory/TieredCompression.js +270 -0
- package/dist/core/memory/TieredCompression.js.map +1 -0
- package/dist/core/memory/index.d.ts +6 -0
- package/dist/core/memory/index.d.ts.map +1 -1
- package/dist/core/memory/index.js +29 -1
- package/dist/core/memory/index.js.map +1 -1
- package/dist/core/metrics/MetricsAggregator.d.ts +228 -0
- package/dist/core/metrics/MetricsAggregator.d.ts.map +1 -0
- package/dist/core/metrics/MetricsAggregator.js +482 -0
- package/dist/core/metrics/MetricsAggregator.js.map +1 -0
- package/dist/core/metrics/index.d.ts +5 -0
- package/dist/core/metrics/index.d.ts.map +1 -0
- package/dist/core/metrics/index.js +11 -0
- package/dist/core/metrics/index.js.map +1 -0
- package/dist/core/optimization/SwarmOptimizer.d.ts +5 -0
- package/dist/core/optimization/SwarmOptimizer.d.ts.map +1 -1
- package/dist/core/optimization/SwarmOptimizer.js +17 -0
- package/dist/core/optimization/SwarmOptimizer.js.map +1 -1
- package/dist/core/orchestration/AdaptiveScheduler.d.ts +190 -0
- package/dist/core/orchestration/AdaptiveScheduler.d.ts.map +1 -0
- package/dist/core/orchestration/AdaptiveScheduler.js +460 -0
- package/dist/core/orchestration/AdaptiveScheduler.js.map +1 -0
- package/dist/core/orchestration/WorkflowOrchestrator.d.ts +13 -0
- package/dist/core/orchestration/WorkflowOrchestrator.d.ts.map +1 -1
- package/dist/core/orchestration/WorkflowOrchestrator.js +32 -0
- package/dist/core/orchestration/WorkflowOrchestrator.js.map +1 -1
- package/dist/core/recovery/CircuitBreaker.d.ts +176 -0
- package/dist/core/recovery/CircuitBreaker.d.ts.map +1 -0
- package/dist/core/recovery/CircuitBreaker.js +382 -0
- package/dist/core/recovery/CircuitBreaker.js.map +1 -0
- package/dist/core/recovery/RecoveryOrchestrator.d.ts +186 -0
- package/dist/core/recovery/RecoveryOrchestrator.d.ts.map +1 -0
- package/dist/core/recovery/RecoveryOrchestrator.js +476 -0
- package/dist/core/recovery/RecoveryOrchestrator.js.map +1 -0
- package/dist/core/recovery/RetryStrategy.d.ts +127 -0
- package/dist/core/recovery/RetryStrategy.d.ts.map +1 -0
- package/dist/core/recovery/RetryStrategy.js +314 -0
- package/dist/core/recovery/RetryStrategy.js.map +1 -0
- package/dist/core/recovery/index.d.ts +8 -0
- package/dist/core/recovery/index.d.ts.map +1 -0
- package/dist/core/recovery/index.js +27 -0
- package/dist/core/recovery/index.js.map +1 -0
- package/dist/core/skills/DependencyResolver.d.ts +99 -0
- package/dist/core/skills/DependencyResolver.d.ts.map +1 -0
- package/dist/core/skills/DependencyResolver.js +260 -0
- package/dist/core/skills/DependencyResolver.js.map +1 -0
- package/dist/core/skills/ManifestGenerator.d.ts +114 -0
- package/dist/core/skills/ManifestGenerator.d.ts.map +1 -0
- package/dist/core/skills/ManifestGenerator.js +449 -0
- package/dist/core/skills/ManifestGenerator.js.map +1 -0
- package/dist/core/skills/index.d.ts +9 -0
- package/dist/core/skills/index.d.ts.map +1 -0
- package/dist/core/skills/index.js +24 -0
- package/dist/core/skills/index.js.map +1 -0
- package/dist/mcp/handlers/chaos/chaos-inject-failure.d.ts +5 -0
- package/dist/mcp/handlers/chaos/chaos-inject-failure.d.ts.map +1 -1
- package/dist/mcp/handlers/chaos/chaos-inject-failure.js +36 -2
- package/dist/mcp/handlers/chaos/chaos-inject-failure.js.map +1 -1
- package/dist/mcp/handlers/chaos/chaos-inject-latency.d.ts +5 -0
- package/dist/mcp/handlers/chaos/chaos-inject-latency.d.ts.map +1 -1
- package/dist/mcp/handlers/chaos/chaos-inject-latency.js +36 -2
- package/dist/mcp/handlers/chaos/chaos-inject-latency.js.map +1 -1
- package/dist/mcp/server.d.ts +9 -9
- package/dist/mcp/server.d.ts.map +1 -1
- package/dist/mcp/server.js +1 -2
- package/dist/mcp/server.js.map +1 -1
- package/dist/types/qx.d.ts +113 -7
- package/dist/types/qx.d.ts.map +1 -1
- package/dist/types/qx.js.map +1 -1
- package/dist/visualization/api/RestEndpoints.js +1 -1
- package/dist/visualization/api/RestEndpoints.js.map +1 -1
- package/package.json +15 -54
|
@@ -1,598 +1,245 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: agentic-quality-engineering
|
|
3
|
-
description:
|
|
3
|
+
description: "AI agents as force multipliers for quality work. Core skill for all 18 QE agents using PACT principles."
|
|
4
|
+
category: qe-core
|
|
5
|
+
priority: critical
|
|
6
|
+
tokenEstimate: 1400
|
|
7
|
+
agents: [qe-test-generator, qe-test-executor, qe-coverage-analyzer, qe-quality-gate, qe-quality-analyzer, qe-performance-tester, qe-security-scanner, qe-requirements-validator, qe-production-intelligence, qe-fleet-commander, qe-deployment-readiness, qe-regression-risk-analyzer, qe-test-data-architect, qe-api-contract-validator, qe-flaky-test-hunter, qe-visual-tester, qe-chaos-engineer, qe-code-complexity]
|
|
8
|
+
implementation_status: optimized
|
|
9
|
+
optimization_version: 1.0
|
|
10
|
+
last_optimized: 2025-12-02
|
|
11
|
+
dependencies: []
|
|
12
|
+
quick_reference_card: true
|
|
13
|
+
tags: [pact, agents, fleet, coordination, autonomous, foundational]
|
|
4
14
|
---
|
|
5
15
|
|
|
6
16
|
# Agentic Quality Engineering
|
|
7
17
|
|
|
8
|
-
|
|
9
|
-
|
|
10
|
-
|
|
11
|
-
|
|
12
|
-
|
|
13
|
-
|
|
14
|
-
|
|
15
|
-
|
|
16
|
-
|
|
17
|
-
|
|
18
|
-
|
|
19
|
-
|
|
20
|
-
|
|
21
|
-
-
|
|
22
|
-
-
|
|
23
|
-
|
|
24
|
-
|
|
25
|
-
|
|
26
|
-
|
|
27
|
-
-
|
|
28
|
-
|
|
29
|
-
|
|
30
|
-
|
|
31
|
-
|
|
32
|
-
|
|
33
|
-
-
|
|
34
|
-
-
|
|
35
|
-
-
|
|
36
|
-
-
|
|
37
|
-
|
|
38
|
-
###
|
|
39
|
-
|
|
40
|
-
|
|
41
|
-
|
|
42
|
-
|
|
43
|
-
-
|
|
44
|
-
-
|
|
45
|
-
|
|
46
|
-
|
|
18
|
+
<default_to_action>
|
|
19
|
+
When implementing agentic QE or coordinating agents:
|
|
20
|
+
1. SPAWN appropriate agent(s) for the task using `Task` tool with agent type
|
|
21
|
+
2. CONFIGURE agent coordination (hierarchical/mesh/sequential)
|
|
22
|
+
3. EXECUTE with PACT principles: Proactive analysis, Autonomous operation, Collaborative feedback, Targeted risk focus
|
|
23
|
+
4. VALIDATE results through quality gates before deployment
|
|
24
|
+
5. LEARN from outcomes - store patterns in `aqe/learning/*` namespace
|
|
25
|
+
|
|
26
|
+
**Quick Agent Selection:**
|
|
27
|
+
- Test generation needed → `qe-test-generator`
|
|
28
|
+
- Coverage gaps → `qe-coverage-analyzer`
|
|
29
|
+
- Quality decision → `qe-quality-gate`
|
|
30
|
+
- Security scan → `qe-security-scanner`
|
|
31
|
+
- Performance test → `qe-performance-tester`
|
|
32
|
+
- Full pipeline → `qe-fleet-commander`
|
|
33
|
+
|
|
34
|
+
**Critical Success Factors:**
|
|
35
|
+
- Agents amplify human expertise, not replace it
|
|
36
|
+
- Human-in-the-loop for critical decisions
|
|
37
|
+
- Measure: bugs caught, time saved, coverage improved
|
|
38
|
+
</default_to_action>
|
|
39
|
+
|
|
40
|
+
## Quick Reference Card
|
|
41
|
+
|
|
42
|
+
### When to Use
|
|
43
|
+
- Designing autonomous testing systems
|
|
44
|
+
- Scaling QE with intelligent agents
|
|
45
|
+
- Implementing multi-agent coordination
|
|
46
|
+
- Building CI/CD quality pipelines
|
|
47
|
+
|
|
48
|
+
### PACT Principles
|
|
49
|
+
| Principle | Agent Behavior | Human Role |
|
|
50
|
+
|-----------|---------------|------------|
|
|
51
|
+
| **P**roactive | Analyze pre-merge, predict risk | Set guardrails |
|
|
52
|
+
| **A**utonomous | Execute tests, fix flaky tests | Review critical |
|
|
53
|
+
| **C**ollaborative | Multi-agent coordination | Provide context |
|
|
54
|
+
| **T**argeted | Risk-based prioritization | Define risk areas |
|
|
55
|
+
|
|
56
|
+
### 18-Agent Fleet
|
|
57
|
+
| Category | Agents | Primary Use |
|
|
58
|
+
|----------|--------|-------------|
|
|
59
|
+
| Core Testing (5) | test-generator, test-executor, coverage-analyzer, quality-gate, quality-analyzer | Daily testing |
|
|
60
|
+
| Performance/Security (2) | performance-tester, security-scanner | Non-functional |
|
|
61
|
+
| Strategic (3) | requirements-validator, production-intelligence, fleet-commander | Planning |
|
|
62
|
+
| Advanced (4) | regression-risk-analyzer, test-data-architect, api-contract-validator, flaky-test-hunter | Specialized |
|
|
63
|
+
| Visual/Chaos (2) | visual-tester, chaos-engineer | Edge cases |
|
|
64
|
+
| Deployment (1) | deployment-readiness | Release |
|
|
65
|
+
| Analysis (1) | code-complexity | Maintainability |
|
|
66
|
+
|
|
67
|
+
### Coordination Patterns
|
|
68
|
+
```
|
|
69
|
+
Hierarchical: fleet-commander → [generators] → [executors] → quality-gate
|
|
70
|
+
Mesh: test-gen ↔ coverage ↔ quality (peer decisions)
|
|
71
|
+
Sequential: risk-analyzer → test-gen → executor → coverage → gate
|
|
72
|
+
```
|
|
73
|
+
|
|
74
|
+
### Success Criteria
|
|
75
|
+
✅ 10x deployment frequency with same/better quality
|
|
76
|
+
✅ Coverage gaps detected in real-time
|
|
77
|
+
✅ Bugs caught pre-production
|
|
78
|
+
❌ Agents acting without human oversight on critical decisions
|
|
79
|
+
❌ Deploying all 18 agents at once (start with 1-2)
|
|
47
80
|
|
|
48
81
|
---
|
|
49
82
|
|
|
50
|
-
##
|
|
51
|
-
|
|
52
|
-
### Multi-Agent Fleet (17 Specialized Agents)
|
|
53
|
-
|
|
54
|
-
**Core Testing Agents (5):**
|
|
55
|
-
- `qe-test-generator` - AI-powered test generation with sublinear optimization
|
|
56
|
-
- `qe-test-executor` - Multi-framework parallel test execution
|
|
57
|
-
- `qe-coverage-analyzer` - Real-time gap detection with O(log n) algorithms
|
|
58
|
-
- `qe-quality-gate` - Intelligent quality gate with risk assessment
|
|
59
|
-
- `qe-quality-analyzer` - Comprehensive quality metrics analysis
|
|
60
|
-
|
|
61
|
-
**Performance & Security (2):**
|
|
62
|
-
- `qe-performance-tester` - Load testing with k6/JMeter/Gatling
|
|
63
|
-
- `qe-security-scanner` - SAST/DAST multi-layer scanning
|
|
64
|
-
|
|
65
|
-
**Strategic Planning (3):**
|
|
66
|
-
- `qe-requirements-validator` - INVEST criteria + BDD generation
|
|
67
|
-
- `qe-production-intelligence` - Production data to test scenarios
|
|
68
|
-
- `qe-fleet-commander` - Hierarchical fleet coordination (50+ agents)
|
|
83
|
+
## Core Concepts
|
|
69
84
|
|
|
70
|
-
|
|
71
|
-
|
|
85
|
+
### QE Evolution
|
|
86
|
+
| Stage | Approach | Limitation |
|
|
87
|
+
|-------|----------|------------|
|
|
88
|
+
| Traditional | Manual everything | Human bottleneck |
|
|
89
|
+
| Automation | Scripts + fixed scenarios | Needs orchestration |
|
|
90
|
+
| **Agentic** | AI agents + human judgment | Requires trust-building |
|
|
72
91
|
|
|
73
|
-
**
|
|
74
|
-
- `qe-regression-risk-analyzer` - ML-driven test selection
|
|
75
|
-
- `qe-test-data-architect` - High-speed realistic data (10k+ records/sec)
|
|
76
|
-
- `qe-api-contract-validator` - Breaking change detection
|
|
77
|
-
- `qe-flaky-test-hunter` - Statistical flakiness detection + auto-fix
|
|
92
|
+
**Core Premise:** Agents amplify human expertise for 10x scale.
|
|
78
93
|
|
|
79
|
-
|
|
80
|
-
- `qe-visual-tester` - Visual regression with AI comparison
|
|
81
|
-
- `qe-chaos-engineer` - Controlled fault injection
|
|
94
|
+
### Key Capabilities
|
|
82
95
|
|
|
83
|
-
|
|
84
|
-
|
|
85
|
-
**Hierarchical:**
|
|
86
|
-
```
|
|
87
|
-
qe-fleet-commander
|
|
88
|
-
├── qe-test-generator → qe-test-executor → qe-coverage-analyzer
|
|
89
|
-
├── qe-security-scanner + qe-performance-tester (parallel)
|
|
90
|
-
└── qe-quality-gate (final validation)
|
|
91
|
-
```
|
|
92
|
-
|
|
93
|
-
**Mesh (Peer-to-Peer):**
|
|
94
|
-
```
|
|
95
|
-
qe-test-generator ↔ qe-coverage-analyzer ↔ qe-quality-analyzer
|
|
96
|
-
↕ ↕ ↕
|
|
97
|
-
qe-requirements-validator ↔ qe-test-executor ↔ qe-quality-gate
|
|
98
|
-
```
|
|
99
|
-
|
|
100
|
-
**Sequential (Pipeline):**
|
|
101
|
-
```
|
|
102
|
-
Code Change → qe-regression-risk-analyzer → qe-test-generator →
|
|
103
|
-
qe-test-executor → qe-coverage-analyzer → qe-quality-gate → Deploy
|
|
104
|
-
```
|
|
105
|
-
|
|
106
|
-
---
|
|
107
|
-
|
|
108
|
-
## Key Capabilities
|
|
109
|
-
|
|
110
|
-
### 1. Intelligent Test Generation
|
|
111
|
-
|
|
112
|
-
**What agents do:**
|
|
113
|
-
- Analyze code changes (git diff)
|
|
114
|
-
- Identify changed functions and dependencies
|
|
115
|
-
- Generate relevant test scenarios
|
|
116
|
-
- Prioritize based on risk and coverage gaps
|
|
117
|
-
|
|
118
|
-
**Example:**
|
|
96
|
+
**1. Intelligent Test Generation**
|
|
119
97
|
```typescript
|
|
120
|
-
// Agent
|
|
121
|
-
|
|
122
|
-
|
|
123
|
-
}
|
|
124
|
-
|
|
125
|
-
// Agent generates:
|
|
126
|
-
// ✓ Happy path test
|
|
127
|
-
// ✓ Invalid token test
|
|
128
|
-
// ✓ Zero/negative amount test
|
|
129
|
-
// ✓ Network timeout test
|
|
130
|
-
// ✓ Idempotency test
|
|
131
|
-
```
|
|
132
|
-
|
|
133
|
-
**Human role:** Review generated tests, add domain-specific edge cases, validate test quality
|
|
134
|
-
|
|
135
|
-
### 2. Pattern Detection in Logs
|
|
136
|
-
|
|
137
|
-
**What agents do:**
|
|
138
|
-
- Scan thousands of log lines in seconds
|
|
139
|
-
- Identify anomaly patterns
|
|
140
|
-
- Correlate errors across services
|
|
141
|
-
- Detect performance degradation trends
|
|
142
|
-
|
|
143
|
-
**Example:**
|
|
144
|
-
```
|
|
145
|
-
Agent finds pattern:
|
|
146
|
-
2025-10-20T10:15:32 [ERROR] Payment timeout (customer_123)
|
|
147
|
-
2025-10-20T10:16:01 [ERROR] Payment timeout (customer_456)
|
|
148
|
-
2025-10-20T10:16:18 [ERROR] Payment timeout (customer_789)
|
|
149
|
-
|
|
150
|
-
Agent analysis:
|
|
151
|
-
→ 15 payment timeouts in 5 minutes
|
|
152
|
-
→ All timeouts to Stripe gateway
|
|
153
|
-
→ Started after deploy at 10:14:00
|
|
154
|
-
→ Recommendation: Rollback deployment
|
|
98
|
+
// Agent analyzes code change, generates targeted tests
|
|
99
|
+
const tests = await qeTestGenerator.generate(prDiff);
|
|
100
|
+
// → Happy path, edge cases, error handling tests
|
|
155
101
|
```
|
|
156
102
|
|
|
157
|
-
**
|
|
103
|
+
**2. Pattern Detection** - Scan logs, find anomalies, correlate errors
|
|
158
104
|
|
|
159
|
-
|
|
105
|
+
**3. Adaptive Strategy** - Adjust test focus based on risk signals
|
|
160
106
|
|
|
161
|
-
**
|
|
162
|
-
- Monitor test results and production incidents
|
|
163
|
-
- Adjust test focus based on risk signals
|
|
164
|
-
- Re-prioritize test execution
|
|
165
|
-
- Recommend new test coverage
|
|
107
|
+
**4. Root Cause Analysis** - Link failures to code changes, suggest fixes
|
|
166
108
|
|
|
167
|
-
|
|
168
|
-
```
|
|
169
|
-
Agent detects:
|
|
170
|
-
- 5 production incidents in checkout (last 7 days)
|
|
171
|
-
- Current test coverage: 60%
|
|
172
|
-
- Flaky test rate: 8%
|
|
173
|
-
|
|
174
|
-
Agent adapts:
|
|
175
|
-
→ Increase checkout test coverage to 90%
|
|
176
|
-
→ Add chaos testing for payment gateway
|
|
177
|
-
→ Fix/quarantine flaky tests
|
|
178
|
-
→ Run checkout tests on every commit
|
|
179
|
-
```
|
|
180
|
-
|
|
181
|
-
**Human role:** Approve strategy changes, validate risk assessment, set guardrails
|
|
182
|
-
|
|
183
|
-
### 4. Root Cause Analysis
|
|
109
|
+
---
|
|
184
110
|
|
|
185
|
-
|
|
186
|
-
- Correlate test failures across test suites
|
|
187
|
-
- Link failures to code changes
|
|
188
|
-
- Identify affected components
|
|
189
|
-
- Suggest likely root causes
|
|
111
|
+
## Agent Coordination
|
|
190
112
|
|
|
191
|
-
|
|
113
|
+
### Memory Namespaces
|
|
192
114
|
```
|
|
193
|
-
Test
|
|
194
|
-
|
|
195
|
-
|
|
196
|
-
|
|
197
|
-
|
|
198
|
-
→ Changed file: order-service.ts
|
|
199
|
-
→ Root cause: Missing null check on line 45
|
|
200
|
-
→ Confidence: 95%
|
|
201
|
-
```
|
|
202
|
-
|
|
203
|
-
**Human role:** Verify root cause, implement fix, validate solution
|
|
204
|
-
|
|
205
|
-
### 5. Documentation Generation
|
|
206
|
-
|
|
207
|
-
**What agents do:**
|
|
208
|
-
- Generate test reports
|
|
209
|
-
- Create API documentation from code
|
|
210
|
-
- Build quality dashboards
|
|
211
|
-
- Write test summaries
|
|
212
|
-
|
|
213
|
-
**Example:**
|
|
214
|
-
```markdown
|
|
215
|
-
# Sprint 42 Quality Report (Agent-Generated)
|
|
216
|
-
|
|
217
|
-
## Test Coverage
|
|
218
|
-
- Unit: 85% (↑ 3% from last sprint)
|
|
219
|
-
- Integration: 72% (↑ 5%)
|
|
220
|
-
- E2E: Critical paths at 100%
|
|
221
|
-
|
|
222
|
-
## Bugs Found
|
|
223
|
-
- Critical: 2 (fixed)
|
|
224
|
-
- High: 5 (4 fixed, 1 in progress)
|
|
225
|
-
- Medium: 12 (triaged)
|
|
226
|
-
|
|
227
|
-
## Risk Assessment
|
|
228
|
-
🔴 Payment gateway timeout (production incident)
|
|
229
|
-
🟡 Checkout flow performance degrading
|
|
230
|
-
🟢 Authentication stable
|
|
115
|
+
aqe/test-plan/* - Test planning decisions
|
|
116
|
+
aqe/coverage/* - Coverage analysis results
|
|
117
|
+
aqe/quality/* - Quality metrics and gates
|
|
118
|
+
aqe/learning/* - Patterns and Q-values
|
|
119
|
+
aqe/coordination/* - Cross-agent state
|
|
231
120
|
```
|
|
232
121
|
|
|
233
|
-
|
|
234
|
-
|
|
235
|
-
|
|
236
|
-
|
|
237
|
-
|
|
238
|
-
|
|
239
|
-
|
|
240
|
-
**Agents act before problems occur:**
|
|
241
|
-
- Analyze code changes pre-merge
|
|
242
|
-
- Predict high-risk areas
|
|
243
|
-
- Generate tests for new code
|
|
244
|
-
- Monitor trends in real-time
|
|
245
|
-
|
|
246
|
-
**Example:** Agent detects increasing error rate and generates alerts before customer impact
|
|
247
|
-
|
|
248
|
-
### Autonomous
|
|
249
|
-
**Agents work independently:**
|
|
250
|
-
- Execute tests without human trigger
|
|
251
|
-
- Prioritize test execution
|
|
252
|
-
- Generate test data
|
|
253
|
-
- Fix flaky tests automatically
|
|
254
|
-
|
|
255
|
-
**Example:** Agent detects flaky test, identifies root cause (timing issue), applies fix, creates PR
|
|
256
|
-
|
|
257
|
-
### Collaborative
|
|
258
|
-
**Agents work with humans and other agents:**
|
|
259
|
-
- Multi-agent coordination (test-gen → test-exec → coverage)
|
|
260
|
-
- Human-in-the-loop for critical decisions
|
|
261
|
-
- Share insights across team
|
|
262
|
-
- Learn from human feedback
|
|
263
|
-
|
|
264
|
-
**Example:** Agent generates tests, human reviews and adds domain knowledge, agent learns patterns
|
|
265
|
-
|
|
266
|
-
### Targeted
|
|
267
|
-
**Agents focus on high-value work:**
|
|
268
|
-
- Risk-based test prioritization
|
|
269
|
-
- Coverage of critical paths
|
|
270
|
-
- Ignore low-risk areas
|
|
271
|
-
- Optimize for impact
|
|
272
|
-
|
|
273
|
-
**Example:** Agent focuses 80% of testing on payment and auth (high risk) vs 20% on admin panel (low risk)
|
|
274
|
-
|
|
275
|
-
---
|
|
276
|
-
|
|
277
|
-
## Using with QE Agents
|
|
278
|
-
|
|
279
|
-
### Agent Assignment by Skill
|
|
280
|
-
|
|
281
|
-
Each of the 17 QE agents uses this foundational skill plus specialized skills:
|
|
122
|
+
### Blackboard Events
|
|
123
|
+
| Event | Trigger | Subscribers |
|
|
124
|
+
|-------|---------|-------------|
|
|
125
|
+
| `test:generated` | New tests created | executor, coverage |
|
|
126
|
+
| `coverage:gap` | Gap detected | test-generator |
|
|
127
|
+
| `quality:decision` | Gate evaluated | fleet-commander |
|
|
128
|
+
| `security:finding` | Vulnerability found | quality-gate |
|
|
282
129
|
|
|
283
|
-
|
|
284
|
-
- `agentic-quality-engineering` (core)
|
|
285
|
-
- `api-testing-patterns`
|
|
286
|
-
- `tdd-london-chicago`
|
|
287
|
-
- `test-automation-strategy`
|
|
288
|
-
|
|
289
|
-
**qe-coverage-analyzer:**
|
|
290
|
-
- `agentic-quality-engineering` (core)
|
|
291
|
-
- `quality-metrics`
|
|
292
|
-
- `risk-based-testing`
|
|
293
|
-
|
|
294
|
-
**qe-flaky-test-hunter:**
|
|
295
|
-
- `agentic-quality-engineering` (core)
|
|
296
|
-
- `exploratory-testing-advanced`
|
|
297
|
-
- `risk-based-testing`
|
|
298
|
-
|
|
299
|
-
**qe-security-scanner:**
|
|
300
|
-
- `agentic-quality-engineering` (core)
|
|
301
|
-
- `security-testing`
|
|
302
|
-
- `risk-based-testing`
|
|
303
|
-
|
|
304
|
-
*See `.claude/agents/` for complete agent definitions and skill mappings.*
|
|
305
|
-
|
|
306
|
-
### Agent Coordination Examples
|
|
307
|
-
|
|
308
|
-
**Example 1: PR Quality Gate**
|
|
130
|
+
### Example: PR Quality Pipeline
|
|
309
131
|
```typescript
|
|
310
|
-
// 1.
|
|
311
|
-
const
|
|
312
|
-
|
|
313
|
-
// 2. qe-test-generator creates targeted tests
|
|
314
|
-
const newTests = await agent.generateTests(riskAreas);
|
|
132
|
+
// 1. Risk analysis
|
|
133
|
+
const risks = await Task("Analyze PR", prDiff, "qe-regression-risk-analyzer");
|
|
315
134
|
|
|
316
|
-
//
|
|
317
|
-
const
|
|
135
|
+
// 2. Generate tests for risks
|
|
136
|
+
const tests = await Task("Generate tests", risks, "qe-test-generator");
|
|
318
137
|
|
|
319
|
-
//
|
|
320
|
-
const
|
|
138
|
+
// 3. Execute + analyze
|
|
139
|
+
const results = await Task("Run tests", tests, "qe-test-executor");
|
|
140
|
+
const coverage = await Task("Check coverage", results, "qe-coverage-analyzer");
|
|
321
141
|
|
|
322
|
-
//
|
|
323
|
-
const decision = await
|
|
324
|
-
// →
|
|
325
|
-
```
|
|
326
|
-
|
|
327
|
-
**Example 2: Production Intelligence Loop**
|
|
328
|
-
```typescript
|
|
329
|
-
// 1. qe-production-intelligence monitors production
|
|
330
|
-
const incidents = await agent.monitorProduction();
|
|
331
|
-
|
|
332
|
-
// 2. Agent converts incident to test scenario
|
|
333
|
-
const testScenario = await agent.incidentToTest(incidents[0]);
|
|
334
|
-
|
|
335
|
-
// 3. qe-test-generator implements test
|
|
336
|
-
const test = await agent.generateTest(testScenario);
|
|
337
|
-
|
|
338
|
-
// 4. qe-test-executor validates fix
|
|
339
|
-
const result = await agent.executeTest(test);
|
|
340
|
-
// → Test now prevents regression
|
|
142
|
+
// 4. Quality decision
|
|
143
|
+
const decision = await Task("Evaluate", {results, coverage}, "qe-quality-gate");
|
|
144
|
+
// → GO/NO-GO with rationale
|
|
341
145
|
```
|
|
342
146
|
|
|
343
147
|
---
|
|
344
148
|
|
|
345
|
-
##
|
|
149
|
+
## Implementation Phases
|
|
346
150
|
|
|
347
|
-
|
|
151
|
+
| Phase | Duration | Goal | Agent(s) |
|
|
152
|
+
|-------|----------|------|----------|
|
|
153
|
+
| Experiment | Weeks 1-4 | Validate one use case | 1 agent |
|
|
154
|
+
| Integrate | Months 2-3 | CI/CD pipeline | 3-4 agents |
|
|
155
|
+
| Scale | Months 4-6 | Multiple use cases | 8+ agents |
|
|
156
|
+
| Evolve | Ongoing | Continuous learning | Full fleet |
|
|
348
157
|
|
|
349
|
-
|
|
350
|
-
|
|
351
|
-
**Pick one agent + one use case:**
|
|
352
|
-
- `qe-test-generator` for PR test generation
|
|
353
|
-
- `qe-coverage-analyzer` for gap detection
|
|
354
|
-
- `qe-quality-gate` for automated quality checks
|
|
355
|
-
|
|
356
|
-
**Measure:**
|
|
357
|
-
- Tests generated per PR
|
|
358
|
-
- Coverage improvements
|
|
359
|
-
- Bugs caught before production
|
|
360
|
-
- Time saved
|
|
361
|
-
|
|
362
|
-
**Example:**
|
|
158
|
+
### Phase 1 Example
|
|
363
159
|
```bash
|
|
364
|
-
# Week 1: Deploy
|
|
160
|
+
# Week 1: Deploy single agent
|
|
365
161
|
aqe agent spawn qe-test-generator
|
|
366
162
|
|
|
367
|
-
#
|
|
368
|
-
# Track:
|
|
163
|
+
# Weeks 2-3: Generate tests for 10 PRs
|
|
164
|
+
# Track: bugs found, test quality, review time
|
|
369
165
|
|
|
370
166
|
# Week 4: Measure impact
|
|
371
167
|
aqe agent metrics qe-test-generator
|
|
372
|
-
#
|
|
373
|
-
```
|
|
374
|
-
|
|
375
|
-
### Phase 2: Integrate (Months 2-3)
|
|
376
|
-
|
|
377
|
-
**Goal:** Build into CI/CD pipeline
|
|
378
|
-
|
|
379
|
-
**Add agents to workflow:**
|
|
380
|
-
```yaml
|
|
381
|
-
# .github/workflows/quality-gate.yml
|
|
382
|
-
name: Agentic Quality Gate
|
|
383
|
-
|
|
384
|
-
on: [pull_request]
|
|
385
|
-
|
|
386
|
-
jobs:
|
|
387
|
-
quality-check:
|
|
388
|
-
runs-on: ubuntu-latest
|
|
389
|
-
steps:
|
|
390
|
-
- name: Analyze Risk
|
|
391
|
-
run: aqe agent run qe-regression-risk-analyzer
|
|
392
|
-
|
|
393
|
-
- name: Generate Tests
|
|
394
|
-
run: aqe agent run qe-test-generator
|
|
395
|
-
|
|
396
|
-
- name: Execute Tests
|
|
397
|
-
run: aqe agent run qe-test-executor
|
|
398
|
-
|
|
399
|
-
- name: Check Coverage
|
|
400
|
-
run: aqe agent run qe-coverage-analyzer
|
|
401
|
-
|
|
402
|
-
- name: Quality Gate
|
|
403
|
-
run: aqe agent run qe-quality-gate
|
|
404
|
-
```
|
|
405
|
-
|
|
406
|
-
**Create feedback loops:**
|
|
407
|
-
- Agents learn from which tests find bugs
|
|
408
|
-
- Humans label false positives
|
|
409
|
-
- System adapts over time
|
|
410
|
-
|
|
411
|
-
### Phase 3: Scale (Months 4-6)
|
|
412
|
-
|
|
413
|
-
**Goal:** Expand to multiple use cases
|
|
414
|
-
|
|
415
|
-
**Add more agents:**
|
|
416
|
-
- Performance testing (`qe-performance-tester`)
|
|
417
|
-
- Security scanning (`qe-security-scanner`)
|
|
418
|
-
- Flaky test detection (`qe-flaky-test-hunter`)
|
|
419
|
-
|
|
420
|
-
**Coordinate agents:**
|
|
421
|
-
```typescript
|
|
422
|
-
// Fleet coordination
|
|
423
|
-
const fleet = await FleetManager.init({
|
|
424
|
-
topology: 'hierarchical',
|
|
425
|
-
agents: [
|
|
426
|
-
'qe-fleet-commander',
|
|
427
|
-
'qe-test-generator',
|
|
428
|
-
'qe-test-executor',
|
|
429
|
-
'qe-coverage-analyzer',
|
|
430
|
-
'qe-security-scanner',
|
|
431
|
-
'qe-quality-gate'
|
|
432
|
-
]
|
|
433
|
-
});
|
|
434
|
-
|
|
435
|
-
// Commander orchestrates all agents
|
|
436
|
-
await fleet.commander.orchestrate(pullRequest);
|
|
437
|
-
```
|
|
438
|
-
|
|
439
|
-
### Phase 4: Evolve (Ongoing)
|
|
440
|
-
|
|
441
|
-
**Goal:** Continuous improvement through learning
|
|
442
|
-
|
|
443
|
-
**Agent learning:**
|
|
444
|
-
- Track success rates
|
|
445
|
-
- Learn from human corrections
|
|
446
|
-
- Adapt to codebase patterns
|
|
447
|
-
- Improve over time
|
|
448
|
-
|
|
449
|
-
**Metrics:**
|
|
450
|
-
```bash
|
|
451
|
-
aqe learn status --agent test-generator
|
|
452
|
-
# Shows: Learning progress, pattern recognition, success rate
|
|
168
|
+
# → Tests: 150, Bugs: 12, Time saved: 8h
|
|
453
169
|
```
|
|
454
170
|
|
|
455
171
|
---
|
|
456
172
|
|
|
457
|
-
##
|
|
458
|
-
|
|
459
|
-
### What Agents Can't Do (Yet)
|
|
460
|
-
|
|
461
|
-
**Business Context:**
|
|
462
|
-
- Agents don't understand "why" features exist
|
|
463
|
-
- Can't prioritize based on business value without guidance
|
|
464
|
-
- Need humans to explain domain constraints
|
|
465
|
-
|
|
466
|
-
**Ethical Judgment:**
|
|
467
|
-
- Agents can't make ethical decisions
|
|
468
|
-
- Can't balance competing priorities (speed vs quality)
|
|
469
|
-
- Need human oversight for critical decisions
|
|
173
|
+
## Limitations & Strengths
|
|
470
174
|
|
|
471
|
-
|
|
472
|
-
-
|
|
473
|
-
-
|
|
474
|
-
-
|
|
175
|
+
### Agents Excel At
|
|
176
|
+
- **Volume**: Scan thousands of logs in seconds
|
|
177
|
+
- **Patterns**: Find correlations humans miss
|
|
178
|
+
- **Tireless**: 24/7 testing and monitoring
|
|
179
|
+
- **Speed**: Instant code change analysis
|
|
475
180
|
|
|
476
|
-
|
|
477
|
-
-
|
|
478
|
-
-
|
|
479
|
-
-
|
|
480
|
-
|
|
481
|
-
### What Agents Excel At
|
|
482
|
-
|
|
483
|
-
**Data Volume:**
|
|
484
|
-
- Scan thousands of log lines in seconds
|
|
485
|
-
- Analyze entire codebases
|
|
486
|
-
- Process metrics from hundreds of services
|
|
487
|
-
|
|
488
|
-
**Pattern Detection:**
|
|
489
|
-
- Find correlations humans would miss
|
|
490
|
-
- Detect subtle anomalies
|
|
491
|
-
- Identify trends over time
|
|
492
|
-
|
|
493
|
-
**Tireless Repetition:**
|
|
494
|
-
- Run tests 24/7
|
|
495
|
-
- Monitor systems continuously
|
|
496
|
-
- Never get bored or tired
|
|
497
|
-
|
|
498
|
-
**Rapid Feedback:**
|
|
499
|
-
- Instant analysis of code changes
|
|
500
|
-
- Real-time test generation
|
|
501
|
-
- Immediate coverage feedback
|
|
181
|
+
### Agents Need Humans For
|
|
182
|
+
- Business context and priorities
|
|
183
|
+
- Ethical judgment and trade-offs
|
|
184
|
+
- Creative exploration ("what if" scenarios)
|
|
185
|
+
- Domain expertise (healthcare, finance, legal)
|
|
502
186
|
|
|
503
187
|
---
|
|
504
188
|
|
|
505
189
|
## Best Practices
|
|
506
190
|
|
|
507
|
-
|
|
508
|
-
|
|
509
|
-
|
|
510
|
-
|
|
511
|
-
|
|
512
|
-
|
|
513
|
-
|
|
191
|
+
| Do | Don't |
|
|
192
|
+
|----|-------|
|
|
193
|
+
| Start with one agent, one use case | Deploy all 18 at once |
|
|
194
|
+
| Build feedback loops early | Deploy and forget |
|
|
195
|
+
| Human reviews agent output | Auto-merge without review |
|
|
196
|
+
| Measure bugs caught, time saved | Track vanity metrics (test count) |
|
|
197
|
+
| Build trust gradually | Give full autonomy immediately |
|
|
514
198
|
|
|
515
|
-
|
|
516
|
-
❌ Deploy and forget
|
|
199
|
+
### Trust Progression
|
|
517
200
|
```
|
|
518
|
-
|
|
519
|
-
|
|
201
|
+
Month 1: Agent suggests → Human decides
|
|
202
|
+
Month 2: Agent acts → Human reviews after
|
|
203
|
+
Month 3: Agent autonomous on low-risk
|
|
204
|
+
Month 4: Agent handles critical with oversight
|
|
520
205
|
```
|
|
521
|
-
✅ Agent generates tests → Human reviews → Agent learns
|
|
522
|
-
❌ Agent generates tests → Auto-merge without review
|
|
523
206
|
|
|
524
|
-
|
|
525
|
-
❌ Agent decides to block deployment autonomously
|
|
207
|
+
---
|
|
526
208
|
|
|
527
|
-
|
|
528
|
-
❌ Agent takes action without human validation
|
|
529
|
-
```
|
|
209
|
+
## Agent Coordination Hints
|
|
530
210
|
|
|
531
|
-
|
|
532
|
-
|
|
533
|
-
|
|
534
|
-
-
|
|
535
|
-
|
|
536
|
-
-
|
|
537
|
-
- Developer confidence (survey)
|
|
538
|
-
|
|
539
|
-
Don't track:
|
|
540
|
-
- Number of tests generated (vanity metric)
|
|
541
|
-
- Agent uptime (not meaningful)
|
|
542
|
-
- Lines of code analyzed (doesn't show value)
|
|
543
|
-
```
|
|
211
|
+
```yaml
|
|
212
|
+
coordination:
|
|
213
|
+
topology: hierarchical
|
|
214
|
+
commander: qe-fleet-commander
|
|
215
|
+
memory_namespace: aqe/coordination
|
|
216
|
+
blackboard_topic: qe-fleet
|
|
544
217
|
|
|
545
|
-
|
|
546
|
-
|
|
547
|
-
|
|
548
|
-
|
|
549
|
-
|
|
550
|
-
|
|
218
|
+
preload_skills:
|
|
219
|
+
- agentic-quality-engineering # Always (this skill)
|
|
220
|
+
- risk-based-testing # For prioritization
|
|
221
|
+
- quality-metrics # For measurement
|
|
222
|
+
|
|
223
|
+
agent_assignments:
|
|
224
|
+
qe-test-generator: [api-testing-patterns, tdd-london-chicago]
|
|
225
|
+
qe-coverage-analyzer: [quality-metrics, risk-based-testing]
|
|
226
|
+
qe-security-scanner: [security-testing, risk-based-testing]
|
|
227
|
+
qe-performance-tester: [performance-testing]
|
|
551
228
|
```
|
|
552
229
|
|
|
553
230
|
---
|
|
554
231
|
|
|
555
232
|
## Related Skills
|
|
556
|
-
|
|
557
|
-
|
|
558
|
-
-
|
|
559
|
-
-
|
|
560
|
-
- [risk-based-testing](../risk-based-testing/) - Focus agents on high-risk areas
|
|
561
|
-
|
|
562
|
-
**Testing Specializations:**
|
|
563
|
-
- [api-testing-patterns](../api-testing-patterns/) - API testing with agents
|
|
564
|
-
- [performance-testing](../performance-testing/) - Load testing automation
|
|
565
|
-
- [security-testing](../security-testing/) - Security scanning agents
|
|
566
|
-
- [test-automation-strategy](../test-automation-strategy/) - Automation best practices
|
|
567
|
-
|
|
568
|
-
**Development Practices:**
|
|
569
|
-
- [tdd-london-chicago](../tdd-london-chicago/) - TDD with agent assistance
|
|
570
|
-
- [xp-practices](../xp-practices/) - Pair programming with agents
|
|
571
|
-
|
|
572
|
-
**Communication:**
|
|
573
|
-
- [technical-writing](../technical-writing/) - Agent-generated documentation
|
|
574
|
-
- [quality-metrics](../quality-metrics/) - Metrics for agent effectiveness
|
|
575
|
-
|
|
576
|
-
---
|
|
233
|
+
- `holistic-testing-pact` - PACT principles deep dive
|
|
234
|
+
- `risk-based-testing` - Prioritize agent focus
|
|
235
|
+
- `quality-metrics` - Measure agent effectiveness
|
|
236
|
+
- `api-testing-patterns`, `security-testing`, `performance-testing` - Specialized testing
|
|
577
237
|
|
|
578
238
|
## Resources
|
|
579
|
-
|
|
580
|
-
|
|
581
|
-
-
|
|
582
|
-
- [Agent Definitions](../../../.claude/agents/)
|
|
583
|
-
- [CLI Reference](../../../src/cli/)
|
|
584
|
-
|
|
585
|
-
**Learning:**
|
|
586
|
-
- Start with `qe-test-generator` for immediate value
|
|
587
|
-
- Use `aqe agent --help` for CLI commands
|
|
588
|
-
- Read agent-specific docs in `.claude/agents/`
|
|
589
|
-
|
|
590
|
-
**Community:**
|
|
591
|
-
- [GitHub Discussions](https://github.com/proffesor-for-testing/agentic-qe-cf/discussions)
|
|
592
|
-
- [Issue Tracker](https://github.com/proffesor-for-testing/agentic-qe-cf/issues)
|
|
239
|
+
- Agent definitions: `.claude/agents/`
|
|
240
|
+
- CLI: `aqe agent --help`
|
|
241
|
+
- Fleet status: `aqe fleet status`
|
|
593
242
|
|
|
594
243
|
---
|
|
595
244
|
|
|
596
|
-
**
|
|
597
|
-
|
|
598
|
-
**Success Metric:** Can your QE team confidently deploy 10x more frequently with the same or better quality? If yes, agentic QE is working.
|
|
245
|
+
**Success Metric:** Deploy 10x more frequently with same or better quality through intelligent agent collaboration.
|