agentic-qe 1.3.3 β 1.3.4
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.claude/agents/qe-api-contract-validator.md +20 -0
- package/.claude/agents/qe-chaos-engineer.md +20 -0
- package/.claude/agents/qe-coverage-analyzer.md +21 -0
- package/.claude/agents/qe-deployment-readiness.md +20 -0
- package/.claude/agents/qe-flaky-test-hunter.md +20 -0
- package/.claude/agents/qe-fleet-commander.md +20 -0
- package/.claude/agents/qe-performance-tester.md +21 -0
- package/.claude/agents/qe-production-intelligence.md +20 -0
- package/.claude/agents/qe-quality-analyzer.md +20 -0
- package/.claude/agents/qe-quality-gate.md +20 -0
- package/.claude/agents/qe-regression-risk-analyzer.md +20 -0
- package/.claude/agents/qe-requirements-validator.md +20 -0
- package/.claude/agents/qe-security-scanner.md +21 -0
- package/.claude/agents/qe-test-data-architect.md +19 -0
- package/.claude/agents/qe-test-executor.md +20 -0
- package/.claude/agents/qe-test-generator.md +22 -0
- package/.claude/agents/qe-visual-tester.md +22 -0
- package/README.md +320 -1019
- package/README.md.backup-20251026 +1366 -0
- package/bin/aqe-mcp +1 -1
- package/dist/agents/BaseAgent.js +1 -1
- package/dist/agents/BaseAgent.js.map +1 -1
- package/dist/agents/CoverageAnalyzerAgent.js +16 -16
- package/dist/agents/CoverageAnalyzerAgent.js.map +1 -1
- package/dist/agents/FlakyTestHunterAgent.js +1 -1
- package/dist/agents/FlakyTestHunterAgent.js.map +1 -1
- package/dist/cli/commands/init.d.ts.map +1 -1
- package/dist/cli/commands/init.js +71 -37
- package/dist/cli/commands/init.js.map +1 -1
- package/dist/core/memory/EnhancedAgentDBService.d.ts +127 -0
- package/dist/core/memory/EnhancedAgentDBService.d.ts.map +1 -0
- package/dist/core/memory/EnhancedAgentDBService.js +298 -0
- package/dist/core/memory/EnhancedAgentDBService.js.map +1 -0
- package/dist/core/neural/NeuralTrainer.js +2 -2
- package/dist/core/neural/NeuralTrainer.js.map +1 -1
- package/dist/learning/FixRecommendationEngine.d.ts +68 -0
- package/dist/learning/FixRecommendationEngine.d.ts.map +1 -0
- package/dist/learning/FixRecommendationEngine.js +500 -0
- package/dist/learning/FixRecommendationEngine.js.map +1 -0
- package/dist/learning/FlakyTestDetector.d.ts +19 -0
- package/dist/learning/FlakyTestDetector.d.ts.map +1 -1
- package/dist/learning/FlakyTestDetector.js +121 -2
- package/dist/learning/FlakyTestDetector.js.map +1 -1
- package/dist/learning/index.d.ts +2 -0
- package/dist/learning/index.d.ts.map +1 -1
- package/dist/learning/index.js +5 -0
- package/dist/learning/index.js.map +1 -1
- package/dist/learning/types.d.ts +11 -0
- package/dist/learning/types.d.ts.map +1 -1
- package/dist/reasoning/PatternQualityScorer.d.ts +134 -0
- package/dist/reasoning/PatternQualityScorer.d.ts.map +1 -0
- package/dist/reasoning/PatternQualityScorer.js +340 -0
- package/dist/reasoning/PatternQualityScorer.js.map +1 -0
- package/dist/reasoning/QEReasoningBank.d.ts +51 -4
- package/dist/reasoning/QEReasoningBank.d.ts.map +1 -1
- package/dist/reasoning/QEReasoningBank.js +173 -11
- package/dist/reasoning/QEReasoningBank.js.map +1 -1
- package/dist/reasoning/VectorSimilarity.d.ts +131 -0
- package/dist/reasoning/VectorSimilarity.d.ts.map +1 -0
- package/dist/reasoning/VectorSimilarity.js +250 -0
- package/dist/reasoning/VectorSimilarity.js.map +1 -0
- package/dist/reasoning/index.d.ts +8 -1
- package/dist/reasoning/index.d.ts.map +1 -1
- package/dist/reasoning/index.js +13 -2
- package/dist/reasoning/index.js.map +1 -1
- package/dist/streaming/BaseStreamHandler.d.ts +89 -0
- package/dist/streaming/BaseStreamHandler.d.ts.map +1 -0
- package/dist/streaming/BaseStreamHandler.js +168 -0
- package/dist/streaming/BaseStreamHandler.js.map +1 -0
- package/dist/streaming/TestGenerateStreamHandler.d.ts +103 -0
- package/dist/streaming/TestGenerateStreamHandler.d.ts.map +1 -0
- package/dist/streaming/TestGenerateStreamHandler.js +321 -0
- package/dist/streaming/TestGenerateStreamHandler.js.map +1 -0
- package/dist/streaming/index.d.ts +16 -0
- package/dist/streaming/index.d.ts.map +1 -0
- package/dist/streaming/index.js +39 -0
- package/dist/streaming/index.js.map +1 -0
- package/dist/utils/__mocks__/Logger.d.ts +26 -0
- package/dist/utils/__mocks__/Logger.d.ts.map +1 -0
- package/dist/utils/__mocks__/Logger.js +42 -0
- package/dist/utils/__mocks__/Logger.js.map +1 -0
- package/package.json +15 -3
- package/dist/agents/mixins/NeuralCapableMixin.d.ts +0 -130
- package/dist/agents/mixins/NeuralCapableMixin.d.ts.map +0 -1
- package/dist/agents/mixins/NeuralCapableMixin.js +0 -358
- package/dist/agents/mixins/NeuralCapableMixin.js.map +0 -1
- package/dist/agents/mixins/QUICCapableMixin.d.ts +0 -34
- package/dist/agents/mixins/QUICCapableMixin.d.ts.map +0 -1
- package/dist/agents/mixins/QUICCapableMixin.js +0 -346
- package/dist/agents/mixins/QUICCapableMixin.js.map +0 -1
- package/dist/core/security/CertificateValidator.d.ts +0 -130
- package/dist/core/security/CertificateValidator.d.ts.map +0 -1
- package/dist/core/security/CertificateValidator.js +0 -376
- package/dist/core/security/CertificateValidator.js.map +0 -1
- package/dist/core/transport/QUICTransport.d.ts +0 -62
- package/dist/core/transport/QUICTransport.d.ts.map +0 -1
- package/dist/core/transport/QUICTransport.js +0 -381
- package/dist/core/transport/QUICTransport.js.map +0 -1
- package/dist/core/transport/SecureQUICTransport.d.ts +0 -71
- package/dist/core/transport/SecureQUICTransport.d.ts.map +0 -1
- package/dist/core/transport/SecureQUICTransport.js +0 -253
- package/dist/core/transport/SecureQUICTransport.js.map +0 -1
- package/dist/learning/AdvancedFeatureExtractor.d.ts +0 -123
- package/dist/learning/AdvancedFeatureExtractor.d.ts.map +0 -1
- package/dist/learning/AdvancedFeatureExtractor.js +0 -423
- package/dist/learning/AdvancedFeatureExtractor.js.map +0 -1
- package/dist/learning/NeuralPatternMatcher.d.ts +0 -184
- package/dist/learning/NeuralPatternMatcher.d.ts.map +0 -1
- package/dist/learning/NeuralPatternMatcher.js +0 -702
- package/dist/learning/NeuralPatternMatcher.js.map +0 -1
- package/dist/learning/NeuralTrainer.d.ts +0 -209
- package/dist/learning/NeuralTrainer.d.ts.map +0 -1
- package/dist/learning/NeuralTrainer.js +0 -478
- package/dist/learning/NeuralTrainer.js.map +0 -1
- package/dist/transport/QUICTransport.d.ts +0 -340
- package/dist/transport/QUICTransport.d.ts.map +0 -1
- package/dist/transport/QUICTransport.js +0 -814
- package/dist/transport/QUICTransport.js.map +0 -1
- package/dist/transport/UDPTransport.d.ts +0 -348
- package/dist/transport/UDPTransport.d.ts.map +0 -1
- package/dist/transport/UDPTransport.js +0 -820
- package/dist/transport/UDPTransport.js.map +0 -1
package/README.md
CHANGED
|
@@ -7,409 +7,280 @@
|
|
|
7
7
|
[](https://www.typescriptlang.org/)
|
|
8
8
|
[](https://nodejs.org/)
|
|
9
9
|
|
|
10
|
-
**Version 1.3.
|
|
10
|
+
**Version 1.3.4** | [Changelog](CHANGELOG.md) | [Issues](https://github.com/proffesor-for-testing/agentic-qe/issues) | [Discussions](https://github.com/proffesor-for-testing/agentic-qe/discussions)
|
|
11
11
|
|
|
12
|
-
> Enterprise-grade test automation with AI learning, comprehensive skills library (
|
|
12
|
+
> Enterprise-grade test automation with AI learning, comprehensive skills library (34 QE skills), and optional 70-81% cost savings through intelligent model routing.
|
|
13
13
|
|
|
14
|
-
π§ **20% Continuous Improvement** | π **
|
|
15
|
-
|
|
16
|
-
[Quick Start](#quick-start) β’ [Documentation](docs/) β’ [Contributing](CONTRIBUTING.md) β’ [Examples](examples/)
|
|
14
|
+
π§ **20% Continuous Improvement** | π **34 World-Class QE Skills** | π― **100% Flaky Test Detection** | π° **70-81% Cost Savings** | π§ **61 MCP Tools**
|
|
17
15
|
|
|
18
16
|
</div>
|
|
19
17
|
|
|
20
18
|
---
|
|
21
19
|
|
|
22
|
-
##
|
|
23
|
-
|
|
24
|
-
**π Critical Bug Fixes**: Fixed missing `memory_store` database table and MCP server startup issues. Fleet initialization now works correctly, and Claude Code MCP integration is reliable with all 52 tools available.
|
|
25
|
-
|
|
26
|
-
### Key Fixes
|
|
27
|
-
|
|
28
|
-
- **Database Schema**: Added missing `memory_store` table for persistent agent memory
|
|
29
|
-
- **MCP Server**: Created standalone `aqe-mcp` binary + fixed module resolution
|
|
30
|
-
- **Impact**: Smooth fleet initialization and reliable Claude Code integration
|
|
20
|
+
## β‘ Quick Start
|
|
31
21
|
|
|
32
|
-
###
|
|
22
|
+
### Install & Initialize
|
|
33
23
|
|
|
34
24
|
```bash
|
|
35
|
-
|
|
36
|
-
|
|
37
|
-
aqe init
|
|
38
|
-
```
|
|
39
|
-
|
|
40
|
-
Update Claude Code MCP config:
|
|
41
|
-
```json
|
|
42
|
-
{
|
|
43
|
-
"mcpServers": {
|
|
44
|
-
"agentic-qe": {
|
|
45
|
-
"command": "aqe-mcp",
|
|
46
|
-
"args": []
|
|
47
|
-
}
|
|
48
|
-
}
|
|
49
|
-
}
|
|
50
|
-
```
|
|
51
|
-
|
|
52
|
-
### Previous Release (v1.3.2)
|
|
25
|
+
# Install globally
|
|
26
|
+
npm install -g agentic-qe
|
|
53
27
|
|
|
54
|
-
|
|
28
|
+
# Initialize your project
|
|
29
|
+
cd your-project
|
|
30
|
+
aqe init
|
|
55
31
|
|
|
56
|
-
|
|
32
|
+
# Add MCP server to Claude Code (optional)
|
|
33
|
+
claude mcp add agentic-qe npx -y agentic-qe mcp:start
|
|
57
34
|
|
|
58
|
-
|
|
35
|
+
# Verify connection
|
|
36
|
+
claude mcp list
|
|
37
|
+
```
|
|
59
38
|
|
|
60
|
-
###
|
|
39
|
+
### Use from Claude Code CLI
|
|
61
40
|
|
|
62
|
-
|
|
63
|
-
**2 days of intensive development** | **Security hardening + 17 new skills** | **11,500+ lines of expert content**
|
|
41
|
+
Ask Claude to use AQE agents directly from your terminal:
|
|
64
42
|
|
|
65
|
-
|
|
43
|
+
```bash
|
|
44
|
+
# Generate comprehensive tests
|
|
45
|
+
claude "Use qe-test-generator to create tests for src/services/user-service.ts with 95% coverage"
|
|
66
46
|
|
|
67
|
-
|
|
68
|
-
|
|
69
|
-
- risk-based-testing, test-automation-strategy, api-testing-patterns, performance-testing, security-testing
|
|
70
|
-
- tdd-london-chicago, xp-practices, code-review-quality, refactoring-patterns, quality-metrics
|
|
71
|
-
- bug-reporting-excellence, technical-writing, consultancy-practices
|
|
47
|
+
# Run quality pipeline
|
|
48
|
+
claude "Initialize AQE fleet: generate tests, execute them, analyze coverage, and run quality gate"
|
|
72
49
|
|
|
73
|
-
|
|
74
|
-
-
|
|
75
|
-
|
|
76
|
-
- **Testing Infrastructure (2)**: test-environment-management, test-reporting-analytics
|
|
50
|
+
# Detect flaky tests
|
|
51
|
+
claude "Use qe-flaky-test-hunter to analyze the last 100 test runs and identify flaky tests"
|
|
52
|
+
```
|
|
77
53
|
|
|
78
|
-
**
|
|
79
|
-
|
|
80
|
-
|
|
81
|
-
- β
|
|
82
|
-
- β
|
|
83
|
-
- β
|
|
84
|
-
- β
|
|
85
|
-
|
|
86
|
-
**Claude Flow Integration Skills (25 skills)**
|
|
87
|
-
- AgentDB: advanced, learning, memory-patterns, optimization, vector-search (5)
|
|
88
|
-
- GitHub: code-review, multi-repo, project-management, release-management, workflow-automation (5)
|
|
89
|
-
- Flow Nexus: neural, platform, swarm (3)
|
|
90
|
-
- Advanced: hive-mind-advanced, hooks-automation, pair-programming, performance-analysis, sparc-methodology, skill-builder, stream-chain, swarm-advanced, swarm-orchestration, reasoningbank-agentdb, reasoningbank-intelligence, verification-quality (12)
|
|
91
|
-
|
|
92
|
-
**Unified CLAUDE.md** - 72 Total Agents (18 QE + 54 Claude Flow)
|
|
93
|
-
|
|
94
|
-
#### π **AgentDB Integration - Production Hardening**
|
|
95
|
-
|
|
96
|
-
**Code Reduction: 2,290+ Lines Removed (95%)**
|
|
97
|
-
- 900 lines: Custom QUIC β AgentDB QUIC sync (<1ms latency, TLS 1.3)
|
|
98
|
-
- 800 lines: Custom neural β AgentDB learning plugins (9 RL algorithms)
|
|
99
|
-
- 896 lines: Mixins removed (QUICCapableMixin, NeuralCapableMixin)
|
|
100
|
-
- 590 lines: Wrapper removed (AgentDBIntegration)
|
|
101
|
-
|
|
102
|
-
**Performance Improvements** β‘
|
|
103
|
-
- QUIC Latency: 6.23ms β <1ms (84% faster)
|
|
104
|
-
- Vector Search: 150ms β 1ms (150x faster)
|
|
105
|
-
- Neural Training: 1000ms β 10-100ms (10-100x faster)
|
|
106
|
-
- Memory Usage: 512MB β 128-16MB (4-32x less)
|
|
107
|
-
- Startup Time: 500ms β 300ms (40% faster)
|
|
108
|
-
|
|
109
|
-
**Security Enhancements** π
|
|
110
|
-
- OWASP Compliance: 70% β 90%+ (+20 points)
|
|
111
|
-
- Vulnerabilities Fixed: 8 total (3 CRITICAL, 5 HIGH)
|
|
112
|
-
- TLS 1.3: Enforced by default
|
|
113
|
-
- Certificate Validation: Mandatory
|
|
114
|
-
|
|
115
|
-
#### β¨ **New Features**
|
|
116
|
-
|
|
117
|
-
**Advanced Search & Indexing**
|
|
118
|
-
- HNSW Indexing: 150x faster vector search (O(log n))
|
|
119
|
-
- Quantization: 4-32x memory reduction
|
|
120
|
-
- Vector Search: Semantic search across all memories
|
|
121
|
-
- Full-Text Search: BM25 ranking
|
|
122
|
-
|
|
123
|
-
**9 Reinforcement Learning Algorithms**
|
|
124
|
-
- Decision Transformer, Q-Learning, SARSA, Actor-Critic
|
|
125
|
-
- DQN, PPO, A3C, REINFORCE, Monte Carlo
|
|
126
|
-
|
|
127
|
-
**QUIC Synchronization**
|
|
128
|
-
- Sub-millisecond latency (<1ms)
|
|
129
|
-
- TLS 1.3 encryption by default
|
|
130
|
-
- Automatic connection recovery
|
|
131
|
-
- Stream multiplexing
|
|
132
|
-
|
|
133
|
-
#### π§ͺ **Test Suite Expansion**
|
|
134
|
-
- 60+ new test files added
|
|
135
|
-
- AgentDB: 6/6 tests (100%)
|
|
136
|
-
- Core: 53/53 tests (100%)
|
|
137
|
-
- Total: 59/59 tests passing (100%)
|
|
138
|
-
- Zero regressions detected
|
|
139
|
-
|
|
140
|
-
#### π§Ή **Repository Cleanup**
|
|
141
|
-
- Documentation: 24 reports archived (4.1MB saved)
|
|
142
|
-
- Dependencies: 89 packages removed (7.3MB saved)
|
|
143
|
-
- Total: 11.4MB savings
|
|
144
|
-
- Clean build: Zero TypeScript errors
|
|
145
|
-
|
|
146
|
-
#### π **Breaking Changes**
|
|
147
|
-
- `enableQUIC()` β `initializeAgentDB({ quic: {...} })`
|
|
148
|
-
- `enableNeural()` β `initializeAgentDB({ learning: {...} })`
|
|
149
|
-
- Removed: QUICTransport, NeuralPatternMatcher, mixins
|
|
150
|
-
- See [Migration Guide](docs/AGENTDB-MIGRATION-GUIDE.md)
|
|
151
|
-
|
|
152
|
-
**Release Score: 90/100** β
| [Complete Changelog](docs/COMPLETE-1.2.0-CHANGELOG.md)
|
|
54
|
+
**What gets initialized:**
|
|
55
|
+
- β
Multi-Model Router (70-81% cost savings - opt-in)
|
|
56
|
+
- β
Learning System (20% improvement target)
|
|
57
|
+
- β
Pattern Bank (cross-project reuse)
|
|
58
|
+
- β
ML Flaky Detection (100% accuracy)
|
|
59
|
+
- β
18 Specialized agent definitions
|
|
60
|
+
- β
8 AQE slash commands
|
|
61
|
+
- β
Configuration directory
|
|
153
62
|
|
|
154
63
|
---
|
|
155
64
|
|
|
156
|
-
##
|
|
157
|
-
|
|
158
|
-
### Intelligence Boost Release (Previous)
|
|
159
|
-
|
|
160
|
-
**Learning System** π§
|
|
161
|
-
- Q-learning reinforcement learning for strategy optimization
|
|
162
|
-
- 20% improvement target tracking with automatic achievement
|
|
163
|
-
- Experience replay buffer (10,000 experiences)
|
|
164
|
-
- Automatic strategy recommendation with 95%+ confidence
|
|
165
|
-
- Cross-agent knowledge sharing
|
|
166
|
-
|
|
167
|
-
**Pattern Bank** π¦
|
|
168
|
-
- Cross-project pattern sharing and reuse
|
|
169
|
-
- 85%+ matching accuracy with AI-powered similarity
|
|
170
|
-
- 6 framework support (Jest, Mocha, Cypress, Vitest, Jasmine, AVA)
|
|
171
|
-
- Automatic pattern extraction from existing tests
|
|
172
|
-
- Pattern quality scoring and versioning
|
|
173
|
-
|
|
174
|
-
**ML Flaky Detection** π―
|
|
175
|
-
- 100% detection accuracy (target: 90%)
|
|
176
|
-
- 0% false positive rate (target: < 5%)
|
|
177
|
-
- Root cause analysis (timing, race conditions, dependencies, isolation)
|
|
178
|
-
- Automated fix recommendations with code examples
|
|
179
|
-
- < 1 second processing time for 1000+ test results
|
|
65
|
+
## β¨ Features
|
|
180
66
|
|
|
181
|
-
|
|
182
|
-
-
|
|
183
|
-
-
|
|
184
|
-
-
|
|
185
|
-
-
|
|
186
|
-
|
|
187
|
-
**Enhanced Agents:**
|
|
188
|
-
- **TestGeneratorAgent**: Pattern-based generation (20%+ faster)
|
|
189
|
-
- **CoverageAnalyzerAgent**: Learning-enhanced analysis
|
|
190
|
-
- **FlakyTestHunterAgent**: ML-based detection (99% accuracy)
|
|
191
|
-
|
|
192
|
-
See [CHANGELOG.md](CHANGELOG.md) for full details.
|
|
193
|
-
|
|
194
|
-
---
|
|
195
|
-
|
|
196
|
-
## π Features
|
|
67
|
+
### π€ Autonomous Agent Fleet
|
|
68
|
+
- **18 Specialized Agents**: Expert agents for every QE domain (test generation, coverage analysis, security scanning, performance testing)
|
|
69
|
+
- **AI-Powered Coordination**: Event-driven architecture with intelligent task distribution
|
|
70
|
+
- **Zero External Dependencies**: Native AQE hooks system (100-500x faster than external coordination)
|
|
71
|
+
- **Scalable**: From single developer projects to enterprise-scale testing infrastructure
|
|
197
72
|
|
|
198
|
-
###
|
|
73
|
+
### π§ Intelligence & Learning (v1.1.0)
|
|
74
|
+
- **Q-Learning System**: 20% improvement target with automatic strategy optimization
|
|
75
|
+
- **Pattern Bank**: 85%+ matching accuracy across 6 test frameworks (Jest, Mocha, Cypress, Vitest, Jasmine, AVA)
|
|
76
|
+
- **ML Flaky Detection**: 100% accuracy with root cause analysis and automated fix recommendations
|
|
77
|
+
- **Continuous Improvement**: A/B testing framework with 95%+ statistical confidence
|
|
78
|
+
- **Experience Replay**: Learn from 10,000+ past executions
|
|
199
79
|
|
|
200
|
-
|
|
201
|
-
- **70-81%
|
|
80
|
+
### π° Cost Optimization (v1.0.5)
|
|
81
|
+
- **Multi-Model Router**: 70-81% cost savings through intelligent model selection (opt-in feature)
|
|
202
82
|
- **4+ AI Models**: GPT-3.5, GPT-4, Claude Haiku, Claude Sonnet 4.5
|
|
203
|
-
- **Smart Routing**: Automatic complexity analysis and model selection
|
|
204
|
-
- **Real-Time Tracking**: Live cost monitoring with
|
|
205
|
-
- **Budget Alerts**: Email, Slack, and webhook notifications
|
|
206
|
-
- **Cost Forecasting**: Predict future costs with 90% confidence
|
|
83
|
+
- **Smart Routing**: Automatic complexity analysis and optimal model selection
|
|
84
|
+
- **Real-Time Tracking**: Live cost monitoring with budget alerts and forecasting
|
|
207
85
|
- **ROI Dashboard**: Track savings vs single-model baseline
|
|
208
86
|
|
|
209
|
-
|
|
210
|
-
- **
|
|
211
|
-
- **
|
|
212
|
-
- **
|
|
213
|
-
- **
|
|
214
|
-
- **
|
|
215
|
-
- **Cancellation Support**: Stop operations mid-stream
|
|
216
|
-
- **Event Piping**: Chain multiple operations together
|
|
87
|
+
### π Comprehensive Testing
|
|
88
|
+
- **Multi-Framework Support**: Jest, Mocha, Cypress, Playwright, Vitest, Jasmine, AVA
|
|
89
|
+
- **Parallel Execution**: 10,000+ concurrent tests with intelligent orchestration
|
|
90
|
+
- **Real-Time Coverage**: O(log n) algorithms for instant gap detection
|
|
91
|
+
- **Performance Testing**: k6, JMeter, Gatling integration
|
|
92
|
+
- **Real-Time Streaming**: Live progress updates for all operations
|
|
217
93
|
|
|
218
|
-
###
|
|
94
|
+
### π 34 QE Skills Library (v1.3.0)
|
|
95
|
+
**95%+ coverage of modern QE practices**
|
|
219
96
|
|
|
220
|
-
|
|
221
|
-
|
|
222
|
-
- **20% Improvement Target**: Automatic tracking and achievement
|
|
223
|
-
- **Experience Replay**: Learn from 10,000+ past executions
|
|
224
|
-
- **Strategy Recommendation**: AI-powered optimal strategy selection
|
|
225
|
-
- **Cross-Agent Sharing**: Agents learn from each other's experiences
|
|
226
|
-
|
|
227
|
-
#### π¦ Pattern Bank
|
|
228
|
-
- **Pattern Extraction**: Automatic extraction from existing tests
|
|
229
|
-
- **85%+ Matching Accuracy**: AI-powered pattern matching
|
|
230
|
-
- **Cross-Project Sharing**: Export/import patterns across teams
|
|
231
|
-
- **6 Framework Support**: Jest, Mocha, Cypress, Vitest, Jasmine, AVA
|
|
232
|
-
- **Quality Scoring**: Automatic pattern quality assessment
|
|
233
|
-
|
|
234
|
-
#### π― ML Flaky Detection
|
|
235
|
-
- **100% Accuracy**: ML-based detection with zero false positives
|
|
236
|
-
- **Root Cause Analysis**: Identify timing, race conditions, dependencies
|
|
237
|
-
- **Automated Fixes**: Generate fix recommendations with code examples
|
|
238
|
-
- **Continuous Monitoring**: Track test reliability over time
|
|
239
|
-
- **< 1 Second Processing**: 8,000+ test results per second
|
|
240
|
-
|
|
241
|
-
#### π Continuous Improvement
|
|
242
|
-
- **A/B Testing**: Compare strategies with statistical confidence
|
|
243
|
-
- **Auto-Optimization**: Apply winning strategies automatically
|
|
244
|
-
- **Failure Pattern Analysis**: Detect and mitigate recurring issues
|
|
245
|
-
- **Performance Benchmarks**: < 50ms pattern matching, < 100ms learning
|
|
246
|
-
|
|
247
|
-
### Core Features
|
|
248
|
-
|
|
249
|
-
#### π€ Autonomous Agent Fleet
|
|
250
|
-
- **18 Specialized Agents**: Each agent is an expert in specific quality engineering domains
|
|
251
|
-
- _Note: 17 QE-specific agents + 1 general-purpose base-template-generator agent_
|
|
252
|
-
- **AQE Hooks System**: 100-500x faster coordination with zero external dependencies
|
|
253
|
-
- **Intelligent Coordination**: Event-driven architecture with automatic task distribution
|
|
254
|
-
- **Scalable**: From single developer projects to enterprise-scale testing infrastructure
|
|
255
|
-
- **Self-Organizing**: Agents autonomously coordinate testing strategies
|
|
256
|
-
- **Type-Safe**: Full TypeScript type checking and IntelliSense support
|
|
97
|
+
<details>
|
|
98
|
+
<summary><b>View All Skills</b></summary>
|
|
257
99
|
|
|
258
|
-
|
|
259
|
-
- **
|
|
260
|
-
- **
|
|
261
|
-
- **
|
|
262
|
-
- **
|
|
100
|
+
**Phase 1: Original Quality Engineering Skills (17 skills)**
|
|
101
|
+
- **Core Testing**: agentic-quality-engineering, holistic-testing-pact, context-driven-testing, exploratory-testing-advanced
|
|
102
|
+
- **Methodologies**: tdd-london-chicago, xp-practices, risk-based-testing, test-automation-strategy
|
|
103
|
+
- **Techniques**: api-testing-patterns, performance-testing, security-testing
|
|
104
|
+
- **Code Quality**: code-review-quality, refactoring-patterns, quality-metrics
|
|
105
|
+
- **Communication**: bug-reporting-excellence, technical-writing, consultancy-practices
|
|
263
106
|
|
|
264
|
-
|
|
265
|
-
- **
|
|
266
|
-
- **
|
|
267
|
-
- **
|
|
268
|
-
- **Visual Regression**: AI-powered screenshot comparison and UI validation
|
|
107
|
+
**Phase 2: Expanded QE Skills Library (17 skills)**
|
|
108
|
+
- **Testing Methodologies (6)**: regression-testing, shift-left-testing, shift-right-testing, test-design-techniques, mutation-testing, test-data-management
|
|
109
|
+
- **Specialized Testing (9)**: accessibility-testing, mobile-testing, database-testing, contract-testing, chaos-engineering-resilience, compatibility-testing, localization-testing, compliance-testing, visual-testing-advanced
|
|
110
|
+
- **Testing Infrastructure (2)**: test-environment-management, test-reporting-analytics
|
|
269
111
|
|
|
270
|
-
|
|
271
|
-
- **API Contract Validation**: Breaking change detection across versions
|
|
272
|
-
- **Test Data Generation**: 10,000+ realistic records per second
|
|
273
|
-
- **Production Intelligence**: Convert production incidents into test scenarios
|
|
274
|
-
- **Chaos Engineering**: Controlled fault injection for resilience testing
|
|
112
|
+
</details>
|
|
275
113
|
|
|
276
114
|
---
|
|
277
115
|
|
|
278
|
-
##
|
|
116
|
+
## π» Usage Examples
|
|
279
117
|
|
|
280
|
-
###
|
|
118
|
+
### Example 1: Single Agent Execution
|
|
281
119
|
|
|
282
|
-
|
|
283
|
-
|
|
284
|
-
|
|
285
|
-
-
|
|
120
|
+
Ask Claude to use a specific agent:
|
|
121
|
+
|
|
122
|
+
```bash
|
|
123
|
+
claude "Use the qe-test-generator agent to create comprehensive tests for src/services/user-service.ts with 95% coverage"
|
|
124
|
+
```
|
|
286
125
|
|
|
287
|
-
|
|
288
|
-
|
|
289
|
-
|
|
290
|
-
|
|
291
|
-
|
|
292
|
-
npx claude-flow@alpha init --force
|
|
293
|
-
```
|
|
126
|
+
**What happens:**
|
|
127
|
+
1. Claude Code spawns qe-test-generator via Task tool
|
|
128
|
+
2. Agent analyzes the source file
|
|
129
|
+
3. Generates tests with pattern matching (Phase 2 feature)
|
|
130
|
+
4. Stores results in memory at `aqe/test-plan/generated`
|
|
294
131
|
|
|
295
|
-
**
|
|
132
|
+
**Output:**
|
|
133
|
+
```
|
|
134
|
+
Generated 42 tests
|
|
135
|
+
Pattern hit rate: 67%
|
|
136
|
+
Time saved: 2.3s
|
|
137
|
+
Quality score: 96%
|
|
138
|
+
```
|
|
296
139
|
|
|
297
|
-
###
|
|
140
|
+
### Example 2: Multi-Agent Parallel Execution
|
|
298
141
|
|
|
299
|
-
|
|
142
|
+
Coordinate multiple agents at once:
|
|
300
143
|
|
|
301
|
-
|
|
144
|
+
```bash
|
|
145
|
+
claude "Initialize the AQE fleet:
|
|
146
|
+
1. Use qe-test-generator to create tests for src/services/*.ts
|
|
147
|
+
2. Use qe-test-executor to run all tests in parallel
|
|
148
|
+
3. Use qe-coverage-analyzer to find gaps with sublinear algorithms
|
|
149
|
+
4. Use qe-quality-gate to validate against 95% threshold"
|
|
150
|
+
```
|
|
302
151
|
|
|
303
|
-
|
|
304
|
-
|
|
305
|
-
|
|
152
|
+
**What happens:**
|
|
153
|
+
1. Claude spawns 4 agents concurrently in a single message
|
|
154
|
+
2. Agents coordinate through `aqe/*` memory namespace
|
|
155
|
+
3. Pipeline: test generator β executor β analyzer β gate
|
|
156
|
+
4. Real-time streaming progress updates
|
|
306
157
|
|
|
307
|
-
|
|
308
|
-
|
|
309
|
-
|
|
158
|
+
**Memory namespaces:**
|
|
159
|
+
- `aqe/test-plan/*` - Test planning and requirements
|
|
160
|
+
- `aqe/coverage/*` - Coverage analysis results
|
|
161
|
+
- `aqe/performance/*` - Performance test data
|
|
162
|
+
- `aqe/quality/*` - Quality metrics
|
|
310
163
|
|
|
311
|
-
|
|
312
|
-
```bash
|
|
313
|
-
npm install --save-dev agentic-qe
|
|
164
|
+
### Example 3: Using Agents with Skills
|
|
314
165
|
|
|
315
|
-
|
|
316
|
-
npx aqe init
|
|
317
|
-
```
|
|
166
|
+
Agents automatically leverage skills:
|
|
318
167
|
|
|
319
|
-
|
|
320
|
-
|
|
321
|
-
|
|
322
|
-
cd agentic-qe
|
|
323
|
-
npm install
|
|
324
|
-
npm run build
|
|
325
|
-
npm link
|
|
326
|
-
```
|
|
168
|
+
```bash
|
|
169
|
+
claude "Use qe-test-generator with shift-left-testing and test-design-techniques skills to create tests before implementing the new payment feature"
|
|
170
|
+
```
|
|
327
171
|
|
|
328
|
-
|
|
172
|
+
**Available skills** (agents auto-select from 34):
|
|
173
|
+
- TDD, API testing, performance, security
|
|
174
|
+
- Accessibility, mobile, chaos engineering
|
|
175
|
+
- Regression, shift-left/right, compliance
|
|
329
176
|
|
|
330
|
-
|
|
331
|
-
- **OS**: Linux, macOS, Windows (via WSL2)
|
|
332
|
-
- **Agent Execution**: Via Claude Code's Task tool or MCP integration
|
|
177
|
+
### Example 4: Full Quality Pipeline
|
|
333
178
|
|
|
334
|
-
|
|
179
|
+
End-to-end quality workflow:
|
|
335
180
|
|
|
336
|
-
|
|
181
|
+
```bash
|
|
182
|
+
claude "Run the full AQE quality pipeline:
|
|
183
|
+
1. qe-requirements-validator - validate requirements are testable
|
|
184
|
+
2. qe-test-generator - generate comprehensive test suite
|
|
185
|
+
3. qe-test-executor - run tests with parallel execution
|
|
186
|
+
4. qe-coverage-analyzer - analyze gaps using O(log n) algorithms
|
|
187
|
+
5. qe-flaky-test-hunter - detect flaky tests with 100% ML accuracy
|
|
188
|
+
6. qe-security-scanner - run SAST/DAST scans
|
|
189
|
+
7. qe-performance-tester - load test critical paths
|
|
190
|
+
8. qe-quality-gate - validate all quality criteria met
|
|
191
|
+
9. qe-deployment-readiness - assess deployment risk"
|
|
192
|
+
```
|
|
337
193
|
|
|
338
|
-
###
|
|
194
|
+
### Example 5: Specialized Testing Scenarios
|
|
339
195
|
|
|
340
196
|
```bash
|
|
341
|
-
#
|
|
342
|
-
|
|
197
|
+
# API contract validation
|
|
198
|
+
claude "Use qe-api-contract-validator to check if the new API changes break any existing contracts"
|
|
343
199
|
|
|
344
|
-
#
|
|
345
|
-
claude
|
|
200
|
+
# Visual regression testing
|
|
201
|
+
claude "Use qe-visual-tester to compare screenshots of the updated dashboard against baseline"
|
|
346
202
|
|
|
347
|
-
#
|
|
348
|
-
claude
|
|
203
|
+
# Chaos engineering
|
|
204
|
+
claude "Use qe-chaos-engineer to inject random failures and validate system resilience"
|
|
205
|
+
|
|
206
|
+
# Flaky test detection with ML
|
|
207
|
+
claude "Use qe-flaky-test-hunter to analyze the last 100 test runs and identify flaky tests with ML-powered root cause analysis"
|
|
349
208
|
```
|
|
350
209
|
|
|
351
|
-
###
|
|
210
|
+
### Example 6: Fleet Coordination at Scale
|
|
352
211
|
|
|
353
212
|
```bash
|
|
354
|
-
#
|
|
355
|
-
|
|
356
|
-
aqe init
|
|
213
|
+
# Coordinate 50+ agents for large projects
|
|
214
|
+
claude "Use qe-fleet-commander to coordinate parallel testing across 8 microservices with 50 agents total"
|
|
357
215
|
```
|
|
358
216
|
|
|
359
|
-
|
|
360
|
-
- β
Multi-Model Router (70-81% cost savings)
|
|
361
|
-
- β
Learning System (20% improvement target)
|
|
362
|
-
- β
Pattern Bank (cross-project reuse)
|
|
363
|
-
- β
ML Flaky Detection (100% accuracy)
|
|
364
|
-
- β
Improvement Loop (A/B testing)
|
|
365
|
-
- β
17 Specialized QE agent definitions (+ 1 general-purpose agent)
|
|
366
|
-
- β
8 AQE slash commands
|
|
367
|
-
- β
Configuration directory
|
|
217
|
+
### MCP Integration Examples
|
|
368
218
|
|
|
369
|
-
|
|
219
|
+
You can also use agents through MCP tools:
|
|
370
220
|
|
|
371
221
|
```bash
|
|
372
|
-
#
|
|
373
|
-
claude
|
|
222
|
+
# Check MCP connection
|
|
223
|
+
claude mcp list
|
|
224
|
+
|
|
225
|
+
# Direct MCP tool usage (in Claude Code)
|
|
226
|
+
# Generate tests
|
|
227
|
+
mcp__agentic_qe__test_generate({
|
|
228
|
+
type: "unit",
|
|
229
|
+
framework: "jest",
|
|
230
|
+
targetFile: "src/user-service.ts"
|
|
231
|
+
})
|
|
232
|
+
|
|
233
|
+
# Execute tests
|
|
234
|
+
mcp__agentic_qe__test_execute({
|
|
235
|
+
parallel: true,
|
|
236
|
+
coverage: true
|
|
237
|
+
})
|
|
238
|
+
|
|
239
|
+
# Analyze coverage
|
|
240
|
+
mcp__agentic_qe__coverage_analyze({
|
|
241
|
+
threshold: 95
|
|
242
|
+
})
|
|
374
243
|
```
|
|
375
244
|
|
|
376
|
-
|
|
377
|
-
- Agents are Claude Code agent definitions (markdown files in `.claude/agents/`)
|
|
378
|
-
- Executed via Claude Code's Task tool OR MCP tools
|
|
379
|
-
- MCP integration enables Claude to orchestrate QE agents directly
|
|
380
|
-
- NOT standalone Node.js processes
|
|
245
|
+
### CLI Direct Usage
|
|
381
246
|
|
|
382
|
-
|
|
247
|
+
```bash
|
|
248
|
+
# Generate tests
|
|
249
|
+
aqe test src/services/user-service.ts
|
|
383
250
|
|
|
384
|
-
|
|
251
|
+
# Analyze coverage
|
|
252
|
+
aqe coverage --threshold 95
|
|
385
253
|
|
|
386
|
-
|
|
254
|
+
# Run quality gate
|
|
255
|
+
aqe quality
|
|
387
256
|
|
|
388
|
-
|
|
257
|
+
# View fleet status
|
|
258
|
+
aqe status --verbose
|
|
389
259
|
|
|
390
|
-
|
|
391
|
-
|
|
260
|
+
# Enable multi-model router (70-81% cost savings)
|
|
261
|
+
aqe routing enable
|
|
262
|
+
|
|
263
|
+
# Start learning system
|
|
392
264
|
aqe learn enable --all
|
|
265
|
+
```
|
|
393
266
|
|
|
394
|
-
|
|
395
|
-
aqe learn status
|
|
267
|
+
### Advanced Patterns
|
|
396
268
|
|
|
397
|
-
|
|
398
|
-
aqe learn history --agent test-generator
|
|
269
|
+
#### Pattern 1: Continuous Learning
|
|
399
270
|
|
|
400
|
-
|
|
401
|
-
|
|
271
|
+
```bash
|
|
272
|
+
# Agents learn from execution
|
|
273
|
+
claude "Use qe-test-generator with learning enabled to create tests, then analyze improvement over time"
|
|
402
274
|
|
|
403
|
-
#
|
|
404
|
-
aqe learn
|
|
275
|
+
# Check learning metrics
|
|
276
|
+
aqe learn status --agent test-generator
|
|
405
277
|
```
|
|
406
278
|
|
|
407
|
-
**Example Output
|
|
279
|
+
**Example Output:**
|
|
408
280
|
```
|
|
409
281
|
π LEARNING STATUS
|
|
410
282
|
|
|
411
283
|
Agent: test-generator
|
|
412
|
-
ββββββββββββββββββββββββββββββββββββββββββββββ
|
|
413
284
|
Status: ENABLED β
|
|
414
285
|
Total Experiences: 247
|
|
415
286
|
Exploration Rate: 15.3%
|
|
@@ -423,30 +294,19 @@ Top Strategies:
|
|
|
423
294
|
1. property-based (confidence: 92%, success: 95%)
|
|
424
295
|
2. mutation-based (confidence: 85%, success: 88%)
|
|
425
296
|
3. example-based (confidence: 78%, success: 82%)
|
|
426
|
-
|
|
427
|
-
Recent Trend: β improving
|
|
428
297
|
```
|
|
429
298
|
|
|
430
|
-
|
|
299
|
+
#### Pattern 2: Pattern Bank Usage
|
|
431
300
|
|
|
432
301
|
```bash
|
|
433
|
-
#
|
|
434
|
-
|
|
435
|
-
|
|
436
|
-
# Search patterns by keyword
|
|
437
|
-
aqe patterns search "null check"
|
|
302
|
+
# Extract and reuse patterns
|
|
303
|
+
claude "Use qe-test-generator to extract test patterns from existing tests, then apply them to new modules"
|
|
438
304
|
|
|
439
|
-
#
|
|
440
|
-
aqe patterns
|
|
441
|
-
|
|
442
|
-
# Share patterns across projects
|
|
443
|
-
aqe patterns share --id pattern-001 --projects proj-a,proj-b
|
|
444
|
-
|
|
445
|
-
# Export patterns
|
|
446
|
-
aqe patterns export --output patterns-backup.json
|
|
305
|
+
# List patterns
|
|
306
|
+
aqe patterns list --framework jest
|
|
447
307
|
```
|
|
448
308
|
|
|
449
|
-
**Example Output
|
|
309
|
+
**Example Output:**
|
|
450
310
|
```
|
|
451
311
|
π¦ PATTERN LIBRARY (247 patterns)
|
|
452
312
|
|
|
@@ -455,67 +315,23 @@ ID | Name | Framework | Quality | Uses
|
|
|
455
315
|
pattern-001| Null Parameter Check | jest | 92% | 142
|
|
456
316
|
pattern-002| Empty Array Handling | jest | 89% | 98
|
|
457
317
|
pattern-003| API Timeout Test | cypress | 95% | 87
|
|
458
|
-
pattern-004| Database Transaction | mocha | 88% | 76
|
|
459
|
-
pattern-005| Async Error Handling | jest | 91% | 65
|
|
460
|
-
|
|
461
|
-
Total: 247 patterns | Average Quality: 87%
|
|
462
|
-
```
|
|
463
|
-
|
|
464
|
-
### Improvement Loop Commands π
|
|
465
|
-
|
|
466
|
-
```bash
|
|
467
|
-
# View improvement status
|
|
468
|
-
aqe improve status
|
|
469
|
-
|
|
470
|
-
# Start improvement loop
|
|
471
|
-
aqe improve start
|
|
472
|
-
|
|
473
|
-
# Run A/B test
|
|
474
|
-
aqe improve ab-test --strategies "property-based,mutation-based" --sample-size 50
|
|
475
|
-
|
|
476
|
-
# View failure patterns
|
|
477
|
-
aqe improve failures
|
|
478
|
-
|
|
479
|
-
# Generate improvement report
|
|
480
|
-
aqe improve report --format html --output improvement-report.html
|
|
481
318
|
```
|
|
482
319
|
|
|
483
|
-
|
|
484
|
-
|
|
485
|
-
## π― Phase 1 Commands (v1.0.5)
|
|
486
|
-
|
|
487
|
-
### Multi-Model Router Commands π°
|
|
320
|
+
#### Pattern 3: Cost Optimization
|
|
488
321
|
|
|
489
322
|
```bash
|
|
490
|
-
# Enable
|
|
323
|
+
# Enable intelligent model routing
|
|
491
324
|
aqe routing enable
|
|
492
325
|
|
|
493
|
-
# View
|
|
494
|
-
|
|
495
|
-
|
|
496
|
-
# Launch real-time cost dashboard
|
|
326
|
+
# View savings
|
|
327
|
+
claude "Check routing status and show cost savings"
|
|
497
328
|
aqe routing dashboard
|
|
498
|
-
|
|
499
|
-
# Generate detailed cost report
|
|
500
|
-
aqe routing report --format html --output report.html
|
|
501
|
-
|
|
502
|
-
# View routing statistics
|
|
503
|
-
aqe routing stats --days 30
|
|
504
|
-
|
|
505
|
-
# Disable routing
|
|
506
|
-
aqe routing disable
|
|
507
329
|
```
|
|
508
330
|
|
|
509
|
-
**Example Output
|
|
331
|
+
**Example Output:**
|
|
510
332
|
```
|
|
511
333
|
β
Multi-Model Router Status
|
|
512
334
|
|
|
513
|
-
Configuration:
|
|
514
|
-
Status: ENABLED β
|
|
515
|
-
Default Model: claude-sonnet-4.5
|
|
516
|
-
Cost Tracking: ENABLED β
|
|
517
|
-
Fallback Chains: ENABLED β
|
|
518
|
-
|
|
519
335
|
Cost Summary (Last 30 Days):
|
|
520
336
|
Total Cost: $127.50
|
|
521
337
|
Baseline Cost: $545.00
|
|
@@ -529,257 +345,20 @@ Model Usage:
|
|
|
529
345
|
ββ gpt-4: 7% (critical tasks)
|
|
530
346
|
```
|
|
531
347
|
|
|
532
|
-
|
|
533
|
-
|
|
534
|
-
### Basic Commands
|
|
535
|
-
|
|
536
|
-
```bash
|
|
537
|
-
# Check fleet status
|
|
538
|
-
aqe status
|
|
539
|
-
|
|
540
|
-
# Generate tests for a module
|
|
541
|
-
aqe test src/services/user-service.ts
|
|
542
|
-
|
|
543
|
-
# Analyze test coverage
|
|
544
|
-
aqe coverage --threshold 95
|
|
545
|
-
|
|
546
|
-
# Run quality gate validation
|
|
547
|
-
aqe quality
|
|
548
|
-
|
|
549
|
-
# Execute comprehensive test suite
|
|
550
|
-
aqe execute --parallel --coverage
|
|
348
|
+
### Pro Tips
|
|
551
349
|
|
|
552
|
-
|
|
553
|
-
aqe
|
|
554
|
-
|
|
555
|
-
|
|
556
|
-
|
|
557
|
-
|
|
558
|
-
## π» Programmatic Usage
|
|
559
|
-
|
|
560
|
-
### Pattern-Based Test Generation (v1.1.0) π
|
|
561
|
-
|
|
562
|
-
```typescript
|
|
563
|
-
import { TestGeneratorAgent, QEReasoningBank } from 'agentic-qe';
|
|
564
|
-
|
|
565
|
-
// Initialize pattern bank
|
|
566
|
-
const bank = new QEReasoningBank();
|
|
567
|
-
|
|
568
|
-
// Create agent with pattern matching
|
|
569
|
-
const agent = new TestGeneratorAgent(
|
|
570
|
-
{ agentId: 'test-gen-1', memoryStore },
|
|
571
|
-
{
|
|
572
|
-
targetCoverage: 95,
|
|
573
|
-
framework: 'jest',
|
|
574
|
-
enablePatterns: true, // β
Enable pattern matching
|
|
575
|
-
enableLearning: true, // β
Enable learning
|
|
576
|
-
reasoningBank: bank
|
|
577
|
-
}
|
|
578
|
-
);
|
|
579
|
-
|
|
580
|
-
// Generate tests with patterns
|
|
581
|
-
const result = await agent.execute({
|
|
582
|
-
type: 'test-generation',
|
|
583
|
-
payload: {
|
|
584
|
-
sourceFile: 'src/myModule.ts',
|
|
585
|
-
framework: 'jest'
|
|
586
|
-
}
|
|
587
|
-
});
|
|
588
|
-
|
|
589
|
-
console.log(`Generated ${result.testsGenerated} tests`);
|
|
590
|
-
console.log(`Pattern hit rate: ${result.patternHitRate}%`);
|
|
591
|
-
console.log(`Time saved: ${result.patterns.savings}ms`);
|
|
592
|
-
console.log(`Quality score: ${result.qualityScore}%`);
|
|
593
|
-
```
|
|
594
|
-
|
|
595
|
-
**Performance Impact:**
|
|
596
|
-
```
|
|
597
|
-
Without Patterns:
|
|
598
|
-
ββ Test Generation: 180ms avg
|
|
599
|
-
ββ Coverage: 78%
|
|
600
|
-
ββ Quality Score: 82%
|
|
601
|
-
|
|
602
|
-
With Patterns:
|
|
603
|
-
ββ Test Generation: 145ms avg (β 19.4%)
|
|
604
|
-
ββ Coverage: 94% (β 20.5%)
|
|
605
|
-
ββ Quality Score: 96% (β 17.1%)
|
|
606
|
-
```
|
|
607
|
-
|
|
608
|
-
### Learning-Enhanced Coverage Analysis (v1.1.0) π
|
|
609
|
-
|
|
610
|
-
```typescript
|
|
611
|
-
import { CoverageAnalyzerAgent, LearningEngine } from 'agentic-qe';
|
|
612
|
-
|
|
613
|
-
// Create learning engine
|
|
614
|
-
const learningEngine = new LearningEngine('coverage-1', memory, {
|
|
615
|
-
enabled: true,
|
|
616
|
-
learningRate: 0.1,
|
|
617
|
-
targetImprovement: 0.20 // 20% improvement target
|
|
618
|
-
});
|
|
619
|
-
|
|
620
|
-
await learningEngine.initialize();
|
|
621
|
-
|
|
622
|
-
// Create agent with learning
|
|
623
|
-
const agent = new CoverageAnalyzerAgent(
|
|
624
|
-
{ agentId: 'coverage-1', memoryStore: memory },
|
|
625
|
-
{
|
|
626
|
-
targetCoverage: 95,
|
|
627
|
-
algorithm: 'sublinear',
|
|
628
|
-
enableLearning: true // β
Enable learning
|
|
629
|
-
}
|
|
630
|
-
);
|
|
631
|
-
|
|
632
|
-
// Analyze coverage (learning happens automatically)
|
|
633
|
-
const analysis = await agent.execute({
|
|
634
|
-
type: 'coverage-analysis',
|
|
635
|
-
payload: {
|
|
636
|
-
coverageReport: './coverage/coverage-final.json',
|
|
637
|
-
threshold: 95
|
|
638
|
-
}
|
|
639
|
-
});
|
|
640
|
-
|
|
641
|
-
// Check learning progress
|
|
642
|
-
const status = await learningEngine.calculateImprovement();
|
|
643
|
-
console.log(`Current improvement: ${(status.improvementRate * 100).toFixed(1)}%`);
|
|
644
|
-
console.log(`Target reached: ${status.targetAchieved ? 'β
YES' : 'β NO'}`);
|
|
645
|
-
console.log(`Confidence: ${(status.confidence * 100).toFixed(1)}%`);
|
|
646
|
-
```
|
|
647
|
-
|
|
648
|
-
### ML Flaky Detection (v1.1.0) π
|
|
649
|
-
|
|
650
|
-
```typescript
|
|
651
|
-
import { FlakyTestHunterAgent, FlakyTestDetector } from 'agentic-qe';
|
|
652
|
-
|
|
653
|
-
// Create detector with ML
|
|
654
|
-
const detector = new FlakyTestDetector({
|
|
655
|
-
minRuns: 5,
|
|
656
|
-
passRateThreshold: 0.8,
|
|
657
|
-
confidenceThreshold: 0.7,
|
|
658
|
-
enableML: true // β
100% accuracy
|
|
659
|
-
});
|
|
660
|
-
|
|
661
|
-
// Create agent
|
|
662
|
-
const agent = new FlakyTestHunterAgent(
|
|
663
|
-
{ agentId: 'flaky-1', memoryStore: memory },
|
|
664
|
-
{
|
|
665
|
-
enableML: true, // β
100% accuracy
|
|
666
|
-
detector
|
|
667
|
-
}
|
|
668
|
-
);
|
|
669
|
-
|
|
670
|
-
// Detect flaky tests
|
|
671
|
-
const result = await agent.execute({
|
|
672
|
-
type: 'flaky-detection',
|
|
673
|
-
payload: {
|
|
674
|
-
testHistory: testResults
|
|
675
|
-
}
|
|
676
|
-
});
|
|
677
|
-
|
|
678
|
-
result.flakyTests.forEach(test => {
|
|
679
|
-
console.log(`π΄ ${test.testName}`);
|
|
680
|
-
console.log(` Pass Rate: ${(test.passRate * 100).toFixed(1)}%`);
|
|
681
|
-
console.log(` Root Cause: ${test.rootCause.cause}`);
|
|
682
|
-
console.log(` ML Confidence: ${(test.rootCause.mlConfidence * 100).toFixed(1)}%`);
|
|
683
|
-
console.log(` Severity: ${test.severity}`);
|
|
684
|
-
console.log(` Fix: ${test.fixRecommendations[0].recommendation}`);
|
|
685
|
-
console.log(` Code Example:`);
|
|
686
|
-
console.log(` ${test.fixRecommendations[0].codeExample}`);
|
|
687
|
-
});
|
|
688
|
-
```
|
|
689
|
-
|
|
690
|
-
**Detection Results:**
|
|
691
|
-
```
|
|
692
|
-
Model Training Complete:
|
|
693
|
-
Accuracy: 100.00% β
Exceeds 90% target by 10%
|
|
694
|
-
Precision: 100.00% β
Perfect precision
|
|
695
|
-
Recall: 100.00% β
Perfect recall
|
|
696
|
-
F1 Score: 100.00% β
Perfect F1
|
|
697
|
-
False Positive Rate: 0.00% β
Well below 5% target
|
|
698
|
-
|
|
699
|
-
Processing 1,200 test results: ~150ms
|
|
700
|
-
Throughput: ~8,000 results/second
|
|
701
|
-
Memory Usage: < 5MB delta
|
|
702
|
-
```
|
|
703
|
-
|
|
704
|
-
### With Multi-Model Router (v1.0.5)
|
|
705
|
-
|
|
706
|
-
```typescript
|
|
707
|
-
import { FleetManager, AdaptiveModelRouter } from 'agentic-qe';
|
|
708
|
-
|
|
709
|
-
// Initialize fleet with cost-optimized routing
|
|
710
|
-
const fleet = new FleetManager({
|
|
711
|
-
maxAgents: 20,
|
|
712
|
-
topology: 'mesh',
|
|
713
|
-
routing: {
|
|
714
|
-
enabled: true,
|
|
715
|
-
defaultModel: 'claude-sonnet-4.5',
|
|
716
|
-
enableCostTracking: true,
|
|
717
|
-
enableFallback: true,
|
|
718
|
-
modelPreferences: {
|
|
719
|
-
simple: 'gpt-3.5-turbo', // 70% cheaper for simple tasks
|
|
720
|
-
medium: 'claude-haiku', // 60% cheaper for standard tests
|
|
721
|
-
complex: 'claude-sonnet-4.5', // Best quality/cost for complex
|
|
722
|
-
critical: 'gpt-4' // Maximum quality when needed
|
|
723
|
-
},
|
|
724
|
-
budgets: {
|
|
725
|
-
daily: 50,
|
|
726
|
-
monthly: 1000
|
|
727
|
-
}
|
|
728
|
-
}
|
|
729
|
-
});
|
|
730
|
-
|
|
731
|
-
await fleet.initialize();
|
|
732
|
-
|
|
733
|
-
// Spawn agent (automatically uses optimal model based on task complexity)
|
|
734
|
-
const testGen = await fleet.spawnAgent('test-generator', {
|
|
735
|
-
targetCoverage: 95,
|
|
736
|
-
framework: 'jest',
|
|
737
|
-
useRouting: true // Enable intelligent model selection
|
|
738
|
-
});
|
|
739
|
-
|
|
740
|
-
// Execute task (router selects cheapest model that meets quality requirements)
|
|
741
|
-
const tests = await testGen.execute({
|
|
742
|
-
sourceFile: 'src/services/user-service.ts',
|
|
743
|
-
testStyle: 'property-based'
|
|
744
|
-
});
|
|
745
|
-
|
|
746
|
-
// Check cost savings
|
|
747
|
-
const savings = await fleet.getRoutingSavings();
|
|
748
|
-
console.log(`π° Total savings: $${savings.total} (${savings.percent}%)`);
|
|
749
|
-
console.log(`π Models used: ${JSON.stringify(savings.modelBreakdown, null, 2)}`);
|
|
750
|
-
```
|
|
751
|
-
|
|
752
|
-
π **[Complete Routing Examples](docs/examples/ROUTING-EXAMPLES.md)** - Advanced programmatic usage
|
|
753
|
-
|
|
754
|
-
---
|
|
755
|
-
|
|
756
|
-
## π Performance Benchmarks
|
|
757
|
-
|
|
758
|
-
| Feature | Target | Actual | Status |
|
|
759
|
-
|---------|--------|--------|--------|
|
|
760
|
-
| **Pattern Matching (p95)** | <50ms | 32ms | β
Exceeded |
|
|
761
|
-
| **Learning Iteration** | <100ms | 68ms | β
Exceeded |
|
|
762
|
-
| **ML Flaky Detection (1000 tests)** | <500ms | 385ms | β
Exceeded |
|
|
763
|
-
| **Agent Memory** | <100MB | 85MB | β
Exceeded |
|
|
764
|
-
| **Cost Savings** | 70%+ | 70-81% | β
Achieved |
|
|
765
|
-
| **Test Improvement** | 20%+ | 23%+ | β
Exceeded |
|
|
766
|
-
| **Flaky Detection Accuracy** | 90%+ | 100% | β
Exceeded |
|
|
767
|
-
| **False Positive Rate** | <5% | 0% | β
Exceeded |
|
|
768
|
-
|
|
769
|
-
### Core Performance
|
|
770
|
-
|
|
771
|
-
- **Test Generation**: 1000+ tests/minute
|
|
772
|
-
- **Parallel Execution**: 10,000+ concurrent tests
|
|
773
|
-
- **Coverage Analysis**: O(log n) complexity
|
|
774
|
-
- **Data Generation**: 10,000+ records/second
|
|
775
|
-
- **Agent Spawning**: <100ms per agent
|
|
776
|
-
- **Memory Efficient**: <2GB for typical projects
|
|
350
|
+
1. **Batch agent operations**: Always spawn multiple agents in one Claude message for parallel execution
|
|
351
|
+
2. **Use memory namespace**: Agents coordinate through `aqe/*` memory keys
|
|
352
|
+
3. **Enable learning**: Add `--enable-learning` to agent commands for continuous improvement
|
|
353
|
+
4. **Check agent status**: Use `aqe status` to see active agents and coordination
|
|
354
|
+
5. **Review agent output**: Agents store detailed results in `.agentic-qe/logs/`
|
|
777
355
|
|
|
778
356
|
---
|
|
779
357
|
|
|
780
358
|
## π€ Agent Types
|
|
781
359
|
|
|
782
|
-
|
|
360
|
+
<details>
|
|
361
|
+
<summary><b>Core Testing Agents (5 agents)</b></summary>
|
|
783
362
|
|
|
784
363
|
| Agent | Purpose | Key Features | Phase 2 Enhancements |
|
|
785
364
|
|-------|---------|-------------|---------------------|
|
|
@@ -789,14 +368,20 @@ console.log(`π Models used: ${JSON.stringify(savings.modelBreakdown, null, 2)
|
|
|
789
368
|
| **quality-gate** | Intelligent validation | ML-driven decisions, risk assessment | β
Flaky test metrics |
|
|
790
369
|
| **quality-analyzer** | Metrics analysis | ESLint, SonarQube, Lighthouse integration | - |
|
|
791
370
|
|
|
792
|
-
|
|
371
|
+
</details>
|
|
372
|
+
|
|
373
|
+
<details>
|
|
374
|
+
<summary><b>Performance & Security (2 agents)</b></summary>
|
|
793
375
|
|
|
794
376
|
| Agent | Purpose | Key Features |
|
|
795
377
|
|-------|---------|-------------|
|
|
796
378
|
| **performance-tester** | Load & stress testing | k6, JMeter, Gatling, bottleneck detection |
|
|
797
379
|
| **security-scanner** | Vulnerability detection | SAST, DAST, dependency scanning |
|
|
798
380
|
|
|
799
|
-
|
|
381
|
+
</details>
|
|
382
|
+
|
|
383
|
+
<details>
|
|
384
|
+
<summary><b>Strategic Planning (3 agents)</b></summary>
|
|
800
385
|
|
|
801
386
|
| Agent | Purpose | Key Features |
|
|
802
387
|
|-------|---------|-------------|
|
|
@@ -804,7 +389,10 @@ console.log(`π Models used: ${JSON.stringify(savings.modelBreakdown, null, 2)
|
|
|
804
389
|
| **production-intelligence** | Incident replay | RUM analysis, anomaly detection |
|
|
805
390
|
| **fleet-commander** | Hierarchical coordination | 50+ agent orchestration |
|
|
806
391
|
|
|
807
|
-
|
|
392
|
+
</details>
|
|
393
|
+
|
|
394
|
+
<details>
|
|
395
|
+
<summary><b>Advanced Testing (4 agents)</b></summary>
|
|
808
396
|
|
|
809
397
|
| Agent | Purpose | Key Features | Phase 2 Enhancements |
|
|
810
398
|
|-------|---------|-------------|---------------------|
|
|
@@ -813,7 +401,10 @@ console.log(`π Models used: ${JSON.stringify(savings.modelBreakdown, null, 2)
|
|
|
813
401
|
| **api-contract-validator** | Breaking change detection | OpenAPI, GraphQL, gRPC | - |
|
|
814
402
|
| **flaky-test-hunter** | Stability analysis | Statistical detection, auto-fix | β
100% accuracy ML detection |
|
|
815
403
|
|
|
816
|
-
|
|
404
|
+
</details>
|
|
405
|
+
|
|
406
|
+
<details>
|
|
407
|
+
<summary><b>Specialized (3 agents)</b></summary>
|
|
817
408
|
|
|
818
409
|
| Agent | Purpose | Key Features |
|
|
819
410
|
|-------|---------|-------------|
|
|
@@ -821,386 +412,113 @@ console.log(`π Models used: ${JSON.stringify(savings.modelBreakdown, null, 2)
|
|
|
821
412
|
| **visual-tester** | UI regression | AI-powered comparison |
|
|
822
413
|
| **chaos-engineer** | Resilience testing | Fault injection, blast radius |
|
|
823
414
|
|
|
824
|
-
|
|
825
|
-
|
|
826
|
-
## ποΈ Architecture
|
|
827
|
-
|
|
828
|
-
### Core Components
|
|
829
|
-
|
|
830
|
-
```
|
|
831
|
-
βββββββββββββββββββββββββββββββββββββββββββββββ
|
|
832
|
-
β Fleet Manager β
|
|
833
|
-
β (Central Coordination & Task Distribution) β
|
|
834
|
-
βββββββββββββββββββββββββββββββββββββββββββββββ
|
|
835
|
-
β
|
|
836
|
-
βββββββββββββΌββββββββββββ
|
|
837
|
-
β β β
|
|
838
|
-
ββββββΌβββββ βββββΌβββββ βββββΌβββββ
|
|
839
|
-
β Agent 1 β β Agent 2β β Agent Nβ
|
|
840
|
-
β Pool β β Pool β β Pool β
|
|
841
|
-
ββββββ¬βββββ βββββ¬βββββ βββββ¬βββββ
|
|
842
|
-
β β β
|
|
843
|
-
ββββββΌβββββββββββΌβββββββββββΌβββββ
|
|
844
|
-
β Event Bus β
|
|
845
|
-
β (Event-Driven Communication) β
|
|
846
|
-
ββββββ¬βββββββββββββββββββββββ¬βββββ
|
|
847
|
-
β β
|
|
848
|
-
ββββββΌβββββ ββββββΌβββββ
|
|
849
|
-
β Memory β βDatabase β
|
|
850
|
-
β Store β β(SQLite) β
|
|
851
|
-
β β β β
|
|
852
|
-
β Phase 2:β β β
|
|
853
|
-
β Learningβ β β
|
|
854
|
-
β Patternsβ β β
|
|
855
|
-
β ML Modelβ β β
|
|
856
|
-
βββββββββββ βββββββββββ
|
|
857
|
-
```
|
|
858
|
-
|
|
859
|
-
### Event-Driven Architecture
|
|
860
|
-
|
|
861
|
-
- **EventBus**: Real-time communication between agents
|
|
862
|
-
- **Task Queue**: Priority-based task scheduling
|
|
863
|
-
- **Memory Store**: Shared context and learning (SwarmMemoryManager)
|
|
864
|
-
- **Persistence**: SQLite for state, metrics, and audit trails
|
|
865
|
-
|
|
866
|
-
### AQE Hooks System
|
|
415
|
+
</details>
|
|
867
416
|
|
|
868
|
-
|
|
417
|
+
<details>
|
|
418
|
+
<summary><b>General Purpose (1 agent)</b></summary>
|
|
869
419
|
|
|
870
|
-
|
|
871
|
-
|
|
872
|
-
|
|
873
|
-
protected async onPreTask(data): Promise<void> { /* prepare */ }
|
|
874
|
-
protected async onPostTask(data): Promise<void> { /* validate */ }
|
|
875
|
-
protected async onTaskError(data): Promise<void> { /* recover */ }
|
|
876
|
-
}
|
|
877
|
-
|
|
878
|
-
// Advanced verification hooks
|
|
879
|
-
const hookManager = new VerificationHookManager(memoryStore);
|
|
880
|
-
await hookManager.executePreTaskVerification({ task, context });
|
|
881
|
-
await hookManager.executePostTaskValidation({ task, result });
|
|
882
|
-
```
|
|
420
|
+
| Agent | Purpose | Key Features |
|
|
421
|
+
|-------|---------|-------------|
|
|
422
|
+
| **base-template-generator** | Agent templates | General-purpose agent creation |
|
|
883
423
|
|
|
884
|
-
|
|
424
|
+
</details>
|
|
885
425
|
|
|
886
|
-
**
|
|
887
|
-
- Full TypeScript type safety
|
|
888
|
-
- Direct SwarmMemoryManager integration
|
|
889
|
-
- Built-in RollbackManager support
|
|
890
|
-
- EventBus coordination
|
|
891
|
-
- Context engineering (pre/post tool-use bundles)
|
|
426
|
+
**Total: 18 Agents** (17 QE-specific + 1 general-purpose)
|
|
892
427
|
|
|
893
428
|
---
|
|
894
429
|
|
|
895
430
|
## π Documentation
|
|
896
431
|
|
|
897
|
-
### π Phase 2 Features (v1.1.0)
|
|
898
|
-
- [Learning System User Guide](docs/guides/LEARNING-SYSTEM-USER-GUIDE.md) - **NEW!** Q-learning and continuous improvement
|
|
899
|
-
- [Pattern Management User Guide](docs/guides/PATTERN-MANAGEMENT-USER-GUIDE.md) - **NEW!** Cross-project pattern sharing
|
|
900
|
-
- [ML Flaky Detection Guide](docs/guides/ML-FLAKY-DETECTION-USER-GUIDE.md) - **NEW!** 100% accurate flaky detection
|
|
901
|
-
- [Performance Improvement Guide](docs/guides/PERFORMANCE-IMPROVEMENT-USER-GUIDE.md) - **NEW!** A/B testing and optimization
|
|
902
|
-
- [Learning System Examples](docs/examples/LEARNING-SYSTEM-EXAMPLES.md) - **NEW!** Learning code examples
|
|
903
|
-
- [Pattern Examples](docs/examples/REASONING-BANK-EXAMPLES.md) - **NEW!** Pattern usage examples
|
|
904
|
-
- [Flaky Detection Examples](docs/examples/FLAKY-DETECTION-ML-EXAMPLES.md) - **NEW!** ML detection examples
|
|
905
|
-
|
|
906
|
-
### Phase 1 Features (v1.0.5)
|
|
907
|
-
- [Multi-Model Router Guide](docs/guides/MULTI-MODEL-ROUTER.md) - Save 70% on AI costs
|
|
908
|
-
- [Streaming API Tutorial](docs/guides/STREAMING-API.md) - Real-time progress updates
|
|
909
|
-
- [Cost Optimization Best Practices](docs/guides/COST-OPTIMIZATION.md) - Maximize ROI
|
|
910
|
-
- [Migration Guide v1.0.5](docs/guides/MIGRATION-V1.0.5.md) - Upgrade guide
|
|
911
|
-
- [Routing API Reference](docs/api/ROUTING-API.md) - Complete API docs
|
|
912
|
-
- [Streaming API Reference](docs/api/STREAMING-API.md) - Complete API docs
|
|
913
|
-
- [Phase 1 Code Examples](docs/examples/ROUTING-EXAMPLES.md) - Working examples
|
|
914
|
-
|
|
915
432
|
### Getting Started
|
|
916
|
-
- [Quick Start Guide](docs/AQE-CLI.md)
|
|
433
|
+
- [Quick Start Guide](docs/AQE-CLI.md) - Get started in 5 minutes
|
|
917
434
|
- [User Guide](docs/USER-GUIDE.md) - Comprehensive workflows and examples
|
|
918
|
-
- [
|
|
435
|
+
- [MCP Integration](docs/guides/MCP-INTEGRATION.md) - Claude Code integration
|
|
919
436
|
- [Configuration Guide](docs/CONFIGURATION.md) - Complete configuration reference
|
|
920
437
|
- [Troubleshooting Guide](docs/TROUBLESHOOTING.md) - Common issues and solutions
|
|
921
438
|
|
|
922
|
-
###
|
|
923
|
-
- [Test Generation](docs/guides/TEST-GENERATION.md)
|
|
924
|
-
- [Coverage Analysis](docs/guides/COVERAGE-ANALYSIS.md)
|
|
925
|
-
- [Quality Gates](docs/guides/QUALITY-GATES.md)
|
|
926
|
-
- [Performance Testing](docs/guides/PERFORMANCE-TESTING.md)
|
|
927
|
-
- [Test Execution](docs/guides/TEST-EXECUTION.md)
|
|
928
|
-
- [MCP Integration](docs/guides/MCP-INTEGRATION.md)
|
|
929
|
-
|
|
930
|
-
### Advanced Topics
|
|
931
|
-
- [API Reference](docs/API.md)
|
|
932
|
-
- [Agent Development](docs/AGENT-DEVELOPMENT.md)
|
|
933
|
-
- [MCP Integration](docs/CLAUDE-MD-INTEGRATION.md)
|
|
934
|
-
- [Best Practices](docs/AI%20%26%20Agentic%20Security%20Best%20Practices.md)
|
|
935
|
-
- [AQE Hooks Guide](docs/AQE-HOOKS-GUIDE.md)
|
|
936
|
-
|
|
937
|
-
### Commands Reference
|
|
938
|
-
- [AQE Commands Overview](docs/QE-COMMANDS-INDEX.md)
|
|
939
|
-
- [Command Specifications](docs/QE-SLASH-COMMANDS-SPECIFICATION.md)
|
|
940
|
-
- [Hooks Architecture](docs/QE_HOOKS_ARCHITECTURE.md)
|
|
439
|
+
### Feature Guides
|
|
941
440
|
|
|
942
|
-
|
|
441
|
+
**Phase 2 Features (v1.1.0)**
|
|
442
|
+
- [Learning System User Guide](docs/guides/LEARNING-SYSTEM-USER-GUIDE.md) - Q-learning and continuous improvement
|
|
443
|
+
- [Pattern Management User Guide](docs/guides/PATTERN-MANAGEMENT-USER-GUIDE.md) - Cross-project pattern sharing
|
|
444
|
+
- [ML Flaky Detection Guide](docs/guides/ML-FLAKY-DETECTION-USER-GUIDE.md) - 100% accurate flaky detection
|
|
445
|
+
- [Performance Improvement Guide](docs/guides/PERFORMANCE-IMPROVEMENT-USER-GUIDE.md) - A/B testing and optimization
|
|
943
446
|
|
|
944
|
-
|
|
447
|
+
**Phase 1 Features (v1.0.5)**
|
|
448
|
+
- [Multi-Model Router Guide](docs/guides/MULTI-MODEL-ROUTER.md) - Save 70% on AI costs
|
|
449
|
+
- [Streaming API Tutorial](docs/guides/STREAMING-API.md) - Real-time progress updates
|
|
450
|
+
- [Cost Optimization Best Practices](docs/guides/COST-OPTIMIZATION.md) - Maximize ROI
|
|
945
451
|
|
|
946
|
-
###
|
|
452
|
+
### Testing Guides
|
|
453
|
+
- [Test Generation](docs/guides/TEST-GENERATION.md) - AI-powered test creation
|
|
454
|
+
- [Coverage Analysis](docs/guides/COVERAGE-ANALYSIS.md) - O(log n) gap detection
|
|
455
|
+
- [Quality Gates](docs/guides/QUALITY-GATES.md) - Intelligent validation
|
|
456
|
+
- [Performance Testing](docs/guides/PERFORMANCE-TESTING.md) - Load and stress testing
|
|
457
|
+
- [Test Execution](docs/guides/TEST-EXECUTION.md) - Parallel orchestration
|
|
947
458
|
|
|
948
|
-
|
|
459
|
+
### Advanced Topics
|
|
460
|
+
- [API Reference](docs/API.md) - Complete API documentation
|
|
461
|
+
- [Agent Development](docs/AGENT-DEVELOPMENT.md) - Create custom agents
|
|
462
|
+
- [Agent Types Overview](docs/Agentic-QE-Fleet-Specification.md) - Complete agent reference
|
|
463
|
+
- [AQE Hooks Guide](docs/AQE-HOOKS-GUIDE.md) - Native coordination system
|
|
464
|
+
- [Best Practices](docs/AI%20%26%20Agentic%20Security%20Best%20Practices.md) - Security and quality
|
|
949
465
|
|
|
950
|
-
|
|
951
|
-
|
|
952
|
-
|
|
953
|
-
|
|
954
|
-
HEARTBEAT_INTERVAL=30000
|
|
955
|
-
|
|
956
|
-
# Phase 2: Learning System
|
|
957
|
-
LEARNING_ENABLED=true
|
|
958
|
-
LEARNING_RATE=0.1
|
|
959
|
-
TARGET_IMPROVEMENT=0.20
|
|
960
|
-
|
|
961
|
-
# Phase 2: Pattern Bank
|
|
962
|
-
PATTERN_MATCHING_ENABLED=true
|
|
963
|
-
MIN_PATTERN_QUALITY=0.8
|
|
964
|
-
|
|
965
|
-
# Phase 2: ML Flaky Detection
|
|
966
|
-
FLAKY_DETECTION_ML=true
|
|
967
|
-
FLAKY_MIN_RUNS=5
|
|
968
|
-
|
|
969
|
-
# Database
|
|
970
|
-
DB_TYPE=sqlite
|
|
971
|
-
DB_FILENAME=./data/fleet.db
|
|
972
|
-
|
|
973
|
-
# Logging
|
|
974
|
-
LOG_LEVEL=info
|
|
975
|
-
LOG_FORMAT=json
|
|
976
|
-
|
|
977
|
-
# API (optional)
|
|
978
|
-
API_PORT=3000
|
|
979
|
-
API_HOST=localhost
|
|
980
|
-
```
|
|
466
|
+
### Commands Reference
|
|
467
|
+
- [AQE Commands Overview](docs/QE-COMMANDS-INDEX.md) - All CLI commands
|
|
468
|
+
- [Command Specifications](docs/QE-SLASH-COMMANDS-SPECIFICATION.md) - Slash command reference
|
|
469
|
+
- [Hooks Architecture](docs/QE_HOOKS_ARCHITECTURE.md) - Coordination architecture
|
|
981
470
|
|
|
982
|
-
###
|
|
983
|
-
|
|
984
|
-
|
|
985
|
-
|
|
986
|
-
|
|
987
|
-
fleet:
|
|
988
|
-
id: "my-project-fleet"
|
|
989
|
-
name: "My Project QE Fleet"
|
|
990
|
-
maxAgents: 20
|
|
991
|
-
topology: mesh
|
|
992
|
-
|
|
993
|
-
agents:
|
|
994
|
-
test-generator:
|
|
995
|
-
count: 3
|
|
996
|
-
config:
|
|
997
|
-
frameworks: [jest, cypress, playwright]
|
|
998
|
-
targetCoverage: 95
|
|
999
|
-
enablePatterns: true # Phase 2: Pattern matching
|
|
1000
|
-
enableLearning: true # Phase 2: Learning
|
|
1001
|
-
|
|
1002
|
-
coverage-analyzer:
|
|
1003
|
-
count: 2
|
|
1004
|
-
config:
|
|
1005
|
-
targetCoverage: 95
|
|
1006
|
-
optimizationAlgorithm: sublinear
|
|
1007
|
-
enableLearning: true # Phase 2: Learning
|
|
1008
|
-
|
|
1009
|
-
flaky-test-hunter:
|
|
1010
|
-
count: 1
|
|
1011
|
-
config:
|
|
1012
|
-
enableML: true # Phase 2: ML detection
|
|
1013
|
-
minRuns: 5
|
|
1014
|
-
passRateThreshold: 0.8
|
|
1015
|
-
|
|
1016
|
-
quality-analyzer:
|
|
1017
|
-
count: 2
|
|
1018
|
-
config:
|
|
1019
|
-
tools: [eslint, sonarqube, lighthouse]
|
|
1020
|
-
thresholds:
|
|
1021
|
-
coverage: 80
|
|
1022
|
-
complexity: 10
|
|
1023
|
-
maintainability: 65
|
|
1024
|
-
|
|
1025
|
-
# Phase 2: Learning Configuration
|
|
1026
|
-
learning:
|
|
1027
|
-
enabled: true
|
|
1028
|
-
learningRate: 0.1
|
|
1029
|
-
discountFactor: 0.95
|
|
1030
|
-
explorationRate: 0.3
|
|
1031
|
-
targetImprovement: 0.20
|
|
1032
|
-
|
|
1033
|
-
# Phase 2: Pattern Bank Configuration
|
|
1034
|
-
patterns:
|
|
1035
|
-
enabled: true
|
|
1036
|
-
minQuality: 0.8
|
|
1037
|
-
frameworks: [jest, mocha, cypress, vitest, jasmine, ava]
|
|
1038
|
-
autoExtract: true
|
|
1039
|
-
|
|
1040
|
-
# Phase 2: Flaky Detection Configuration
|
|
1041
|
-
flakyDetection:
|
|
1042
|
-
enableML: true
|
|
1043
|
-
minRuns: 5
|
|
1044
|
-
passRateThreshold: 0.8
|
|
1045
|
-
confidenceThreshold: 0.7
|
|
1046
|
-
```
|
|
471
|
+
### Code Examples
|
|
472
|
+
- [Learning System Examples](docs/examples/LEARNING-SYSTEM-EXAMPLES.md) - Learning code examples
|
|
473
|
+
- [Pattern Examples](docs/examples/REASONING-BANK-EXAMPLES.md) - Pattern usage examples
|
|
474
|
+
- [Flaky Detection Examples](docs/examples/FLAKY-DETECTION-ML-EXAMPLES.md) - ML detection examples
|
|
475
|
+
- [Routing Examples](docs/examples/ROUTING-EXAMPLES.md) - Cost optimization examples
|
|
1047
476
|
|
|
1048
477
|
---
|
|
1049
478
|
|
|
1050
|
-
##
|
|
1051
|
-
|
|
1052
|
-
### Example 1: Pattern-Based Test Generation
|
|
1053
|
-
|
|
1054
|
-
```typescript
|
|
1055
|
-
import { QEReasoningBank, PatternExtractor, TestGeneratorAgent } from 'agentic-qe';
|
|
1056
|
-
|
|
1057
|
-
// Initialize components
|
|
1058
|
-
const bank = new QEReasoningBank();
|
|
1059
|
-
const extractor = new PatternExtractor({ minQuality: 0.8 });
|
|
1060
|
-
|
|
1061
|
-
// Extract patterns from existing tests
|
|
1062
|
-
const patterns = await extractor.extractFromDirectory('./tests', {
|
|
1063
|
-
recursive: true,
|
|
1064
|
-
filePattern: '**/*.test.ts'
|
|
1065
|
-
});
|
|
1066
|
-
|
|
1067
|
-
console.log(`Extracted ${patterns.length} patterns`);
|
|
1068
|
-
|
|
1069
|
-
// Store patterns in bank
|
|
1070
|
-
for (const pattern of patterns) {
|
|
1071
|
-
await bank.storePattern(pattern);
|
|
1072
|
-
}
|
|
1073
|
-
|
|
1074
|
-
// Use patterns in test generation
|
|
1075
|
-
const testGen = new TestGeneratorAgent(
|
|
1076
|
-
{ agentId: 'test-gen-1', memoryStore },
|
|
1077
|
-
{
|
|
1078
|
-
targetCoverage: 95,
|
|
1079
|
-
framework: 'jest',
|
|
1080
|
-
usePatterns: true,
|
|
1081
|
-
reasoningBank: bank
|
|
1082
|
-
}
|
|
1083
|
-
);
|
|
1084
|
-
|
|
1085
|
-
const result = await testGen.execute({
|
|
1086
|
-
type: 'test-generation',
|
|
1087
|
-
payload: {
|
|
1088
|
-
sourceFile: 'src/user-service.ts',
|
|
1089
|
-
framework: 'jest'
|
|
1090
|
-
}
|
|
1091
|
-
});
|
|
1092
|
-
|
|
1093
|
-
console.log(`Generated ${result.testsGenerated} tests using ${result.patternsUsed.length} patterns`);
|
|
1094
|
-
```
|
|
1095
|
-
|
|
1096
|
-
### Example 2: Learning-Enhanced Coverage
|
|
1097
|
-
|
|
1098
|
-
```typescript
|
|
1099
|
-
import {
|
|
1100
|
-
CoverageAnalyzerAgent,
|
|
1101
|
-
LearningEngine,
|
|
1102
|
-
PerformanceTracker,
|
|
1103
|
-
SwarmMemoryManager
|
|
1104
|
-
} from 'agentic-qe';
|
|
1105
|
-
|
|
1106
|
-
// Initialize components
|
|
1107
|
-
const memory = new SwarmMemoryManager({ databasePath: './.aqe/memory.db' });
|
|
1108
|
-
await memory.initialize();
|
|
1109
|
-
|
|
1110
|
-
const learningEngine = new LearningEngine('coverage-1', memory, {
|
|
1111
|
-
enabled: true,
|
|
1112
|
-
learningRate: 0.1,
|
|
1113
|
-
explorationRate: 0.3
|
|
1114
|
-
});
|
|
1115
|
-
|
|
1116
|
-
const performanceTracker = new PerformanceTracker('coverage-1', memory, {
|
|
1117
|
-
targetImprovement: 0.20,
|
|
1118
|
-
snapshotWindow: 100
|
|
1119
|
-
});
|
|
1120
|
-
|
|
1121
|
-
await learningEngine.initialize();
|
|
1122
|
-
await performanceTracker.initialize();
|
|
1123
|
-
|
|
1124
|
-
// Create agent with learning
|
|
1125
|
-
const agent = new CoverageAnalyzerAgent(
|
|
1126
|
-
{
|
|
1127
|
-
agentId: 'coverage-1',
|
|
1128
|
-
memoryStore: memory
|
|
1129
|
-
},
|
|
1130
|
-
{
|
|
1131
|
-
targetCoverage: 95,
|
|
1132
|
-
algorithm: 'sublinear',
|
|
1133
|
-
enableLearning: true
|
|
1134
|
-
}
|
|
1135
|
-
);
|
|
1136
|
-
|
|
1137
|
-
// Execute 100 tasks to build learning data
|
|
1138
|
-
for (let i = 0; i < 100; i++) {
|
|
1139
|
-
const result = await agent.execute({
|
|
1140
|
-
type: 'coverage-analysis',
|
|
1141
|
-
payload: {
|
|
1142
|
-
coverageReport: `./coverage/report-${i}.json`,
|
|
1143
|
-
threshold: 95
|
|
1144
|
-
}
|
|
1145
|
-
});
|
|
1146
|
-
|
|
1147
|
-
console.log(`Task ${i + 1}/100: gaps=${result.gaps.length}`);
|
|
1148
|
-
}
|
|
1149
|
-
|
|
1150
|
-
// Check improvement
|
|
1151
|
-
const improvement = await performanceTracker.calculateImprovement();
|
|
1152
|
-
console.log(`\nπ― Final Results:`);
|
|
1153
|
-
console.log(`Improvement Rate: ${improvement.improvementRate.toFixed(2)}%`);
|
|
1154
|
-
console.log(`Target Achieved: ${improvement.targetAchieved ? 'β
YES' : 'β NO'}`);
|
|
1155
|
-
```
|
|
1156
|
-
|
|
1157
|
-
### Example 3: ML Flaky Detection
|
|
1158
|
-
|
|
1159
|
-
```typescript
|
|
1160
|
-
import { FlakyTestDetector } from 'agentic-qe';
|
|
1161
|
-
|
|
1162
|
-
const detector = new FlakyTestDetector({
|
|
1163
|
-
minRuns: 5,
|
|
1164
|
-
passRateThreshold: 0.8,
|
|
1165
|
-
confidenceThreshold: 0.7
|
|
1166
|
-
});
|
|
1167
|
-
|
|
1168
|
-
const flakyTests = await detector.detectFlakyTests(testHistory);
|
|
479
|
+
## π Performance Benchmarks
|
|
1169
480
|
|
|
1170
|
-
|
|
1171
|
-
|
|
1172
|
-
|
|
1173
|
-
|
|
1174
|
-
|
|
1175
|
-
|
|
1176
|
-
|
|
481
|
+
| Feature | Target | Actual | Status |
|
|
482
|
+
|---------|--------|--------|--------|
|
|
483
|
+
| **Pattern Matching (p95)** | <50ms | 32ms | β
Exceeded |
|
|
484
|
+
| **Learning Iteration** | <100ms | 68ms | β
Exceeded |
|
|
485
|
+
| **ML Flaky Detection (1000 tests)** | <500ms | 385ms | β
Exceeded |
|
|
486
|
+
| **Agent Memory** | <100MB | 85MB | β
Exceeded |
|
|
487
|
+
| **Cost Savings** | 70%+ | 70-81% | β
Achieved |
|
|
488
|
+
| **Test Improvement** | 20%+ | 23%+ | β
Exceeded |
|
|
489
|
+
| **Flaky Detection Accuracy** | 90%+ | 100% | β
Exceeded |
|
|
490
|
+
| **False Positive Rate** | <5% | 0% | β
Exceeded |
|
|
1177
491
|
|
|
1178
|
-
|
|
492
|
+
### Core Performance
|
|
493
|
+
- **Test Generation**: 1000+ tests/minute
|
|
494
|
+
- **Parallel Execution**: 10,000+ concurrent tests
|
|
495
|
+
- **Coverage Analysis**: O(log n) complexity
|
|
496
|
+
- **Data Generation**: 10,000+ records/second
|
|
497
|
+
- **Agent Spawning**: <100ms per agent
|
|
498
|
+
- **Memory Efficient**: <2GB for typical projects
|
|
1179
499
|
|
|
1180
500
|
---
|
|
1181
501
|
|
|
1182
|
-
##
|
|
502
|
+
## π Recent Changes
|
|
1183
503
|
|
|
1184
|
-
###
|
|
504
|
+
### v1.3.4 (2025-10-26) - Feature Implementation & Test Coverage
|
|
1185
505
|
|
|
1186
|
-
|
|
1187
|
-
|
|
1188
|
-
|
|
506
|
+
**New Features:**
|
|
507
|
+
- Multi-Model Router: 85.7% cost savings (exceeds 70-81% promise)
|
|
508
|
+
- Pattern Bank with vector similarity (85%+ accuracy)
|
|
509
|
+
- ML Flaky Detection with automated fix recommendations (90%+ accuracy)
|
|
510
|
+
- Streaming API with AsyncGenerator pattern
|
|
511
|
+
- AgentDB Integration (150x vector search speedup)
|
|
1189
512
|
|
|
1190
|
-
|
|
1191
|
-
|
|
1192
|
-
|
|
513
|
+
**Improvements:**
|
|
514
|
+
- Test coverage improved from 1.67% to 50-70% (30-40x increase)
|
|
515
|
+
- Fixed 328 import paths across 122 test files
|
|
516
|
+
- Documentation consistency fixes (6 count errors corrected)
|
|
517
|
+
- All 17 QE agents updated with Phase 2 skill references
|
|
1193
518
|
|
|
1194
|
-
|
|
519
|
+
**Impact:** Production-ready release with 92/100 quality score, zero breaking changes, 100% backward compatible.
|
|
1195
520
|
|
|
1196
|
-
|
|
1197
|
-
# Configure production environment
|
|
1198
|
-
cp .env.example .env.production
|
|
1199
|
-
# Edit .env.production with secure credentials
|
|
1200
|
-
|
|
1201
|
-
# Deploy
|
|
1202
|
-
docker-compose -f docker-compose.yml -f docker-compose.prod.yml up -d
|
|
1203
|
-
```
|
|
521
|
+
[View Complete Changelog](CHANGELOG.md)
|
|
1204
522
|
|
|
1205
523
|
---
|
|
1206
524
|
|
|
@@ -1242,37 +560,19 @@ npm test
|
|
|
1242
560
|
```
|
|
1243
561
|
agentic-qe/
|
|
1244
562
|
βββ src/
|
|
1245
|
-
β βββ agents/ # Agent implementation classes
|
|
563
|
+
β βββ agents/ # Agent implementation classes
|
|
1246
564
|
β βββ core/ # Core fleet management
|
|
1247
|
-
β β βββ FleetManager.ts
|
|
1248
|
-
β β βββ Agent.ts
|
|
1249
|
-
β β βββ Task.ts
|
|
1250
|
-
β β βββ EventBus.ts
|
|
1251
|
-
β β βββ MemoryManager.ts
|
|
1252
565
|
β βββ learning/ # Phase 2: Learning system
|
|
1253
|
-
β β βββ LearningEngine.ts
|
|
1254
|
-
β β βββ PerformanceTracker.ts
|
|
1255
|
-
β β βββ ImprovementLoop.ts
|
|
1256
|
-
β β βββ FlakyTestDetector.ts
|
|
1257
|
-
β β βββ FlakyPredictionModel.ts
|
|
1258
566
|
β βββ reasoning/ # Phase 2: Pattern bank
|
|
1259
|
-
β β βββ QEReasoningBank.ts
|
|
1260
|
-
β β βββ PatternExtractor.ts
|
|
1261
|
-
β β βββ PatternMatcher.ts
|
|
1262
567
|
β βββ cli/ # Command-line interface
|
|
1263
568
|
β βββ mcp/ # Model Context Protocol server
|
|
1264
569
|
β βββ types/ # TypeScript type definitions
|
|
1265
570
|
β βββ utils/ # Shared utilities
|
|
1266
571
|
βββ tests/ # Comprehensive test suites
|
|
1267
|
-
β βββ unit/
|
|
1268
|
-
β βββ integration/
|
|
1269
|
-
β βββ e2e/
|
|
1270
|
-
β βββ performance/
|
|
1271
|
-
β βββ benchmarks/ # Phase 2: Performance benchmarks
|
|
1272
572
|
βββ examples/ # Usage examples
|
|
1273
573
|
βββ docs/ # Documentation
|
|
1274
574
|
βββ .claude/ # Agent & command definitions
|
|
1275
|
-
β βββ agents/ #
|
|
575
|
+
β βββ agents/ # 18 agent definitions
|
|
1276
576
|
β βββ commands/ # 8 AQE slash commands
|
|
1277
577
|
βββ config/ # Configuration files
|
|
1278
578
|
```
|
|
@@ -1304,21 +604,6 @@ We welcome contributions! Please see [CONTRIBUTING.md](CONTRIBUTING.md) for deta
|
|
|
1304
604
|
|
|
1305
605
|
---
|
|
1306
606
|
|
|
1307
|
-
## π License
|
|
1308
|
-
|
|
1309
|
-
This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.
|
|
1310
|
-
|
|
1311
|
-
---
|
|
1312
|
-
|
|
1313
|
-
## π Acknowledgments
|
|
1314
|
-
|
|
1315
|
-
- Built with TypeScript, Node.js, and better-sqlite3
|
|
1316
|
-
- Inspired by autonomous agent architectures and swarm intelligence
|
|
1317
|
-
- Integrates with Jest, Cypress, Playwright, k6, SonarQube, and more
|
|
1318
|
-
- Compatible with Claude Code via Model Context Protocol (MCP)
|
|
1319
|
-
|
|
1320
|
-
---
|
|
1321
|
-
|
|
1322
607
|
## π Support
|
|
1323
608
|
|
|
1324
609
|
- **Documentation**: [docs/](docs/)
|
|
@@ -1330,18 +615,19 @@ This project is licensed under the MIT License - see the [LICENSE](LICENSE) file
|
|
|
1330
615
|
|
|
1331
616
|
## πΊοΈ Roadmap
|
|
1332
617
|
|
|
1333
|
-
### Current (v1.
|
|
618
|
+
### Current (v1.3)
|
|
1334
619
|
- β
Learning System with Q-learning
|
|
1335
620
|
- β
Pattern Bank with cross-project sharing
|
|
1336
621
|
- β
ML Flaky Detection (100% accuracy)
|
|
1337
622
|
- β
Continuous Improvement Loop
|
|
1338
|
-
- β
|
|
623
|
+
- β
18 specialized agents
|
|
1339
624
|
- β
Multi-framework test execution
|
|
1340
625
|
- β
Real-time coverage analysis
|
|
1341
626
|
- β
MCP integration
|
|
1342
627
|
- β
Multi-model router (70-81% cost savings)
|
|
628
|
+
- β
34 QE skills library
|
|
1343
629
|
|
|
1344
|
-
### Planned (v1.
|
|
630
|
+
### Planned (v1.4)
|
|
1345
631
|
- π Web dashboard for visualization
|
|
1346
632
|
- π GraphQL API
|
|
1347
633
|
- π CI/CD integrations (GitHub Actions, GitLab CI)
|
|
@@ -1357,6 +643,21 @@ This project is licensed under the MIT License - see the [LICENSE](LICENSE) file
|
|
|
1357
643
|
|
|
1358
644
|
---
|
|
1359
645
|
|
|
646
|
+
## π License
|
|
647
|
+
|
|
648
|
+
This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.
|
|
649
|
+
|
|
650
|
+
---
|
|
651
|
+
|
|
652
|
+
## π Acknowledgments
|
|
653
|
+
|
|
654
|
+
- Built with TypeScript, Node.js, and better-sqlite3
|
|
655
|
+
- Inspired by autonomous agent architectures and swarm intelligence
|
|
656
|
+
- Integrates with Jest, Cypress, Playwright, k6, SonarQube, and more
|
|
657
|
+
- Compatible with Claude Code via Model Context Protocol (MCP)
|
|
658
|
+
|
|
659
|
+
---
|
|
660
|
+
|
|
1360
661
|
<div align="center">
|
|
1361
662
|
|
|
1362
663
|
**Made with β€οΈ by the Agentic QE Team**
|