agentic-qe 1.3.2 β 1.3.4
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.claude/agents/qe-api-contract-validator.md +20 -0
- package/.claude/agents/qe-chaos-engineer.md +20 -0
- package/.claude/agents/qe-coverage-analyzer.md +21 -0
- package/.claude/agents/qe-deployment-readiness.md +20 -0
- package/.claude/agents/qe-flaky-test-hunter.md +20 -0
- package/.claude/agents/qe-fleet-commander.md +20 -0
- package/.claude/agents/qe-performance-tester.md +21 -0
- package/.claude/agents/qe-production-intelligence.md +20 -0
- package/.claude/agents/qe-quality-analyzer.md +20 -0
- package/.claude/agents/qe-quality-gate.md +20 -0
- package/.claude/agents/qe-regression-risk-analyzer.md +20 -0
- package/.claude/agents/qe-requirements-validator.md +20 -0
- package/.claude/agents/qe-security-scanner.md +21 -0
- package/.claude/agents/qe-test-data-architect.md +19 -0
- package/.claude/agents/qe-test-executor.md +20 -0
- package/.claude/agents/qe-test-generator.md +22 -0
- package/.claude/agents/qe-visual-tester.md +22 -0
- package/CHANGELOG.md +114 -0
- package/README.md +322 -991
- package/README.md.backup-20251026 +1366 -0
- package/bin/aqe-mcp +65 -0
- package/dist/agents/BaseAgent.js +1 -1
- package/dist/agents/BaseAgent.js.map +1 -1
- package/dist/agents/CoverageAnalyzerAgent.js +16 -16
- package/dist/agents/CoverageAnalyzerAgent.js.map +1 -1
- package/dist/agents/FlakyTestHunterAgent.js +1 -1
- package/dist/agents/FlakyTestHunterAgent.js.map +1 -1
- package/dist/cli/commands/init.d.ts.map +1 -1
- package/dist/cli/commands/init.js +71 -37
- package/dist/cli/commands/init.js.map +1 -1
- package/dist/core/memory/EnhancedAgentDBService.d.ts +127 -0
- package/dist/core/memory/EnhancedAgentDBService.d.ts.map +1 -0
- package/dist/core/memory/EnhancedAgentDBService.js +298 -0
- package/dist/core/memory/EnhancedAgentDBService.js.map +1 -0
- package/dist/core/neural/NeuralTrainer.js +2 -2
- package/dist/core/neural/NeuralTrainer.js.map +1 -1
- package/dist/learning/FixRecommendationEngine.d.ts +68 -0
- package/dist/learning/FixRecommendationEngine.d.ts.map +1 -0
- package/dist/learning/FixRecommendationEngine.js +500 -0
- package/dist/learning/FixRecommendationEngine.js.map +1 -0
- package/dist/learning/FlakyTestDetector.d.ts +19 -0
- package/dist/learning/FlakyTestDetector.d.ts.map +1 -1
- package/dist/learning/FlakyTestDetector.js +121 -2
- package/dist/learning/FlakyTestDetector.js.map +1 -1
- package/dist/learning/index.d.ts +2 -0
- package/dist/learning/index.d.ts.map +1 -1
- package/dist/learning/index.js +5 -0
- package/dist/learning/index.js.map +1 -1
- package/dist/learning/types.d.ts +11 -0
- package/dist/learning/types.d.ts.map +1 -1
- package/dist/reasoning/PatternQualityScorer.d.ts +134 -0
- package/dist/reasoning/PatternQualityScorer.d.ts.map +1 -0
- package/dist/reasoning/PatternQualityScorer.js +340 -0
- package/dist/reasoning/PatternQualityScorer.js.map +1 -0
- package/dist/reasoning/QEReasoningBank.d.ts +51 -4
- package/dist/reasoning/QEReasoningBank.d.ts.map +1 -1
- package/dist/reasoning/QEReasoningBank.js +173 -11
- package/dist/reasoning/QEReasoningBank.js.map +1 -1
- package/dist/reasoning/VectorSimilarity.d.ts +131 -0
- package/dist/reasoning/VectorSimilarity.d.ts.map +1 -0
- package/dist/reasoning/VectorSimilarity.js +250 -0
- package/dist/reasoning/VectorSimilarity.js.map +1 -0
- package/dist/reasoning/index.d.ts +8 -1
- package/dist/reasoning/index.d.ts.map +1 -1
- package/dist/reasoning/index.js +13 -2
- package/dist/reasoning/index.js.map +1 -1
- package/dist/streaming/BaseStreamHandler.d.ts +89 -0
- package/dist/streaming/BaseStreamHandler.d.ts.map +1 -0
- package/dist/streaming/BaseStreamHandler.js +168 -0
- package/dist/streaming/BaseStreamHandler.js.map +1 -0
- package/dist/streaming/TestGenerateStreamHandler.d.ts +103 -0
- package/dist/streaming/TestGenerateStreamHandler.d.ts.map +1 -0
- package/dist/streaming/TestGenerateStreamHandler.js +321 -0
- package/dist/streaming/TestGenerateStreamHandler.js.map +1 -0
- package/dist/streaming/index.d.ts +16 -0
- package/dist/streaming/index.d.ts.map +1 -0
- package/dist/streaming/index.js +39 -0
- package/dist/streaming/index.js.map +1 -0
- package/dist/utils/Database.d.ts.map +1 -1
- package/dist/utils/Database.js +15 -1
- package/dist/utils/Database.js.map +1 -1
- package/dist/utils/__mocks__/Logger.d.ts +26 -0
- package/dist/utils/__mocks__/Logger.d.ts.map +1 -0
- package/dist/utils/__mocks__/Logger.js +42 -0
- package/dist/utils/__mocks__/Logger.js.map +1 -0
- package/package.json +19 -6
- package/dist/agents/mixins/NeuralCapableMixin.d.ts +0 -130
- package/dist/agents/mixins/NeuralCapableMixin.d.ts.map +0 -1
- package/dist/agents/mixins/NeuralCapableMixin.js +0 -358
- package/dist/agents/mixins/NeuralCapableMixin.js.map +0 -1
- package/dist/agents/mixins/QUICCapableMixin.d.ts +0 -34
- package/dist/agents/mixins/QUICCapableMixin.d.ts.map +0 -1
- package/dist/agents/mixins/QUICCapableMixin.js +0 -346
- package/dist/agents/mixins/QUICCapableMixin.js.map +0 -1
- package/dist/core/security/CertificateValidator.d.ts +0 -130
- package/dist/core/security/CertificateValidator.d.ts.map +0 -1
- package/dist/core/security/CertificateValidator.js +0 -376
- package/dist/core/security/CertificateValidator.js.map +0 -1
- package/dist/core/transport/QUICTransport.d.ts +0 -62
- package/dist/core/transport/QUICTransport.d.ts.map +0 -1
- package/dist/core/transport/QUICTransport.js +0 -381
- package/dist/core/transport/QUICTransport.js.map +0 -1
- package/dist/core/transport/SecureQUICTransport.d.ts +0 -71
- package/dist/core/transport/SecureQUICTransport.d.ts.map +0 -1
- package/dist/core/transport/SecureQUICTransport.js +0 -253
- package/dist/core/transport/SecureQUICTransport.js.map +0 -1
- package/dist/learning/AdvancedFeatureExtractor.d.ts +0 -123
- package/dist/learning/AdvancedFeatureExtractor.d.ts.map +0 -1
- package/dist/learning/AdvancedFeatureExtractor.js +0 -423
- package/dist/learning/AdvancedFeatureExtractor.js.map +0 -1
- package/dist/learning/NeuralPatternMatcher.d.ts +0 -184
- package/dist/learning/NeuralPatternMatcher.d.ts.map +0 -1
- package/dist/learning/NeuralPatternMatcher.js +0 -702
- package/dist/learning/NeuralPatternMatcher.js.map +0 -1
- package/dist/learning/NeuralTrainer.d.ts +0 -209
- package/dist/learning/NeuralTrainer.d.ts.map +0 -1
- package/dist/learning/NeuralTrainer.js +0 -478
- package/dist/learning/NeuralTrainer.js.map +0 -1
- package/dist/transport/QUICTransport.d.ts +0 -340
- package/dist/transport/QUICTransport.d.ts.map +0 -1
- package/dist/transport/QUICTransport.js +0 -814
- package/dist/transport/QUICTransport.js.map +0 -1
- package/dist/transport/UDPTransport.d.ts +0 -348
- package/dist/transport/UDPTransport.d.ts.map +0 -1
- package/dist/transport/UDPTransport.js +0 -820
- package/dist/transport/UDPTransport.js.map +0 -1
package/README.md
CHANGED
|
@@ -7,379 +7,280 @@
|
|
|
7
7
|
[](https://www.typescriptlang.org/)
|
|
8
8
|
[](https://nodejs.org/)
|
|
9
9
|
|
|
10
|
-
**Version 1.3.
|
|
10
|
+
**Version 1.3.4** | [Changelog](CHANGELOG.md) | [Issues](https://github.com/proffesor-for-testing/agentic-qe/issues) | [Discussions](https://github.com/proffesor-for-testing/agentic-qe/discussions)
|
|
11
11
|
|
|
12
|
-
> Enterprise-grade test automation with AI learning, comprehensive skills library (
|
|
12
|
+
> Enterprise-grade test automation with AI learning, comprehensive skills library (34 QE skills), and optional 70-81% cost savings through intelligent model routing.
|
|
13
13
|
|
|
14
|
-
π§ **20% Continuous Improvement** | π **
|
|
15
|
-
|
|
16
|
-
[Quick Start](#quick-start) β’ [Documentation](docs/) β’ [Contributing](CONTRIBUTING.md) β’ [Examples](examples/)
|
|
14
|
+
π§ **20% Continuous Improvement** | π **34 World-Class QE Skills** | π― **100% Flaky Test Detection** | π° **70-81% Cost Savings** | π§ **61 MCP Tools**
|
|
17
15
|
|
|
18
16
|
</div>
|
|
19
17
|
|
|
20
18
|
---
|
|
21
19
|
|
|
22
|
-
##
|
|
23
|
-
|
|
24
|
-
**π Security Release**: Fixed all 4 open CodeQL security alerts - achieving **100% alert resolution (26/26 fixed)**. Critical fixes include elimination of cryptographic randomness bias, enhanced prototype pollution prevention, and comprehensive security test coverage.
|
|
25
|
-
|
|
26
|
-
### Previous Release (v1.3.1)
|
|
27
|
-
|
|
28
|
-
**Critical Bug Fix**: Fixed `aqe init` command that was using hardcoded versions (1.0.5, 1.1.0) instead of reading from `package.json`. All generated config files now correctly sync with the package version.
|
|
29
|
-
|
|
30
|
-
### Previous Release (v1.3.0)
|
|
31
|
-
|
|
32
|
-
### Security Hardening + Skills Expansion (Oct 23-24, 2025)
|
|
33
|
-
**2 days of intensive development** | **Security hardening + 17 new skills** | **11,500+ lines of expert content**
|
|
34
|
-
|
|
35
|
-
#### π **60 Claude Skills Total (35 QE-Specific)**
|
|
36
|
-
|
|
37
|
-
**Phase 1: Original Quality Engineering Skills (18 skills)** - World-class v1.0.0 β¨
|
|
38
|
-
- agentic-quality-engineering, holistic-testing-pact, context-driven-testing, exploratory-testing-advanced
|
|
39
|
-
- risk-based-testing, test-automation-strategy, api-testing-patterns, performance-testing, security-testing
|
|
40
|
-
- tdd-london-chicago, xp-practices, code-review-quality, refactoring-patterns, quality-metrics
|
|
41
|
-
- bug-reporting-excellence, technical-writing, consultancy-practices
|
|
20
|
+
## β‘ Quick Start
|
|
42
21
|
|
|
43
|
-
|
|
44
|
-
- **Testing Methodologies (6)**: regression-testing, shift-left-testing, shift-right-testing, test-design-techniques, mutation-testing, test-data-management
|
|
45
|
-
- **Specialized Testing (9)**: accessibility-testing, mobile-testing, database-testing, contract-testing, chaos-engineering-resilience, compatibility-testing, localization-testing, compliance-testing, visual-testing-advanced
|
|
46
|
-
- **Testing Infrastructure (2)**: test-environment-management, test-reporting-analytics
|
|
22
|
+
### Install & Initialize
|
|
47
23
|
|
|
48
|
-
|
|
49
|
-
|
|
50
|
-
|
|
51
|
-
- β
107 unique tags, 156 cross-references
|
|
52
|
-
- β
Semantic versioning (v1.0.0)
|
|
53
|
-
- β
34x speedup with 13 parallel agents
|
|
54
|
-
- β
Quality: 52% β 100% (+48%)
|
|
55
|
-
|
|
56
|
-
**Claude Flow Integration Skills (25 skills)**
|
|
57
|
-
- AgentDB: advanced, learning, memory-patterns, optimization, vector-search (5)
|
|
58
|
-
- GitHub: code-review, multi-repo, project-management, release-management, workflow-automation (5)
|
|
59
|
-
- Flow Nexus: neural, platform, swarm (3)
|
|
60
|
-
- Advanced: hive-mind-advanced, hooks-automation, pair-programming, performance-analysis, sparc-methodology, skill-builder, stream-chain, swarm-advanced, swarm-orchestration, reasoningbank-agentdb, reasoningbank-intelligence, verification-quality (12)
|
|
61
|
-
|
|
62
|
-
**Unified CLAUDE.md** - 72 Total Agents (18 QE + 54 Claude Flow)
|
|
63
|
-
|
|
64
|
-
#### π **AgentDB Integration - Production Hardening**
|
|
65
|
-
|
|
66
|
-
**Code Reduction: 2,290+ Lines Removed (95%)**
|
|
67
|
-
- 900 lines: Custom QUIC β AgentDB QUIC sync (<1ms latency, TLS 1.3)
|
|
68
|
-
- 800 lines: Custom neural β AgentDB learning plugins (9 RL algorithms)
|
|
69
|
-
- 896 lines: Mixins removed (QUICCapableMixin, NeuralCapableMixin)
|
|
70
|
-
- 590 lines: Wrapper removed (AgentDBIntegration)
|
|
71
|
-
|
|
72
|
-
**Performance Improvements** β‘
|
|
73
|
-
- QUIC Latency: 6.23ms β <1ms (84% faster)
|
|
74
|
-
- Vector Search: 150ms β 1ms (150x faster)
|
|
75
|
-
- Neural Training: 1000ms β 10-100ms (10-100x faster)
|
|
76
|
-
- Memory Usage: 512MB β 128-16MB (4-32x less)
|
|
77
|
-
- Startup Time: 500ms β 300ms (40% faster)
|
|
78
|
-
|
|
79
|
-
**Security Enhancements** π
|
|
80
|
-
- OWASP Compliance: 70% β 90%+ (+20 points)
|
|
81
|
-
- Vulnerabilities Fixed: 8 total (3 CRITICAL, 5 HIGH)
|
|
82
|
-
- TLS 1.3: Enforced by default
|
|
83
|
-
- Certificate Validation: Mandatory
|
|
84
|
-
|
|
85
|
-
#### β¨ **New Features**
|
|
86
|
-
|
|
87
|
-
**Advanced Search & Indexing**
|
|
88
|
-
- HNSW Indexing: 150x faster vector search (O(log n))
|
|
89
|
-
- Quantization: 4-32x memory reduction
|
|
90
|
-
- Vector Search: Semantic search across all memories
|
|
91
|
-
- Full-Text Search: BM25 ranking
|
|
92
|
-
|
|
93
|
-
**9 Reinforcement Learning Algorithms**
|
|
94
|
-
- Decision Transformer, Q-Learning, SARSA, Actor-Critic
|
|
95
|
-
- DQN, PPO, A3C, REINFORCE, Monte Carlo
|
|
96
|
-
|
|
97
|
-
**QUIC Synchronization**
|
|
98
|
-
- Sub-millisecond latency (<1ms)
|
|
99
|
-
- TLS 1.3 encryption by default
|
|
100
|
-
- Automatic connection recovery
|
|
101
|
-
- Stream multiplexing
|
|
102
|
-
|
|
103
|
-
#### π§ͺ **Test Suite Expansion**
|
|
104
|
-
- 60+ new test files added
|
|
105
|
-
- AgentDB: 6/6 tests (100%)
|
|
106
|
-
- Core: 53/53 tests (100%)
|
|
107
|
-
- Total: 59/59 tests passing (100%)
|
|
108
|
-
- Zero regressions detected
|
|
109
|
-
|
|
110
|
-
#### π§Ή **Repository Cleanup**
|
|
111
|
-
- Documentation: 24 reports archived (4.1MB saved)
|
|
112
|
-
- Dependencies: 89 packages removed (7.3MB saved)
|
|
113
|
-
- Total: 11.4MB savings
|
|
114
|
-
- Clean build: Zero TypeScript errors
|
|
115
|
-
|
|
116
|
-
#### π **Breaking Changes**
|
|
117
|
-
- `enableQUIC()` β `initializeAgentDB({ quic: {...} })`
|
|
118
|
-
- `enableNeural()` β `initializeAgentDB({ learning: {...} })`
|
|
119
|
-
- Removed: QUICTransport, NeuralPatternMatcher, mixins
|
|
120
|
-
- See [Migration Guide](docs/AGENTDB-MIGRATION-GUIDE.md)
|
|
121
|
-
|
|
122
|
-
**Release Score: 90/100** β
| [Complete Changelog](docs/COMPLETE-1.2.0-CHANGELOG.md)
|
|
24
|
+
```bash
|
|
25
|
+
# Install globally
|
|
26
|
+
npm install -g agentic-qe
|
|
123
27
|
|
|
124
|
-
|
|
28
|
+
# Initialize your project
|
|
29
|
+
cd your-project
|
|
30
|
+
aqe init
|
|
125
31
|
|
|
126
|
-
|
|
32
|
+
# Add MCP server to Claude Code (optional)
|
|
33
|
+
claude mcp add agentic-qe npx -y agentic-qe mcp:start
|
|
127
34
|
|
|
128
|
-
|
|
35
|
+
# Verify connection
|
|
36
|
+
claude mcp list
|
|
37
|
+
```
|
|
129
38
|
|
|
130
|
-
|
|
131
|
-
- Q-learning reinforcement learning for strategy optimization
|
|
132
|
-
- 20% improvement target tracking with automatic achievement
|
|
133
|
-
- Experience replay buffer (10,000 experiences)
|
|
134
|
-
- Automatic strategy recommendation with 95%+ confidence
|
|
135
|
-
- Cross-agent knowledge sharing
|
|
39
|
+
### Use from Claude Code CLI
|
|
136
40
|
|
|
137
|
-
|
|
138
|
-
- Cross-project pattern sharing and reuse
|
|
139
|
-
- 85%+ matching accuracy with AI-powered similarity
|
|
140
|
-
- 6 framework support (Jest, Mocha, Cypress, Vitest, Jasmine, AVA)
|
|
141
|
-
- Automatic pattern extraction from existing tests
|
|
142
|
-
- Pattern quality scoring and versioning
|
|
41
|
+
Ask Claude to use AQE agents directly from your terminal:
|
|
143
42
|
|
|
144
|
-
|
|
145
|
-
|
|
146
|
-
-
|
|
147
|
-
- Root cause analysis (timing, race conditions, dependencies, isolation)
|
|
148
|
-
- Automated fix recommendations with code examples
|
|
149
|
-
- < 1 second processing time for 1000+ test results
|
|
43
|
+
```bash
|
|
44
|
+
# Generate comprehensive tests
|
|
45
|
+
claude "Use qe-test-generator to create tests for src/services/user-service.ts with 95% coverage"
|
|
150
46
|
|
|
151
|
-
|
|
152
|
-
|
|
153
|
-
- Auto-optimization with statistical confidence (95%+)
|
|
154
|
-
- Failure pattern analysis and mitigation
|
|
155
|
-
- Performance benchmarks (< 50ms pattern matching, < 100ms learning)
|
|
47
|
+
# Run quality pipeline
|
|
48
|
+
claude "Initialize AQE fleet: generate tests, execute them, analyze coverage, and run quality gate"
|
|
156
49
|
|
|
157
|
-
|
|
158
|
-
|
|
159
|
-
|
|
160
|
-
- **FlakyTestHunterAgent**: ML-based detection (99% accuracy)
|
|
50
|
+
# Detect flaky tests
|
|
51
|
+
claude "Use qe-flaky-test-hunter to analyze the last 100 test runs and identify flaky tests"
|
|
52
|
+
```
|
|
161
53
|
|
|
162
|
-
|
|
54
|
+
**What gets initialized:**
|
|
55
|
+
- β
Multi-Model Router (70-81% cost savings - opt-in)
|
|
56
|
+
- β
Learning System (20% improvement target)
|
|
57
|
+
- β
Pattern Bank (cross-project reuse)
|
|
58
|
+
- β
ML Flaky Detection (100% accuracy)
|
|
59
|
+
- β
18 Specialized agent definitions
|
|
60
|
+
- β
8 AQE slash commands
|
|
61
|
+
- β
Configuration directory
|
|
163
62
|
|
|
164
63
|
---
|
|
165
64
|
|
|
166
|
-
##
|
|
65
|
+
## β¨ Features
|
|
66
|
+
|
|
67
|
+
### π€ Autonomous Agent Fleet
|
|
68
|
+
- **18 Specialized Agents**: Expert agents for every QE domain (test generation, coverage analysis, security scanning, performance testing)
|
|
69
|
+
- **AI-Powered Coordination**: Event-driven architecture with intelligent task distribution
|
|
70
|
+
- **Zero External Dependencies**: Native AQE hooks system (100-500x faster than external coordination)
|
|
71
|
+
- **Scalable**: From single developer projects to enterprise-scale testing infrastructure
|
|
167
72
|
|
|
168
|
-
###
|
|
73
|
+
### π§ Intelligence & Learning (v1.1.0)
|
|
74
|
+
- **Q-Learning System**: 20% improvement target with automatic strategy optimization
|
|
75
|
+
- **Pattern Bank**: 85%+ matching accuracy across 6 test frameworks (Jest, Mocha, Cypress, Vitest, Jasmine, AVA)
|
|
76
|
+
- **ML Flaky Detection**: 100% accuracy with root cause analysis and automated fix recommendations
|
|
77
|
+
- **Continuous Improvement**: A/B testing framework with 95%+ statistical confidence
|
|
78
|
+
- **Experience Replay**: Learn from 10,000+ past executions
|
|
169
79
|
|
|
170
|
-
|
|
171
|
-
- **70-81%
|
|
80
|
+
### π° Cost Optimization (v1.0.5)
|
|
81
|
+
- **Multi-Model Router**: 70-81% cost savings through intelligent model selection (opt-in feature)
|
|
172
82
|
- **4+ AI Models**: GPT-3.5, GPT-4, Claude Haiku, Claude Sonnet 4.5
|
|
173
|
-
- **Smart Routing**: Automatic complexity analysis and model selection
|
|
174
|
-
- **Real-Time Tracking**: Live cost monitoring with
|
|
175
|
-
- **Budget Alerts**: Email, Slack, and webhook notifications
|
|
176
|
-
- **Cost Forecasting**: Predict future costs with 90% confidence
|
|
83
|
+
- **Smart Routing**: Automatic complexity analysis and optimal model selection
|
|
84
|
+
- **Real-Time Tracking**: Live cost monitoring with budget alerts and forecasting
|
|
177
85
|
- **ROI Dashboard**: Track savings vs single-model baseline
|
|
178
86
|
|
|
179
|
-
|
|
180
|
-
- **
|
|
181
|
-
- **
|
|
182
|
-
- **
|
|
183
|
-
- **
|
|
184
|
-
- **
|
|
185
|
-
- **Cancellation Support**: Stop operations mid-stream
|
|
186
|
-
- **Event Piping**: Chain multiple operations together
|
|
87
|
+
### π Comprehensive Testing
|
|
88
|
+
- **Multi-Framework Support**: Jest, Mocha, Cypress, Playwright, Vitest, Jasmine, AVA
|
|
89
|
+
- **Parallel Execution**: 10,000+ concurrent tests with intelligent orchestration
|
|
90
|
+
- **Real-Time Coverage**: O(log n) algorithms for instant gap detection
|
|
91
|
+
- **Performance Testing**: k6, JMeter, Gatling integration
|
|
92
|
+
- **Real-Time Streaming**: Live progress updates for all operations
|
|
187
93
|
|
|
188
|
-
###
|
|
94
|
+
### π 34 QE Skills Library (v1.3.0)
|
|
95
|
+
**95%+ coverage of modern QE practices**
|
|
189
96
|
|
|
190
|
-
|
|
191
|
-
|
|
192
|
-
- **20% Improvement Target**: Automatic tracking and achievement
|
|
193
|
-
- **Experience Replay**: Learn from 10,000+ past executions
|
|
194
|
-
- **Strategy Recommendation**: AI-powered optimal strategy selection
|
|
195
|
-
- **Cross-Agent Sharing**: Agents learn from each other's experiences
|
|
196
|
-
|
|
197
|
-
#### π¦ Pattern Bank
|
|
198
|
-
- **Pattern Extraction**: Automatic extraction from existing tests
|
|
199
|
-
- **85%+ Matching Accuracy**: AI-powered pattern matching
|
|
200
|
-
- **Cross-Project Sharing**: Export/import patterns across teams
|
|
201
|
-
- **6 Framework Support**: Jest, Mocha, Cypress, Vitest, Jasmine, AVA
|
|
202
|
-
- **Quality Scoring**: Automatic pattern quality assessment
|
|
203
|
-
|
|
204
|
-
#### π― ML Flaky Detection
|
|
205
|
-
- **100% Accuracy**: ML-based detection with zero false positives
|
|
206
|
-
- **Root Cause Analysis**: Identify timing, race conditions, dependencies
|
|
207
|
-
- **Automated Fixes**: Generate fix recommendations with code examples
|
|
208
|
-
- **Continuous Monitoring**: Track test reliability over time
|
|
209
|
-
- **< 1 Second Processing**: 8,000+ test results per second
|
|
210
|
-
|
|
211
|
-
#### π Continuous Improvement
|
|
212
|
-
- **A/B Testing**: Compare strategies with statistical confidence
|
|
213
|
-
- **Auto-Optimization**: Apply winning strategies automatically
|
|
214
|
-
- **Failure Pattern Analysis**: Detect and mitigate recurring issues
|
|
215
|
-
- **Performance Benchmarks**: < 50ms pattern matching, < 100ms learning
|
|
216
|
-
|
|
217
|
-
### Core Features
|
|
218
|
-
|
|
219
|
-
#### π€ Autonomous Agent Fleet
|
|
220
|
-
- **18 Specialized Agents**: Each agent is an expert in specific quality engineering domains
|
|
221
|
-
- _Note: 17 QE-specific agents + 1 general-purpose base-template-generator agent_
|
|
222
|
-
- **AQE Hooks System**: 100-500x faster coordination with zero external dependencies
|
|
223
|
-
- **Intelligent Coordination**: Event-driven architecture with automatic task distribution
|
|
224
|
-
- **Scalable**: From single developer projects to enterprise-scale testing infrastructure
|
|
225
|
-
- **Self-Organizing**: Agents autonomously coordinate testing strategies
|
|
226
|
-
- **Type-Safe**: Full TypeScript type checking and IntelliSense support
|
|
97
|
+
<details>
|
|
98
|
+
<summary><b>View All Skills</b></summary>
|
|
227
99
|
|
|
228
|
-
|
|
229
|
-
- **
|
|
230
|
-
- **
|
|
231
|
-
- **
|
|
232
|
-
- **
|
|
100
|
+
**Phase 1: Original Quality Engineering Skills (17 skills)**
|
|
101
|
+
- **Core Testing**: agentic-quality-engineering, holistic-testing-pact, context-driven-testing, exploratory-testing-advanced
|
|
102
|
+
- **Methodologies**: tdd-london-chicago, xp-practices, risk-based-testing, test-automation-strategy
|
|
103
|
+
- **Techniques**: api-testing-patterns, performance-testing, security-testing
|
|
104
|
+
- **Code Quality**: code-review-quality, refactoring-patterns, quality-metrics
|
|
105
|
+
- **Communication**: bug-reporting-excellence, technical-writing, consultancy-practices
|
|
233
106
|
|
|
234
|
-
|
|
235
|
-
- **
|
|
236
|
-
- **
|
|
237
|
-
- **
|
|
238
|
-
- **Visual Regression**: AI-powered screenshot comparison and UI validation
|
|
107
|
+
**Phase 2: Expanded QE Skills Library (17 skills)**
|
|
108
|
+
- **Testing Methodologies (6)**: regression-testing, shift-left-testing, shift-right-testing, test-design-techniques, mutation-testing, test-data-management
|
|
109
|
+
- **Specialized Testing (9)**: accessibility-testing, mobile-testing, database-testing, contract-testing, chaos-engineering-resilience, compatibility-testing, localization-testing, compliance-testing, visual-testing-advanced
|
|
110
|
+
- **Testing Infrastructure (2)**: test-environment-management, test-reporting-analytics
|
|
239
111
|
|
|
240
|
-
|
|
241
|
-
- **API Contract Validation**: Breaking change detection across versions
|
|
242
|
-
- **Test Data Generation**: 10,000+ realistic records per second
|
|
243
|
-
- **Production Intelligence**: Convert production incidents into test scenarios
|
|
244
|
-
- **Chaos Engineering**: Controlled fault injection for resilience testing
|
|
112
|
+
</details>
|
|
245
113
|
|
|
246
114
|
---
|
|
247
115
|
|
|
248
|
-
##
|
|
116
|
+
## π» Usage Examples
|
|
249
117
|
|
|
250
|
-
###
|
|
118
|
+
### Example 1: Single Agent Execution
|
|
251
119
|
|
|
252
|
-
|
|
253
|
-
- **Claude Code**: Install from [claude.ai/code](https://claude.ai/code)
|
|
254
|
-
- **Node.js**: 18.0 or higher
|
|
255
|
-
- **npm**: 8.0 or higher
|
|
120
|
+
Ask Claude to use a specific agent:
|
|
256
121
|
|
|
257
|
-
|
|
258
|
-
-
|
|
259
|
-
|
|
260
|
-
npm install -g @claude/flow
|
|
261
|
-
# or
|
|
262
|
-
npx claude-flow@alpha init --force
|
|
263
|
-
```
|
|
122
|
+
```bash
|
|
123
|
+
claude "Use the qe-test-generator agent to create comprehensive tests for src/services/user-service.ts with 95% coverage"
|
|
124
|
+
```
|
|
264
125
|
|
|
265
|
-
**
|
|
126
|
+
**What happens:**
|
|
127
|
+
1. Claude Code spawns qe-test-generator via Task tool
|
|
128
|
+
2. Agent analyzes the source file
|
|
129
|
+
3. Generates tests with pattern matching (Phase 2 feature)
|
|
130
|
+
4. Stores results in memory at `aqe/test-plan/generated`
|
|
266
131
|
|
|
267
|
-
|
|
132
|
+
**Output:**
|
|
133
|
+
```
|
|
134
|
+
Generated 42 tests
|
|
135
|
+
Pattern hit rate: 67%
|
|
136
|
+
Time saved: 2.3s
|
|
137
|
+
Quality score: 96%
|
|
138
|
+
```
|
|
268
139
|
|
|
269
|
-
|
|
140
|
+
### Example 2: Multi-Agent Parallel Execution
|
|
270
141
|
|
|
271
|
-
|
|
142
|
+
Coordinate multiple agents at once:
|
|
272
143
|
|
|
273
|
-
|
|
274
|
-
|
|
275
|
-
|
|
144
|
+
```bash
|
|
145
|
+
claude "Initialize the AQE fleet:
|
|
146
|
+
1. Use qe-test-generator to create tests for src/services/*.ts
|
|
147
|
+
2. Use qe-test-executor to run all tests in parallel
|
|
148
|
+
3. Use qe-coverage-analyzer to find gaps with sublinear algorithms
|
|
149
|
+
4. Use qe-quality-gate to validate against 95% threshold"
|
|
150
|
+
```
|
|
276
151
|
|
|
277
|
-
|
|
278
|
-
|
|
279
|
-
|
|
152
|
+
**What happens:**
|
|
153
|
+
1. Claude spawns 4 agents concurrently in a single message
|
|
154
|
+
2. Agents coordinate through `aqe/*` memory namespace
|
|
155
|
+
3. Pipeline: test generator β executor β analyzer β gate
|
|
156
|
+
4. Real-time streaming progress updates
|
|
280
157
|
|
|
281
|
-
|
|
282
|
-
|
|
283
|
-
|
|
158
|
+
**Memory namespaces:**
|
|
159
|
+
- `aqe/test-plan/*` - Test planning and requirements
|
|
160
|
+
- `aqe/coverage/*` - Coverage analysis results
|
|
161
|
+
- `aqe/performance/*` - Performance test data
|
|
162
|
+
- `aqe/quality/*` - Quality metrics
|
|
284
163
|
|
|
285
|
-
|
|
286
|
-
npx aqe init
|
|
287
|
-
```
|
|
164
|
+
### Example 3: Using Agents with Skills
|
|
288
165
|
|
|
289
|
-
|
|
290
|
-
```bash
|
|
291
|
-
git clone https://github.com/proffesor-for-testing/agentic-qe.git
|
|
292
|
-
cd agentic-qe
|
|
293
|
-
npm install
|
|
294
|
-
npm run build
|
|
295
|
-
npm link
|
|
296
|
-
```
|
|
166
|
+
Agents automatically leverage skills:
|
|
297
167
|
|
|
298
|
-
|
|
168
|
+
```bash
|
|
169
|
+
claude "Use qe-test-generator with shift-left-testing and test-design-techniques skills to create tests before implementing the new payment feature"
|
|
170
|
+
```
|
|
299
171
|
|
|
300
|
-
|
|
301
|
-
-
|
|
302
|
-
-
|
|
172
|
+
**Available skills** (agents auto-select from 34):
|
|
173
|
+
- TDD, API testing, performance, security
|
|
174
|
+
- Accessibility, mobile, chaos engineering
|
|
175
|
+
- Regression, shift-left/right, compliance
|
|
303
176
|
|
|
304
|
-
|
|
177
|
+
### Example 4: Full Quality Pipeline
|
|
305
178
|
|
|
306
|
-
|
|
179
|
+
End-to-end quality workflow:
|
|
180
|
+
|
|
181
|
+
```bash
|
|
182
|
+
claude "Run the full AQE quality pipeline:
|
|
183
|
+
1. qe-requirements-validator - validate requirements are testable
|
|
184
|
+
2. qe-test-generator - generate comprehensive test suite
|
|
185
|
+
3. qe-test-executor - run tests with parallel execution
|
|
186
|
+
4. qe-coverage-analyzer - analyze gaps using O(log n) algorithms
|
|
187
|
+
5. qe-flaky-test-hunter - detect flaky tests with 100% ML accuracy
|
|
188
|
+
6. qe-security-scanner - run SAST/DAST scans
|
|
189
|
+
7. qe-performance-tester - load test critical paths
|
|
190
|
+
8. qe-quality-gate - validate all quality criteria met
|
|
191
|
+
9. qe-deployment-readiness - assess deployment risk"
|
|
192
|
+
```
|
|
307
193
|
|
|
308
|
-
###
|
|
194
|
+
### Example 5: Specialized Testing Scenarios
|
|
309
195
|
|
|
310
196
|
```bash
|
|
311
|
-
#
|
|
312
|
-
|
|
197
|
+
# API contract validation
|
|
198
|
+
claude "Use qe-api-contract-validator to check if the new API changes break any existing contracts"
|
|
313
199
|
|
|
314
|
-
#
|
|
315
|
-
claude
|
|
200
|
+
# Visual regression testing
|
|
201
|
+
claude "Use qe-visual-tester to compare screenshots of the updated dashboard against baseline"
|
|
316
202
|
|
|
317
|
-
#
|
|
318
|
-
claude
|
|
203
|
+
# Chaos engineering
|
|
204
|
+
claude "Use qe-chaos-engineer to inject random failures and validate system resilience"
|
|
205
|
+
|
|
206
|
+
# Flaky test detection with ML
|
|
207
|
+
claude "Use qe-flaky-test-hunter to analyze the last 100 test runs and identify flaky tests with ML-powered root cause analysis"
|
|
319
208
|
```
|
|
320
209
|
|
|
321
|
-
###
|
|
210
|
+
### Example 6: Fleet Coordination at Scale
|
|
322
211
|
|
|
323
212
|
```bash
|
|
324
|
-
#
|
|
325
|
-
|
|
326
|
-
aqe init
|
|
213
|
+
# Coordinate 50+ agents for large projects
|
|
214
|
+
claude "Use qe-fleet-commander to coordinate parallel testing across 8 microservices with 50 agents total"
|
|
327
215
|
```
|
|
328
216
|
|
|
329
|
-
|
|
330
|
-
- β
Multi-Model Router (70-81% cost savings)
|
|
331
|
-
- β
Learning System (20% improvement target)
|
|
332
|
-
- β
Pattern Bank (cross-project reuse)
|
|
333
|
-
- β
ML Flaky Detection (100% accuracy)
|
|
334
|
-
- β
Improvement Loop (A/B testing)
|
|
335
|
-
- β
17 Specialized QE agent definitions (+ 1 general-purpose agent)
|
|
336
|
-
- β
8 AQE slash commands
|
|
337
|
-
- β
Configuration directory
|
|
217
|
+
### MCP Integration Examples
|
|
338
218
|
|
|
339
|
-
|
|
219
|
+
You can also use agents through MCP tools:
|
|
340
220
|
|
|
341
221
|
```bash
|
|
342
|
-
#
|
|
343
|
-
claude
|
|
222
|
+
# Check MCP connection
|
|
223
|
+
claude mcp list
|
|
224
|
+
|
|
225
|
+
# Direct MCP tool usage (in Claude Code)
|
|
226
|
+
# Generate tests
|
|
227
|
+
mcp__agentic_qe__test_generate({
|
|
228
|
+
type: "unit",
|
|
229
|
+
framework: "jest",
|
|
230
|
+
targetFile: "src/user-service.ts"
|
|
231
|
+
})
|
|
232
|
+
|
|
233
|
+
# Execute tests
|
|
234
|
+
mcp__agentic_qe__test_execute({
|
|
235
|
+
parallel: true,
|
|
236
|
+
coverage: true
|
|
237
|
+
})
|
|
238
|
+
|
|
239
|
+
# Analyze coverage
|
|
240
|
+
mcp__agentic_qe__coverage_analyze({
|
|
241
|
+
threshold: 95
|
|
242
|
+
})
|
|
344
243
|
```
|
|
345
244
|
|
|
346
|
-
|
|
347
|
-
|
|
348
|
-
|
|
349
|
-
|
|
350
|
-
|
|
245
|
+
### CLI Direct Usage
|
|
246
|
+
|
|
247
|
+
```bash
|
|
248
|
+
# Generate tests
|
|
249
|
+
aqe test src/services/user-service.ts
|
|
351
250
|
|
|
352
|
-
|
|
251
|
+
# Analyze coverage
|
|
252
|
+
aqe coverage --threshold 95
|
|
353
253
|
|
|
354
|
-
|
|
254
|
+
# Run quality gate
|
|
255
|
+
aqe quality
|
|
355
256
|
|
|
356
|
-
|
|
257
|
+
# View fleet status
|
|
258
|
+
aqe status --verbose
|
|
357
259
|
|
|
358
|
-
|
|
260
|
+
# Enable multi-model router (70-81% cost savings)
|
|
261
|
+
aqe routing enable
|
|
359
262
|
|
|
360
|
-
|
|
361
|
-
# Enable learning for all agents
|
|
263
|
+
# Start learning system
|
|
362
264
|
aqe learn enable --all
|
|
265
|
+
```
|
|
363
266
|
|
|
364
|
-
|
|
365
|
-
aqe learn status
|
|
267
|
+
### Advanced Patterns
|
|
366
268
|
|
|
367
|
-
|
|
368
|
-
aqe learn history --agent test-generator
|
|
269
|
+
#### Pattern 1: Continuous Learning
|
|
369
270
|
|
|
370
|
-
|
|
371
|
-
|
|
271
|
+
```bash
|
|
272
|
+
# Agents learn from execution
|
|
273
|
+
claude "Use qe-test-generator with learning enabled to create tests, then analyze improvement over time"
|
|
372
274
|
|
|
373
|
-
#
|
|
374
|
-
aqe learn
|
|
275
|
+
# Check learning metrics
|
|
276
|
+
aqe learn status --agent test-generator
|
|
375
277
|
```
|
|
376
278
|
|
|
377
|
-
**Example Output
|
|
279
|
+
**Example Output:**
|
|
378
280
|
```
|
|
379
281
|
π LEARNING STATUS
|
|
380
282
|
|
|
381
283
|
Agent: test-generator
|
|
382
|
-
ββββββββββββββββββββββββββββββββββββββββββββββ
|
|
383
284
|
Status: ENABLED β
|
|
384
285
|
Total Experiences: 247
|
|
385
286
|
Exploration Rate: 15.3%
|
|
@@ -393,30 +294,19 @@ Top Strategies:
|
|
|
393
294
|
1. property-based (confidence: 92%, success: 95%)
|
|
394
295
|
2. mutation-based (confidence: 85%, success: 88%)
|
|
395
296
|
3. example-based (confidence: 78%, success: 82%)
|
|
396
|
-
|
|
397
|
-
Recent Trend: β improving
|
|
398
297
|
```
|
|
399
298
|
|
|
400
|
-
|
|
299
|
+
#### Pattern 2: Pattern Bank Usage
|
|
401
300
|
|
|
402
301
|
```bash
|
|
403
|
-
#
|
|
404
|
-
|
|
405
|
-
|
|
406
|
-
# Search patterns by keyword
|
|
407
|
-
aqe patterns search "null check"
|
|
408
|
-
|
|
409
|
-
# Extract patterns from tests
|
|
410
|
-
aqe patterns extract --path tests/ --framework jest
|
|
302
|
+
# Extract and reuse patterns
|
|
303
|
+
claude "Use qe-test-generator to extract test patterns from existing tests, then apply them to new modules"
|
|
411
304
|
|
|
412
|
-
#
|
|
413
|
-
aqe patterns
|
|
414
|
-
|
|
415
|
-
# Export patterns
|
|
416
|
-
aqe patterns export --output patterns-backup.json
|
|
305
|
+
# List patterns
|
|
306
|
+
aqe patterns list --framework jest
|
|
417
307
|
```
|
|
418
308
|
|
|
419
|
-
**Example Output
|
|
309
|
+
**Example Output:**
|
|
420
310
|
```
|
|
421
311
|
π¦ PATTERN LIBRARY (247 patterns)
|
|
422
312
|
|
|
@@ -425,67 +315,23 @@ ID | Name | Framework | Quality | Uses
|
|
|
425
315
|
pattern-001| Null Parameter Check | jest | 92% | 142
|
|
426
316
|
pattern-002| Empty Array Handling | jest | 89% | 98
|
|
427
317
|
pattern-003| API Timeout Test | cypress | 95% | 87
|
|
428
|
-
pattern-004| Database Transaction | mocha | 88% | 76
|
|
429
|
-
pattern-005| Async Error Handling | jest | 91% | 65
|
|
430
|
-
|
|
431
|
-
Total: 247 patterns | Average Quality: 87%
|
|
432
318
|
```
|
|
433
319
|
|
|
434
|
-
|
|
435
|
-
|
|
436
|
-
```bash
|
|
437
|
-
# View improvement status
|
|
438
|
-
aqe improve status
|
|
439
|
-
|
|
440
|
-
# Start improvement loop
|
|
441
|
-
aqe improve start
|
|
442
|
-
|
|
443
|
-
# Run A/B test
|
|
444
|
-
aqe improve ab-test --strategies "property-based,mutation-based" --sample-size 50
|
|
445
|
-
|
|
446
|
-
# View failure patterns
|
|
447
|
-
aqe improve failures
|
|
448
|
-
|
|
449
|
-
# Generate improvement report
|
|
450
|
-
aqe improve report --format html --output improvement-report.html
|
|
451
|
-
```
|
|
452
|
-
|
|
453
|
-
---
|
|
454
|
-
|
|
455
|
-
## π― Phase 1 Commands (v1.0.5)
|
|
456
|
-
|
|
457
|
-
### Multi-Model Router Commands π°
|
|
320
|
+
#### Pattern 3: Cost Optimization
|
|
458
321
|
|
|
459
322
|
```bash
|
|
460
|
-
# Enable
|
|
323
|
+
# Enable intelligent model routing
|
|
461
324
|
aqe routing enable
|
|
462
325
|
|
|
463
|
-
# View
|
|
464
|
-
|
|
465
|
-
|
|
466
|
-
# Launch real-time cost dashboard
|
|
326
|
+
# View savings
|
|
327
|
+
claude "Check routing status and show cost savings"
|
|
467
328
|
aqe routing dashboard
|
|
468
|
-
|
|
469
|
-
# Generate detailed cost report
|
|
470
|
-
aqe routing report --format html --output report.html
|
|
471
|
-
|
|
472
|
-
# View routing statistics
|
|
473
|
-
aqe routing stats --days 30
|
|
474
|
-
|
|
475
|
-
# Disable routing
|
|
476
|
-
aqe routing disable
|
|
477
329
|
```
|
|
478
330
|
|
|
479
|
-
**Example Output
|
|
331
|
+
**Example Output:**
|
|
480
332
|
```
|
|
481
333
|
β
Multi-Model Router Status
|
|
482
334
|
|
|
483
|
-
Configuration:
|
|
484
|
-
Status: ENABLED β
|
|
485
|
-
Default Model: claude-sonnet-4.5
|
|
486
|
-
Cost Tracking: ENABLED β
|
|
487
|
-
Fallback Chains: ENABLED β
|
|
488
|
-
|
|
489
335
|
Cost Summary (Last 30 Days):
|
|
490
336
|
Total Cost: $127.50
|
|
491
337
|
Baseline Cost: $545.00
|
|
@@ -499,257 +345,20 @@ Model Usage:
|
|
|
499
345
|
ββ gpt-4: 7% (critical tasks)
|
|
500
346
|
```
|
|
501
347
|
|
|
502
|
-
|
|
503
|
-
|
|
504
|
-
### Basic Commands
|
|
505
|
-
|
|
506
|
-
```bash
|
|
507
|
-
# Check fleet status
|
|
508
|
-
aqe status
|
|
509
|
-
|
|
510
|
-
# Generate tests for a module
|
|
511
|
-
aqe test src/services/user-service.ts
|
|
512
|
-
|
|
513
|
-
# Analyze test coverage
|
|
514
|
-
aqe coverage --threshold 95
|
|
515
|
-
|
|
516
|
-
# Run quality gate validation
|
|
517
|
-
aqe quality
|
|
518
|
-
|
|
519
|
-
# Execute comprehensive test suite
|
|
520
|
-
aqe execute --parallel --coverage
|
|
521
|
-
|
|
522
|
-
# View all commands
|
|
523
|
-
aqe help
|
|
524
|
-
```
|
|
525
|
-
|
|
526
|
-
---
|
|
527
|
-
|
|
528
|
-
## π» Programmatic Usage
|
|
529
|
-
|
|
530
|
-
### Pattern-Based Test Generation (v1.1.0) π
|
|
531
|
-
|
|
532
|
-
```typescript
|
|
533
|
-
import { TestGeneratorAgent, QEReasoningBank } from 'agentic-qe';
|
|
534
|
-
|
|
535
|
-
// Initialize pattern bank
|
|
536
|
-
const bank = new QEReasoningBank();
|
|
537
|
-
|
|
538
|
-
// Create agent with pattern matching
|
|
539
|
-
const agent = new TestGeneratorAgent(
|
|
540
|
-
{ agentId: 'test-gen-1', memoryStore },
|
|
541
|
-
{
|
|
542
|
-
targetCoverage: 95,
|
|
543
|
-
framework: 'jest',
|
|
544
|
-
enablePatterns: true, // β
Enable pattern matching
|
|
545
|
-
enableLearning: true, // β
Enable learning
|
|
546
|
-
reasoningBank: bank
|
|
547
|
-
}
|
|
548
|
-
);
|
|
549
|
-
|
|
550
|
-
// Generate tests with patterns
|
|
551
|
-
const result = await agent.execute({
|
|
552
|
-
type: 'test-generation',
|
|
553
|
-
payload: {
|
|
554
|
-
sourceFile: 'src/myModule.ts',
|
|
555
|
-
framework: 'jest'
|
|
556
|
-
}
|
|
557
|
-
});
|
|
558
|
-
|
|
559
|
-
console.log(`Generated ${result.testsGenerated} tests`);
|
|
560
|
-
console.log(`Pattern hit rate: ${result.patternHitRate}%`);
|
|
561
|
-
console.log(`Time saved: ${result.patterns.savings}ms`);
|
|
562
|
-
console.log(`Quality score: ${result.qualityScore}%`);
|
|
563
|
-
```
|
|
564
|
-
|
|
565
|
-
**Performance Impact:**
|
|
566
|
-
```
|
|
567
|
-
Without Patterns:
|
|
568
|
-
ββ Test Generation: 180ms avg
|
|
569
|
-
ββ Coverage: 78%
|
|
570
|
-
ββ Quality Score: 82%
|
|
571
|
-
|
|
572
|
-
With Patterns:
|
|
573
|
-
ββ Test Generation: 145ms avg (β 19.4%)
|
|
574
|
-
ββ Coverage: 94% (β 20.5%)
|
|
575
|
-
ββ Quality Score: 96% (β 17.1%)
|
|
576
|
-
```
|
|
577
|
-
|
|
578
|
-
### Learning-Enhanced Coverage Analysis (v1.1.0) π
|
|
579
|
-
|
|
580
|
-
```typescript
|
|
581
|
-
import { CoverageAnalyzerAgent, LearningEngine } from 'agentic-qe';
|
|
582
|
-
|
|
583
|
-
// Create learning engine
|
|
584
|
-
const learningEngine = new LearningEngine('coverage-1', memory, {
|
|
585
|
-
enabled: true,
|
|
586
|
-
learningRate: 0.1,
|
|
587
|
-
targetImprovement: 0.20 // 20% improvement target
|
|
588
|
-
});
|
|
589
|
-
|
|
590
|
-
await learningEngine.initialize();
|
|
591
|
-
|
|
592
|
-
// Create agent with learning
|
|
593
|
-
const agent = new CoverageAnalyzerAgent(
|
|
594
|
-
{ agentId: 'coverage-1', memoryStore: memory },
|
|
595
|
-
{
|
|
596
|
-
targetCoverage: 95,
|
|
597
|
-
algorithm: 'sublinear',
|
|
598
|
-
enableLearning: true // β
Enable learning
|
|
599
|
-
}
|
|
600
|
-
);
|
|
601
|
-
|
|
602
|
-
// Analyze coverage (learning happens automatically)
|
|
603
|
-
const analysis = await agent.execute({
|
|
604
|
-
type: 'coverage-analysis',
|
|
605
|
-
payload: {
|
|
606
|
-
coverageReport: './coverage/coverage-final.json',
|
|
607
|
-
threshold: 95
|
|
608
|
-
}
|
|
609
|
-
});
|
|
610
|
-
|
|
611
|
-
// Check learning progress
|
|
612
|
-
const status = await learningEngine.calculateImprovement();
|
|
613
|
-
console.log(`Current improvement: ${(status.improvementRate * 100).toFixed(1)}%`);
|
|
614
|
-
console.log(`Target reached: ${status.targetAchieved ? 'β
YES' : 'β NO'}`);
|
|
615
|
-
console.log(`Confidence: ${(status.confidence * 100).toFixed(1)}%`);
|
|
616
|
-
```
|
|
617
|
-
|
|
618
|
-
### ML Flaky Detection (v1.1.0) π
|
|
619
|
-
|
|
620
|
-
```typescript
|
|
621
|
-
import { FlakyTestHunterAgent, FlakyTestDetector } from 'agentic-qe';
|
|
622
|
-
|
|
623
|
-
// Create detector with ML
|
|
624
|
-
const detector = new FlakyTestDetector({
|
|
625
|
-
minRuns: 5,
|
|
626
|
-
passRateThreshold: 0.8,
|
|
627
|
-
confidenceThreshold: 0.7,
|
|
628
|
-
enableML: true // β
100% accuracy
|
|
629
|
-
});
|
|
630
|
-
|
|
631
|
-
// Create agent
|
|
632
|
-
const agent = new FlakyTestHunterAgent(
|
|
633
|
-
{ agentId: 'flaky-1', memoryStore: memory },
|
|
634
|
-
{
|
|
635
|
-
enableML: true, // β
100% accuracy
|
|
636
|
-
detector
|
|
637
|
-
}
|
|
638
|
-
);
|
|
639
|
-
|
|
640
|
-
// Detect flaky tests
|
|
641
|
-
const result = await agent.execute({
|
|
642
|
-
type: 'flaky-detection',
|
|
643
|
-
payload: {
|
|
644
|
-
testHistory: testResults
|
|
645
|
-
}
|
|
646
|
-
});
|
|
647
|
-
|
|
648
|
-
result.flakyTests.forEach(test => {
|
|
649
|
-
console.log(`π΄ ${test.testName}`);
|
|
650
|
-
console.log(` Pass Rate: ${(test.passRate * 100).toFixed(1)}%`);
|
|
651
|
-
console.log(` Root Cause: ${test.rootCause.cause}`);
|
|
652
|
-
console.log(` ML Confidence: ${(test.rootCause.mlConfidence * 100).toFixed(1)}%`);
|
|
653
|
-
console.log(` Severity: ${test.severity}`);
|
|
654
|
-
console.log(` Fix: ${test.fixRecommendations[0].recommendation}`);
|
|
655
|
-
console.log(` Code Example:`);
|
|
656
|
-
console.log(` ${test.fixRecommendations[0].codeExample}`);
|
|
657
|
-
});
|
|
658
|
-
```
|
|
659
|
-
|
|
660
|
-
**Detection Results:**
|
|
661
|
-
```
|
|
662
|
-
Model Training Complete:
|
|
663
|
-
Accuracy: 100.00% β
Exceeds 90% target by 10%
|
|
664
|
-
Precision: 100.00% β
Perfect precision
|
|
665
|
-
Recall: 100.00% β
Perfect recall
|
|
666
|
-
F1 Score: 100.00% β
Perfect F1
|
|
667
|
-
False Positive Rate: 0.00% β
Well below 5% target
|
|
668
|
-
|
|
669
|
-
Processing 1,200 test results: ~150ms
|
|
670
|
-
Throughput: ~8,000 results/second
|
|
671
|
-
Memory Usage: < 5MB delta
|
|
672
|
-
```
|
|
673
|
-
|
|
674
|
-
### With Multi-Model Router (v1.0.5)
|
|
675
|
-
|
|
676
|
-
```typescript
|
|
677
|
-
import { FleetManager, AdaptiveModelRouter } from 'agentic-qe';
|
|
678
|
-
|
|
679
|
-
// Initialize fleet with cost-optimized routing
|
|
680
|
-
const fleet = new FleetManager({
|
|
681
|
-
maxAgents: 20,
|
|
682
|
-
topology: 'mesh',
|
|
683
|
-
routing: {
|
|
684
|
-
enabled: true,
|
|
685
|
-
defaultModel: 'claude-sonnet-4.5',
|
|
686
|
-
enableCostTracking: true,
|
|
687
|
-
enableFallback: true,
|
|
688
|
-
modelPreferences: {
|
|
689
|
-
simple: 'gpt-3.5-turbo', // 70% cheaper for simple tasks
|
|
690
|
-
medium: 'claude-haiku', // 60% cheaper for standard tests
|
|
691
|
-
complex: 'claude-sonnet-4.5', // Best quality/cost for complex
|
|
692
|
-
critical: 'gpt-4' // Maximum quality when needed
|
|
693
|
-
},
|
|
694
|
-
budgets: {
|
|
695
|
-
daily: 50,
|
|
696
|
-
monthly: 1000
|
|
697
|
-
}
|
|
698
|
-
}
|
|
699
|
-
});
|
|
700
|
-
|
|
701
|
-
await fleet.initialize();
|
|
702
|
-
|
|
703
|
-
// Spawn agent (automatically uses optimal model based on task complexity)
|
|
704
|
-
const testGen = await fleet.spawnAgent('test-generator', {
|
|
705
|
-
targetCoverage: 95,
|
|
706
|
-
framework: 'jest',
|
|
707
|
-
useRouting: true // Enable intelligent model selection
|
|
708
|
-
});
|
|
709
|
-
|
|
710
|
-
// Execute task (router selects cheapest model that meets quality requirements)
|
|
711
|
-
const tests = await testGen.execute({
|
|
712
|
-
sourceFile: 'src/services/user-service.ts',
|
|
713
|
-
testStyle: 'property-based'
|
|
714
|
-
});
|
|
715
|
-
|
|
716
|
-
// Check cost savings
|
|
717
|
-
const savings = await fleet.getRoutingSavings();
|
|
718
|
-
console.log(`π° Total savings: $${savings.total} (${savings.percent}%)`);
|
|
719
|
-
console.log(`π Models used: ${JSON.stringify(savings.modelBreakdown, null, 2)}`);
|
|
720
|
-
```
|
|
348
|
+
### Pro Tips
|
|
721
349
|
|
|
722
|
-
|
|
723
|
-
|
|
724
|
-
|
|
725
|
-
|
|
726
|
-
|
|
727
|
-
|
|
728
|
-
| Feature | Target | Actual | Status |
|
|
729
|
-
|---------|--------|--------|--------|
|
|
730
|
-
| **Pattern Matching (p95)** | <50ms | 32ms | β
Exceeded |
|
|
731
|
-
| **Learning Iteration** | <100ms | 68ms | β
Exceeded |
|
|
732
|
-
| **ML Flaky Detection (1000 tests)** | <500ms | 385ms | β
Exceeded |
|
|
733
|
-
| **Agent Memory** | <100MB | 85MB | β
Exceeded |
|
|
734
|
-
| **Cost Savings** | 70%+ | 70-81% | β
Achieved |
|
|
735
|
-
| **Test Improvement** | 20%+ | 23%+ | β
Exceeded |
|
|
736
|
-
| **Flaky Detection Accuracy** | 90%+ | 100% | β
Exceeded |
|
|
737
|
-
| **False Positive Rate** | <5% | 0% | β
Exceeded |
|
|
738
|
-
|
|
739
|
-
### Core Performance
|
|
740
|
-
|
|
741
|
-
- **Test Generation**: 1000+ tests/minute
|
|
742
|
-
- **Parallel Execution**: 10,000+ concurrent tests
|
|
743
|
-
- **Coverage Analysis**: O(log n) complexity
|
|
744
|
-
- **Data Generation**: 10,000+ records/second
|
|
745
|
-
- **Agent Spawning**: <100ms per agent
|
|
746
|
-
- **Memory Efficient**: <2GB for typical projects
|
|
350
|
+
1. **Batch agent operations**: Always spawn multiple agents in one Claude message for parallel execution
|
|
351
|
+
2. **Use memory namespace**: Agents coordinate through `aqe/*` memory keys
|
|
352
|
+
3. **Enable learning**: Add `--enable-learning` to agent commands for continuous improvement
|
|
353
|
+
4. **Check agent status**: Use `aqe status` to see active agents and coordination
|
|
354
|
+
5. **Review agent output**: Agents store detailed results in `.agentic-qe/logs/`
|
|
747
355
|
|
|
748
356
|
---
|
|
749
357
|
|
|
750
358
|
## π€ Agent Types
|
|
751
359
|
|
|
752
|
-
|
|
360
|
+
<details>
|
|
361
|
+
<summary><b>Core Testing Agents (5 agents)</b></summary>
|
|
753
362
|
|
|
754
363
|
| Agent | Purpose | Key Features | Phase 2 Enhancements |
|
|
755
364
|
|-------|---------|-------------|---------------------|
|
|
@@ -759,14 +368,20 @@ console.log(`π Models used: ${JSON.stringify(savings.modelBreakdown, null, 2)
|
|
|
759
368
|
| **quality-gate** | Intelligent validation | ML-driven decisions, risk assessment | β
Flaky test metrics |
|
|
760
369
|
| **quality-analyzer** | Metrics analysis | ESLint, SonarQube, Lighthouse integration | - |
|
|
761
370
|
|
|
762
|
-
|
|
371
|
+
</details>
|
|
372
|
+
|
|
373
|
+
<details>
|
|
374
|
+
<summary><b>Performance & Security (2 agents)</b></summary>
|
|
763
375
|
|
|
764
376
|
| Agent | Purpose | Key Features |
|
|
765
377
|
|-------|---------|-------------|
|
|
766
378
|
| **performance-tester** | Load & stress testing | k6, JMeter, Gatling, bottleneck detection |
|
|
767
379
|
| **security-scanner** | Vulnerability detection | SAST, DAST, dependency scanning |
|
|
768
380
|
|
|
769
|
-
|
|
381
|
+
</details>
|
|
382
|
+
|
|
383
|
+
<details>
|
|
384
|
+
<summary><b>Strategic Planning (3 agents)</b></summary>
|
|
770
385
|
|
|
771
386
|
| Agent | Purpose | Key Features |
|
|
772
387
|
|-------|---------|-------------|
|
|
@@ -774,7 +389,10 @@ console.log(`π Models used: ${JSON.stringify(savings.modelBreakdown, null, 2)
|
|
|
774
389
|
| **production-intelligence** | Incident replay | RUM analysis, anomaly detection |
|
|
775
390
|
| **fleet-commander** | Hierarchical coordination | 50+ agent orchestration |
|
|
776
391
|
|
|
777
|
-
|
|
392
|
+
</details>
|
|
393
|
+
|
|
394
|
+
<details>
|
|
395
|
+
<summary><b>Advanced Testing (4 agents)</b></summary>
|
|
778
396
|
|
|
779
397
|
| Agent | Purpose | Key Features | Phase 2 Enhancements |
|
|
780
398
|
|-------|---------|-------------|---------------------|
|
|
@@ -783,7 +401,10 @@ console.log(`π Models used: ${JSON.stringify(savings.modelBreakdown, null, 2)
|
|
|
783
401
|
| **api-contract-validator** | Breaking change detection | OpenAPI, GraphQL, gRPC | - |
|
|
784
402
|
| **flaky-test-hunter** | Stability analysis | Statistical detection, auto-fix | β
100% accuracy ML detection |
|
|
785
403
|
|
|
786
|
-
|
|
404
|
+
</details>
|
|
405
|
+
|
|
406
|
+
<details>
|
|
407
|
+
<summary><b>Specialized (3 agents)</b></summary>
|
|
787
408
|
|
|
788
409
|
| Agent | Purpose | Key Features |
|
|
789
410
|
|-------|---------|-------------|
|
|
@@ -791,386 +412,113 @@ console.log(`π Models used: ${JSON.stringify(savings.modelBreakdown, null, 2)
|
|
|
791
412
|
| **visual-tester** | UI regression | AI-powered comparison |
|
|
792
413
|
| **chaos-engineer** | Resilience testing | Fault injection, blast radius |
|
|
793
414
|
|
|
794
|
-
|
|
795
|
-
|
|
796
|
-
## ποΈ Architecture
|
|
797
|
-
|
|
798
|
-
### Core Components
|
|
799
|
-
|
|
800
|
-
```
|
|
801
|
-
βββββββββββββββββββββββββββββββββββββββββββββββ
|
|
802
|
-
β Fleet Manager β
|
|
803
|
-
β (Central Coordination & Task Distribution) β
|
|
804
|
-
βββββββββββββββββββββββββββββββββββββββββββββββ
|
|
805
|
-
β
|
|
806
|
-
βββββββββββββΌββββββββββββ
|
|
807
|
-
β β β
|
|
808
|
-
ββββββΌβββββ βββββΌβββββ βββββΌβββββ
|
|
809
|
-
β Agent 1 β β Agent 2β β Agent Nβ
|
|
810
|
-
β Pool β β Pool β β Pool β
|
|
811
|
-
ββββββ¬βββββ βββββ¬βββββ βββββ¬βββββ
|
|
812
|
-
β β β
|
|
813
|
-
ββββββΌβββββββββββΌβββββββββββΌβββββ
|
|
814
|
-
β Event Bus β
|
|
815
|
-
β (Event-Driven Communication) β
|
|
816
|
-
ββββββ¬βββββββββββββββββββββββ¬βββββ
|
|
817
|
-
β β
|
|
818
|
-
ββββββΌβββββ ββββββΌβββββ
|
|
819
|
-
β Memory β βDatabase β
|
|
820
|
-
β Store β β(SQLite) β
|
|
821
|
-
β β β β
|
|
822
|
-
β Phase 2:β β β
|
|
823
|
-
β Learningβ β β
|
|
824
|
-
β Patternsβ β β
|
|
825
|
-
β ML Modelβ β β
|
|
826
|
-
βββββββββββ βββββββββββ
|
|
827
|
-
```
|
|
828
|
-
|
|
829
|
-
### Event-Driven Architecture
|
|
415
|
+
</details>
|
|
830
416
|
|
|
831
|
-
|
|
832
|
-
|
|
833
|
-
- **Memory Store**: Shared context and learning (SwarmMemoryManager)
|
|
834
|
-
- **Persistence**: SQLite for state, metrics, and audit trails
|
|
417
|
+
<details>
|
|
418
|
+
<summary><b>General Purpose (1 agent)</b></summary>
|
|
835
419
|
|
|
836
|
-
|
|
837
|
-
|
|
838
|
-
|
|
839
|
-
|
|
840
|
-
```typescript
|
|
841
|
-
// Automatic lifecycle hooks in every agent (aqe-hooks protocol)
|
|
842
|
-
class QEAgent extends BaseAgent {
|
|
843
|
-
protected async onPreTask(data): Promise<void> { /* prepare */ }
|
|
844
|
-
protected async onPostTask(data): Promise<void> { /* validate */ }
|
|
845
|
-
protected async onTaskError(data): Promise<void> { /* recover */ }
|
|
846
|
-
}
|
|
847
|
-
|
|
848
|
-
// Advanced verification hooks
|
|
849
|
-
const hookManager = new VerificationHookManager(memoryStore);
|
|
850
|
-
await hookManager.executePreTaskVerification({ task, context });
|
|
851
|
-
await hookManager.executePostTaskValidation({ task, result });
|
|
852
|
-
```
|
|
420
|
+
| Agent | Purpose | Key Features |
|
|
421
|
+
|-------|---------|-------------|
|
|
422
|
+
| **base-template-generator** | Agent templates | General-purpose agent creation |
|
|
853
423
|
|
|
854
|
-
|
|
424
|
+
</details>
|
|
855
425
|
|
|
856
|
-
**
|
|
857
|
-
- Full TypeScript type safety
|
|
858
|
-
- Direct SwarmMemoryManager integration
|
|
859
|
-
- Built-in RollbackManager support
|
|
860
|
-
- EventBus coordination
|
|
861
|
-
- Context engineering (pre/post tool-use bundles)
|
|
426
|
+
**Total: 18 Agents** (17 QE-specific + 1 general-purpose)
|
|
862
427
|
|
|
863
428
|
---
|
|
864
429
|
|
|
865
430
|
## π Documentation
|
|
866
431
|
|
|
867
|
-
### π Phase 2 Features (v1.1.0)
|
|
868
|
-
- [Learning System User Guide](docs/guides/LEARNING-SYSTEM-USER-GUIDE.md) - **NEW!** Q-learning and continuous improvement
|
|
869
|
-
- [Pattern Management User Guide](docs/guides/PATTERN-MANAGEMENT-USER-GUIDE.md) - **NEW!** Cross-project pattern sharing
|
|
870
|
-
- [ML Flaky Detection Guide](docs/guides/ML-FLAKY-DETECTION-USER-GUIDE.md) - **NEW!** 100% accurate flaky detection
|
|
871
|
-
- [Performance Improvement Guide](docs/guides/PERFORMANCE-IMPROVEMENT-USER-GUIDE.md) - **NEW!** A/B testing and optimization
|
|
872
|
-
- [Learning System Examples](docs/examples/LEARNING-SYSTEM-EXAMPLES.md) - **NEW!** Learning code examples
|
|
873
|
-
- [Pattern Examples](docs/examples/REASONING-BANK-EXAMPLES.md) - **NEW!** Pattern usage examples
|
|
874
|
-
- [Flaky Detection Examples](docs/examples/FLAKY-DETECTION-ML-EXAMPLES.md) - **NEW!** ML detection examples
|
|
875
|
-
|
|
876
|
-
### Phase 1 Features (v1.0.5)
|
|
877
|
-
- [Multi-Model Router Guide](docs/guides/MULTI-MODEL-ROUTER.md) - Save 70% on AI costs
|
|
878
|
-
- [Streaming API Tutorial](docs/guides/STREAMING-API.md) - Real-time progress updates
|
|
879
|
-
- [Cost Optimization Best Practices](docs/guides/COST-OPTIMIZATION.md) - Maximize ROI
|
|
880
|
-
- [Migration Guide v1.0.5](docs/guides/MIGRATION-V1.0.5.md) - Upgrade guide
|
|
881
|
-
- [Routing API Reference](docs/api/ROUTING-API.md) - Complete API docs
|
|
882
|
-
- [Streaming API Reference](docs/api/STREAMING-API.md) - Complete API docs
|
|
883
|
-
- [Phase 1 Code Examples](docs/examples/ROUTING-EXAMPLES.md) - Working examples
|
|
884
|
-
|
|
885
432
|
### Getting Started
|
|
886
|
-
- [Quick Start Guide](docs/AQE-CLI.md)
|
|
433
|
+
- [Quick Start Guide](docs/AQE-CLI.md) - Get started in 5 minutes
|
|
887
434
|
- [User Guide](docs/USER-GUIDE.md) - Comprehensive workflows and examples
|
|
888
|
-
- [
|
|
435
|
+
- [MCP Integration](docs/guides/MCP-INTEGRATION.md) - Claude Code integration
|
|
889
436
|
- [Configuration Guide](docs/CONFIGURATION.md) - Complete configuration reference
|
|
890
437
|
- [Troubleshooting Guide](docs/TROUBLESHOOTING.md) - Common issues and solutions
|
|
891
438
|
|
|
892
|
-
###
|
|
893
|
-
- [Test Generation](docs/guides/TEST-GENERATION.md)
|
|
894
|
-
- [Coverage Analysis](docs/guides/COVERAGE-ANALYSIS.md)
|
|
895
|
-
- [Quality Gates](docs/guides/QUALITY-GATES.md)
|
|
896
|
-
- [Performance Testing](docs/guides/PERFORMANCE-TESTING.md)
|
|
897
|
-
- [Test Execution](docs/guides/TEST-EXECUTION.md)
|
|
898
|
-
- [MCP Integration](docs/guides/MCP-INTEGRATION.md)
|
|
899
|
-
|
|
900
|
-
### Advanced Topics
|
|
901
|
-
- [API Reference](docs/API.md)
|
|
902
|
-
- [Agent Development](docs/AGENT-DEVELOPMENT.md)
|
|
903
|
-
- [MCP Integration](docs/CLAUDE-MD-INTEGRATION.md)
|
|
904
|
-
- [Best Practices](docs/AI%20%26%20Agentic%20Security%20Best%20Practices.md)
|
|
905
|
-
- [AQE Hooks Guide](docs/AQE-HOOKS-GUIDE.md)
|
|
906
|
-
|
|
907
|
-
### Commands Reference
|
|
908
|
-
- [AQE Commands Overview](docs/QE-COMMANDS-INDEX.md)
|
|
909
|
-
- [Command Specifications](docs/QE-SLASH-COMMANDS-SPECIFICATION.md)
|
|
910
|
-
- [Hooks Architecture](docs/QE_HOOKS_ARCHITECTURE.md)
|
|
439
|
+
### Feature Guides
|
|
911
440
|
|
|
912
|
-
|
|
441
|
+
**Phase 2 Features (v1.1.0)**
|
|
442
|
+
- [Learning System User Guide](docs/guides/LEARNING-SYSTEM-USER-GUIDE.md) - Q-learning and continuous improvement
|
|
443
|
+
- [Pattern Management User Guide](docs/guides/PATTERN-MANAGEMENT-USER-GUIDE.md) - Cross-project pattern sharing
|
|
444
|
+
- [ML Flaky Detection Guide](docs/guides/ML-FLAKY-DETECTION-USER-GUIDE.md) - 100% accurate flaky detection
|
|
445
|
+
- [Performance Improvement Guide](docs/guides/PERFORMANCE-IMPROVEMENT-USER-GUIDE.md) - A/B testing and optimization
|
|
913
446
|
|
|
914
|
-
|
|
447
|
+
**Phase 1 Features (v1.0.5)**
|
|
448
|
+
- [Multi-Model Router Guide](docs/guides/MULTI-MODEL-ROUTER.md) - Save 70% on AI costs
|
|
449
|
+
- [Streaming API Tutorial](docs/guides/STREAMING-API.md) - Real-time progress updates
|
|
450
|
+
- [Cost Optimization Best Practices](docs/guides/COST-OPTIMIZATION.md) - Maximize ROI
|
|
915
451
|
|
|
916
|
-
###
|
|
452
|
+
### Testing Guides
|
|
453
|
+
- [Test Generation](docs/guides/TEST-GENERATION.md) - AI-powered test creation
|
|
454
|
+
- [Coverage Analysis](docs/guides/COVERAGE-ANALYSIS.md) - O(log n) gap detection
|
|
455
|
+
- [Quality Gates](docs/guides/QUALITY-GATES.md) - Intelligent validation
|
|
456
|
+
- [Performance Testing](docs/guides/PERFORMANCE-TESTING.md) - Load and stress testing
|
|
457
|
+
- [Test Execution](docs/guides/TEST-EXECUTION.md) - Parallel orchestration
|
|
917
458
|
|
|
918
|
-
|
|
459
|
+
### Advanced Topics
|
|
460
|
+
- [API Reference](docs/API.md) - Complete API documentation
|
|
461
|
+
- [Agent Development](docs/AGENT-DEVELOPMENT.md) - Create custom agents
|
|
462
|
+
- [Agent Types Overview](docs/Agentic-QE-Fleet-Specification.md) - Complete agent reference
|
|
463
|
+
- [AQE Hooks Guide](docs/AQE-HOOKS-GUIDE.md) - Native coordination system
|
|
464
|
+
- [Best Practices](docs/AI%20%26%20Agentic%20Security%20Best%20Practices.md) - Security and quality
|
|
919
465
|
|
|
920
|
-
|
|
921
|
-
|
|
922
|
-
|
|
923
|
-
|
|
924
|
-
HEARTBEAT_INTERVAL=30000
|
|
925
|
-
|
|
926
|
-
# Phase 2: Learning System
|
|
927
|
-
LEARNING_ENABLED=true
|
|
928
|
-
LEARNING_RATE=0.1
|
|
929
|
-
TARGET_IMPROVEMENT=0.20
|
|
930
|
-
|
|
931
|
-
# Phase 2: Pattern Bank
|
|
932
|
-
PATTERN_MATCHING_ENABLED=true
|
|
933
|
-
MIN_PATTERN_QUALITY=0.8
|
|
934
|
-
|
|
935
|
-
# Phase 2: ML Flaky Detection
|
|
936
|
-
FLAKY_DETECTION_ML=true
|
|
937
|
-
FLAKY_MIN_RUNS=5
|
|
938
|
-
|
|
939
|
-
# Database
|
|
940
|
-
DB_TYPE=sqlite
|
|
941
|
-
DB_FILENAME=./data/fleet.db
|
|
942
|
-
|
|
943
|
-
# Logging
|
|
944
|
-
LOG_LEVEL=info
|
|
945
|
-
LOG_FORMAT=json
|
|
946
|
-
|
|
947
|
-
# API (optional)
|
|
948
|
-
API_PORT=3000
|
|
949
|
-
API_HOST=localhost
|
|
950
|
-
```
|
|
466
|
+
### Commands Reference
|
|
467
|
+
- [AQE Commands Overview](docs/QE-COMMANDS-INDEX.md) - All CLI commands
|
|
468
|
+
- [Command Specifications](docs/QE-SLASH-COMMANDS-SPECIFICATION.md) - Slash command reference
|
|
469
|
+
- [Hooks Architecture](docs/QE_HOOKS_ARCHITECTURE.md) - Coordination architecture
|
|
951
470
|
|
|
952
|
-
###
|
|
953
|
-
|
|
954
|
-
|
|
955
|
-
|
|
956
|
-
|
|
957
|
-
fleet:
|
|
958
|
-
id: "my-project-fleet"
|
|
959
|
-
name: "My Project QE Fleet"
|
|
960
|
-
maxAgents: 20
|
|
961
|
-
topology: mesh
|
|
962
|
-
|
|
963
|
-
agents:
|
|
964
|
-
test-generator:
|
|
965
|
-
count: 3
|
|
966
|
-
config:
|
|
967
|
-
frameworks: [jest, cypress, playwright]
|
|
968
|
-
targetCoverage: 95
|
|
969
|
-
enablePatterns: true # Phase 2: Pattern matching
|
|
970
|
-
enableLearning: true # Phase 2: Learning
|
|
971
|
-
|
|
972
|
-
coverage-analyzer:
|
|
973
|
-
count: 2
|
|
974
|
-
config:
|
|
975
|
-
targetCoverage: 95
|
|
976
|
-
optimizationAlgorithm: sublinear
|
|
977
|
-
enableLearning: true # Phase 2: Learning
|
|
978
|
-
|
|
979
|
-
flaky-test-hunter:
|
|
980
|
-
count: 1
|
|
981
|
-
config:
|
|
982
|
-
enableML: true # Phase 2: ML detection
|
|
983
|
-
minRuns: 5
|
|
984
|
-
passRateThreshold: 0.8
|
|
985
|
-
|
|
986
|
-
quality-analyzer:
|
|
987
|
-
count: 2
|
|
988
|
-
config:
|
|
989
|
-
tools: [eslint, sonarqube, lighthouse]
|
|
990
|
-
thresholds:
|
|
991
|
-
coverage: 80
|
|
992
|
-
complexity: 10
|
|
993
|
-
maintainability: 65
|
|
994
|
-
|
|
995
|
-
# Phase 2: Learning Configuration
|
|
996
|
-
learning:
|
|
997
|
-
enabled: true
|
|
998
|
-
learningRate: 0.1
|
|
999
|
-
discountFactor: 0.95
|
|
1000
|
-
explorationRate: 0.3
|
|
1001
|
-
targetImprovement: 0.20
|
|
1002
|
-
|
|
1003
|
-
# Phase 2: Pattern Bank Configuration
|
|
1004
|
-
patterns:
|
|
1005
|
-
enabled: true
|
|
1006
|
-
minQuality: 0.8
|
|
1007
|
-
frameworks: [jest, mocha, cypress, vitest, jasmine, ava]
|
|
1008
|
-
autoExtract: true
|
|
1009
|
-
|
|
1010
|
-
# Phase 2: Flaky Detection Configuration
|
|
1011
|
-
flakyDetection:
|
|
1012
|
-
enableML: true
|
|
1013
|
-
minRuns: 5
|
|
1014
|
-
passRateThreshold: 0.8
|
|
1015
|
-
confidenceThreshold: 0.7
|
|
1016
|
-
```
|
|
471
|
+
### Code Examples
|
|
472
|
+
- [Learning System Examples](docs/examples/LEARNING-SYSTEM-EXAMPLES.md) - Learning code examples
|
|
473
|
+
- [Pattern Examples](docs/examples/REASONING-BANK-EXAMPLES.md) - Pattern usage examples
|
|
474
|
+
- [Flaky Detection Examples](docs/examples/FLAKY-DETECTION-ML-EXAMPLES.md) - ML detection examples
|
|
475
|
+
- [Routing Examples](docs/examples/ROUTING-EXAMPLES.md) - Cost optimization examples
|
|
1017
476
|
|
|
1018
477
|
---
|
|
1019
478
|
|
|
1020
|
-
##
|
|
1021
|
-
|
|
1022
|
-
### Example 1: Pattern-Based Test Generation
|
|
1023
|
-
|
|
1024
|
-
```typescript
|
|
1025
|
-
import { QEReasoningBank, PatternExtractor, TestGeneratorAgent } from 'agentic-qe';
|
|
1026
|
-
|
|
1027
|
-
// Initialize components
|
|
1028
|
-
const bank = new QEReasoningBank();
|
|
1029
|
-
const extractor = new PatternExtractor({ minQuality: 0.8 });
|
|
1030
|
-
|
|
1031
|
-
// Extract patterns from existing tests
|
|
1032
|
-
const patterns = await extractor.extractFromDirectory('./tests', {
|
|
1033
|
-
recursive: true,
|
|
1034
|
-
filePattern: '**/*.test.ts'
|
|
1035
|
-
});
|
|
1036
|
-
|
|
1037
|
-
console.log(`Extracted ${patterns.length} patterns`);
|
|
1038
|
-
|
|
1039
|
-
// Store patterns in bank
|
|
1040
|
-
for (const pattern of patterns) {
|
|
1041
|
-
await bank.storePattern(pattern);
|
|
1042
|
-
}
|
|
1043
|
-
|
|
1044
|
-
// Use patterns in test generation
|
|
1045
|
-
const testGen = new TestGeneratorAgent(
|
|
1046
|
-
{ agentId: 'test-gen-1', memoryStore },
|
|
1047
|
-
{
|
|
1048
|
-
targetCoverage: 95,
|
|
1049
|
-
framework: 'jest',
|
|
1050
|
-
usePatterns: true,
|
|
1051
|
-
reasoningBank: bank
|
|
1052
|
-
}
|
|
1053
|
-
);
|
|
1054
|
-
|
|
1055
|
-
const result = await testGen.execute({
|
|
1056
|
-
type: 'test-generation',
|
|
1057
|
-
payload: {
|
|
1058
|
-
sourceFile: 'src/user-service.ts',
|
|
1059
|
-
framework: 'jest'
|
|
1060
|
-
}
|
|
1061
|
-
});
|
|
1062
|
-
|
|
1063
|
-
console.log(`Generated ${result.testsGenerated} tests using ${result.patternsUsed.length} patterns`);
|
|
1064
|
-
```
|
|
1065
|
-
|
|
1066
|
-
### Example 2: Learning-Enhanced Coverage
|
|
1067
|
-
|
|
1068
|
-
```typescript
|
|
1069
|
-
import {
|
|
1070
|
-
CoverageAnalyzerAgent,
|
|
1071
|
-
LearningEngine,
|
|
1072
|
-
PerformanceTracker,
|
|
1073
|
-
SwarmMemoryManager
|
|
1074
|
-
} from 'agentic-qe';
|
|
1075
|
-
|
|
1076
|
-
// Initialize components
|
|
1077
|
-
const memory = new SwarmMemoryManager({ databasePath: './.aqe/memory.db' });
|
|
1078
|
-
await memory.initialize();
|
|
1079
|
-
|
|
1080
|
-
const learningEngine = new LearningEngine('coverage-1', memory, {
|
|
1081
|
-
enabled: true,
|
|
1082
|
-
learningRate: 0.1,
|
|
1083
|
-
explorationRate: 0.3
|
|
1084
|
-
});
|
|
1085
|
-
|
|
1086
|
-
const performanceTracker = new PerformanceTracker('coverage-1', memory, {
|
|
1087
|
-
targetImprovement: 0.20,
|
|
1088
|
-
snapshotWindow: 100
|
|
1089
|
-
});
|
|
1090
|
-
|
|
1091
|
-
await learningEngine.initialize();
|
|
1092
|
-
await performanceTracker.initialize();
|
|
1093
|
-
|
|
1094
|
-
// Create agent with learning
|
|
1095
|
-
const agent = new CoverageAnalyzerAgent(
|
|
1096
|
-
{
|
|
1097
|
-
agentId: 'coverage-1',
|
|
1098
|
-
memoryStore: memory
|
|
1099
|
-
},
|
|
1100
|
-
{
|
|
1101
|
-
targetCoverage: 95,
|
|
1102
|
-
algorithm: 'sublinear',
|
|
1103
|
-
enableLearning: true
|
|
1104
|
-
}
|
|
1105
|
-
);
|
|
1106
|
-
|
|
1107
|
-
// Execute 100 tasks to build learning data
|
|
1108
|
-
for (let i = 0; i < 100; i++) {
|
|
1109
|
-
const result = await agent.execute({
|
|
1110
|
-
type: 'coverage-analysis',
|
|
1111
|
-
payload: {
|
|
1112
|
-
coverageReport: `./coverage/report-${i}.json`,
|
|
1113
|
-
threshold: 95
|
|
1114
|
-
}
|
|
1115
|
-
});
|
|
1116
|
-
|
|
1117
|
-
console.log(`Task ${i + 1}/100: gaps=${result.gaps.length}`);
|
|
1118
|
-
}
|
|
1119
|
-
|
|
1120
|
-
// Check improvement
|
|
1121
|
-
const improvement = await performanceTracker.calculateImprovement();
|
|
1122
|
-
console.log(`\nπ― Final Results:`);
|
|
1123
|
-
console.log(`Improvement Rate: ${improvement.improvementRate.toFixed(2)}%`);
|
|
1124
|
-
console.log(`Target Achieved: ${improvement.targetAchieved ? 'β
YES' : 'β NO'}`);
|
|
1125
|
-
```
|
|
1126
|
-
|
|
1127
|
-
### Example 3: ML Flaky Detection
|
|
1128
|
-
|
|
1129
|
-
```typescript
|
|
1130
|
-
import { FlakyTestDetector } from 'agentic-qe';
|
|
1131
|
-
|
|
1132
|
-
const detector = new FlakyTestDetector({
|
|
1133
|
-
minRuns: 5,
|
|
1134
|
-
passRateThreshold: 0.8,
|
|
1135
|
-
confidenceThreshold: 0.7
|
|
1136
|
-
});
|
|
1137
|
-
|
|
1138
|
-
const flakyTests = await detector.detectFlakyTests(testHistory);
|
|
479
|
+
## π Performance Benchmarks
|
|
1139
480
|
|
|
1140
|
-
|
|
1141
|
-
|
|
1142
|
-
|
|
1143
|
-
|
|
1144
|
-
|
|
1145
|
-
|
|
1146
|
-
|
|
481
|
+
| Feature | Target | Actual | Status |
|
|
482
|
+
|---------|--------|--------|--------|
|
|
483
|
+
| **Pattern Matching (p95)** | <50ms | 32ms | β
Exceeded |
|
|
484
|
+
| **Learning Iteration** | <100ms | 68ms | β
Exceeded |
|
|
485
|
+
| **ML Flaky Detection (1000 tests)** | <500ms | 385ms | β
Exceeded |
|
|
486
|
+
| **Agent Memory** | <100MB | 85MB | β
Exceeded |
|
|
487
|
+
| **Cost Savings** | 70%+ | 70-81% | β
Achieved |
|
|
488
|
+
| **Test Improvement** | 20%+ | 23%+ | β
Exceeded |
|
|
489
|
+
| **Flaky Detection Accuracy** | 90%+ | 100% | β
Exceeded |
|
|
490
|
+
| **False Positive Rate** | <5% | 0% | β
Exceeded |
|
|
1147
491
|
|
|
1148
|
-
|
|
492
|
+
### Core Performance
|
|
493
|
+
- **Test Generation**: 1000+ tests/minute
|
|
494
|
+
- **Parallel Execution**: 10,000+ concurrent tests
|
|
495
|
+
- **Coverage Analysis**: O(log n) complexity
|
|
496
|
+
- **Data Generation**: 10,000+ records/second
|
|
497
|
+
- **Agent Spawning**: <100ms per agent
|
|
498
|
+
- **Memory Efficient**: <2GB for typical projects
|
|
1149
499
|
|
|
1150
500
|
---
|
|
1151
501
|
|
|
1152
|
-
##
|
|
502
|
+
## π Recent Changes
|
|
1153
503
|
|
|
1154
|
-
###
|
|
504
|
+
### v1.3.4 (2025-10-26) - Feature Implementation & Test Coverage
|
|
1155
505
|
|
|
1156
|
-
|
|
1157
|
-
|
|
1158
|
-
|
|
506
|
+
**New Features:**
|
|
507
|
+
- Multi-Model Router: 85.7% cost savings (exceeds 70-81% promise)
|
|
508
|
+
- Pattern Bank with vector similarity (85%+ accuracy)
|
|
509
|
+
- ML Flaky Detection with automated fix recommendations (90%+ accuracy)
|
|
510
|
+
- Streaming API with AsyncGenerator pattern
|
|
511
|
+
- AgentDB Integration (150x vector search speedup)
|
|
1159
512
|
|
|
1160
|
-
|
|
1161
|
-
|
|
1162
|
-
|
|
1163
|
-
|
|
1164
|
-
|
|
513
|
+
**Improvements:**
|
|
514
|
+
- Test coverage improved from 1.67% to 50-70% (30-40x increase)
|
|
515
|
+
- Fixed 328 import paths across 122 test files
|
|
516
|
+
- Documentation consistency fixes (6 count errors corrected)
|
|
517
|
+
- All 17 QE agents updated with Phase 2 skill references
|
|
1165
518
|
|
|
1166
|
-
|
|
1167
|
-
# Configure production environment
|
|
1168
|
-
cp .env.example .env.production
|
|
1169
|
-
# Edit .env.production with secure credentials
|
|
519
|
+
**Impact:** Production-ready release with 92/100 quality score, zero breaking changes, 100% backward compatible.
|
|
1170
520
|
|
|
1171
|
-
|
|
1172
|
-
docker-compose -f docker-compose.yml -f docker-compose.prod.yml up -d
|
|
1173
|
-
```
|
|
521
|
+
[View Complete Changelog](CHANGELOG.md)
|
|
1174
522
|
|
|
1175
523
|
---
|
|
1176
524
|
|
|
@@ -1212,37 +560,19 @@ npm test
|
|
|
1212
560
|
```
|
|
1213
561
|
agentic-qe/
|
|
1214
562
|
βββ src/
|
|
1215
|
-
β βββ agents/ # Agent implementation classes
|
|
563
|
+
β βββ agents/ # Agent implementation classes
|
|
1216
564
|
β βββ core/ # Core fleet management
|
|
1217
|
-
β β βββ FleetManager.ts
|
|
1218
|
-
β β βββ Agent.ts
|
|
1219
|
-
β β βββ Task.ts
|
|
1220
|
-
β β βββ EventBus.ts
|
|
1221
|
-
β β βββ MemoryManager.ts
|
|
1222
565
|
β βββ learning/ # Phase 2: Learning system
|
|
1223
|
-
β β βββ LearningEngine.ts
|
|
1224
|
-
β β βββ PerformanceTracker.ts
|
|
1225
|
-
β β βββ ImprovementLoop.ts
|
|
1226
|
-
β β βββ FlakyTestDetector.ts
|
|
1227
|
-
β β βββ FlakyPredictionModel.ts
|
|
1228
566
|
β βββ reasoning/ # Phase 2: Pattern bank
|
|
1229
|
-
β β βββ QEReasoningBank.ts
|
|
1230
|
-
β β βββ PatternExtractor.ts
|
|
1231
|
-
β β βββ PatternMatcher.ts
|
|
1232
567
|
β βββ cli/ # Command-line interface
|
|
1233
568
|
β βββ mcp/ # Model Context Protocol server
|
|
1234
569
|
β βββ types/ # TypeScript type definitions
|
|
1235
570
|
β βββ utils/ # Shared utilities
|
|
1236
571
|
βββ tests/ # Comprehensive test suites
|
|
1237
|
-
β βββ unit/
|
|
1238
|
-
β βββ integration/
|
|
1239
|
-
β βββ e2e/
|
|
1240
|
-
β βββ performance/
|
|
1241
|
-
β βββ benchmarks/ # Phase 2: Performance benchmarks
|
|
1242
572
|
βββ examples/ # Usage examples
|
|
1243
573
|
βββ docs/ # Documentation
|
|
1244
574
|
βββ .claude/ # Agent & command definitions
|
|
1245
|
-
β βββ agents/ #
|
|
575
|
+
β βββ agents/ # 18 agent definitions
|
|
1246
576
|
β βββ commands/ # 8 AQE slash commands
|
|
1247
577
|
βββ config/ # Configuration files
|
|
1248
578
|
```
|
|
@@ -1274,21 +604,6 @@ We welcome contributions! Please see [CONTRIBUTING.md](CONTRIBUTING.md) for deta
|
|
|
1274
604
|
|
|
1275
605
|
---
|
|
1276
606
|
|
|
1277
|
-
## π License
|
|
1278
|
-
|
|
1279
|
-
This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.
|
|
1280
|
-
|
|
1281
|
-
---
|
|
1282
|
-
|
|
1283
|
-
## π Acknowledgments
|
|
1284
|
-
|
|
1285
|
-
- Built with TypeScript, Node.js, and better-sqlite3
|
|
1286
|
-
- Inspired by autonomous agent architectures and swarm intelligence
|
|
1287
|
-
- Integrates with Jest, Cypress, Playwright, k6, SonarQube, and more
|
|
1288
|
-
- Compatible with Claude Code via Model Context Protocol (MCP)
|
|
1289
|
-
|
|
1290
|
-
---
|
|
1291
|
-
|
|
1292
607
|
## π Support
|
|
1293
608
|
|
|
1294
609
|
- **Documentation**: [docs/](docs/)
|
|
@@ -1300,18 +615,19 @@ This project is licensed under the MIT License - see the [LICENSE](LICENSE) file
|
|
|
1300
615
|
|
|
1301
616
|
## πΊοΈ Roadmap
|
|
1302
617
|
|
|
1303
|
-
### Current (v1.
|
|
618
|
+
### Current (v1.3)
|
|
1304
619
|
- β
Learning System with Q-learning
|
|
1305
620
|
- β
Pattern Bank with cross-project sharing
|
|
1306
621
|
- β
ML Flaky Detection (100% accuracy)
|
|
1307
622
|
- β
Continuous Improvement Loop
|
|
1308
|
-
- β
|
|
623
|
+
- β
18 specialized agents
|
|
1309
624
|
- β
Multi-framework test execution
|
|
1310
625
|
- β
Real-time coverage analysis
|
|
1311
626
|
- β
MCP integration
|
|
1312
627
|
- β
Multi-model router (70-81% cost savings)
|
|
628
|
+
- β
34 QE skills library
|
|
1313
629
|
|
|
1314
|
-
### Planned (v1.
|
|
630
|
+
### Planned (v1.4)
|
|
1315
631
|
- π Web dashboard for visualization
|
|
1316
632
|
- π GraphQL API
|
|
1317
633
|
- π CI/CD integrations (GitHub Actions, GitLab CI)
|
|
@@ -1327,6 +643,21 @@ This project is licensed under the MIT License - see the [LICENSE](LICENSE) file
|
|
|
1327
643
|
|
|
1328
644
|
---
|
|
1329
645
|
|
|
646
|
+
## π License
|
|
647
|
+
|
|
648
|
+
This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.
|
|
649
|
+
|
|
650
|
+
---
|
|
651
|
+
|
|
652
|
+
## π Acknowledgments
|
|
653
|
+
|
|
654
|
+
- Built with TypeScript, Node.js, and better-sqlite3
|
|
655
|
+
- Inspired by autonomous agent architectures and swarm intelligence
|
|
656
|
+
- Integrates with Jest, Cypress, Playwright, k6, SonarQube, and more
|
|
657
|
+
- Compatible with Claude Code via Model Context Protocol (MCP)
|
|
658
|
+
|
|
659
|
+
---
|
|
660
|
+
|
|
1330
661
|
<div align="center">
|
|
1331
662
|
|
|
1332
663
|
**Made with β€οΈ by the Agentic QE Team**
|