agentic-qe 1.4.0 → 1.4.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +234 -0
- package/README.md +18 -85
- package/dist/agents/index.d.ts.map +1 -1
- package/dist/agents/index.js +14 -15
- package/dist/agents/index.js.map +1 -1
- package/dist/cli/commands/config/set.d.ts.map +1 -1
- package/dist/cli/commands/config/set.js +10 -2
- package/dist/cli/commands/config/set.js.map +1 -1
- package/dist/cli/commands/init.d.ts.map +1 -1
- package/dist/cli/commands/init.js +26 -3
- package/dist/cli/commands/init.js.map +1 -1
- package/dist/core/Agent.d.ts +1 -1
- package/dist/core/Agent.d.ts.map +1 -1
- package/dist/core/Agent.js +2 -2
- package/dist/core/Agent.js.map +1 -1
- package/dist/core/MemoryManager.d.ts.map +1 -1
- package/dist/core/MemoryManager.js +4 -1
- package/dist/core/MemoryManager.js.map +1 -1
- package/dist/core/hooks/RollbackManager.js +2 -2
- package/dist/core/hooks/RollbackManager.js.map +1 -1
- package/dist/learning/LearningEngine.d.ts +21 -4
- package/dist/learning/LearningEngine.d.ts.map +1 -1
- package/dist/learning/LearningEngine.js +118 -131
- package/dist/learning/LearningEngine.js.map +1 -1
- package/dist/learning/LearningPersistenceAdapter.d.ts +84 -0
- package/dist/learning/LearningPersistenceAdapter.d.ts.map +1 -0
- package/dist/learning/LearningPersistenceAdapter.js +202 -0
- package/dist/learning/LearningPersistenceAdapter.js.map +1 -0
- package/dist/learning/index.d.ts +1 -0
- package/dist/learning/index.d.ts.map +1 -1
- package/dist/learning/index.js +1 -0
- package/dist/learning/index.js.map +1 -1
- package/dist/mcp/handlers/analysis/coverage-analyze-sublinear-handler.d.ts.map +1 -1
- package/dist/mcp/handlers/analysis/coverage-analyze-sublinear-handler.js +3 -7
- package/dist/mcp/handlers/analysis/coverage-analyze-sublinear-handler.js.map +1 -1
- package/dist/mcp/handlers/analysis/coverage-gaps-detect-handler.d.ts.map +1 -1
- package/dist/mcp/handlers/analysis/coverage-gaps-detect-handler.js +3 -7
- package/dist/mcp/handlers/analysis/coverage-gaps-detect-handler.js.map +1 -1
- package/dist/mcp/handlers/analysis/performance-benchmark-run-handler.d.ts.map +1 -1
- package/dist/mcp/handlers/analysis/performance-benchmark-run-handler.js +3 -7
- package/dist/mcp/handlers/analysis/performance-benchmark-run-handler.js.map +1 -1
- package/dist/mcp/handlers/analysis/performance-monitor-realtime-handler.d.ts.map +1 -1
- package/dist/mcp/handlers/analysis/performance-monitor-realtime-handler.js +3 -7
- package/dist/mcp/handlers/analysis/performance-monitor-realtime-handler.js.map +1 -1
- package/dist/mcp/handlers/analysis/security-scan-comprehensive-handler.d.ts.map +1 -1
- package/dist/mcp/handlers/analysis/security-scan-comprehensive-handler.js +3 -7
- package/dist/mcp/handlers/analysis/security-scan-comprehensive-handler.js.map +1 -1
- package/dist/mcp/handlers/base-handler.d.ts +30 -0
- package/dist/mcp/handlers/base-handler.d.ts.map +1 -1
- package/dist/mcp/handlers/base-handler.js +42 -0
- package/dist/mcp/handlers/base-handler.js.map +1 -1
- package/dist/mcp/handlers/coordination/event-emit.d.ts.map +1 -1
- package/dist/mcp/handlers/coordination/event-emit.js +4 -10
- package/dist/mcp/handlers/coordination/event-emit.js.map +1 -1
- package/dist/mcp/handlers/coordination/event-subscribe.d.ts.map +1 -1
- package/dist/mcp/handlers/coordination/event-subscribe.js +3 -9
- package/dist/mcp/handlers/coordination/event-subscribe.js.map +1 -1
- package/dist/mcp/handlers/coordination/task-status.d.ts.map +1 -1
- package/dist/mcp/handlers/coordination/task-status.js +4 -10
- package/dist/mcp/handlers/coordination/task-status.js.map +1 -1
- package/dist/mcp/handlers/coordination/workflow-checkpoint.d.ts.map +1 -1
- package/dist/mcp/handlers/coordination/workflow-checkpoint.js +4 -10
- package/dist/mcp/handlers/coordination/workflow-checkpoint.js.map +1 -1
- package/dist/mcp/handlers/coordination/workflow-create.d.ts.map +1 -1
- package/dist/mcp/handlers/coordination/workflow-create.js +4 -10
- package/dist/mcp/handlers/coordination/workflow-create.js.map +1 -1
- package/dist/mcp/handlers/coordination/workflow-execute.d.ts.map +1 -1
- package/dist/mcp/handlers/coordination/workflow-execute.js +4 -10
- package/dist/mcp/handlers/coordination/workflow-execute.js.map +1 -1
- package/dist/mcp/handlers/coordination/workflow-resume.d.ts.map +1 -1
- package/dist/mcp/handlers/coordination/workflow-resume.js +4 -10
- package/dist/mcp/handlers/coordination/workflow-resume.js.map +1 -1
- package/dist/mcp/handlers/memory/memory-backup.d.ts.map +1 -1
- package/dist/mcp/handlers/memory/memory-backup.js +3 -7
- package/dist/mcp/handlers/memory/memory-backup.js.map +1 -1
- package/dist/mcp/handlers/memory/memory-query.d.ts.map +1 -1
- package/dist/mcp/handlers/memory/memory-query.js +8 -8
- package/dist/mcp/handlers/memory/memory-query.js.map +1 -1
- package/dist/mcp/handlers/memory/memory-retrieve.d.ts.map +1 -1
- package/dist/mcp/handlers/memory/memory-retrieve.js +3 -7
- package/dist/mcp/handlers/memory/memory-retrieve.js.map +1 -1
- package/dist/mcp/handlers/memory/memory-share.d.ts.map +1 -1
- package/dist/mcp/handlers/memory/memory-share.js +3 -7
- package/dist/mcp/handlers/memory/memory-share.js.map +1 -1
- package/dist/mcp/handlers/memory/memory-store.d.ts.map +1 -1
- package/dist/mcp/handlers/memory/memory-store.js +3 -7
- package/dist/mcp/handlers/memory/memory-store.js.map +1 -1
- package/dist/mcp/handlers/prediction/deployment-readiness-check.d.ts.map +1 -1
- package/dist/mcp/handlers/prediction/deployment-readiness-check.js +4 -8
- package/dist/mcp/handlers/prediction/deployment-readiness-check.js.map +1 -1
- package/dist/mcp/handlers/prediction/flaky-test-detect.d.ts.map +1 -1
- package/dist/mcp/handlers/prediction/flaky-test-detect.js +4 -8
- package/dist/mcp/handlers/prediction/flaky-test-detect.js.map +1 -1
- package/dist/mcp/handlers/prediction/predict-defects-ai.d.ts.map +1 -1
- package/dist/mcp/handlers/prediction/predict-defects-ai.js +4 -8
- package/dist/mcp/handlers/prediction/predict-defects-ai.js.map +1 -1
- package/dist/mcp/handlers/prediction/regression-risk-analyze.d.ts.map +1 -1
- package/dist/mcp/handlers/prediction/regression-risk-analyze.js +4 -8
- package/dist/mcp/handlers/prediction/regression-risk-analyze.js.map +1 -1
- package/dist/mcp/handlers/prediction/visual-test-regression.d.ts.map +1 -1
- package/dist/mcp/handlers/prediction/visual-test-regression.js +4 -8
- package/dist/mcp/handlers/prediction/visual-test-regression.js.map +1 -1
- package/dist/mcp/handlers/quality/quality-decision-make.d.ts.map +1 -1
- package/dist/mcp/handlers/quality/quality-decision-make.js +9 -19
- package/dist/mcp/handlers/quality/quality-decision-make.js.map +1 -1
- package/dist/mcp/handlers/quality/quality-gate-execute.d.ts.map +1 -1
- package/dist/mcp/handlers/quality/quality-gate-execute.js +10 -22
- package/dist/mcp/handlers/quality/quality-gate-execute.js.map +1 -1
- package/dist/mcp/handlers/quality/quality-policy-check.d.ts.map +1 -1
- package/dist/mcp/handlers/quality/quality-policy-check.js +5 -15
- package/dist/mcp/handlers/quality/quality-policy-check.js.map +1 -1
- package/dist/mcp/handlers/quality/quality-risk-assess.d.ts.map +1 -1
- package/dist/mcp/handlers/quality/quality-risk-assess.js +5 -15
- package/dist/mcp/handlers/quality/quality-risk-assess.js.map +1 -1
- package/dist/mcp/handlers/quality/quality-validate-metrics.d.ts.map +1 -1
- package/dist/mcp/handlers/quality/quality-validate-metrics.js +4 -14
- package/dist/mcp/handlers/quality/quality-validate-metrics.js.map +1 -1
- package/dist/mcp/handlers/test/test-coverage-detailed.d.ts.map +1 -1
- package/dist/mcp/handlers/test/test-coverage-detailed.js +8 -12
- package/dist/mcp/handlers/test/test-coverage-detailed.js.map +1 -1
- package/dist/mcp/handlers/test/test-execute-parallel.d.ts.map +1 -1
- package/dist/mcp/handlers/test/test-execute-parallel.js +8 -12
- package/dist/mcp/handlers/test/test-execute-parallel.js.map +1 -1
- package/dist/mcp/handlers/test/test-generate-enhanced.d.ts.map +1 -1
- package/dist/mcp/handlers/test/test-generate-enhanced.js +4 -8
- package/dist/mcp/handlers/test/test-generate-enhanced.js.map +1 -1
- package/dist/mcp/handlers/test/test-optimize-sublinear.d.ts.map +1 -1
- package/dist/mcp/handlers/test/test-optimize-sublinear.js +8 -12
- package/dist/mcp/handlers/test/test-optimize-sublinear.js.map +1 -1
- package/dist/mcp/handlers/test/test-report-comprehensive.d.ts.map +1 -1
- package/dist/mcp/handlers/test/test-report-comprehensive.js +4 -8
- package/dist/mcp/handlers/test/test-report-comprehensive.js.map +1 -1
- package/dist/mcp/services/AgentRegistry.d.ts.map +1 -1
- package/dist/mcp/services/AgentRegistry.js +16 -4
- package/dist/mcp/services/AgentRegistry.js.map +1 -1
- package/dist/utils/Database.d.ts.map +1 -1
- package/dist/utils/Database.js +17 -4
- package/dist/utils/Database.js.map +1 -1
- package/dist/utils/__mocks__/Logger.d.ts.map +1 -1
- package/dist/utils/__mocks__/Logger.js +2 -0
- package/dist/utils/__mocks__/Logger.js.map +1 -1
- package/package.json +11 -10
- package/.claude/aqe-fleet.json +0 -47
- package/.claude/helpers/checkpoint-manager.sh +0 -251
- package/.claude/helpers/github-safe.js +0 -106
- package/.claude/helpers/github-setup.sh +0 -28
- package/.claude/helpers/quick-start.sh +0 -19
- package/.claude/helpers/setup-mcp.sh +0 -18
- package/.claude/helpers/standard-checkpoint-hooks.sh +0 -179
- package/.claude/settings.json +0 -153
- package/.claude/settings.local.json +0 -10
- package/.claude/statusline-command.sh +0 -176
- package/README.md.backup-20251026 +0 -1366
- package/dist/migrations/001_add_learning_tables.d.ts +0 -63
- package/dist/migrations/001_add_learning_tables.d.ts.map +0 -1
- package/dist/migrations/001_add_learning_tables.js +0 -287
- package/dist/migrations/001_add_learning_tables.js.map +0 -1
|
@@ -1,1366 +0,0 @@
|
|
|
1
|
-
# Agentic Quality Engineering Fleet
|
|
2
|
-
|
|
3
|
-
<div align="center">
|
|
4
|
-
|
|
5
|
-
[](https://www.npmjs.com/package/agentic-qe)
|
|
6
|
-
[](https://opensource.org/licenses/MIT)
|
|
7
|
-
[](https://www.typescriptlang.org/)
|
|
8
|
-
[](https://nodejs.org/)
|
|
9
|
-
|
|
10
|
-
**Version 1.3.3** - Critical Database & MCP Server Fixes
|
|
11
|
-
|
|
12
|
-
> Enterprise-grade test automation with AI learning, comprehensive skills library (34 QE skills), and optional 70-81% cost savings through intelligent model routing (opt-in feature).
|
|
13
|
-
|
|
14
|
-
🧠 **20% Continuous Improvement** | 📚 **34 World-Class QE Skills** | 🎯 **100% Flaky Test Detection** | 💰 **70-81% Cost Savings (opt-in)** | 🔒 **100% CodeQL Resolution** | 🔧 **61 MCP Tools**
|
|
15
|
-
|
|
16
|
-
[Quick Start](#quick-start) • [Documentation](docs/) • [Contributing](CONTRIBUTING.md) • [Examples](examples/)
|
|
17
|
-
|
|
18
|
-
</div>
|
|
19
|
-
|
|
20
|
-
---
|
|
21
|
-
|
|
22
|
-
## 🎉 What's New in v1.3.3
|
|
23
|
-
|
|
24
|
-
**🐛 Critical Bug Fixes**: Fixed missing `memory_store` database table and MCP server startup issues. Fleet initialization now works correctly, and Claude Code MCP integration is reliable with all 52 tools available.
|
|
25
|
-
|
|
26
|
-
### Key Fixes
|
|
27
|
-
|
|
28
|
-
- **Database Schema**: Added missing `memory_store` table for persistent agent memory
|
|
29
|
-
- **MCP Server**: Created standalone `aqe-mcp` binary + fixed module resolution
|
|
30
|
-
- **Impact**: Smooth fleet initialization and reliable Claude Code integration
|
|
31
|
-
|
|
32
|
-
### Migration from v1.3.2
|
|
33
|
-
|
|
34
|
-
```bash
|
|
35
|
-
npm install -g agentic-qe@latest
|
|
36
|
-
rm -rf ./data/*.db ./.agentic-qe/*.db
|
|
37
|
-
aqe init
|
|
38
|
-
```
|
|
39
|
-
|
|
40
|
-
Update Claude Code MCP config:
|
|
41
|
-
```json
|
|
42
|
-
{
|
|
43
|
-
"mcpServers": {
|
|
44
|
-
"agentic-qe": {
|
|
45
|
-
"command": "aqe-mcp",
|
|
46
|
-
"args": []
|
|
47
|
-
}
|
|
48
|
-
}
|
|
49
|
-
}
|
|
50
|
-
```
|
|
51
|
-
|
|
52
|
-
### Previous Release (v1.3.2)
|
|
53
|
-
|
|
54
|
-
**🔐 Security Release**: Fixed all 4 open CodeQL security alerts - achieving **100% alert resolution (26/26 fixed)**. Critical fixes include elimination of cryptographic randomness bias, enhanced prototype pollution prevention, and comprehensive security test coverage.
|
|
55
|
-
|
|
56
|
-
### Previous Release (v1.3.1)
|
|
57
|
-
|
|
58
|
-
**Critical Bug Fix**: Fixed `aqe init` command that was using hardcoded versions (1.0.5, 1.1.0) instead of reading from `package.json`. All generated config files now correctly sync with the package version.
|
|
59
|
-
|
|
60
|
-
### Previous Release (v1.3.0)
|
|
61
|
-
|
|
62
|
-
### Security Hardening + Skills Expansion (Oct 23-24, 2025)
|
|
63
|
-
**2 days of intensive development** | **Security hardening + 17 new skills** | **11,500+ lines of expert content**
|
|
64
|
-
|
|
65
|
-
#### 🎓 **59 Claude Skills Total (34 QE-Specific)**
|
|
66
|
-
|
|
67
|
-
**Phase 1: Original Quality Engineering Skills (17 skills)** - World-class v1.0.0 ✨
|
|
68
|
-
- agentic-quality-engineering, holistic-testing-pact, context-driven-testing, exploratory-testing-advanced
|
|
69
|
-
- risk-based-testing, test-automation-strategy, api-testing-patterns, performance-testing, security-testing
|
|
70
|
-
- tdd-london-chicago, xp-practices, code-review-quality, refactoring-patterns, quality-metrics
|
|
71
|
-
- bug-reporting-excellence, technical-writing, consultancy-practices
|
|
72
|
-
|
|
73
|
-
**Phase 2: Expanded QE Skills Library (17 NEW skills)** - v1.0.0 🆕
|
|
74
|
-
- **Testing Methodologies (6)**: regression-testing, shift-left-testing, shift-right-testing, test-design-techniques, mutation-testing, test-data-management
|
|
75
|
-
- **Specialized Testing (9)**: accessibility-testing, mobile-testing, database-testing, contract-testing, chaos-engineering-resilience, compatibility-testing, localization-testing, compliance-testing, visual-testing-advanced
|
|
76
|
-
- **Testing Infrastructure (2)**: test-environment-management, test-reporting-analytics
|
|
77
|
-
|
|
78
|
-
**Total QE Skills: 34 (95%+ coverage of modern QE practices)** 🏆
|
|
79
|
-
|
|
80
|
-
**Skill Optimization Achievements:**
|
|
81
|
-
- ✅ 107 unique tags, 156 cross-references
|
|
82
|
-
- ✅ Semantic versioning (v1.0.0)
|
|
83
|
-
- ✅ 34x speedup with 13 parallel agents
|
|
84
|
-
- ✅ Quality: 52% → 100% (+48%)
|
|
85
|
-
|
|
86
|
-
**Claude Flow Integration Skills (25 skills)**
|
|
87
|
-
- AgentDB: advanced, learning, memory-patterns, optimization, vector-search (5)
|
|
88
|
-
- GitHub: code-review, multi-repo, project-management, release-management, workflow-automation (5)
|
|
89
|
-
- Flow Nexus: neural, platform, swarm (3)
|
|
90
|
-
- Advanced: hive-mind-advanced, hooks-automation, pair-programming, performance-analysis, sparc-methodology, skill-builder, stream-chain, swarm-advanced, swarm-orchestration, reasoningbank-agentdb, reasoningbank-intelligence, verification-quality (12)
|
|
91
|
-
|
|
92
|
-
**Unified CLAUDE.md** - 72 Total Agents (18 QE + 54 Claude Flow)
|
|
93
|
-
|
|
94
|
-
#### 🚀 **AgentDB Integration - Production Hardening**
|
|
95
|
-
|
|
96
|
-
**Code Reduction: 2,290+ Lines Removed (95%)**
|
|
97
|
-
- 900 lines: Custom QUIC → AgentDB QUIC sync (<1ms latency, TLS 1.3)
|
|
98
|
-
- 800 lines: Custom neural → AgentDB learning plugins (9 RL algorithms)
|
|
99
|
-
- 896 lines: Mixins removed (QUICCapableMixin, NeuralCapableMixin)
|
|
100
|
-
- 590 lines: Wrapper removed (AgentDBIntegration)
|
|
101
|
-
|
|
102
|
-
**Performance Improvements** ⚡
|
|
103
|
-
- QUIC Latency: 6.23ms → <1ms (84% faster)
|
|
104
|
-
- Vector Search: 150ms → 1ms (150x faster)
|
|
105
|
-
- Neural Training: 1000ms → 10-100ms (10-100x faster)
|
|
106
|
-
- Memory Usage: 512MB → 128-16MB (4-32x less)
|
|
107
|
-
- Startup Time: 500ms → 300ms (40% faster)
|
|
108
|
-
|
|
109
|
-
**Security Enhancements** 🔒
|
|
110
|
-
- OWASP Compliance: 70% → 90%+ (+20 points)
|
|
111
|
-
- Vulnerabilities Fixed: 8 total (3 CRITICAL, 5 HIGH)
|
|
112
|
-
- TLS 1.3: Enforced by default
|
|
113
|
-
- Certificate Validation: Mandatory
|
|
114
|
-
|
|
115
|
-
#### ✨ **New Features**
|
|
116
|
-
|
|
117
|
-
**Advanced Search & Indexing**
|
|
118
|
-
- HNSW Indexing: 150x faster vector search (O(log n))
|
|
119
|
-
- Quantization: 4-32x memory reduction
|
|
120
|
-
- Vector Search: Semantic search across all memories
|
|
121
|
-
- Full-Text Search: BM25 ranking
|
|
122
|
-
|
|
123
|
-
**9 Reinforcement Learning Algorithms**
|
|
124
|
-
- Decision Transformer, Q-Learning, SARSA, Actor-Critic
|
|
125
|
-
- DQN, PPO, A3C, REINFORCE, Monte Carlo
|
|
126
|
-
|
|
127
|
-
**QUIC Synchronization**
|
|
128
|
-
- Sub-millisecond latency (<1ms)
|
|
129
|
-
- TLS 1.3 encryption by default
|
|
130
|
-
- Automatic connection recovery
|
|
131
|
-
- Stream multiplexing
|
|
132
|
-
|
|
133
|
-
#### 🧪 **Test Suite Expansion**
|
|
134
|
-
- 60+ new test files added
|
|
135
|
-
- AgentDB: 6/6 tests (100%)
|
|
136
|
-
- Core: 53/53 tests (100%)
|
|
137
|
-
- Total: 59/59 tests passing (100%)
|
|
138
|
-
- Zero regressions detected
|
|
139
|
-
|
|
140
|
-
#### 🧹 **Repository Cleanup**
|
|
141
|
-
- Documentation: 24 reports archived (4.1MB saved)
|
|
142
|
-
- Dependencies: 89 packages removed (7.3MB saved)
|
|
143
|
-
- Total: 11.4MB savings
|
|
144
|
-
- Clean build: Zero TypeScript errors
|
|
145
|
-
|
|
146
|
-
#### 💔 **Breaking Changes**
|
|
147
|
-
- `enableQUIC()` → `initializeAgentDB({ quic: {...} })`
|
|
148
|
-
- `enableNeural()` → `initializeAgentDB({ learning: {...} })`
|
|
149
|
-
- Removed: QUICTransport, NeuralPatternMatcher, mixins
|
|
150
|
-
- See [Migration Guide](docs/AGENTDB-MIGRATION-GUIDE.md)
|
|
151
|
-
|
|
152
|
-
**Release Score: 90/100** ✅ | [Complete Changelog](docs/COMPLETE-1.2.0-CHANGELOG.md)
|
|
153
|
-
|
|
154
|
-
---
|
|
155
|
-
|
|
156
|
-
## 🎉 What's in v1.1.0
|
|
157
|
-
|
|
158
|
-
### Intelligence Boost Release (Previous)
|
|
159
|
-
|
|
160
|
-
**Learning System** 🧠
|
|
161
|
-
- Q-learning reinforcement learning for strategy optimization
|
|
162
|
-
- 20% improvement target tracking with automatic achievement
|
|
163
|
-
- Experience replay buffer (10,000 experiences)
|
|
164
|
-
- Automatic strategy recommendation with 95%+ confidence
|
|
165
|
-
- Cross-agent knowledge sharing
|
|
166
|
-
|
|
167
|
-
**Pattern Bank** 📦
|
|
168
|
-
- Cross-project pattern sharing and reuse
|
|
169
|
-
- 85%+ matching accuracy with AI-powered similarity
|
|
170
|
-
- 6 framework support (Jest, Mocha, Cypress, Vitest, Jasmine, AVA)
|
|
171
|
-
- Automatic pattern extraction from existing tests
|
|
172
|
-
- Pattern quality scoring and versioning
|
|
173
|
-
|
|
174
|
-
**ML Flaky Detection** 🎯
|
|
175
|
-
- 100% detection accuracy (target: 90%)
|
|
176
|
-
- 0% false positive rate (target: < 5%)
|
|
177
|
-
- Root cause analysis (timing, race conditions, dependencies, isolation)
|
|
178
|
-
- Automated fix recommendations with code examples
|
|
179
|
-
- < 1 second processing time for 1000+ test results
|
|
180
|
-
|
|
181
|
-
**Continuous Improvement** 🔄
|
|
182
|
-
- A/B testing framework for strategy comparison
|
|
183
|
-
- Auto-optimization with statistical confidence (95%+)
|
|
184
|
-
- Failure pattern analysis and mitigation
|
|
185
|
-
- Performance benchmarks (< 50ms pattern matching, < 100ms learning)
|
|
186
|
-
|
|
187
|
-
**Enhanced Agents:**
|
|
188
|
-
- **TestGeneratorAgent**: Pattern-based generation (20%+ faster)
|
|
189
|
-
- **CoverageAnalyzerAgent**: Learning-enhanced analysis
|
|
190
|
-
- **FlakyTestHunterAgent**: ML-based detection (99% accuracy)
|
|
191
|
-
|
|
192
|
-
See [CHANGELOG.md](CHANGELOG.md) for full details.
|
|
193
|
-
|
|
194
|
-
---
|
|
195
|
-
|
|
196
|
-
## 🚀 Features
|
|
197
|
-
|
|
198
|
-
### Phase 1: Cost Optimization (v1.0.5)
|
|
199
|
-
|
|
200
|
-
#### 💰 Multi-Model Router
|
|
201
|
-
- **70-81% Cost Savings**: Intelligent AI model selection saves $417+ per month
|
|
202
|
-
- **4+ AI Models**: GPT-3.5, GPT-4, Claude Haiku, Claude Sonnet 4.5
|
|
203
|
-
- **Smart Routing**: Automatic complexity analysis and model selection
|
|
204
|
-
- **Real-Time Tracking**: Live cost monitoring with daily/monthly budgets
|
|
205
|
-
- **Budget Alerts**: Email, Slack, and webhook notifications
|
|
206
|
-
- **Cost Forecasting**: Predict future costs with 90% confidence
|
|
207
|
-
- **ROI Dashboard**: Track savings vs single-model baseline
|
|
208
|
-
|
|
209
|
-
#### 📊 Real-Time Streaming
|
|
210
|
-
- **Live Progress Updates**: Real-time feedback for all operations
|
|
211
|
-
- **Test Generation Streaming**: See tests as they're created
|
|
212
|
-
- **Test Execution Streaming**: Live pass/fail updates
|
|
213
|
-
- **Coverage Streaming**: Real-time gap detection
|
|
214
|
-
- **Progress Bars**: Beautiful terminal progress visualization
|
|
215
|
-
- **Cancellation Support**: Stop operations mid-stream
|
|
216
|
-
- **Event Piping**: Chain multiple operations together
|
|
217
|
-
|
|
218
|
-
### Phase 2: Intelligence Boost (v1.1.0) 🆕
|
|
219
|
-
|
|
220
|
-
#### 🧠 Learning System
|
|
221
|
-
- **Q-Learning Algorithm**: Reinforcement learning for strategy optimization
|
|
222
|
-
- **20% Improvement Target**: Automatic tracking and achievement
|
|
223
|
-
- **Experience Replay**: Learn from 10,000+ past executions
|
|
224
|
-
- **Strategy Recommendation**: AI-powered optimal strategy selection
|
|
225
|
-
- **Cross-Agent Sharing**: Agents learn from each other's experiences
|
|
226
|
-
|
|
227
|
-
#### 📦 Pattern Bank
|
|
228
|
-
- **Pattern Extraction**: Automatic extraction from existing tests
|
|
229
|
-
- **85%+ Matching Accuracy**: AI-powered pattern matching
|
|
230
|
-
- **Cross-Project Sharing**: Export/import patterns across teams
|
|
231
|
-
- **6 Framework Support**: Jest, Mocha, Cypress, Vitest, Jasmine, AVA
|
|
232
|
-
- **Quality Scoring**: Automatic pattern quality assessment
|
|
233
|
-
|
|
234
|
-
#### 🎯 ML Flaky Detection
|
|
235
|
-
- **100% Accuracy**: ML-based detection with zero false positives
|
|
236
|
-
- **Root Cause Analysis**: Identify timing, race conditions, dependencies
|
|
237
|
-
- **Automated Fixes**: Generate fix recommendations with code examples
|
|
238
|
-
- **Continuous Monitoring**: Track test reliability over time
|
|
239
|
-
- **< 1 Second Processing**: 8,000+ test results per second
|
|
240
|
-
|
|
241
|
-
#### 🔄 Continuous Improvement
|
|
242
|
-
- **A/B Testing**: Compare strategies with statistical confidence
|
|
243
|
-
- **Auto-Optimization**: Apply winning strategies automatically
|
|
244
|
-
- **Failure Pattern Analysis**: Detect and mitigate recurring issues
|
|
245
|
-
- **Performance Benchmarks**: < 50ms pattern matching, < 100ms learning
|
|
246
|
-
|
|
247
|
-
### Core Features
|
|
248
|
-
|
|
249
|
-
#### 🤖 Autonomous Agent Fleet
|
|
250
|
-
- **18 Specialized Agents**: Each agent is an expert in specific quality engineering domains
|
|
251
|
-
- _Note: 17 QE-specific agents + 1 general-purpose base-template-generator agent_
|
|
252
|
-
- **AQE Hooks System**: 100-500x faster coordination with zero external dependencies
|
|
253
|
-
- **Intelligent Coordination**: Event-driven architecture with automatic task distribution
|
|
254
|
-
- **Scalable**: From single developer projects to enterprise-scale testing infrastructure
|
|
255
|
-
- **Self-Organizing**: Agents autonomously coordinate testing strategies
|
|
256
|
-
- **Type-Safe**: Full TypeScript type checking and IntelliSense support
|
|
257
|
-
|
|
258
|
-
#### 🧪 Comprehensive Testing
|
|
259
|
-
- **AI-Powered Test Generation**: Generate comprehensive test suites automatically
|
|
260
|
-
- **Multi-Framework Support**: Jest, Mocha, Cypress, Playwright, Vitest, Jasmine, AVA
|
|
261
|
-
- **Parallel Execution**: Execute thousands of tests concurrently with intelligent orchestration
|
|
262
|
-
- **Real-Time Coverage Analysis**: O(log n) algorithms for instant coverage gap detection
|
|
263
|
-
|
|
264
|
-
#### 🎯 Quality Intelligence
|
|
265
|
-
- **Smart Quality Gates**: ML-driven quality assessment with risk scoring
|
|
266
|
-
- **Security Scanning**: SAST, DAST, dependency analysis, and container security
|
|
267
|
-
- **Performance Testing**: Load testing with k6, JMeter, and Gatling integration
|
|
268
|
-
- **Visual Regression**: AI-powered screenshot comparison and UI validation
|
|
269
|
-
|
|
270
|
-
#### ⚡ Advanced Capabilities
|
|
271
|
-
- **API Contract Validation**: Breaking change detection across versions
|
|
272
|
-
- **Test Data Generation**: 10,000+ realistic records per second
|
|
273
|
-
- **Production Intelligence**: Convert production incidents into test scenarios
|
|
274
|
-
- **Chaos Engineering**: Controlled fault injection for resilience testing
|
|
275
|
-
|
|
276
|
-
---
|
|
277
|
-
|
|
278
|
-
## 📦 Prerequisites & Installation
|
|
279
|
-
|
|
280
|
-
### Prerequisites
|
|
281
|
-
|
|
282
|
-
#### Required
|
|
283
|
-
- **Claude Code**: Install from [claude.ai/code](https://claude.ai/code)
|
|
284
|
-
- **Node.js**: 18.0 or higher
|
|
285
|
-
- **npm**: 8.0 or higher
|
|
286
|
-
|
|
287
|
-
#### Optional (Advanced Features)
|
|
288
|
-
- **Claude Flow**: For optional MCP coordination features
|
|
289
|
-
```bash
|
|
290
|
-
npm install -g @claude/flow
|
|
291
|
-
# or
|
|
292
|
-
npx claude-flow@alpha init --force
|
|
293
|
-
```
|
|
294
|
-
|
|
295
|
-
**Note**: AQE hooks system requires NO external dependencies. All coordination features are built-in with TypeScript.
|
|
296
|
-
|
|
297
|
-
### Installation Steps
|
|
298
|
-
|
|
299
|
-
1. **Install Claude Code** globally or in your workspace
|
|
300
|
-
|
|
301
|
-
2. **Install Agentic QE**
|
|
302
|
-
|
|
303
|
-
**Global Installation** (Recommended)
|
|
304
|
-
```bash
|
|
305
|
-
npm install -g agentic-qe
|
|
306
|
-
|
|
307
|
-
# Verify installation
|
|
308
|
-
aqe --version
|
|
309
|
-
```
|
|
310
|
-
|
|
311
|
-
**Project Installation**
|
|
312
|
-
```bash
|
|
313
|
-
npm install --save-dev agentic-qe
|
|
314
|
-
|
|
315
|
-
# Use with npx
|
|
316
|
-
npx aqe init
|
|
317
|
-
```
|
|
318
|
-
|
|
319
|
-
3. **Local Development**
|
|
320
|
-
```bash
|
|
321
|
-
git clone https://github.com/proffesor-for-testing/agentic-qe.git
|
|
322
|
-
cd agentic-qe
|
|
323
|
-
npm install
|
|
324
|
-
npm run build
|
|
325
|
-
npm link
|
|
326
|
-
```
|
|
327
|
-
|
|
328
|
-
### System Requirements
|
|
329
|
-
|
|
330
|
-
- **Memory**: 2GB+ recommended for large test suites
|
|
331
|
-
- **OS**: Linux, macOS, Windows (via WSL2)
|
|
332
|
-
- **Agent Execution**: Via Claude Code's Task tool or MCP integration
|
|
333
|
-
|
|
334
|
-
---
|
|
335
|
-
|
|
336
|
-
## ⚡ Quick Start
|
|
337
|
-
|
|
338
|
-
### 1. Install & Setup MCP Integration
|
|
339
|
-
|
|
340
|
-
```bash
|
|
341
|
-
# Install Agentic QE
|
|
342
|
-
npm install -g agentic-qe
|
|
343
|
-
|
|
344
|
-
# Add MCP server to Claude Code
|
|
345
|
-
claude mcp add agentic-qe npx -y agentic-qe mcp:start
|
|
346
|
-
|
|
347
|
-
# Verify connection
|
|
348
|
-
claude mcp list
|
|
349
|
-
```
|
|
350
|
-
|
|
351
|
-
### 2. Initialize Your Project (v1.1.0)
|
|
352
|
-
|
|
353
|
-
```bash
|
|
354
|
-
# Initialize with Phase 1 + Phase 2 features
|
|
355
|
-
cd your-project
|
|
356
|
-
aqe init
|
|
357
|
-
```
|
|
358
|
-
|
|
359
|
-
**What gets initialized:**
|
|
360
|
-
- ✅ Multi-Model Router (70-81% cost savings)
|
|
361
|
-
- ✅ Learning System (20% improvement target)
|
|
362
|
-
- ✅ Pattern Bank (cross-project reuse)
|
|
363
|
-
- ✅ ML Flaky Detection (100% accuracy)
|
|
364
|
-
- ✅ Improvement Loop (A/B testing)
|
|
365
|
-
- ✅ 17 Specialized QE agent definitions (+ 1 general-purpose agent)
|
|
366
|
-
- ✅ 8 AQE slash commands
|
|
367
|
-
- ✅ Configuration directory
|
|
368
|
-
|
|
369
|
-
### 3. Use from Claude Code CLI
|
|
370
|
-
|
|
371
|
-
```bash
|
|
372
|
-
# Ask Claude to generate tests using AQE agents with patterns
|
|
373
|
-
claude "Initialize AQE fleet and generate comprehensive tests for src/services/user-service.ts with 95% coverage using pattern matching"
|
|
374
|
-
```
|
|
375
|
-
|
|
376
|
-
**Agent Execution Model:**
|
|
377
|
-
- Agents are Claude Code agent definitions (markdown files in `.claude/agents/`)
|
|
378
|
-
- Executed via Claude Code's Task tool OR MCP tools
|
|
379
|
-
- MCP integration enables Claude to orchestrate QE agents directly
|
|
380
|
-
- NOT standalone Node.js processes
|
|
381
|
-
|
|
382
|
-
📖 **[Complete MCP Integration Guide](docs/guides/MCP-INTEGRATION.md)** - Detailed setup, examples, and use cases
|
|
383
|
-
|
|
384
|
-
---
|
|
385
|
-
|
|
386
|
-
## 🎯 Phase 2 Commands (v1.1.0)
|
|
387
|
-
|
|
388
|
-
### Learning System Commands 🧠
|
|
389
|
-
|
|
390
|
-
```bash
|
|
391
|
-
# Enable learning for all agents
|
|
392
|
-
aqe learn enable --all
|
|
393
|
-
|
|
394
|
-
# View learning metrics
|
|
395
|
-
aqe learn status
|
|
396
|
-
|
|
397
|
-
# View learning history
|
|
398
|
-
aqe learn history --agent test-generator
|
|
399
|
-
|
|
400
|
-
# Manual training
|
|
401
|
-
aqe learn train --agent test-generator
|
|
402
|
-
|
|
403
|
-
# Export learning data
|
|
404
|
-
aqe learn export --agent test-generator --output learning-state.json
|
|
405
|
-
```
|
|
406
|
-
|
|
407
|
-
**Example Output** - `aqe learn status`:
|
|
408
|
-
```
|
|
409
|
-
📊 LEARNING STATUS
|
|
410
|
-
|
|
411
|
-
Agent: test-generator
|
|
412
|
-
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
|
|
413
|
-
Status: ENABLED ✅
|
|
414
|
-
Total Experiences: 247
|
|
415
|
-
Exploration Rate: 15.3%
|
|
416
|
-
|
|
417
|
-
Performance:
|
|
418
|
-
├─ Average Reward: 1.23
|
|
419
|
-
├─ Success Rate: 87.5%
|
|
420
|
-
└─ Improvement Rate: 18.7% (↑ target: 20%)
|
|
421
|
-
|
|
422
|
-
Top Strategies:
|
|
423
|
-
1. property-based (confidence: 92%, success: 95%)
|
|
424
|
-
2. mutation-based (confidence: 85%, success: 88%)
|
|
425
|
-
3. example-based (confidence: 78%, success: 82%)
|
|
426
|
-
|
|
427
|
-
Recent Trend: ↗ improving
|
|
428
|
-
```
|
|
429
|
-
|
|
430
|
-
### Pattern Management Commands 📦
|
|
431
|
-
|
|
432
|
-
```bash
|
|
433
|
-
# List all patterns
|
|
434
|
-
aqe patterns list
|
|
435
|
-
|
|
436
|
-
# Search patterns by keyword
|
|
437
|
-
aqe patterns search "null check"
|
|
438
|
-
|
|
439
|
-
# Extract patterns from tests
|
|
440
|
-
aqe patterns extract --path tests/ --framework jest
|
|
441
|
-
|
|
442
|
-
# Share patterns across projects
|
|
443
|
-
aqe patterns share --id pattern-001 --projects proj-a,proj-b
|
|
444
|
-
|
|
445
|
-
# Export patterns
|
|
446
|
-
aqe patterns export --output patterns-backup.json
|
|
447
|
-
```
|
|
448
|
-
|
|
449
|
-
**Example Output** - `aqe patterns list`:
|
|
450
|
-
```
|
|
451
|
-
📦 PATTERN LIBRARY (247 patterns)
|
|
452
|
-
|
|
453
|
-
ID | Name | Framework | Quality | Uses
|
|
454
|
-
-----------|---------------------------|-----------|---------|-----
|
|
455
|
-
pattern-001| Null Parameter Check | jest | 92% | 142
|
|
456
|
-
pattern-002| Empty Array Handling | jest | 89% | 98
|
|
457
|
-
pattern-003| API Timeout Test | cypress | 95% | 87
|
|
458
|
-
pattern-004| Database Transaction | mocha | 88% | 76
|
|
459
|
-
pattern-005| Async Error Handling | jest | 91% | 65
|
|
460
|
-
|
|
461
|
-
Total: 247 patterns | Average Quality: 87%
|
|
462
|
-
```
|
|
463
|
-
|
|
464
|
-
### Improvement Loop Commands 🔄
|
|
465
|
-
|
|
466
|
-
```bash
|
|
467
|
-
# View improvement status
|
|
468
|
-
aqe improve status
|
|
469
|
-
|
|
470
|
-
# Start improvement loop
|
|
471
|
-
aqe improve start
|
|
472
|
-
|
|
473
|
-
# Run A/B test
|
|
474
|
-
aqe improve ab-test --strategies "property-based,mutation-based" --sample-size 50
|
|
475
|
-
|
|
476
|
-
# View failure patterns
|
|
477
|
-
aqe improve failures
|
|
478
|
-
|
|
479
|
-
# Generate improvement report
|
|
480
|
-
aqe improve report --format html --output improvement-report.html
|
|
481
|
-
```
|
|
482
|
-
|
|
483
|
-
---
|
|
484
|
-
|
|
485
|
-
## 🎯 Phase 1 Commands (v1.0.5)
|
|
486
|
-
|
|
487
|
-
### Multi-Model Router Commands 💰
|
|
488
|
-
|
|
489
|
-
```bash
|
|
490
|
-
# Enable cost-optimized routing (70-81% savings)
|
|
491
|
-
aqe routing enable
|
|
492
|
-
|
|
493
|
-
# View current configuration and savings
|
|
494
|
-
aqe routing status
|
|
495
|
-
|
|
496
|
-
# Launch real-time cost dashboard
|
|
497
|
-
aqe routing dashboard
|
|
498
|
-
|
|
499
|
-
# Generate detailed cost report
|
|
500
|
-
aqe routing report --format html --output report.html
|
|
501
|
-
|
|
502
|
-
# View routing statistics
|
|
503
|
-
aqe routing stats --days 30
|
|
504
|
-
|
|
505
|
-
# Disable routing
|
|
506
|
-
aqe routing disable
|
|
507
|
-
```
|
|
508
|
-
|
|
509
|
-
**Example Output** - `aqe routing status`:
|
|
510
|
-
```
|
|
511
|
-
✅ Multi-Model Router Status
|
|
512
|
-
|
|
513
|
-
Configuration:
|
|
514
|
-
Status: ENABLED ✓
|
|
515
|
-
Default Model: claude-sonnet-4.5
|
|
516
|
-
Cost Tracking: ENABLED ✓
|
|
517
|
-
Fallback Chains: ENABLED ✓
|
|
518
|
-
|
|
519
|
-
Cost Summary (Last 30 Days):
|
|
520
|
-
Total Cost: $127.50
|
|
521
|
-
Baseline Cost: $545.00
|
|
522
|
-
Savings: $417.50 (76.6%)
|
|
523
|
-
Budget Status: ON TRACK ✓
|
|
524
|
-
|
|
525
|
-
Model Usage:
|
|
526
|
-
├─ gpt-3.5-turbo: 42% (simple tasks)
|
|
527
|
-
├─ claude-haiku: 31% (medium tasks)
|
|
528
|
-
├─ claude-sonnet-4.5: 20% (complex tasks)
|
|
529
|
-
└─ gpt-4: 7% (critical tasks)
|
|
530
|
-
```
|
|
531
|
-
|
|
532
|
-
📚 **[Complete Routing Examples](docs/examples/ROUTING-EXAMPLES.md)** - CLI and programmatic usage
|
|
533
|
-
|
|
534
|
-
### Basic Commands
|
|
535
|
-
|
|
536
|
-
```bash
|
|
537
|
-
# Check fleet status
|
|
538
|
-
aqe status
|
|
539
|
-
|
|
540
|
-
# Generate tests for a module
|
|
541
|
-
aqe test src/services/user-service.ts
|
|
542
|
-
|
|
543
|
-
# Analyze test coverage
|
|
544
|
-
aqe coverage --threshold 95
|
|
545
|
-
|
|
546
|
-
# Run quality gate validation
|
|
547
|
-
aqe quality
|
|
548
|
-
|
|
549
|
-
# Execute comprehensive test suite
|
|
550
|
-
aqe execute --parallel --coverage
|
|
551
|
-
|
|
552
|
-
# View all commands
|
|
553
|
-
aqe help
|
|
554
|
-
```
|
|
555
|
-
|
|
556
|
-
---
|
|
557
|
-
|
|
558
|
-
## 💻 Programmatic Usage
|
|
559
|
-
|
|
560
|
-
### Pattern-Based Test Generation (v1.1.0) 🆕
|
|
561
|
-
|
|
562
|
-
```typescript
|
|
563
|
-
import { TestGeneratorAgent, QEReasoningBank } from 'agentic-qe';
|
|
564
|
-
|
|
565
|
-
// Initialize pattern bank
|
|
566
|
-
const bank = new QEReasoningBank();
|
|
567
|
-
|
|
568
|
-
// Create agent with pattern matching
|
|
569
|
-
const agent = new TestGeneratorAgent(
|
|
570
|
-
{ agentId: 'test-gen-1', memoryStore },
|
|
571
|
-
{
|
|
572
|
-
targetCoverage: 95,
|
|
573
|
-
framework: 'jest',
|
|
574
|
-
enablePatterns: true, // ✅ Enable pattern matching
|
|
575
|
-
enableLearning: true, // ✅ Enable learning
|
|
576
|
-
reasoningBank: bank
|
|
577
|
-
}
|
|
578
|
-
);
|
|
579
|
-
|
|
580
|
-
// Generate tests with patterns
|
|
581
|
-
const result = await agent.execute({
|
|
582
|
-
type: 'test-generation',
|
|
583
|
-
payload: {
|
|
584
|
-
sourceFile: 'src/myModule.ts',
|
|
585
|
-
framework: 'jest'
|
|
586
|
-
}
|
|
587
|
-
});
|
|
588
|
-
|
|
589
|
-
console.log(`Generated ${result.testsGenerated} tests`);
|
|
590
|
-
console.log(`Pattern hit rate: ${result.patternHitRate}%`);
|
|
591
|
-
console.log(`Time saved: ${result.patterns.savings}ms`);
|
|
592
|
-
console.log(`Quality score: ${result.qualityScore}%`);
|
|
593
|
-
```
|
|
594
|
-
|
|
595
|
-
**Performance Impact:**
|
|
596
|
-
```
|
|
597
|
-
Without Patterns:
|
|
598
|
-
├─ Test Generation: 180ms avg
|
|
599
|
-
├─ Coverage: 78%
|
|
600
|
-
└─ Quality Score: 82%
|
|
601
|
-
|
|
602
|
-
With Patterns:
|
|
603
|
-
├─ Test Generation: 145ms avg (↓ 19.4%)
|
|
604
|
-
├─ Coverage: 94% (↑ 20.5%)
|
|
605
|
-
└─ Quality Score: 96% (↑ 17.1%)
|
|
606
|
-
```
|
|
607
|
-
|
|
608
|
-
### Learning-Enhanced Coverage Analysis (v1.1.0) 🆕
|
|
609
|
-
|
|
610
|
-
```typescript
|
|
611
|
-
import { CoverageAnalyzerAgent, LearningEngine } from 'agentic-qe';
|
|
612
|
-
|
|
613
|
-
// Create learning engine
|
|
614
|
-
const learningEngine = new LearningEngine('coverage-1', memory, {
|
|
615
|
-
enabled: true,
|
|
616
|
-
learningRate: 0.1,
|
|
617
|
-
targetImprovement: 0.20 // 20% improvement target
|
|
618
|
-
});
|
|
619
|
-
|
|
620
|
-
await learningEngine.initialize();
|
|
621
|
-
|
|
622
|
-
// Create agent with learning
|
|
623
|
-
const agent = new CoverageAnalyzerAgent(
|
|
624
|
-
{ agentId: 'coverage-1', memoryStore: memory },
|
|
625
|
-
{
|
|
626
|
-
targetCoverage: 95,
|
|
627
|
-
algorithm: 'sublinear',
|
|
628
|
-
enableLearning: true // ✅ Enable learning
|
|
629
|
-
}
|
|
630
|
-
);
|
|
631
|
-
|
|
632
|
-
// Analyze coverage (learning happens automatically)
|
|
633
|
-
const analysis = await agent.execute({
|
|
634
|
-
type: 'coverage-analysis',
|
|
635
|
-
payload: {
|
|
636
|
-
coverageReport: './coverage/coverage-final.json',
|
|
637
|
-
threshold: 95
|
|
638
|
-
}
|
|
639
|
-
});
|
|
640
|
-
|
|
641
|
-
// Check learning progress
|
|
642
|
-
const status = await learningEngine.calculateImprovement();
|
|
643
|
-
console.log(`Current improvement: ${(status.improvementRate * 100).toFixed(1)}%`);
|
|
644
|
-
console.log(`Target reached: ${status.targetAchieved ? '✅ YES' : '❌ NO'}`);
|
|
645
|
-
console.log(`Confidence: ${(status.confidence * 100).toFixed(1)}%`);
|
|
646
|
-
```
|
|
647
|
-
|
|
648
|
-
### ML Flaky Detection (v1.1.0) 🆕
|
|
649
|
-
|
|
650
|
-
```typescript
|
|
651
|
-
import { FlakyTestHunterAgent, FlakyTestDetector } from 'agentic-qe';
|
|
652
|
-
|
|
653
|
-
// Create detector with ML
|
|
654
|
-
const detector = new FlakyTestDetector({
|
|
655
|
-
minRuns: 5,
|
|
656
|
-
passRateThreshold: 0.8,
|
|
657
|
-
confidenceThreshold: 0.7,
|
|
658
|
-
enableML: true // ✅ 100% accuracy
|
|
659
|
-
});
|
|
660
|
-
|
|
661
|
-
// Create agent
|
|
662
|
-
const agent = new FlakyTestHunterAgent(
|
|
663
|
-
{ agentId: 'flaky-1', memoryStore: memory },
|
|
664
|
-
{
|
|
665
|
-
enableML: true, // ✅ 100% accuracy
|
|
666
|
-
detector
|
|
667
|
-
}
|
|
668
|
-
);
|
|
669
|
-
|
|
670
|
-
// Detect flaky tests
|
|
671
|
-
const result = await agent.execute({
|
|
672
|
-
type: 'flaky-detection',
|
|
673
|
-
payload: {
|
|
674
|
-
testHistory: testResults
|
|
675
|
-
}
|
|
676
|
-
});
|
|
677
|
-
|
|
678
|
-
result.flakyTests.forEach(test => {
|
|
679
|
-
console.log(`🔴 ${test.testName}`);
|
|
680
|
-
console.log(` Pass Rate: ${(test.passRate * 100).toFixed(1)}%`);
|
|
681
|
-
console.log(` Root Cause: ${test.rootCause.cause}`);
|
|
682
|
-
console.log(` ML Confidence: ${(test.rootCause.mlConfidence * 100).toFixed(1)}%`);
|
|
683
|
-
console.log(` Severity: ${test.severity}`);
|
|
684
|
-
console.log(` Fix: ${test.fixRecommendations[0].recommendation}`);
|
|
685
|
-
console.log(` Code Example:`);
|
|
686
|
-
console.log(` ${test.fixRecommendations[0].codeExample}`);
|
|
687
|
-
});
|
|
688
|
-
```
|
|
689
|
-
|
|
690
|
-
**Detection Results:**
|
|
691
|
-
```
|
|
692
|
-
Model Training Complete:
|
|
693
|
-
Accuracy: 100.00% ✅ Exceeds 90% target by 10%
|
|
694
|
-
Precision: 100.00% ✅ Perfect precision
|
|
695
|
-
Recall: 100.00% ✅ Perfect recall
|
|
696
|
-
F1 Score: 100.00% ✅ Perfect F1
|
|
697
|
-
False Positive Rate: 0.00% ✅ Well below 5% target
|
|
698
|
-
|
|
699
|
-
Processing 1,200 test results: ~150ms
|
|
700
|
-
Throughput: ~8,000 results/second
|
|
701
|
-
Memory Usage: < 5MB delta
|
|
702
|
-
```
|
|
703
|
-
|
|
704
|
-
### With Multi-Model Router (v1.0.5)
|
|
705
|
-
|
|
706
|
-
```typescript
|
|
707
|
-
import { FleetManager, AdaptiveModelRouter } from 'agentic-qe';
|
|
708
|
-
|
|
709
|
-
// Initialize fleet with cost-optimized routing
|
|
710
|
-
const fleet = new FleetManager({
|
|
711
|
-
maxAgents: 20,
|
|
712
|
-
topology: 'mesh',
|
|
713
|
-
routing: {
|
|
714
|
-
enabled: true,
|
|
715
|
-
defaultModel: 'claude-sonnet-4.5',
|
|
716
|
-
enableCostTracking: true,
|
|
717
|
-
enableFallback: true,
|
|
718
|
-
modelPreferences: {
|
|
719
|
-
simple: 'gpt-3.5-turbo', // 70% cheaper for simple tasks
|
|
720
|
-
medium: 'claude-haiku', // 60% cheaper for standard tests
|
|
721
|
-
complex: 'claude-sonnet-4.5', // Best quality/cost for complex
|
|
722
|
-
critical: 'gpt-4' // Maximum quality when needed
|
|
723
|
-
},
|
|
724
|
-
budgets: {
|
|
725
|
-
daily: 50,
|
|
726
|
-
monthly: 1000
|
|
727
|
-
}
|
|
728
|
-
}
|
|
729
|
-
});
|
|
730
|
-
|
|
731
|
-
await fleet.initialize();
|
|
732
|
-
|
|
733
|
-
// Spawn agent (automatically uses optimal model based on task complexity)
|
|
734
|
-
const testGen = await fleet.spawnAgent('test-generator', {
|
|
735
|
-
targetCoverage: 95,
|
|
736
|
-
framework: 'jest',
|
|
737
|
-
useRouting: true // Enable intelligent model selection
|
|
738
|
-
});
|
|
739
|
-
|
|
740
|
-
// Execute task (router selects cheapest model that meets quality requirements)
|
|
741
|
-
const tests = await testGen.execute({
|
|
742
|
-
sourceFile: 'src/services/user-service.ts',
|
|
743
|
-
testStyle: 'property-based'
|
|
744
|
-
});
|
|
745
|
-
|
|
746
|
-
// Check cost savings
|
|
747
|
-
const savings = await fleet.getRoutingSavings();
|
|
748
|
-
console.log(`💰 Total savings: $${savings.total} (${savings.percent}%)`);
|
|
749
|
-
console.log(`📊 Models used: ${JSON.stringify(savings.modelBreakdown, null, 2)}`);
|
|
750
|
-
```
|
|
751
|
-
|
|
752
|
-
📚 **[Complete Routing Examples](docs/examples/ROUTING-EXAMPLES.md)** - Advanced programmatic usage
|
|
753
|
-
|
|
754
|
-
---
|
|
755
|
-
|
|
756
|
-
## 📊 Performance Benchmarks
|
|
757
|
-
|
|
758
|
-
| Feature | Target | Actual | Status |
|
|
759
|
-
|---------|--------|--------|--------|
|
|
760
|
-
| **Pattern Matching (p95)** | <50ms | 32ms | ✅ Exceeded |
|
|
761
|
-
| **Learning Iteration** | <100ms | 68ms | ✅ Exceeded |
|
|
762
|
-
| **ML Flaky Detection (1000 tests)** | <500ms | 385ms | ✅ Exceeded |
|
|
763
|
-
| **Agent Memory** | <100MB | 85MB | ✅ Exceeded |
|
|
764
|
-
| **Cost Savings** | 70%+ | 70-81% | ✅ Achieved |
|
|
765
|
-
| **Test Improvement** | 20%+ | 23%+ | ✅ Exceeded |
|
|
766
|
-
| **Flaky Detection Accuracy** | 90%+ | 100% | ✅ Exceeded |
|
|
767
|
-
| **False Positive Rate** | <5% | 0% | ✅ Exceeded |
|
|
768
|
-
|
|
769
|
-
### Core Performance
|
|
770
|
-
|
|
771
|
-
- **Test Generation**: 1000+ tests/minute
|
|
772
|
-
- **Parallel Execution**: 10,000+ concurrent tests
|
|
773
|
-
- **Coverage Analysis**: O(log n) complexity
|
|
774
|
-
- **Data Generation**: 10,000+ records/second
|
|
775
|
-
- **Agent Spawning**: <100ms per agent
|
|
776
|
-
- **Memory Efficient**: <2GB for typical projects
|
|
777
|
-
|
|
778
|
-
---
|
|
779
|
-
|
|
780
|
-
## 🤖 Agent Types
|
|
781
|
-
|
|
782
|
-
### Core Testing Agents
|
|
783
|
-
|
|
784
|
-
| Agent | Purpose | Key Features | Phase 2 Enhancements |
|
|
785
|
-
|-------|---------|-------------|---------------------|
|
|
786
|
-
| **test-generator** | AI-powered test creation | Property-based testing, edge case detection | ✅ Pattern matching, Learning |
|
|
787
|
-
| **test-executor** | Multi-framework execution | Parallel processing, retry logic, reporting | - |
|
|
788
|
-
| **coverage-analyzer** | Real-time gap analysis | O(log n) algorithms, trend tracking | ✅ Learning, Pattern recommendations |
|
|
789
|
-
| **quality-gate** | Intelligent validation | ML-driven decisions, risk assessment | ✅ Flaky test metrics |
|
|
790
|
-
| **quality-analyzer** | Metrics analysis | ESLint, SonarQube, Lighthouse integration | - |
|
|
791
|
-
|
|
792
|
-
### Performance & Security
|
|
793
|
-
|
|
794
|
-
| Agent | Purpose | Key Features |
|
|
795
|
-
|-------|---------|-------------|
|
|
796
|
-
| **performance-tester** | Load & stress testing | k6, JMeter, Gatling, bottleneck detection |
|
|
797
|
-
| **security-scanner** | Vulnerability detection | SAST, DAST, dependency scanning |
|
|
798
|
-
|
|
799
|
-
### Strategic Planning
|
|
800
|
-
|
|
801
|
-
| Agent | Purpose | Key Features |
|
|
802
|
-
|-------|---------|-------------|
|
|
803
|
-
| **requirements-validator** | Testability analysis | INVEST criteria, BDD generation |
|
|
804
|
-
| **production-intelligence** | Incident replay | RUM analysis, anomaly detection |
|
|
805
|
-
| **fleet-commander** | Hierarchical coordination | 50+ agent orchestration |
|
|
806
|
-
|
|
807
|
-
### Advanced Testing
|
|
808
|
-
|
|
809
|
-
| Agent | Purpose | Key Features | Phase 2 Enhancements |
|
|
810
|
-
|-------|---------|-------------|---------------------|
|
|
811
|
-
| **regression-risk-analyzer** | Smart test selection | ML patterns, AST analysis | ✅ Pattern matching |
|
|
812
|
-
| **test-data-architect** | Realistic data generation | 10k+ records/sec, GDPR compliant | - |
|
|
813
|
-
| **api-contract-validator** | Breaking change detection | OpenAPI, GraphQL, gRPC | - |
|
|
814
|
-
| **flaky-test-hunter** | Stability analysis | Statistical detection, auto-fix | ✅ 100% accuracy ML detection |
|
|
815
|
-
|
|
816
|
-
### Specialized
|
|
817
|
-
|
|
818
|
-
| Agent | Purpose | Key Features |
|
|
819
|
-
|-------|---------|-------------|
|
|
820
|
-
| **deployment-readiness** | Release validation | Multi-factor risk scoring |
|
|
821
|
-
| **visual-tester** | UI regression | AI-powered comparison |
|
|
822
|
-
| **chaos-engineer** | Resilience testing | Fault injection, blast radius |
|
|
823
|
-
|
|
824
|
-
---
|
|
825
|
-
|
|
826
|
-
## 🏗️ Architecture
|
|
827
|
-
|
|
828
|
-
### Core Components
|
|
829
|
-
|
|
830
|
-
```
|
|
831
|
-
┌─────────────────────────────────────────────┐
|
|
832
|
-
│ Fleet Manager │
|
|
833
|
-
│ (Central Coordination & Task Distribution) │
|
|
834
|
-
└─────────────────────────────────────────────┘
|
|
835
|
-
│
|
|
836
|
-
┌───────────┼───────────┐
|
|
837
|
-
│ │ │
|
|
838
|
-
┌────▼────┐ ┌───▼────┐ ┌───▼────┐
|
|
839
|
-
│ Agent 1 │ │ Agent 2│ │ Agent N│
|
|
840
|
-
│ Pool │ │ Pool │ │ Pool │
|
|
841
|
-
└────┬────┘ └───┬────┘ └───┬────┘
|
|
842
|
-
│ │ │
|
|
843
|
-
┌────▼──────────▼──────────▼────┐
|
|
844
|
-
│ Event Bus │
|
|
845
|
-
│ (Event-Driven Communication) │
|
|
846
|
-
└────┬──────────────────────┬────┘
|
|
847
|
-
│ │
|
|
848
|
-
┌────▼────┐ ┌────▼────┐
|
|
849
|
-
│ Memory │ │Database │
|
|
850
|
-
│ Store │ │(SQLite) │
|
|
851
|
-
│ │ │ │
|
|
852
|
-
│ Phase 2:│ │ │
|
|
853
|
-
│ Learning│ │ │
|
|
854
|
-
│ Patterns│ │ │
|
|
855
|
-
│ ML Model│ │ │
|
|
856
|
-
└─────────┘ └─────────┘
|
|
857
|
-
```
|
|
858
|
-
|
|
859
|
-
### Event-Driven Architecture
|
|
860
|
-
|
|
861
|
-
- **EventBus**: Real-time communication between agents
|
|
862
|
-
- **Task Queue**: Priority-based task scheduling
|
|
863
|
-
- **Memory Store**: Shared context and learning (SwarmMemoryManager)
|
|
864
|
-
- **Persistence**: SQLite for state, metrics, and audit trails
|
|
865
|
-
|
|
866
|
-
### AQE Hooks System
|
|
867
|
-
|
|
868
|
-
**Zero Dependencies** - Built-in TypeScript hooks for agent coordination:
|
|
869
|
-
|
|
870
|
-
```typescript
|
|
871
|
-
// Automatic lifecycle hooks in every agent (aqe-hooks protocol)
|
|
872
|
-
class QEAgent extends BaseAgent {
|
|
873
|
-
protected async onPreTask(data): Promise<void> { /* prepare */ }
|
|
874
|
-
protected async onPostTask(data): Promise<void> { /* validate */ }
|
|
875
|
-
protected async onTaskError(data): Promise<void> { /* recover */ }
|
|
876
|
-
}
|
|
877
|
-
|
|
878
|
-
// Advanced verification hooks
|
|
879
|
-
const hookManager = new VerificationHookManager(memoryStore);
|
|
880
|
-
await hookManager.executePreTaskVerification({ task, context });
|
|
881
|
-
await hookManager.executePostTaskValidation({ task, result });
|
|
882
|
-
```
|
|
883
|
-
|
|
884
|
-
**Performance**: 100-500x faster than external hooks (<1ms vs 100-500ms)
|
|
885
|
-
|
|
886
|
-
**Features**:
|
|
887
|
-
- Full TypeScript type safety
|
|
888
|
-
- Direct SwarmMemoryManager integration
|
|
889
|
-
- Built-in RollbackManager support
|
|
890
|
-
- EventBus coordination
|
|
891
|
-
- Context engineering (pre/post tool-use bundles)
|
|
892
|
-
|
|
893
|
-
---
|
|
894
|
-
|
|
895
|
-
## 📖 Documentation
|
|
896
|
-
|
|
897
|
-
### 🆕 Phase 2 Features (v1.1.0)
|
|
898
|
-
- [Learning System User Guide](docs/guides/LEARNING-SYSTEM-USER-GUIDE.md) - **NEW!** Q-learning and continuous improvement
|
|
899
|
-
- [Pattern Management User Guide](docs/guides/PATTERN-MANAGEMENT-USER-GUIDE.md) - **NEW!** Cross-project pattern sharing
|
|
900
|
-
- [ML Flaky Detection Guide](docs/guides/ML-FLAKY-DETECTION-USER-GUIDE.md) - **NEW!** 100% accurate flaky detection
|
|
901
|
-
- [Performance Improvement Guide](docs/guides/PERFORMANCE-IMPROVEMENT-USER-GUIDE.md) - **NEW!** A/B testing and optimization
|
|
902
|
-
- [Learning System Examples](docs/examples/LEARNING-SYSTEM-EXAMPLES.md) - **NEW!** Learning code examples
|
|
903
|
-
- [Pattern Examples](docs/examples/REASONING-BANK-EXAMPLES.md) - **NEW!** Pattern usage examples
|
|
904
|
-
- [Flaky Detection Examples](docs/examples/FLAKY-DETECTION-ML-EXAMPLES.md) - **NEW!** ML detection examples
|
|
905
|
-
|
|
906
|
-
### Phase 1 Features (v1.0.5)
|
|
907
|
-
- [Multi-Model Router Guide](docs/guides/MULTI-MODEL-ROUTER.md) - Save 70% on AI costs
|
|
908
|
-
- [Streaming API Tutorial](docs/guides/STREAMING-API.md) - Real-time progress updates
|
|
909
|
-
- [Cost Optimization Best Practices](docs/guides/COST-OPTIMIZATION.md) - Maximize ROI
|
|
910
|
-
- [Migration Guide v1.0.5](docs/guides/MIGRATION-V1.0.5.md) - Upgrade guide
|
|
911
|
-
- [Routing API Reference](docs/api/ROUTING-API.md) - Complete API docs
|
|
912
|
-
- [Streaming API Reference](docs/api/STREAMING-API.md) - Complete API docs
|
|
913
|
-
- [Phase 1 Code Examples](docs/examples/ROUTING-EXAMPLES.md) - Working examples
|
|
914
|
-
|
|
915
|
-
### Getting Started
|
|
916
|
-
- [Quick Start Guide](docs/AQE-CLI.md)
|
|
917
|
-
- [User Guide](docs/USER-GUIDE.md) - Comprehensive workflows and examples
|
|
918
|
-
- [Agent Types Overview](docs/Agentic-QE-Fleet-Specification.md)
|
|
919
|
-
- [Configuration Guide](docs/CONFIGURATION.md) - Complete configuration reference
|
|
920
|
-
- [Troubleshooting Guide](docs/TROUBLESHOOTING.md) - Common issues and solutions
|
|
921
|
-
|
|
922
|
-
### User Guides
|
|
923
|
-
- [Test Generation](docs/guides/TEST-GENERATION.md)
|
|
924
|
-
- [Coverage Analysis](docs/guides/COVERAGE-ANALYSIS.md)
|
|
925
|
-
- [Quality Gates](docs/guides/QUALITY-GATES.md)
|
|
926
|
-
- [Performance Testing](docs/guides/PERFORMANCE-TESTING.md)
|
|
927
|
-
- [Test Execution](docs/guides/TEST-EXECUTION.md)
|
|
928
|
-
- [MCP Integration](docs/guides/MCP-INTEGRATION.md)
|
|
929
|
-
|
|
930
|
-
### Advanced Topics
|
|
931
|
-
- [API Reference](docs/API.md)
|
|
932
|
-
- [Agent Development](docs/AGENT-DEVELOPMENT.md)
|
|
933
|
-
- [MCP Integration](docs/CLAUDE-MD-INTEGRATION.md)
|
|
934
|
-
- [Best Practices](docs/AI%20%26%20Agentic%20Security%20Best%20Practices.md)
|
|
935
|
-
- [AQE Hooks Guide](docs/AQE-HOOKS-GUIDE.md)
|
|
936
|
-
|
|
937
|
-
### Commands Reference
|
|
938
|
-
- [AQE Commands Overview](docs/QE-COMMANDS-INDEX.md)
|
|
939
|
-
- [Command Specifications](docs/QE-SLASH-COMMANDS-SPECIFICATION.md)
|
|
940
|
-
- [Hooks Architecture](docs/QE_HOOKS_ARCHITECTURE.md)
|
|
941
|
-
|
|
942
|
-
---
|
|
943
|
-
|
|
944
|
-
## 🔧 Configuration
|
|
945
|
-
|
|
946
|
-
### Environment Variables
|
|
947
|
-
|
|
948
|
-
Create a `.env` file in your project root:
|
|
949
|
-
|
|
950
|
-
```bash
|
|
951
|
-
# Fleet Configuration
|
|
952
|
-
FLEET_ID=my-project-fleet
|
|
953
|
-
MAX_AGENTS=20
|
|
954
|
-
HEARTBEAT_INTERVAL=30000
|
|
955
|
-
|
|
956
|
-
# Phase 2: Learning System
|
|
957
|
-
LEARNING_ENABLED=true
|
|
958
|
-
LEARNING_RATE=0.1
|
|
959
|
-
TARGET_IMPROVEMENT=0.20
|
|
960
|
-
|
|
961
|
-
# Phase 2: Pattern Bank
|
|
962
|
-
PATTERN_MATCHING_ENABLED=true
|
|
963
|
-
MIN_PATTERN_QUALITY=0.8
|
|
964
|
-
|
|
965
|
-
# Phase 2: ML Flaky Detection
|
|
966
|
-
FLAKY_DETECTION_ML=true
|
|
967
|
-
FLAKY_MIN_RUNS=5
|
|
968
|
-
|
|
969
|
-
# Database
|
|
970
|
-
DB_TYPE=sqlite
|
|
971
|
-
DB_FILENAME=./data/fleet.db
|
|
972
|
-
|
|
973
|
-
# Logging
|
|
974
|
-
LOG_LEVEL=info
|
|
975
|
-
LOG_FORMAT=json
|
|
976
|
-
|
|
977
|
-
# API (optional)
|
|
978
|
-
API_PORT=3000
|
|
979
|
-
API_HOST=localhost
|
|
980
|
-
```
|
|
981
|
-
|
|
982
|
-
### Fleet Configuration
|
|
983
|
-
|
|
984
|
-
Create `config/fleet.yaml`:
|
|
985
|
-
|
|
986
|
-
```yaml
|
|
987
|
-
fleet:
|
|
988
|
-
id: "my-project-fleet"
|
|
989
|
-
name: "My Project QE Fleet"
|
|
990
|
-
maxAgents: 20
|
|
991
|
-
topology: mesh
|
|
992
|
-
|
|
993
|
-
agents:
|
|
994
|
-
test-generator:
|
|
995
|
-
count: 3
|
|
996
|
-
config:
|
|
997
|
-
frameworks: [jest, cypress, playwright]
|
|
998
|
-
targetCoverage: 95
|
|
999
|
-
enablePatterns: true # Phase 2: Pattern matching
|
|
1000
|
-
enableLearning: true # Phase 2: Learning
|
|
1001
|
-
|
|
1002
|
-
coverage-analyzer:
|
|
1003
|
-
count: 2
|
|
1004
|
-
config:
|
|
1005
|
-
targetCoverage: 95
|
|
1006
|
-
optimizationAlgorithm: sublinear
|
|
1007
|
-
enableLearning: true # Phase 2: Learning
|
|
1008
|
-
|
|
1009
|
-
flaky-test-hunter:
|
|
1010
|
-
count: 1
|
|
1011
|
-
config:
|
|
1012
|
-
enableML: true # Phase 2: ML detection
|
|
1013
|
-
minRuns: 5
|
|
1014
|
-
passRateThreshold: 0.8
|
|
1015
|
-
|
|
1016
|
-
quality-analyzer:
|
|
1017
|
-
count: 2
|
|
1018
|
-
config:
|
|
1019
|
-
tools: [eslint, sonarqube, lighthouse]
|
|
1020
|
-
thresholds:
|
|
1021
|
-
coverage: 80
|
|
1022
|
-
complexity: 10
|
|
1023
|
-
maintainability: 65
|
|
1024
|
-
|
|
1025
|
-
# Phase 2: Learning Configuration
|
|
1026
|
-
learning:
|
|
1027
|
-
enabled: true
|
|
1028
|
-
learningRate: 0.1
|
|
1029
|
-
discountFactor: 0.95
|
|
1030
|
-
explorationRate: 0.3
|
|
1031
|
-
targetImprovement: 0.20
|
|
1032
|
-
|
|
1033
|
-
# Phase 2: Pattern Bank Configuration
|
|
1034
|
-
patterns:
|
|
1035
|
-
enabled: true
|
|
1036
|
-
minQuality: 0.8
|
|
1037
|
-
frameworks: [jest, mocha, cypress, vitest, jasmine, ava]
|
|
1038
|
-
autoExtract: true
|
|
1039
|
-
|
|
1040
|
-
# Phase 2: Flaky Detection Configuration
|
|
1041
|
-
flakyDetection:
|
|
1042
|
-
enableML: true
|
|
1043
|
-
minRuns: 5
|
|
1044
|
-
passRateThreshold: 0.8
|
|
1045
|
-
confidenceThreshold: 0.7
|
|
1046
|
-
```
|
|
1047
|
-
|
|
1048
|
-
---
|
|
1049
|
-
|
|
1050
|
-
## 🧪 Examples
|
|
1051
|
-
|
|
1052
|
-
### Example 1: Pattern-Based Test Generation
|
|
1053
|
-
|
|
1054
|
-
```typescript
|
|
1055
|
-
import { QEReasoningBank, PatternExtractor, TestGeneratorAgent } from 'agentic-qe';
|
|
1056
|
-
|
|
1057
|
-
// Initialize components
|
|
1058
|
-
const bank = new QEReasoningBank();
|
|
1059
|
-
const extractor = new PatternExtractor({ minQuality: 0.8 });
|
|
1060
|
-
|
|
1061
|
-
// Extract patterns from existing tests
|
|
1062
|
-
const patterns = await extractor.extractFromDirectory('./tests', {
|
|
1063
|
-
recursive: true,
|
|
1064
|
-
filePattern: '**/*.test.ts'
|
|
1065
|
-
});
|
|
1066
|
-
|
|
1067
|
-
console.log(`Extracted ${patterns.length} patterns`);
|
|
1068
|
-
|
|
1069
|
-
// Store patterns in bank
|
|
1070
|
-
for (const pattern of patterns) {
|
|
1071
|
-
await bank.storePattern(pattern);
|
|
1072
|
-
}
|
|
1073
|
-
|
|
1074
|
-
// Use patterns in test generation
|
|
1075
|
-
const testGen = new TestGeneratorAgent(
|
|
1076
|
-
{ agentId: 'test-gen-1', memoryStore },
|
|
1077
|
-
{
|
|
1078
|
-
targetCoverage: 95,
|
|
1079
|
-
framework: 'jest',
|
|
1080
|
-
usePatterns: true,
|
|
1081
|
-
reasoningBank: bank
|
|
1082
|
-
}
|
|
1083
|
-
);
|
|
1084
|
-
|
|
1085
|
-
const result = await testGen.execute({
|
|
1086
|
-
type: 'test-generation',
|
|
1087
|
-
payload: {
|
|
1088
|
-
sourceFile: 'src/user-service.ts',
|
|
1089
|
-
framework: 'jest'
|
|
1090
|
-
}
|
|
1091
|
-
});
|
|
1092
|
-
|
|
1093
|
-
console.log(`Generated ${result.testsGenerated} tests using ${result.patternsUsed.length} patterns`);
|
|
1094
|
-
```
|
|
1095
|
-
|
|
1096
|
-
### Example 2: Learning-Enhanced Coverage
|
|
1097
|
-
|
|
1098
|
-
```typescript
|
|
1099
|
-
import {
|
|
1100
|
-
CoverageAnalyzerAgent,
|
|
1101
|
-
LearningEngine,
|
|
1102
|
-
PerformanceTracker,
|
|
1103
|
-
SwarmMemoryManager
|
|
1104
|
-
} from 'agentic-qe';
|
|
1105
|
-
|
|
1106
|
-
// Initialize components
|
|
1107
|
-
const memory = new SwarmMemoryManager({ databasePath: './.aqe/memory.db' });
|
|
1108
|
-
await memory.initialize();
|
|
1109
|
-
|
|
1110
|
-
const learningEngine = new LearningEngine('coverage-1', memory, {
|
|
1111
|
-
enabled: true,
|
|
1112
|
-
learningRate: 0.1,
|
|
1113
|
-
explorationRate: 0.3
|
|
1114
|
-
});
|
|
1115
|
-
|
|
1116
|
-
const performanceTracker = new PerformanceTracker('coverage-1', memory, {
|
|
1117
|
-
targetImprovement: 0.20,
|
|
1118
|
-
snapshotWindow: 100
|
|
1119
|
-
});
|
|
1120
|
-
|
|
1121
|
-
await learningEngine.initialize();
|
|
1122
|
-
await performanceTracker.initialize();
|
|
1123
|
-
|
|
1124
|
-
// Create agent with learning
|
|
1125
|
-
const agent = new CoverageAnalyzerAgent(
|
|
1126
|
-
{
|
|
1127
|
-
agentId: 'coverage-1',
|
|
1128
|
-
memoryStore: memory
|
|
1129
|
-
},
|
|
1130
|
-
{
|
|
1131
|
-
targetCoverage: 95,
|
|
1132
|
-
algorithm: 'sublinear',
|
|
1133
|
-
enableLearning: true
|
|
1134
|
-
}
|
|
1135
|
-
);
|
|
1136
|
-
|
|
1137
|
-
// Execute 100 tasks to build learning data
|
|
1138
|
-
for (let i = 0; i < 100; i++) {
|
|
1139
|
-
const result = await agent.execute({
|
|
1140
|
-
type: 'coverage-analysis',
|
|
1141
|
-
payload: {
|
|
1142
|
-
coverageReport: `./coverage/report-${i}.json`,
|
|
1143
|
-
threshold: 95
|
|
1144
|
-
}
|
|
1145
|
-
});
|
|
1146
|
-
|
|
1147
|
-
console.log(`Task ${i + 1}/100: gaps=${result.gaps.length}`);
|
|
1148
|
-
}
|
|
1149
|
-
|
|
1150
|
-
// Check improvement
|
|
1151
|
-
const improvement = await performanceTracker.calculateImprovement();
|
|
1152
|
-
console.log(`\n🎯 Final Results:`);
|
|
1153
|
-
console.log(`Improvement Rate: ${improvement.improvementRate.toFixed(2)}%`);
|
|
1154
|
-
console.log(`Target Achieved: ${improvement.targetAchieved ? '✅ YES' : '❌ NO'}`);
|
|
1155
|
-
```
|
|
1156
|
-
|
|
1157
|
-
### Example 3: ML Flaky Detection
|
|
1158
|
-
|
|
1159
|
-
```typescript
|
|
1160
|
-
import { FlakyTestDetector } from 'agentic-qe';
|
|
1161
|
-
|
|
1162
|
-
const detector = new FlakyTestDetector({
|
|
1163
|
-
minRuns: 5,
|
|
1164
|
-
passRateThreshold: 0.8,
|
|
1165
|
-
confidenceThreshold: 0.7
|
|
1166
|
-
});
|
|
1167
|
-
|
|
1168
|
-
const flakyTests = await detector.detectFlakyTests(testHistory);
|
|
1169
|
-
|
|
1170
|
-
flakyTests.forEach(test => {
|
|
1171
|
-
console.log(`🔴 ${test.name}: ${(test.passRate * 100).toFixed(1)}%`);
|
|
1172
|
-
console.log(` Pattern: ${test.failurePattern}`);
|
|
1173
|
-
console.log(` Severity: ${test.severity}`);
|
|
1174
|
-
console.log(` Fix: ${test.recommendation.suggestedFix}`);
|
|
1175
|
-
});
|
|
1176
|
-
```
|
|
1177
|
-
|
|
1178
|
-
More examples in [examples/](examples/)
|
|
1179
|
-
|
|
1180
|
-
---
|
|
1181
|
-
|
|
1182
|
-
## 🐳 Docker Deployment
|
|
1183
|
-
|
|
1184
|
-
### Quick Start
|
|
1185
|
-
|
|
1186
|
-
```bash
|
|
1187
|
-
# Start with SQLite (development)
|
|
1188
|
-
docker-compose up -d
|
|
1189
|
-
|
|
1190
|
-
# Start with PostgreSQL (production)
|
|
1191
|
-
docker-compose --profile postgres up -d
|
|
1192
|
-
```
|
|
1193
|
-
|
|
1194
|
-
### Production Deployment
|
|
1195
|
-
|
|
1196
|
-
```bash
|
|
1197
|
-
# Configure production environment
|
|
1198
|
-
cp .env.example .env.production
|
|
1199
|
-
# Edit .env.production with secure credentials
|
|
1200
|
-
|
|
1201
|
-
# Deploy
|
|
1202
|
-
docker-compose -f docker-compose.yml -f docker-compose.prod.yml up -d
|
|
1203
|
-
```
|
|
1204
|
-
|
|
1205
|
-
---
|
|
1206
|
-
|
|
1207
|
-
## 🚀 Development
|
|
1208
|
-
|
|
1209
|
-
### Setup
|
|
1210
|
-
|
|
1211
|
-
```bash
|
|
1212
|
-
# Clone repository
|
|
1213
|
-
git clone https://github.com/proffesor-for-testing/agentic-qe.git
|
|
1214
|
-
cd agentic-qe
|
|
1215
|
-
|
|
1216
|
-
# Install dependencies
|
|
1217
|
-
npm install
|
|
1218
|
-
|
|
1219
|
-
# Build
|
|
1220
|
-
npm run build
|
|
1221
|
-
|
|
1222
|
-
# Run tests
|
|
1223
|
-
npm test
|
|
1224
|
-
```
|
|
1225
|
-
|
|
1226
|
-
### Available Scripts
|
|
1227
|
-
|
|
1228
|
-
| Script | Description |
|
|
1229
|
-
|--------|-------------|
|
|
1230
|
-
| `npm run build` | Compile TypeScript to JavaScript |
|
|
1231
|
-
| `npm run dev` | Development mode with hot reload |
|
|
1232
|
-
| `npm test` | Run all test suites |
|
|
1233
|
-
| `npm run test:unit` | Unit tests only |
|
|
1234
|
-
| `npm run test:integration` | Integration tests |
|
|
1235
|
-
| `npm run test:coverage` | Generate coverage report |
|
|
1236
|
-
| `npm run lint` | ESLint code checking |
|
|
1237
|
-
| `npm run lint:fix` | Auto-fix linting issues |
|
|
1238
|
-
| `npm run typecheck` | TypeScript type checking |
|
|
1239
|
-
|
|
1240
|
-
### Project Structure
|
|
1241
|
-
|
|
1242
|
-
```
|
|
1243
|
-
agentic-qe/
|
|
1244
|
-
├── src/
|
|
1245
|
-
│ ├── agents/ # Agent implementation classes (BaseAgent, LearningAgent, etc.)
|
|
1246
|
-
│ ├── core/ # Core fleet management
|
|
1247
|
-
│ │ ├── FleetManager.ts
|
|
1248
|
-
│ │ ├── Agent.ts
|
|
1249
|
-
│ │ ├── Task.ts
|
|
1250
|
-
│ │ ├── EventBus.ts
|
|
1251
|
-
│ │ └── MemoryManager.ts
|
|
1252
|
-
│ ├── learning/ # Phase 2: Learning system
|
|
1253
|
-
│ │ ├── LearningEngine.ts
|
|
1254
|
-
│ │ ├── PerformanceTracker.ts
|
|
1255
|
-
│ │ ├── ImprovementLoop.ts
|
|
1256
|
-
│ │ ├── FlakyTestDetector.ts
|
|
1257
|
-
│ │ └── FlakyPredictionModel.ts
|
|
1258
|
-
│ ├── reasoning/ # Phase 2: Pattern bank
|
|
1259
|
-
│ │ ├── QEReasoningBank.ts
|
|
1260
|
-
│ │ ├── PatternExtractor.ts
|
|
1261
|
-
│ │ └── PatternMatcher.ts
|
|
1262
|
-
│ ├── cli/ # Command-line interface
|
|
1263
|
-
│ ├── mcp/ # Model Context Protocol server
|
|
1264
|
-
│ ├── types/ # TypeScript type definitions
|
|
1265
|
-
│ └── utils/ # Shared utilities
|
|
1266
|
-
├── tests/ # Comprehensive test suites
|
|
1267
|
-
│ ├── unit/
|
|
1268
|
-
│ ├── integration/
|
|
1269
|
-
│ ├── e2e/
|
|
1270
|
-
│ ├── performance/
|
|
1271
|
-
│ └── benchmarks/ # Phase 2: Performance benchmarks
|
|
1272
|
-
├── examples/ # Usage examples
|
|
1273
|
-
├── docs/ # Documentation
|
|
1274
|
-
├── .claude/ # Agent & command definitions
|
|
1275
|
-
│ ├── agents/ # 17 QE agent definitions (+ 1 general-purpose)
|
|
1276
|
-
│ └── commands/ # 8 AQE slash commands
|
|
1277
|
-
└── config/ # Configuration files
|
|
1278
|
-
```
|
|
1279
|
-
|
|
1280
|
-
---
|
|
1281
|
-
|
|
1282
|
-
## 🤝 Contributing
|
|
1283
|
-
|
|
1284
|
-
We welcome contributions! Please see [CONTRIBUTING.md](CONTRIBUTING.md) for details.
|
|
1285
|
-
|
|
1286
|
-
### Quick Contribution Guide
|
|
1287
|
-
|
|
1288
|
-
1. Fork the repository
|
|
1289
|
-
2. Create a feature branch (`git checkout -b feature/amazing-feature`)
|
|
1290
|
-
3. Make your changes
|
|
1291
|
-
4. Add tests for new functionality
|
|
1292
|
-
5. Ensure all tests pass (`npm test`)
|
|
1293
|
-
6. Commit your changes (`git commit -m 'feat: add amazing feature'`)
|
|
1294
|
-
7. Push to your branch (`git push origin feature/amazing-feature`)
|
|
1295
|
-
8. Open a Pull Request
|
|
1296
|
-
|
|
1297
|
-
### Development Guidelines
|
|
1298
|
-
|
|
1299
|
-
- Follow the existing code style
|
|
1300
|
-
- Write comprehensive tests
|
|
1301
|
-
- Update documentation
|
|
1302
|
-
- Use conventional commits
|
|
1303
|
-
- Ensure TypeScript types are accurate
|
|
1304
|
-
|
|
1305
|
-
---
|
|
1306
|
-
|
|
1307
|
-
## 📝 License
|
|
1308
|
-
|
|
1309
|
-
This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.
|
|
1310
|
-
|
|
1311
|
-
---
|
|
1312
|
-
|
|
1313
|
-
## 🙏 Acknowledgments
|
|
1314
|
-
|
|
1315
|
-
- Built with TypeScript, Node.js, and better-sqlite3
|
|
1316
|
-
- Inspired by autonomous agent architectures and swarm intelligence
|
|
1317
|
-
- Integrates with Jest, Cypress, Playwright, k6, SonarQube, and more
|
|
1318
|
-
- Compatible with Claude Code via Model Context Protocol (MCP)
|
|
1319
|
-
|
|
1320
|
-
---
|
|
1321
|
-
|
|
1322
|
-
## 📞 Support
|
|
1323
|
-
|
|
1324
|
-
- **Documentation**: [docs/](docs/)
|
|
1325
|
-
- **Issues**: [GitHub Issues](https://github.com/proffesor-for-testing/agentic-qe/issues)
|
|
1326
|
-
- **Discussions**: [GitHub Discussions](https://github.com/proffesor-for-testing/agentic-qe/discussions)
|
|
1327
|
-
- **Email**: support@agentic-qe.com
|
|
1328
|
-
|
|
1329
|
-
---
|
|
1330
|
-
|
|
1331
|
-
## 🗺️ Roadmap
|
|
1332
|
-
|
|
1333
|
-
### Current (v1.1)
|
|
1334
|
-
- ✅ Learning System with Q-learning
|
|
1335
|
-
- ✅ Pattern Bank with cross-project sharing
|
|
1336
|
-
- ✅ ML Flaky Detection (100% accuracy)
|
|
1337
|
-
- ✅ Continuous Improvement Loop
|
|
1338
|
-
- ✅ 17 specialized QE agents
|
|
1339
|
-
- ✅ Multi-framework test execution
|
|
1340
|
-
- ✅ Real-time coverage analysis
|
|
1341
|
-
- ✅ MCP integration
|
|
1342
|
-
- ✅ Multi-model router (70-81% cost savings)
|
|
1343
|
-
|
|
1344
|
-
### Planned (v1.2)
|
|
1345
|
-
- 🔄 Web dashboard for visualization
|
|
1346
|
-
- 🔄 GraphQL API
|
|
1347
|
-
- 🔄 CI/CD integrations (GitHub Actions, GitLab CI)
|
|
1348
|
-
- 🔄 Enhanced pattern adaptation across frameworks
|
|
1349
|
-
- 🔄 Real-time collaboration features
|
|
1350
|
-
|
|
1351
|
-
### Future (v2.0)
|
|
1352
|
-
- 📋 Natural language test generation
|
|
1353
|
-
- 📋 Self-healing test suites
|
|
1354
|
-
- 📋 Multi-language support (Python, Java, Go)
|
|
1355
|
-
- 📋 Advanced analytics and insights
|
|
1356
|
-
- 📋 Cloud deployment support
|
|
1357
|
-
|
|
1358
|
-
---
|
|
1359
|
-
|
|
1360
|
-
<div align="center">
|
|
1361
|
-
|
|
1362
|
-
**Made with ❤️ by the Agentic QE Team**
|
|
1363
|
-
|
|
1364
|
-
[⭐ Star us on GitHub](https://github.com/proffesor-for-testing/agentic-qe) • [🐦 Follow on Twitter](https://twitter.com/agenticqe)
|
|
1365
|
-
|
|
1366
|
-
</div>
|