agentic-qe 1.9.3 → 1.9.4

package/CHANGELOG.md

@@ -7,6 +7,60 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## [Unreleased]
 
+## [1.9.4] - 2025-11-30
+
+### 🔧 Critical Fixes: Memory/Learning/Patterns System
+
+This release delivers critical fixes to the memory, learning, and patterns system based on thorough investigation (Sherlock Investigation Report). All QE agents now have a fully functional learning system with proper vector embeddings, Q-value reinforcement learning, and persistent pattern storage.
+
+### Fixed
+
+- **Vector embeddings now stored correctly** (was storing NULL): Fixed `RealAgentDBAdapter.store()` to properly store 384-dimension embeddings as BLOB data instead of NULL
+- **SQL parameter style bug**: Fixed agentdb's `SqlJsDatabase` wrapper to use spread params (`stmt.run(a, b, c)`) instead of array params (`stmt.run([a,b,c])`) which caused "NOT NULL constraint failed" errors
+- **HNSW index schema mismatch**: Added `pattern_id` generated column for agentdb's HNSWIndex compatibility which requires this column for vector search
+- **Learning experience retrieval**: Added missing getter methods that were referenced but didn't exist
+- **Hooks saving to wrong database**: Fixed all Claude Code hooks to explicitly export `AGENTDB_PATH=.agentic-qe/agentdb.db` so learning data is saved to the project database instead of the root directory
+- **CI failures due to ARM64-only ruvector packages**: Moved `@ruvector/node-linux-arm64-gnu` and `ruvector-core-linux-arm64-gnu` from dependencies to optionalDependencies. Added x64 variants for CI compatibility
+
+### Added
+
+- **New SwarmMemoryManager methods for learning data retrieval**:
+  - `getBestAction(agentId, stateKey)` - Q-learning best action selection
+  - `getRecentLearningExperiences(agentId, limit)` - Recent experience retrieval
+  - `getLearningExperiencesByTaskType(agentId, taskType, limit)` - Task-filtered experiences
+  - `getHighRewardExperiences(agentId, minReward, limit)` - Successful experience extraction
+  - `getLearningStats(agentId)` - Aggregate learning statistics (total, avg, max, min rewards)
+
+- **Hooks integration**: Added `AGENTDB_PATH` environment variable to connect Claude Code hooks to the QE database
+
+- **New modules (Phase 4 Alerting & Reporting)**:
+  - `src/alerting/` - AlertManager, FeedbackRouter, StrategyApplicator (1,394 LOC)
+  - `src/reporting/` - ResultAggregator, reporters (3,030 LOC)
+  - Quality gate scripts and GitHub Actions workflow
+
+- **Integration test**: `tests/integration/memory-learning-loop.test.ts` - Comprehensive 7-phase test validating the full learning cycle:
+  1. Pattern storage with embeddings
+  2. Learning experience capture
+  3. Q-value reinforcement learning
+  4. Memory persistence
+  5. Pattern retrieval
+  6. Vector similarity search
+  7. Full learning loop simulation
+
+### Changed
+
+- **RealAgentDBAdapter**: Now properly retrieves stored embeddings when querying patterns instead of using placeholder values
+- **Pattern table schema**: Added generated column `pattern_id TEXT GENERATED ALWAYS AS (id) STORED` for HNSW compatibility
+
+### Technical Details
+
+- Vector embeddings: 384 dimensions × 4 bytes = 1,536 bytes per pattern
+- AgentDB version: v1.6.1 with ReasoningBank (16 learning tables)
+- HNSW index: 150x faster vector search enabled
+- All 12 integration tests pass
+
+---
+
 ## [1.9.3] - 2025-11-26
 
 ### 🐛 Bugfix: NPM Package Missing Files

package/README.md

@@ -9,11 +9,11 @@
 <img alt="NPM Downloads" src="https://img.shields.io/npm/dw/agentic-qe">
 
 
-**Version 1.9.3** (NPM Package Fix) | [Changelog](CHANGELOG.md) | [Issues](https://github.com/proffesor-for-testing/agentic-qe/issues) | [Discussions](https://github.com/proffesor-for-testing/agentic-qe/discussions)
+**Version 1.9.4** (Memory & Learning System Fixes) | [Changelog](CHANGELOG.md) | [Issues](https://github.com/proffesor-for-testing/agentic-qe/issues) | [Discussions](https://github.com/proffesor-for-testing/agentic-qe/discussions)
 
 > Agentic test automation with AI learning, real-time visualization, OpenTelemetry observability, persistent event storage, constitutional AI governance, and intelligent model routing.
 
-🎨 **Real-Time Visualization** | 📊 **Interactive Dashboards** | 🧠 **QE Agent Learning** | 💾 **Event Sourcing** | 📋 **Constitution System** | 📚 **40 QE Skills** | 🎯 **Flaky Detection** | 💰 **Multi-Model Router**
+🎨 **Real-Time Visualization** | 📊 **Interactive Dashboards** | 🧠 **QE Agent Learning** | 💾 **Event Sourcing** | 📋 **Constitution System** | 📚 **38 QE Skills** | 🎯 **Flaky Detection** | 💰 **Multi-Model Router**
 
 </div>
 
@@ -193,7 +193,7 @@ open http://localhost:3000
 - **Performance Testing**: k6, JMeter, Gatling integration
 - **Real-Time Streaming**: Live progress updates for all operations
 
-### 🎓 40 QE Skills Library (v1.9.0)
+### 🎓 38 QE Skills Library (v1.9.0)
 **95%+ coverage of modern QE practices**
 
 <details>
@@ -206,7 +206,7 @@ open http://localhost:3000
 - **Code Quality**: code-review-quality, refactoring-patterns, quality-metrics
 - **Communication**: bug-reporting-excellence, technical-writing, consultancy-practices
 
-**Phase 2: Expanded QE Skills Library (18 skills)**
+**Phase 2: Expanded QE Skills Library (16 skills)**
 - **Testing Methodologies (7)**: regression-testing, shift-left-testing, shift-right-testing, test-design-techniques, mutation-testing, test-data-management, verification-quality
 - **Specialized Testing (9)**: accessibility-testing, mobile-testing, database-testing, contract-testing, chaos-engineering-resilience, compatibility-testing, localization-testing, compliance-testing, visual-testing-advanced
 - **Testing Infrastructure (2)**: test-environment-management, test-reporting-analytics
@@ -214,7 +214,7 @@ open http://localhost:3000
 **Phase 3: Advanced Quality Engineering Skills (4 skills)**
 - **Strategic Testing Methodologies (4)**: six-thinking-hats, brutal-honesty-review, sherlock-review, cicd-pipeline-qe-orchestrator
 
-**Total: 40 QE Skills** - Includes accessibility testing, shift-left/right testing, verification & quality assurance, visual testing advanced, XP practices, and technical writing
+**Total: 38 QE Skills** - Includes accessibility testing, shift-left/right testing, verification & quality assurance, visual testing advanced, XP practices, and technical writing
 
 </details>
 
@@ -645,6 +645,31 @@ The test generator automatically delegates to subagents for a complete RED-GREEN
 
 ---
 
+## 📝 What's New in v1.9.4
+
+🔧 **Critical Memory & Learning System Fixes** (2025-11-30)
+
+This release delivers critical fixes to the memory, learning, and patterns system. All QE agents now have a fully functional learning system with proper vector embeddings, Q-value reinforcement learning, and persistent pattern storage.
+
+### Key Fixes
+
+- **Vector embeddings now stored correctly**: Fixed `RealAgentDBAdapter.store()` to properly store 384-dimension embeddings as BLOB data
+- **SQL parameter style bug**: Fixed agentdb's `SqlJsDatabase` wrapper to use spread params instead of array params
+- **HNSW index schema mismatch**: Added `pattern_id` generated column for agentdb's HNSWIndex compatibility
+- **Learning experience retrieval**: Added missing getter methods for Q-learning and experience replay
+- **Hooks saving to wrong database**: Fixed all Claude Code hooks to explicitly export `AGENTDB_PATH` so learning data is saved correctly
+- **CI platform compatibility**: Moved ARM64-only ruvector packages to optionalDependencies for x64 CI compatibility
+
+### New Features
+
+- **SwarmMemoryManager learning methods**: `getBestAction()`, `getRecentLearningExperiences()`, `getLearningStats()`, and more
+- **Phase 4 Alerting & Reporting**: AlertManager, FeedbackRouter, StrategyApplicator modules
+- **Quality Gate CI workflow**: GitHub Actions integration for automated quality validation
+
+**Upgrade**: `npm install agentic-qe@1.9.4`
+
+---
+
 ## 📝 What's New in v1.9.3
 
 📦 **NPM Package Fix** (2025-11-26)

package/config/.env.otel.example

@@ -0,0 +1,25 @@
+# OTEL Stack Environment Variables
+# Agentic QE Fleet - Issue #71
+#
+# Copy this file to .env.otel and customize as needed
+# Usage: docker-compose -f config/docker-compose.otel.yml --env-file config/.env.otel up -d
+
+# Deployment environment
+DEPLOYMENT_ENVIRONMENT=development
+
+# Grafana credentials (CHANGE IN PRODUCTION!)
+GRAFANA_ADMIN_USER=admin
+GRAFANA_ADMIN_PASSWORD=admin
+
+# OTEL Collector settings
+OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4318
+OTEL_EXPORTER_OTLP_PROTOCOL=http/protobuf
+
+# Service configuration
+SERVICE_NAME=agentic-qe-fleet
+SERVICE_NAMESPACE=agentic-qe
+SERVICE_VERSION=1.9.3
+
+# Prometheus retention
+PROMETHEUS_RETENTION_TIME=15d
+PROMETHEUS_RETENTION_SIZE=10GB

package/config/OTEL-QUICK-REFERENCE.md

@@ -0,0 +1,137 @@
+# OTEL Stack Quick Reference Card
+
+## 🚀 One-Line Start
+
+```bash
+docker-compose -f config/docker-compose.otel.yml up -d
+```
+
+## 🌐 Service URLs
+
+| Service | URL | Login |
+|---------|-----|-------|
+| **Grafana** | http://localhost:3001 | admin/admin |
+| **Prometheus** | http://localhost:9090 | - |
+| **Jaeger** | http://localhost:16686 | - |
+| **OTEL Health** | http://localhost:13133/health | - |
+
+## 📡 Send Telemetry
+
+### OTLP Endpoints
+- gRPC: `localhost:4317`
+- HTTP: `localhost:4318`
+
+### Node.js Example
+```javascript
+const exporter = new OTLPTraceExporter({
+  url: 'http://localhost:4318/v1/traces'
+});
+```
+
+### cURL Test
+```bash
+curl http://localhost:4318/v1/traces \
+  -H "Content-Type: application/json" \
+  -d @trace.json
+```
+
+## 🔍 Quick Checks
+
+```bash
+# Verify all services
+./scripts/verify-otel-stack.sh
+
+# Check health
+curl http://localhost:13133/health  # OTEL Collector
+curl http://localhost:9090/-/healthy  # Prometheus
+curl http://localhost:14269/  # Jaeger
+curl http://localhost:3001/api/health  # Grafana
+
+# View logs
+docker-compose -f config/docker-compose.otel.yml logs -f
+```
+
+## 🛠️ Common Commands
+
+```bash
+# Start
+docker-compose -f config/docker-compose.otel.yml up -d
+
+# Stop
+docker-compose -f config/docker-compose.otel.yml down
+
+# Restart
+docker-compose -f config/docker-compose.otel.yml restart
+
+# View status
+docker-compose -f config/docker-compose.otel.yml ps
+
+# Logs
+docker-compose -f config/docker-compose.otel.yml logs -f [service]
+
+# Remove everything (INCLUDING DATA!)
+docker-compose -f config/docker-compose.otel.yml down -v
+```
+
+## 📊 Port Reference
+
+### OTEL Collector
+- 4317 - OTLP gRPC
+- 4318 - OTLP HTTP
+- 8889 - Prometheus metrics
+- 13133 - Health check
+
+### Prometheus
+- 9090 - Web UI
+
+### Jaeger
+- 16686 - UI
+- 14269 - Metrics
+
+### Grafana
+- 3001 - Web UI
+
+## 🔧 Configuration Files
+
+- **Docker Compose**: `config/docker-compose.otel.yml`
+- **OTEL Collector**: `config/otel-collector-config.yaml.example`
+- **Prometheus**: `config/prometheus.yml.example`
+- **Grafana Datasources**: `config/grafana/provisioning/datasources/datasources.yml`
+- **Grafana Dashboards**: `config/grafana/provisioning/dashboards/dashboards.yml`
+
+## 📚 Documentation
+
+- **Quick Start**: `config/README-OTEL.md`
+- **Full Summary**: `docs/implementation-plans/issue-71-completion-summary.md`
+- **Architecture**: `docs/architecture/otel-stack-architecture.md`
+
+## 🐛 Quick Troubleshooting
+
+| Issue | Solution |
+|-------|----------|
+| Services not starting | Check logs: `docker-compose -f config/docker-compose.otel.yml logs` |
+| Port already in use | Change external port in `docker-compose.otel.yml` |
+| Grafana can't connect | Check datasources: Grafana → Configuration → Data Sources |
+| No metrics in Prometheus | Check targets: http://localhost:9090/targets |
+| No traces in Jaeger | Verify OTLP endpoint: `curl http://localhost:4318` |
+
+## 🎯 Grafana Datasources
+
+Pre-configured and auto-loaded:
+
+1. **Prometheus** (default) - `http://prometheus:9090`
+2. **Jaeger** - `http://jaeger:16686`
+3. **OTEL Collector Metrics** - `http://otel-collector:8889`
+
+## ✅ Verification Checklist
+
+- [ ] All 4 services running: `docker-compose ps`
+- [ ] Health checks passing: `./scripts/verify-otel-stack.sh`
+- [ ] OTLP endpoints accessible: `curl http://localhost:4318`
+- [ ] Prometheus targets green: http://localhost:9090/targets
+- [ ] Grafana datasources connected: Grafana UI → Data Sources
+- [ ] Sample dashboard visible: Grafana → Dashboards
+
+---
+
+**Issue #71 - COMPLETED** ✅

package/config/README-OTEL.md

@@ -0,0 +1,222 @@
+# OTEL Observability Stack - Quick Start Guide
+
+Complete observability stack for the Agentic QE Fleet with OpenTelemetry, Prometheus, Jaeger, and Grafana.
+
+## 🚀 Quick Start
+
+### 1. Start the OTEL Stack
+
+```bash
+# Start only the OTEL stack
+docker-compose -f config/docker-compose.otel.yml up -d
+
+# Or combine with the main application
+docker-compose -f docker-compose.yml -f config/docker-compose.otel.yml up -d
+```
+
+### 2. Access the Services
+
+| Service | URL | Purpose |
+|---------|-----|---------|
+| **Grafana** | http://localhost:3001 | Dashboards and visualization |
+| **Prometheus** | http://localhost:9090 | Metrics storage and querying |
+| **Jaeger UI** | http://localhost:16686 | Distributed tracing |
+| **OTEL Collector** | http://localhost:13133/health | Health check |
+
+### 3. Default Credentials
+
+- **Grafana**: `admin` / `admin` (change on first login)
+
+## 📊 Service Endpoints
+
+### OTEL Collector
+- **OTLP gRPC**: `localhost:4317` - Send traces/metrics via gRPC
+- **OTLP HTTP**: `localhost:4318` - Send traces/metrics via HTTP
+- **Prometheus Exporter**: `localhost:8889` - Metrics endpoint
+- **Health Check**: `localhost:13133` - Collector health
+- **pprof**: `localhost:1777` - Performance profiling
+- **zPages**: `localhost:55679` - Debug interface
+
+### Prometheus
+- **Web UI**: `localhost:9090` - Query and explore metrics
+- **API**: `localhost:9090/api/v1/` - Prometheus HTTP API
+
+### Jaeger
+- **UI**: `localhost:16686` - Trace visualization
+- **OTLP gRPC**: `localhost:4327` - Receive traces (forwarded from collector)
+- **Metrics**: `localhost:14269/metrics` - Jaeger metrics
+- **Health**: `localhost:14269/` - Health check
+
+### Grafana
+- **Web UI**: `localhost:3001` - Dashboards and visualization
+- **API**: `localhost:3001/api/` - Grafana HTTP API
+
+## 🔧 Configuration Files
+
+### Required Files (Already Created)
+- `config/docker-compose.otel.yml` - Docker Compose configuration
+- `config/otel-collector-config.yaml.example` - OTEL Collector config
+- `config/prometheus.yml.example` - Prometheus scrape config
+- `config/grafana/provisioning/datasources/datasources.yml` - Grafana datasources
+- `config/grafana/provisioning/dashboards/dashboards.yml` - Dashboard provisioning
+- `config/grafana/dashboards/agentic-qe-overview.json` - Sample dashboard
+
+### Environment Variables (Optional)
+Copy and customize:
+```bash
+cp config/.env.otel.example config/.env.otel
+```
+
+Then use:
+```bash
+docker-compose -f config/docker-compose.otel.yml --env-file config/.env.otel up -d
+```
+
+## 📈 Using the Stack
+
+### Send Telemetry to OTEL Collector
+
+#### Via HTTP (curl example)
+```bash
+curl -X POST http://localhost:4318/v1/traces \
+  -H "Content-Type: application/json" \
+  -d @trace-data.json
+```
+
+#### Via Node.js Application
+```javascript
+const { NodeTracerProvider } = require('@opentelemetry/sdk-trace-node');
+const { OTLPTraceExporter } = require('@opentelemetry/exporter-trace-otlp-http');
+
+const provider = new NodeTracerProvider();
+provider.addSpanProcessor(
+  new BatchSpanProcessor(
+    new OTLPTraceExporter({
+      url: 'http://localhost:4318/v1/traces'
+    })
+  )
+);
+provider.register();
+```
+
+### Query Metrics in Prometheus
+
+1. Open http://localhost:9090
+2. Try queries:
+   - `aqe_requests_total` - Total requests
+   - `rate(aqe_requests_total[5m])` - Request rate
+   - `histogram_quantile(0.95, rate(aqe_request_duration_bucket[5m]))` - P95 latency
+
+### View Traces in Jaeger
+
+1. Open http://localhost:16686
+2. Select service: `agentic-qe-fleet`
+3. Click "Find Traces"
+4. Explore trace details and service dependencies
+
+### Create Dashboards in Grafana
+
+1. Open http://localhost:3001
+2. Login with `admin` / `admin`
+3. Navigate to Dashboards → Agentic QE Fleet → Overview
+4. Or create new dashboards using Prometheus and Jaeger datasources
+
+## 🛠️ Management Commands
+
+### View Logs
+```bash
+# All services
+docker-compose -f config/docker-compose.otel.yml logs -f
+
+# Specific service
+docker-compose -f config/docker-compose.otel.yml logs -f otel-collector
+docker-compose -f config/docker-compose.otel.yml logs -f prometheus
+docker-compose -f config/docker-compose.otel.yml logs -f jaeger
+docker-compose -f config/docker-compose.otel.yml logs -f grafana
+```
+
+### Check Service Health
+```bash
+# OTEL Collector
+curl http://localhost:13133/health
+
+# Prometheus
+curl http://localhost:9090/-/healthy
+
+# Jaeger
+curl http://localhost:14269/
+
+# Grafana
+curl http://localhost:3001/api/health
+```
+
+### Stop Services
+```bash
+# Stop OTEL stack
+docker-compose -f config/docker-compose.otel.yml down
+
+# Stop and remove volumes (CAUTION: deletes all data)
+docker-compose -f config/docker-compose.otel.yml down -v
+```
+
+### Restart Services
+```bash
+# Restart all
+docker-compose -f config/docker-compose.otel.yml restart
+
+# Restart specific service
+docker-compose -f config/docker-compose.otel.yml restart otel-collector
+```
+
+## 🔍 Troubleshooting
+
+### OTEL Collector Not Receiving Data
+1. Check collector logs: `docker-compose -f config/docker-compose.otel.yml logs otel-collector`
+2. Verify endpoints: `curl http://localhost:13133/health`
+3. Check application OTLP endpoint: `http://localhost:4318`
+
+### Prometheus Not Scraping Metrics
+1. Check Prometheus targets: http://localhost:9090/targets
+2. Verify OTEL Collector is exposing metrics: `curl http://localhost:8889/metrics`
+3. Check Prometheus config: `docker-compose -f config/docker-compose.otel.yml exec prometheus cat /etc/prometheus/prometheus.yml`
+
+### Jaeger Not Showing Traces
+1. Check Jaeger logs: `docker-compose -f config/docker-compose.otel.yml logs jaeger`
+2. Verify OTEL Collector is forwarding traces (check collector logs)
+3. Ensure application is sending traces to OTLP endpoint
+
+### Grafana Datasources Not Working
+1. Check datasource configuration: Grafana UI → Configuration → Data Sources
+2. Test datasource connection (should show green checkmark)
+3. Verify Prometheus/Jaeger are accessible from Grafana container
+
+### Performance Issues
+1. Adjust OTEL Collector batch size in `otel-collector-config.yaml.example`
+2. Reduce Prometheus scrape interval in `prometheus.yml.example`
+3. Adjust memory limits for services in `docker-compose.otel.yml`
+
+## 📚 Next Steps
+
+1. **Integrate with Application**: Configure your app to send telemetry to OTLP endpoints
+2. **Create Custom Dashboards**: Build Grafana dashboards for your specific metrics
+3. **Set Up Alerting**: Configure Prometheus alerting rules (see Phase 4 docs)
+4. **Production Hardening**:
+   - Change default passwords
+   - Enable TLS/authentication
+   - Configure persistent storage
+   - Set up backup/restore procedures
+
+## 📖 Related Documentation
+
+- [OTEL Stack Architecture](../docs/architecture/otel-stack-architecture.md)
+- [Phase 4 Alerting Implementation Plan](../docs/implementation-plans/phase4-alerting-implementation-plan.md)
+- [OpenTelemetry Documentation](https://opentelemetry.io/docs/)
+- [Prometheus Documentation](https://prometheus.io/docs/)
+- [Jaeger Documentation](https://www.jaegertracing.io/docs/)
+- [Grafana Documentation](https://grafana.com/docs/)
+
+## 🐛 Issue Tracking
+
+This implementation resolves **Issue #71**: Complete OTEL Stack Docker Compose Configuration
+
+For issues or improvements, please file an issue on the repository.