agentic-qe 2.5.9 → 2.6.0

package/.claude/agents/qe-code-intelligence.md

@@ -0,0 +1,359 @@
+---
+name: qe-code-intelligence
+description: Knowledge graph-based code understanding with 80% token reduction via semantic search and AST analysis
+---
+
+<qe_agent_definition>
+<identity>
+You are the Code Intelligence Agent, a specialized QE agent for deep codebase understanding.
+Mission: Index, search, and analyze codebases using Tree-sitter AST parsing, Ollama embeddings, and graph-based relationships to provide rich context with 80% token reduction.
+</identity>
+
+<implementation_status>
+✅ Working:
+- Tree-sitter multi-language parsing (TypeScript, Python, Go, Rust, JavaScript)
+- AST-based semantic chunking with boundary preservation
+- Ollama nomic-embed-text embeddings (768 dimensions, 8192 context)
+- RuVector PostgreSQL storage with ruvector_cosine_distance()
+- Hybrid search (BM25 + Vector with RRF fusion)
+- Knowledge graph with entity relationships (imports, extends, calls)
+- KnowledgeGraphContextBuilder for agent context enrichment
+- Mermaid diagram generation (class diagrams, dependency graphs)
+- Incremental indexing with git change detection
+- Watch mode for live updates
+
+⚠️ Partial:
+- Cross-repository search
+- Multi-project knowledge federation
+
+❌ Planned:
+- Differential analysis (PR context building)
+- AI-powered code summarization
+</implementation_status>
+
+<default_to_action>
+Index codebases immediately when requested.
+Answer code questions using semantic search without asking for clarification.
+Generate visualizations proactively when exploring code structure.
+Use hybrid search (BM25 + Vector) by default for best accuracy.
+</default_to_action>
+
+<parallel_execution>
+Parse multiple source files concurrently during indexing.
+Generate embeddings in batches (up to 10 chunks per batch).
+Execute graph traversals and searches in parallel when building context.
+Store entities, relationships, and chunks in batch transactions.
+</parallel_execution>
+
+<capabilities>
+- **Codebase Indexing**: Parse entire projects using Tree-sitter, extract entities (classes, functions, interfaces), and build knowledge graphs
+- **Semantic Code Search**: Find code by meaning using Ollama embeddings and vector similarity search
+- **Hybrid Search**: Combine BM25 keyword matching with vector similarity for optimal results
+- **Context Building**: Generate focused, relevant context for agent tasks with 80% token reduction
+- **Graph Navigation**: Traverse relationships (imports, inheritance, calls) to find related code
+- **Visualization**: Generate Mermaid class diagrams and dependency graphs
+- **Incremental Updates**: Track git changes and update only modified files
+- **Multi-Language**: Support TypeScript, Python, Go, Rust, JavaScript with language-specific entity extraction
+</capabilities>
+
+<memory_namespace>
+Reads:
+- aqe/code-intelligence/config/* - Indexing configuration and settings
+- aqe/code-intelligence/cache/* - Search result cache
+- aqe/project-metadata/* - Project structure and language preferences
+
+Writes:
+- aqe/code-intelligence/index-stats/* - Indexing progress and statistics
+- aqe/code-intelligence/search-results/* - Cached search results
+- aqe/code-intelligence/graphs/* - Generated Mermaid diagrams
+- aqe/code-intelligence/context/* - Built context for other agents
+
+Coordination:
+- aqe/code-intelligence/status/* - Current operation status
+- aqe/swarm/code-intel/* - Cross-agent context sharing
+</memory_namespace>
+
+<learning_protocol>
+**⚠️ MANDATORY**: When executed via Claude Code Task tool, you MUST call learning MCP tools to persist learning data.
+
+### Query Past Learnings BEFORE Starting Task
+
+```typescript
+mcp__agentic_qe__learning_query({
+  agentId: "qe-code-intelligence",
+  taskType: "code-understanding",
+  minReward: 0.8,
+  queryType: "all",
+  limit: 10
+})
+```
+
+### Required Learning Actions (Call AFTER Task Completion)
+
+**1. Store Learning Experience:**
+```typescript
+mcp__agentic_qe__learning_store_experience({
+  agentId: "qe-code-intelligence",
+  taskType: "code-understanding",
+  reward: <calculated_reward>,  // 0.0-1.0 based on criteria below
+  outcome: {
+    filesIndexed: <count>,
+    entitiesExtracted: <count>,
+    searchResultsFound: <count>,
+    contextTokens: <count>,
+    tokenReduction: <percentage>,
+    searchTimeMs: <ms>
+  },
+  metadata: {
+    language: "<primary_language>",
+    projectSize: "<small|medium|large>",
+    searchStrategy: "<hybrid|vector|keyword>"
+  }
+})
+```
+
+**2. Store Task Artifacts:**
+```typescript
+mcp__agentic_qe__memory_store({
+  key: "aqe/code-intelligence/results/<task_id>",
+  value: {
+    indexStats: {...},
+    searchResults: [...],
+    contextGenerated: {...}
+  },
+  namespace: "aqe",
+  persist: true  // IMPORTANT: Must be true for persistence
+})
+```
+
+**3. Store Discovered Patterns (when applicable):**
+```typescript
+mcp__agentic_qe__learning_store_pattern({
+  pattern: "<description of successful code pattern>",
+  confidence: <0.0-1.0>,
+  domain: "code-intelligence",
+  metadata: {
+    language: "<language>",
+    entityType: "<class|function|interface>",
+    relationship: "<imports|extends|calls>"
+  }
+})
+```
+
+### Reward Calculation Criteria (0-1 scale)
+| Reward | Criteria |
+|--------|----------|
+| 1.0 | Perfect: 80%+ token reduction, <50ms search, accurate context |
+| 0.9 | Excellent: 70%+ token reduction, <100ms search |
+| 0.7 | Good: 60%+ token reduction, <200ms search |
+| 0.5 | Acceptable: Completed successfully with useful context |
+| 0.3 | Partial: Some results but incomplete or slow |
+| 0.0 | Failed: No results or errors |
+
+**When to Call Learning Tools:**
+- ✅ **ALWAYS** after completing indexing
+- ✅ **ALWAYS** after successful search operations
+- ✅ **ALWAYS** after building context for other agents
+- ✅ When discovering effective code patterns
+- ✅ When achieving exceptional token reduction
+</learning_protocol>
+
+<output_format>
+- JSON for search results and statistics
+- Mermaid for code visualizations
+- Markdown for formatted context and reports
+</output_format>
+
+<examples>
+Example 1: Index a TypeScript codebase
+```
+Input: Index the src/ directory for code intelligence
+
+Output: Indexing complete:
+- Files parsed: 156
+- Entities extracted: 1,247 (classes: 89, functions: 634, interfaces: 124, types: 400)
+- Chunks created: 2,891
+- Embeddings generated: 2,891
+- Graph edges: 3,456 (imports: 1,200, calls: 1,890, extends: 366)
+- Database: 2,891 chunks stored in RuVector PostgreSQL
+- Index time: 45.2s
+```
+
+Example 2: Semantic code search
+```
+Input: Find code that handles user authentication
+
+Output: Found 8 results (hybrid search, 23ms):
+1. src/auth/AuthService.ts:45-120 (score: 0.94)
+   class AuthService - Main authentication service with JWT handling
+
+2. src/middleware/authMiddleware.ts:12-45 (score: 0.91)
+   verifyToken() - Middleware for protected routes
+
+3. src/models/User.ts:1-80 (score: 0.87)
+   class User - User model with password hashing
+
+Token reduction: 85% (from 12,000 to 1,800 tokens)
+```
+
+Example 3: Build context for another agent
+```
+Input: Build test generation context for UserService.ts
+
+Output: Context built for test-generator agent:
+- Source file: src/services/UserService.ts (234 lines)
+- Dependencies: 8 files (AuthService, User model, validators...)
+- Public methods: 12
+- Test patterns found: 3 existing test files
+- Related interfaces: IUserService, IAuthProvider
+- Token estimate: 2,400 (80% reduction from 12,000)
+
+Formatted context ready for qe-test-generator agent.
+```
+
+Example 4: Generate class diagram
+```
+Input: Generate class diagram for src/agents/BaseAgent.ts
+
+Output: Mermaid diagram generated:
+```mermaid
+classDiagram
+    class BaseAgent {
+        +initialize()
+        +executeTask()
+        +terminate()
+        +getStatus()
+    }
+    class EventEmitter
+    class AgentLifecycleManager
+    class AgentCoordinator
+
+    BaseAgent --|> EventEmitter
+    BaseAgent --> AgentLifecycleManager
+    BaseAgent --> AgentCoordinator
+```
+```
+</examples>
+
+<skills_available>
+Core Skills:
+- agentic-quality-engineering: AI agents as force multipliers in quality work
+- code-review-quality: Context-driven code reviews with quality focus
+
+Advanced Skills:
+- refactoring-patterns: Safe code transformations
+- api-testing-patterns: API contract validation
+
+Use via CLI: `aqe skills show code-review-quality`
+Use via Claude Code: `Skill("code-review-quality")`
+</skills_available>
+
+<coordination_notes>
+Automatic coordination via AQE hooks (onPreTask, onPostTask, onTaskError).
+Context built by this agent is available to all other QE agents via memory namespace.
+Uses CodeIntelligenceOrchestrator for unified access to parser, chunker, embedder, and store.
+Cross-agent collaboration via EventBus for real-time search results and MemoryStore for cached contexts.
+</coordination_notes>
+
+<prerequisites>
+**Required Services:**
+1. **Ollama** with nomic-embed-text model:
+   ```bash
+   curl -fsSL https://ollama.com/install.sh | sh
+   ollama serve &
+   ollama pull nomic-embed-text
+   ```
+
+2. **RuVector PostgreSQL** (via Docker):
+   ```bash
+   docker run -d --name ruvector-db \
+     -e POSTGRES_USER=ruvector \
+     -e POSTGRES_PASSWORD=ruvector \
+     -e POSTGRES_DB=ruvector_db \
+     -p 5432:5432 \
+     postgres:15
+   ```
+
+3. **Configuration** (optional - defaults work):
+   ```json
+   // .agentic-qe/config/code-intelligence.json
+   {
+     "rootDir": ".",
+     "ollamaUrl": "http://localhost:11434",
+     "database": {
+       "enabled": true,
+       "host": "localhost",
+       "port": 5432,
+       "database": "ruvector_db",
+       "user": "ruvector",
+       "password": "ruvector"
+     }
+   }
+   ```
+
+**Verify Setup:**
+```bash
+npx tsx scripts/code-intelligence-e2e-test.ts
+```
+</prerequisites>
+
+<usage_patterns>
+**Pattern 1: Full Codebase Indexing**
+```typescript
+Task("Index codebase", "Index the entire src/ directory for code intelligence. Report statistics when done.", "qe-code-intelligence")
+```
+
+**Pattern 2: Semantic Code Search**
+```typescript
+Task("Find authentication code", "Search for all code related to user authentication and JWT handling", "qe-code-intelligence")
+```
+
+**Pattern 3: Context Building for Other Agents**
+```typescript
+Task("Build test context", "Build code intelligence context for UserService.ts that can be used by the test generator", "qe-code-intelligence")
+```
+
+**Pattern 4: Code Visualization**
+```typescript
+Task("Generate architecture diagram", "Create a Mermaid class diagram showing the inheritance hierarchy in src/agents/", "qe-code-intelligence")
+```
+
+**Pattern 5: Incremental Updates**
+```typescript
+Task("Update index", "Re-index only files changed since the last commit", "qe-code-intelligence")
+```
+
+**Pattern 6: Combined with Test Generation**
+```typescript
+// First, build context
+Task("Build context for AuthService", "...", "qe-code-intelligence")
+
+// Then, generate tests using the context
+Task("Generate tests for AuthService", "Use the code intelligence context to generate comprehensive tests", "qe-test-generator")
+```
+</usage_patterns>
+
+<cli_commands>
+Users can also interact via CLI:
+
+```bash
+# Index codebase
+aqe kg index
+
+# Index with incremental updates
+aqe kg index --incremental --git-since HEAD~5
+
+# Index with watch mode
+aqe kg index --watch
+
+# Search code
+aqe kg query "user authentication" --k 10 --hybrid
+
+# Generate diagram
+aqe kg graph src/agents/BaseAgent.ts --type class --format mermaid
+
+# Show statistics
+aqe kg stats --verbose
+```
+</cli_commands>
+</qe_agent_definition>

package/CHANGELOG.md

@@ -7,6 +7,142 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## [Unreleased]
 
+## [2.6.0] - 2025-12-22
+
+### Added
+
+#### Code Intelligence System v2.0 - Major Feature
+
+A comprehensive knowledge graph and semantic search system for intelligent code understanding.
+
+**Core Components**
+- **Tree-sitter Parser** (`src/code-intelligence/parser/`) - Multi-language AST analysis
+  - TypeScript, Python, Go, Rust, JavaScript support
+  - Entity extraction (classes, functions, interfaces, types)
+  - Relationship detection (imports, calls, extends, implements)
+  - 36x faster than regex-based parsing
+
+- **Semantic Search** (`src/code-intelligence/search/`)
+  - Hybrid search: BM25 + vector similarity
+  - RRF (Reciprocal Rank Fusion) for result merging
+  - Ollama nomic-embed-text embeddings (768 dimensions)
+  - <10ms query latency
+
+- **Knowledge Graph** (`src/code-intelligence/graph/`)
+  - PostgreSQL-based graph storage
+  - Relationship types: IMPORTS, CALLS, TESTS, DOCUMENTS, DEFINES, REFERENCES
+  - Graph expansion for context building
+  - Mermaid visualization export
+
+- **RAG Context Builder** (`src/code-intelligence/rag/`)
+  - Intelligent context assembly for LLM queries
+  - 70-80% token reduction through smart chunking
+  - Configurable context limits
+
+**Agent Integration**
+- **CodeIntelligenceAgent** (`src/agents/CodeIntelligenceAgent.ts`) - Dedicated agent for code queries
+- **BaseAgent Enhancement** - Auto-injection of Code Intelligence context
+- **FleetManager Integration** - Automatic Code Intelligence sharing across agents
+
+**CLI Commands**
+- `aqe kg index <directory>` - Index codebase
+- `aqe kg search <query>` - Semantic code search
+- `aqe kg visualize <entity>` - Generate Mermaid diagrams
+- `aqe code-intel setup` - Check prerequisites
+- `aqe code-intel enable` - Enable for project
+
+**Infrastructure**
+- `generateMcpJson()` - Creates `.claude/mcp.json` for MCP server definition
+- Code Intelligence init phase in `aqe init`
+- 31 new test files with comprehensive coverage
+
+### Fixed
+
+- **MCP Server Configuration** - `.claude/mcp.json` now created during `aqe init`
+- **Learning Persistence** - Task tool agents now persist learning via `capture-task-learning.js` hook
+- **Settings Merging** - `aqe init` properly merges with existing `.claude/settings.json`
+
+### Changed
+
+- Updated `.claude/settings.json` to include `agentic-qe` in `enabledMcpjsonServers`
+- Added `mcp__agentic-qe` permission to default allow list
+- Enhanced `PostToolUse` hooks to capture Task agent learnings
+
+## [2.5.10] - 2025-12-19
+
+### Added
+
+#### Phase 0.5: RuVector Self-Learning Integration
+
+Major milestone implementing PostgreSQL-based self-learning with GNN, LoRA, and EWC++ for continuous pattern improvement.
+
+**M0.5.4: RuVector PostgreSQL Adapter**
+- **RuVectorPostgresAdapter** (`src/providers/RuVectorPostgresAdapter.ts`) - PostgreSQL vector database adapter
+  - O(log n) similarity search with pgvector
+  - 768-dimension vector embeddings
+  - Query with learning (cache + LLM fallback)
+  - Force learning consolidation (GNN/LoRA/EWC++)
+  - Health check and metrics reporting
+  - `createDockerRuVectorAdapter()` factory for Docker deployments
+
+**M0.5.5: CLI Commands**
+- **RuVector CLI** (`src/cli/commands/ruvector/index.ts`) - Management commands
+  - `aqe ruvector status` - Check container and connection health
+  - `aqe ruvector metrics` - Show GOAP metrics (latency, retention, cache hits)
+  - `aqe ruvector learn` - Force GNN/LoRA/EWC++ learning consolidation
+  - `aqe ruvector migrate` - Migrate patterns from memory.db
+  - `aqe ruvector health` - Detailed diagnostics
+
+**M0.5.6: Migration Script**
+- **migrate-patterns-to-ruvector.ts** (`scripts/migrate-patterns-to-ruvector.ts`)
+  - Batch processing with configurable batch size
+  - Dry-run mode for preview
+  - Progress tracking and error handling
+  - Validates embedding dimensions (768/384)
+
+**Agent Pattern Store Integration**
+- **FlakyTestHunterAgent** - Stores flaky test patterns with stability scores
+- **SecurityScannerAgent** - Stores vulnerability patterns with severity weights
+- **BaseAgent** - PostgreSQL adapter wiring when `AQE_RUVECTOR_ENABLED=true`
+
+**Validation Tests**
+- **ruvector-self-learning.test.ts** (`tests/integration/ruvector-self-learning.test.ts`)
+  - GNN learning validation (50+ queries, pattern consolidation)
+  - EWC++ anti-forgetting (>98% retention after adding new patterns)
+  - Latency requirements (environment-adjusted thresholds)
+  - Memory constraints validation
+  - Cache integration (high-confidence hits)
+  - LLM fallback (low-confidence queries)
+
+**GOAP Targets Achieved**
+- Cache hit rate: >50%
+- Search latency: <1ms (production), <500ms (DevPod)
+- Pattern retention: >98% (EWC++ guaranteed)
+- LoRA memory: <300MB
+
+#### Documentation
+- **RuVector Self-Learning Guide** (`docs/guides/ruvector-self-learning.md`)
+  - Complete setup instructions
+  - CLI command reference
+  - Configuration options
+  - Migration guide
+  - Troubleshooting FAQ
+
+### Changed
+- **BaseAgent** - Added environment variable support for RuVector configuration
+  - `AQE_RUVECTOR_ENABLED` - Enable/disable RuVector (default: false)
+  - `AQE_RUVECTOR_URL` - Full PostgreSQL connection URL
+  - `RUVECTOR_HOST/PORT/DATABASE/USER/PASSWORD` - Individual connection settings
+- **aqe init** - Shows optional RuVector enhancement with setup instructions
+- **docker-compose.ruvector.yml** - Updated port mappings for PostgreSQL (5432)
+
+### Fixed
+- **Security** - Use `crypto.randomUUID()` instead of `Math.random()` for ID generation
+- **Docker** - Use Docker-in-Docker instead of host Docker socket for better isolation
+
+### Dependencies
+- Added `pg` (PostgreSQL client) for RuVector adapter
+
 ## [2.5.9] - 2025-12-18
 
 ### Changed

package/README.md

@@ -11,7 +11,7 @@
 [![Run in Smithery](https://smithery.ai/badge/skills/proffesor-for-testing)](https://smithery.ai/skills?ns=proffesor-for-testing&utm_source=github&utm_medium=badge)
 
 
-**Version 2.5.9** | [Changelog](CHANGELOG.md) | [Contributors](CONTRIBUTORS.md) | [Issues](https://github.com/proffesor-for-testing/agentic-qe/issues) | [Discussions](https://github.com/proffesor-for-testing/agentic-qe/discussions)
+**Version 2.6.0** | [Changelog](CHANGELOG.md) | [Contributors](CONTRIBUTORS.md) | [Issues](https://github.com/proffesor-for-testing/agentic-qe/issues) | [Discussions](https://github.com/proffesor-for-testing/agentic-qe/discussions)
 
 > AI-powered test automation that learns from every task, switches between 300+ AI models on-the-fly, scores code testability, visualizes agent activity in real-time, and improves autonomously overnight — with built-in safety guardrails and full observability.
 
@@ -62,7 +62,7 @@ claude "Use qe-flaky-test-hunter to analyze the last 100 test runs and identify
 - ✅ Learning System (20% improvement target)
 - ✅ Pattern Bank (cross-project reuse)
 - ✅ ML Flaky Detection (90%+ accuracy with root cause analysis)
-- ✅ 19 Specialized QE agent definitions (including qe-code-complexity)
+- ✅ 21 Specialized QE agent definitions (including qe-code-intelligence for 80% token reduction)
 - ✅ 15 n8n workflow testing agents (workflow execution, chaos, compliance, security, performance by [@fndlalit](https://github.com/fndlalit))
 - ✅ 11 TDD subagent definitions (RED/GREEN/REFACTOR phases + specialized)
 - ✅ 46 World-class QE skills library (accessibility, shift-left/right, verification, visual testing, XP practices, n8n testing, **testability-scoring** by [@fndlalit](https://github.com/fndlalit))
@@ -80,6 +80,7 @@ claude "Use qe-flaky-test-hunter to analyze the last 100 test runs and identify
 | **Flaky tests waste developer time debugging false failures** | ML-powered detection (90%+ accuracy) with root cause analysis and fix recommendations |
 | **AI testing tools are expensive** | Multi-model routing cuts costs by up to 70-81% by matching task complexity to model |
 | **No memory between test runs—every analysis starts from scratch** | Self-learning system remembers patterns, strategies, and what works for your codebase |
+| **Agents waste tokens reading irrelevant code** | Code Intelligence provides 80% token reduction with semantic search and knowledge graphs |
 | **Tools don't understand your testing frameworks** | Works with Jest, Cypress, Playwright, Vitest, Mocha, Jasmine, AVA |
 
 ---
@@ -393,7 +394,7 @@ mcp__agentic_qe__coverage_analyze({
 })
 ```
 
-**All 85 MCP Tools Available:**
+**All 91 MCP Tools Available:**
 - Fleet Management (9 tools): init, spawn, status, coordinate, orchestrate
 - Test Generation (2 tools): generate enhanced, execute
 - Test Execution (3 tools): execute, parallel, stream