agentic-qe 2.3.5 → 2.5.0

package/.claude/agents/qx-partner.md

@@ -29,8 +29,15 @@ Mission: Solve oracle problems when quality criteria are unclear, find balance b
 - Memory coordination via AQE hooks
 - Learning protocol integration
 
+✅ Enhanced with Vibium (v2.2):
+- Live browser automation for real-time UX analysis via MCP
+- Automated competitor QX benchmarking across multiple sites
+- Visual evidence capture (screenshots) for UX validation
+- Runtime oracle detection by navigating actual user flows
+- Element interaction quality assessment (accessibility, bounding boxes)
+- Real browser testing for authentic user experience validation
+
 ⚠️ Partial:
-- Competitive analysis across competitor sites
 - Real-time collaboration with UX/QA agents
 
 ❌ Planned:
@@ -65,6 +72,11 @@ Batch memory operations for analyses, recommendations, and reports in single tra
 - **Testability Integration**: Combine with testability scoring (10 Principles) for holistic quality insights
 - **Comprehensive Reports**: Detailed markdown reports with findings, issues, and recommendations per heuristic
 - **Collaborative QX**: Coordinate with Visual Tester (UX) and Quality Analyzer (QA) agents
+- **Vibium Browser Automation**: Live browser control via MCP for real-time UX validation and competitor analysis
+- **Visual Evidence Capture**: Automated screenshot capture for UX issue documentation and validation
+- **Runtime Oracle Detection**: Navigate actual user flows to detect quality criteria conflicts in real-time
+- **Competitor QX Benchmarking**: Automated analysis across multiple competitor sites for comparative insights
+- **Element Quality Assessment**: Evaluate accessibility attributes, bounding boxes, and interaction quality
 </capabilities>
 
 <memory_namespace>
@@ -74,6 +86,7 @@ Reads:
 - aqe/qx/oracle-patterns - Known oracle problem patterns
 - aqe/testability/* - Testability scoring results for integration
 - aqe/learning/patterns/qx/* - Learned QX strategies
+- aqe/vibium/browser-sessions - Active Vibium browser sessions
 
 Writes:
 - aqe/qx/analysis-results - QX analysis results with scoring
@@ -81,11 +94,15 @@ Writes:
 - aqe/qx/recommendations - QX improvement recommendations
 - aqe/qx/impact-reports - Impact analysis reports
 - aqe/qx/balance-assessments - User vs business balance results
+- aqe/qx/screenshots - Visual evidence from Vibium browser automation
+- aqe/qx/competitor-benchmarks - Competitor QX analysis results
+- aqe/qx/runtime-flows - Recorded user flow analysis results
 
 Coordination:
 - aqe/qx/status - Current QX analysis status
 - aqe/qx/alerts - Critical oracle problem alerts
 - aqe/swarm/qx/* - Cross-agent coordination with Visual Tester and Quality Analyzer
+- aqe/vibium/coordination - Browser automation coordination state
 </memory_namespace>
 
 <learning_protocol>
@@ -180,24 +197,101 @@ mcp__agentic_qe__learning_store_pattern({
 - HTML for visual impact analysis with diagrams
 </output_format>
 
+<vibium_integration>
+## Vibium Browser Automation via MCP
+
+Vibium is integrated via MCP tools for live browser automation. Available tools:
+
+**Installation:**
+```bash
+claude mcp add vibium -- npx -y vibium
+```
+
+**MCP Tools:**
+- `browser_launch` - Launch headless Chrome browser
+- `browser_navigate` - Navigate to URL
+- `browser_find` - Find element by CSS selector
+- `browser_click` - Click element
+- `browser_type` - Type text into element
+- `browser_screenshot` - Capture PNG screenshot (returns base64)
+- `browser_quit` - Close browser gracefully
+
+**QX Analysis Integration Patterns:**
+
+1. **Live Oracle Detection:**
+```typescript
+// Navigate actual user flow to detect conflicts
+const browser = await browser_launch();
+await browser_navigate({ url: "https://example.com/checkout" });
+const addToCart = await browser_find({ selector: "[data-testid='add-to-cart']" });
+await browser_click({ elementId: addToCart.elementId });
+const screenshot = await browser_screenshot();
+// Analyze screenshot for UX friction points
+await browser_quit();
+```
+
+2. **Competitor Benchmarking:**
+```typescript
+// Automated QX comparison across competitors
+const competitors = ["competitor1.com", "competitor2.com", "competitor3.com"];
+const qxScores = [];
+for (const site of competitors) {
+  const browser = await browser_launch();
+  await browser_navigate({ url: `https://${site}` });
+  const qxScore = await analyzePageQX(browser);
+  qxScores.push({ site, score: qxScore });
+  await browser_quit();
+}
+```
+
+3. **Visual Evidence Capture:**
+```typescript
+// Document UX issues with screenshots
+const browser = await browser_launch();
+await browser_navigate({ url: "https://example.com/form" });
+const errorState = await browser_screenshot();
+// Store in memory: aqe/qx/screenshots/<issue-id>
+await memory_store({
+  key: "aqe/qx/screenshots/error-state-001",
+  value: { png: errorState.png, timestamp: Date.now() }
+});
+```
+
+**When to Use Vibium:**
+- ✅ Live UX validation on real websites
+- ✅ Competitor QX benchmarking automation
+- ✅ Visual evidence capture for oracle problems
+- ✅ Runtime accessibility attribute extraction
+- ✅ Element interaction quality assessment
+
+**Current Limitations (V1):**
+- ⚠️ Console/network APIs not yet available (use Playwright for performance metrics)
+- ⚠️ Manual element finding only (AI locators coming in V2)
+- ⚠️ Single browser instance per process
+</vibium_integration>
+
 <examples>
-Example 1: Oracle problem detection
+Example 1: Oracle problem detection with Vibium
 ```
-Input: Analyze checkout redesign for mystore.com
+Input: Analyze checkout redesign for mystore.com with live validation
 - Context: Mobile-first, conversion-focused
 - Stakeholders: Users, Business, Support
+- Mode: Live browser automation
 
 Output: QX Analysis Results
 - QX Score: 72/100 (C)
-- Oracle Problems: 2 detected
+- Oracle Problems: 2 detected (with visual evidence)
   1. HIGH: User convenience vs business revenue conflict
      - One-click checkout reduces upsell opportunities
      - Resolution: A/B test with revenue tracking
+     - Evidence: Screenshot showing no cross-sell section
   2. MEDIUM: Missing mobile usability data
      - Cannot validate mobile-first assumption
      - Resolution: Conduct mobile user research
+     - Evidence: Vibium captured desktop-first layout
 - Balance: Slightly favors business (User: 68, Business: 81)
 - Recommendations: 8 prioritized actions
+- Visual Evidence: 3 screenshots stored in aqe/qx/screenshots/
 ```
 
 Example 2: User-business balance analysis
@@ -239,6 +333,28 @@ Output: Impact Analysis
   - Combined Insight: Low observability may mask user anxiety issues
 - Net Impact Score: 62/100 (Proceed with caution)
 ```
+
+Example 4: Competitor QX benchmarking with Vibium
+```
+Input: Benchmark our checkout against top 3 competitors
+- Target: example.com/checkout
+- Competitors: competitor1.com, competitor2.com, competitor3.com
+- Mode: Automated Vibium analysis
+
+Output: Competitor Benchmark Results
+- Our QX Score: 78/100 (B)
+- Competitor Scores:
+  1. competitor1.com: 85/100 (B+) - Superior mobile UX
+  2. competitor2.com: 72/100 (C) - Poor error messaging
+  3. competitor3.com: 81/100 (B) - Better guest checkout flow
+- Oracle Problems Detected:
+  1. HIGH: We lack social login (all competitors have it)
+  2. MEDIUM: Our mobile layout is desktop-first
+- Competitive Gaps: 3 critical UX features missing
+- Recommendations: 5 high-impact improvements
+- Visual Evidence: 12 comparative screenshots
+- Time to Benchmark: 45 seconds (automated via Vibium)
+```
 </examples>
 
 <skills_available>

package/.claude/skills/testability-scoring/SKILL.md

@@ -1,17 +1,18 @@
 ---
 name: testability-scoring
-description: "AI-powered testability assessment using 10 principles of intrinsic testability with Playwright. Evaluates web applications against Observability, Controllability, Algorithmic Simplicity, Transparency, Stability, Explainability, Unbugginess, Smallness, Decomposability, and Similarity. Use when assessing software testability, evaluating test readiness, identifying testability improvements, or generating testability reports."
+description: "AI-powered testability assessment using 10 principles of intrinsic testability with Playwright and optional Vibium integration. Evaluates web applications against Observability, Controllability, Algorithmic Simplicity, Transparency, Stability, Explainability, Unbugginess, Smallness, Decomposability, and Similarity. Use when assessing software testability, evaluating test readiness, identifying testability improvements, or generating testability reports."
 category: testing-methodologies
 priority: high
 tokenEstimate: 1100
 agents: [qe-quality-analyzer, qx-partner, qe-visual-tester]
 implementation_status: optimized
-optimization_version: 2.1
-last_optimized: 2025-12-03
+optimization_version: 2.2
+last_optimized: 2025-12-12
 dependencies: []
 quick_reference_card: true
-tags: [testability, scoring, playwright, assessment, 10-principles, intrinsic-testability, james-bach, michael-bolton]
+tags: [testability, scoring, playwright, vibium, assessment, 10-principles, intrinsic-testability, james-bach, michael-bolton]
 contributor: "@fndlalit"
+vibium_integration: optional
 ---
 
 # Testability Scoring
@@ -175,6 +176,97 @@ const qxReport = await Task("Full QX Analysis", {
 
 ---
 
+## Vibium Integration (Optional)
+
+### Overview
+Vibium browser automation can be used alongside Playwright for enhanced testability assessment. While **Playwright remains the primary engine**, Vibium offers complementary capabilities for certain metrics.
+
+**Installation:**
+```bash
+claude mcp add vibium -- npx -y vibium
+```
+
+### Vibium-Enhanced Metrics
+
+| Principle | Vibium Enhancement | Benefit |
+|-----------|-------------------|---------|
+| **Observability** | Auto-wait duration tracking | Measures DOM stability (30s timeout, 100ms polling) |
+| **Controllability** | Element interaction success rate | Validates automation readiness via MCP |
+| **Stability** | Screenshot consistency | Visual regression detection for layout stability |
+| **Explainability** | Element attribute extraction | ARIA labels, semantic HTML validation |
+
+### When to Use Vibium
+
+✅ **USE Vibium for:**
+- Element stability metrics (auto-wait duration analysis)
+- Visual consistency checks (screenshot comparison)
+- MCP-native AI agent integration
+- Lightweight Docker images (400MB vs 1.2GB)
+
+❌ **USE Playwright for:**
+- Console error detection (Vibium V1 lacks console API)
+- Network performance metrics (BiDi network APIs coming in V2)
+- Comprehensive browser coverage (Firefox, Safari)
+- Production-proven stability (Vibium V1 released Dec 2024)
+
+### Hybrid Assessment Example
+
+```typescript
+// Testability assessment using both engines
+const assessment = {
+  // Playwright: Comprehensive metrics
+  playwright: await runPlaywrightAssessment(url),
+
+  // Vibium: Stability metrics
+  vibium: {
+    elementStability: await measureAutoWaitDuration(url),
+    visualConsistency: await compareScreenshots(url),
+    accessibilityAttributes: await extractARIALabels(url)
+  }
+};
+
+// Enhanced Observability Score
+const observability =
+  (assessment.playwright.consoleErrors * 0.6) +
+  (assessment.vibium.elementStability * 0.4);
+```
+
+### Vibium MCP Tools for Testability
+
+```typescript
+// 1. Element Stability Measurement
+const browser = await browser_launch();
+await browser_navigate({ url });
+const startTime = Date.now();
+const element = await browser_find({ selector: ".critical-element" });
+const autoWaitDuration = Date.now() - startTime;
+// Lower duration = better stability
+
+// 2. Visual Consistency Check
+const screenshot1 = await browser_screenshot();
+await browser_navigate({ url }); // Reload
+const screenshot2 = await browser_screenshot();
+const visualDiff = compareImages(screenshot1.png, screenshot2.png);
+// Lower diff = better stability
+
+// 3. Accessibility Attribute Extraction
+const elements = await browser_find({ selector: "button, a, input" });
+const ariaLabels = elements.map(el => el.attributes["aria-label"]);
+const semanticScore = (ariaLabels.filter(Boolean).length / elements.length) * 100;
+```
+
+### Migration Strategy
+
+**Current (V2.2):** Hybrid approach
+- Playwright: Primary engine for all 10 principles
+- Vibium: Optional enhancement for stability metrics
+
+**Future (V3.0):** When Vibium V2 ships
+- Evaluate Vibium as primary engine if:
+  - Console/Network APIs available
+  - Production stability proven
+  - Community adoption increases
+
 ## Agent Coordination Hints
 
 ### Memory Namespace
@@ -183,7 +275,8 @@ aqe/testability/
 ├── assessments/*       - Assessment results by URL
 ├── historical/*        - Historical scores for trend analysis
 ├── recommendations/*   - Improvement recommendations
-└── integration/*       - QX integration data
+├── integration/*       - QX integration data
+└── vibium/*           - Vibium-specific metrics (optional)
 ```
 
 ### Fleet Coordination
@@ -209,6 +302,8 @@ const testabilityFleet = await FleetManager.coordinate({
 | Partial results | Check console errors, increase network timeout |
 | Report not opening | Use `AUTO_OPEN=false`, open manually |
 | Config not updating | Use `TEST_URL` env var instead |
+| Vibium not available | Install via `claude mcp add vibium -- npx -y vibium` (optional) |
+| Hybrid mode errors | Vibium is optional; assessments work without it |
 
 ---
 
@@ -227,9 +322,15 @@ const testabilityFleet = await FleetManager.coordinate({
 
 ### Implementation
 - Based on https://github.com/fndlalit/testability-scorer (contributed by [@fndlalit](https://github.com/fndlalit))
-- Playwright v1.49.0+ with AI capabilities
+- Playwright v1.49.0+ with AI capabilities (primary engine)
+- Vibium v1.0+ with MCP integration (optional enhancement)
 - Chart.js for radar visualizations
 
+### Vibium Resources
+- GitHub: https://github.com/VibiumDev/vibium
+- MCP Integration: `claude mcp add vibium -- npx -y vibium`
+- Created by Jason Huggins (creator of Selenium/Appium)
+
 ---
 
 ## Remember

package/CHANGELOG.md

@@ -7,6 +7,139 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## [Unreleased]
 
+## [2.5.0] - 2025-12-13
+
+### Added
+
+#### AccessibilityAllyAgent - Intelligent Accessibility Testing (PR #129)
+*Contributed by [@fndlalit](https://github.com/fndlalit)*
+
+- **New Agent: `qe-a11y-ally`** - Comprehensive WCAG 2.2 compliance testing
+  - WCAG 2.2 Level A, AA, AAA validation using axe-core
+  - Context-aware ARIA label generation based on element semantics
+  - Intelligent remediation suggestions with code examples
+  - Keyboard navigation and screen reader testing
+  - Color contrast optimization with specific fix recommendations
+- **AI Video Analysis** - Multi-provider cascade for accessibility
+  - Vision API support: OpenAI → Anthropic → Ollama → moondream
+  - WebVTT caption generation for videos
+  - Automated audio description suggestions
+- **EU Compliance Support**
+  - EN 301 549 European accessibility standard mapping
+  - EU Accessibility Act compliance checking
+- **ARIA Authoring Practices Guide (APG)**
+  - Pattern suggestions for common UI components
+  - Accessible name computation (AccName)
+- **10 New MCP Accessibility Tools**
+  - `scan-comprehensive` - Full WCAG 2.2 scan
+  - `remediation-code-generator` - Auto-fix code generation
+  - `html-report-generator` - Detailed HTML reports
+  - `markdown-report-generator` - Markdown reports
+  - `video-vision-analyzer` - AI video accessibility analysis
+  - `webvtt-generator` - Caption file generation
+  - `accname-computation` - Accessible name calculation
+  - `apg-patterns` - ARIA pattern suggestions
+  - `en-301-549-mapping` - EU standard mapping
+  - `eu-accessibility-act` - EU Act compliance
+
+**Agent count increased from 19 → 20 QE agents**
+
+#### G4: Unified Memory Architecture - BinaryCache Integration
+- **BinaryCache Integration** with UnifiedMemoryCoordinator for TRM pattern caching
+- `cacheTRMPattern()` - Cache TRM patterns with binary serialization
+- `getCachedTRMPattern()` - Retrieve cached patterns with O(1) key access
+- `persistBinaryCache()` - Persist cache to disk with atomic writes
+- `getBinaryCacheMetrics()` - Cache statistics (hit rate, miss rate, entries)
+- `invalidateBinaryCache()` - Selective cache invalidation with triggers
+- 6x faster pattern loading compared to JSON serialization
+
+#### G6: OpenRouter Provider with Model Hot-Swap
+- **OpenRouterProvider** - Full `ILLMProvider` implementation for OpenRouter API
+  - 300+ model access via unified interface
+  - Model hot-swapping at runtime without restart
+  - Auto-routing with cost optimization (`auto` model)
+  - Vision, streaming, and embeddings support
+  - Cost tracking per model with request counting
+- **Smart Environment Detection** - Automatic provider selection
+  - Claude Code + ANTHROPIC_API_KEY → Claude
+  - OPENROUTER_API_KEY → OpenRouter (300+ models)
+  - ANTHROPIC_API_KEY → Claude
+  - ruvLLM available → Local inference
+- **LLMProviderFactory** enhancements
+  - `hotSwapModel(model)` - Switch models at runtime
+  - `getCurrentModel()` - Get active model name
+  - `listAvailableModels()` - List available OpenRouter models
+  - `detectEnvironment()` - Get environment signals
+- New helper functions in providers module:
+  - `createOpenRouterWithAutoRoute()` - Create auto-routing provider
+  - `hotSwapModel()`, `getCurrentModel()`, `listAvailableModels()`
+
+#### Environment Variables
+- `OPENROUTER_API_KEY` - OpenRouter API key
+- `OPENROUTER_DEFAULT_MODEL` - Default model (default: `auto`)
+- `OPENROUTER_SITE_URL` - Your site URL for rankings
+- `OPENROUTER_SITE_NAME` - Your site name
+- `LLM_PROVIDER` - Force specific provider (`claude`, `openrouter`, `ruvllm`, `auto`)
+
+### Files Added
+- `src/providers/OpenRouterProvider.ts` - OpenRouter provider (~500 LOC)
+- `tests/providers/OpenRouterProvider.test.ts` - 25 unit tests
+
+### Files Modified
+- `src/core/memory/UnifiedMemoryCoordinator.ts` - BinaryCache integration
+- `src/providers/LLMProviderFactory.ts` - OpenRouter + hot-swap + smart detection
+- `src/providers/index.ts` - New exports
+
+### Tests
+- OpenRouterProvider: 25 tests (metadata, init, completion, cost, hot-swap, discovery, health, embeddings, tokens, shutdown)
+
+## [2.4.0] - 2025-12-13
+
+### Added
+
+#### Binary Metadata Cache (Performance)
+- **BinaryMetadataCache** - MessagePack-serialized cache with 6x faster pattern loading
+- Lazy deserialization for O(1) key access without full cache decode
+- Automatic compression for entries > 1KB
+- File-based persistence with atomic writes
+- Stats tracking: hit rate, miss rate, eviction count
+- New file: `src/core/cache/BinaryMetadataCache.ts`
+
+#### AI-Friendly Output Mode
+- **AIOutputFormatter** - Structured JSON output optimized for AI consumption
+- `--ai-output` flag for CLI commands
+- `--ai-output-format` option: `json` (default), `yaml`, `markdown`
+- Schema-validated responses with metadata
+- New file: `src/output/AIOutputFormatter.ts`
+
+#### Automated Benchmarks in CI
+- **Benchmark Suite** - Comprehensive performance benchmarks
+- Automated baseline collection and regression detection
+- CI workflow integration with `benchmark.yml`
+- Historical tracking with JSON baselines
+- New files: `benchmarks/suite.ts`, `benchmarks/baseline-collector.ts`
+
+#### Strategy-Based Agent Architecture (Foundation)
+- **Strategy Pattern** for BaseAgent decomposition
+- LifecycleStrategy, MemoryStrategy, LearningStrategy, CoordinationStrategy interfaces
+- Adapter layer bridging existing services to strategies
+- BaseAgent reduced from 1,569 → 1,005 LOC (36% reduction)
+- Removed deprecated AgentDB direct methods
+- Simplified onPreTask/onPostTask hooks
+
+### Fixed
+- AdapterConfigValidator tests using correct `validateOrThrow()` method
+- QXPartnerAgent tests using correct `store/retrieve` memory methods
+- FleetCommanderAgent lifecycle test expecting IDLE after initialization
+- Added `getAgentId()` method for backward compatibility
+- Race condition tests updated for AgentDB adapter deprecation
+
+### Tests
+- 425 new tests for performance infrastructure
+- Strategy pattern tests: 92 passing
+- Agent tests: 166 passing
+- Adapter fail-fast tests: 17 passing
+
 ## [2.3.5] - 2025-12-12
 
 ### Added

package/README.md

@@ -9,9 +9,9 @@
 <img alt="NPM Downloads" src="https://img.shields.io/npm/dw/agentic-qe">
 
 
-**Version 2.3.5** | [Changelog](CHANGELOG.md) | [Contributors](CONTRIBUTORS.md) | [Issues](https://github.com/proffesor-for-testing/agentic-qe/issues) | [Discussions](https://github.com/proffesor-for-testing/agentic-qe/discussions)
+**Version 2.5.0** | [Changelog](CHANGELOG.md) | [Contributors](CONTRIBUTORS.md) | [Issues](https://github.com/proffesor-for-testing/agentic-qe/issues) | [Discussions](https://github.com/proffesor-for-testing/agentic-qe/discussions)
 
-> Agentic test automation with AI learning, real-time visualization, QUIC transport, testability scoring, OpenTelemetry observability, persistent event storage, constitutional AI governance, and intelligent model routing.
+> AI-powered test automation that learns from every task, switches between 300+ AI models on-the-fly, scores code testability, visualizes agent activity in real-time, and improves autonomously overnight — with built-in safety guardrails and full observability.
 
 🎨 **Real-Time Visualization** | 📊 **Testability Scoring** | 🧠 **QE Agent Learning** | 🚀 **QUIC Transport** | 📋 **Constitution System** | 📚 **41 QE Skills** | 🎯 **Flaky Detection** | 💰 **Multi-Model Router**
 
@@ -147,7 +147,7 @@ open http://localhost:3000
 ---
 
 ### 🤖 Autonomous Agent Fleet
-- **19 Specialized Agents**: Expert agents for every QE domain (test generation, coverage analysis, security scanning, performance testing, code complexity analysis, QX analysis)
+- **20 Specialized Agents**: Expert agents for every QE domain (test generation, coverage analysis, security scanning, performance testing, code complexity analysis, QX analysis, accessibility)
 - **11 TDD Subagents**: Specialized subagents for Test-Driven Development workflow (RED/GREEN/REFACTOR phases + quality validation + analysis)
 - **AI-Powered Coordination**: Event-driven architecture with intelligent task distribution
 - **Zero External Dependencies**: Native AQE hooks system (100-500x faster than external coordination)
@@ -390,7 +390,7 @@ mcp__agentic_qe__coverage_analyze({
 })
 ```
 
-**All 84 MCP Tools Available:**
+**All 85 MCP Tools Available:**
 - Fleet Management (9 tools): init, spawn, status, coordinate, orchestrate
 - Test Generation (2 tools): generate enhanced, execute
 - Test Execution (3 tools): execute, parallel, stream
@@ -579,13 +579,14 @@ Model Usage:
 </details>
 
 <details>
-<summary><b>Specialized (3 agents)</b></summary>
+<summary><b>Specialized (4 agents)</b></summary>
 
 | Agent | Purpose | Key Features |
 |-------|---------|-------------|
 | **deployment-readiness** | Release validation | Multi-factor risk scoring |
 | **visual-tester** | UI regression | AI-powered comparison |
 | **chaos-engineer** | Resilience testing | Fault injection, blast radius |
+| **a11y-ally** | Accessibility testing | WCAG 2.2, AI video analysis, EU compliance |
 
 </details>
 
@@ -598,7 +599,7 @@ Model Usage:
 
 </details>
 
-**Total: 30 Agents** (19 main agents + 11 TDD subagents)
+**Total: 31 Agents** (20 main agents + 11 TDD subagents)
 
 ### TDD Subagents (11 specialized)