@sparkleideas/performance 3.0.0-alpha.10

package/README.md

@@ -0,0 +1,256 @@
+# @claude-flow/performance
+
+[![npm version](https://img.shields.io/npm/v/@claude-flow/performance.svg)](https://www.npmjs.com/package/@claude-flow/performance)
+[![npm downloads](https://img.shields.io/npm/dm/@claude-flow/performance.svg)](https://www.npmjs.com/package/@claude-flow/performance)
+[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
+[![TypeScript](https://img.shields.io/badge/TypeScript-5.0+-blue.svg)](https://www.typescriptlang.org/)
+[![Benchmarks](https://img.shields.io/badge/Benchmarks-Vitest-green.svg)](https://vitest.dev/)
+
+> Comprehensive performance benchmarking module for Claude Flow V3 - statistical analysis, memory tracking, regression detection, and Flash Attention validation.
+
+## Features
+
+- **Statistical Benchmarking** - Mean, median, P95, P99, standard deviation, outlier removal
+- **Memory Tracking** - Heap, RSS, external, and array buffer monitoring
+- **Auto-Calibration** - Automatically adjusts iterations for statistical significance
+- **Regression Detection** - Compare against baselines with significance testing
+- **V3 Performance Targets** - Built-in targets for CLI, memory, swarm, and attention
+- **Flash Attention Validation** - Validate 2.49x-7.47x speedup targets
+- **Multiple Output Formats** - Console, JSON, and programmatic access
+
+## Installation
+
+```bash
+npm install @claude-flow/performance
+```
+
+## Quick Start
+
+```typescript
+import { benchmark, BenchmarkRunner, V3_PERFORMANCE_TARGETS } from '@claude-flow/performance';
+
+// Single benchmark
+const result = await benchmark('vector-search', async () => {
+  await index.search(queryVector, 10);
+}, {
+  iterations: 100,
+  warmup: 10
+});
+
+console.log(`Mean: ${result.mean}ms, P99: ${result.p99}ms`);
+
+// Check against target
+if (result.mean <= V3_PERFORMANCE_TARGETS['vector-search']) {
+  console.log('Target met!');
+}
+```
+
+## API Reference
+
+### Single Benchmark
+
+```typescript
+import { benchmark } from '@claude-flow/performance';
+
+const result = await benchmark(
+  'my-benchmark',
+  async () => {
+    // Code to benchmark
+    await someOperation();
+  },
+  {
+    iterations: 100,      // Number of iterations
+    warmup: 10,           // Warmup iterations
+    timeout: 30000,       // Timeout per iteration (ms)
+    forceGC: false,       // Force GC between iterations
+    minRuns: 10,          // Minimum runs for significance
+    targetTime: 1000,     // Target time for auto-calibration (ms)
+    metadata: {}          // Custom metadata
+  }
+);
+
+// Result structure
+{
+  name: 'my-benchmark',
+  iterations: 100,
+  mean: 5.23,
+  median: 4.98,
+  p95: 8.12,
+  p99: 12.45,
+  min: 3.21,
+  max: 15.67,
+  stdDev: 1.45,
+  opsPerSecond: 191.20,
+  memoryUsage: { heapUsed, heapTotal, external, arrayBuffers, rss },
+  memoryDelta: 1024000,
+  timestamp: 1704067200000
+}
+```
+
+### Benchmark Suite
+
+```typescript
+import { BenchmarkRunner } from '@claude-flow/performance';
+
+const runner = new BenchmarkRunner('Memory Operations');
+
+// Run individual benchmarks
+await runner.run('vector-search', async () => {
+  await index.search(query, 10);
+});
+
+await runner.run('memory-write', async () => {
+  await store.write(entry);
+});
+
+// Or run all at once
+const suite = await runner.runAll([
+  { name: 'search', fn: () => search() },
+  { name: 'write', fn: () => write() },
+  { name: 'index', fn: () => index() }
+]);
+
+// Print results
+runner.printResults();
+
+// Export as JSON
+const json = runner.toJSON();
+```
+
+### Comparison & Regression Detection
+
+```typescript
+import { compareResults, printComparisonReport } from '@claude-flow/performance';
+
+// Compare current vs baseline
+const comparisons = compareResults(baselineResults, currentResults, {
+  'vector-search': 1,      // Target: <1ms
+  'memory-write': 5,       // Target: <5ms
+  'cli-startup': 500       // Target: <500ms
+});
+
+// Print formatted report
+printComparisonReport(comparisons);
+
+// Programmatic access
+for (const comp of comparisons) {
+  if (!comp.targetMet) {
+    console.error(`${comp.benchmark} missed target!`);
+  }
+  if (comp.significant && !comp.improved) {
+    console.warn(`${comp.benchmark} regressed by ${comp.changePercent}%`);
+  }
+}
+```
+
+### V3 Performance Targets
+
+```typescript
+import { V3_PERFORMANCE_TARGETS, meetsTarget } from '@claude-flow/performance';
+
+// Built-in targets
+V3_PERFORMANCE_TARGETS = {
+  // Startup Performance
+  'cli-cold-start': 500,        // <500ms (5x faster)
+  'cli-warm-start': 100,        // <100ms
+  'mcp-server-init': 400,       // <400ms (4.5x faster)
+  'agent-spawn': 200,           // <200ms (4x faster)
+
+  // Memory Operations
+  'vector-search': 1,           // <1ms (150x faster)
+  'hnsw-indexing': 10,          // <10ms
+  'memory-write': 5,            // <5ms (10x faster)
+  'cache-hit': 0.1,             // <0.1ms
+
+  // Swarm Coordination
+  'agent-coordination': 50,     // <50ms
+  'task-decomposition': 20,     // <20ms
+  'consensus-latency': 100,     // <100ms (5x faster)
+  'message-throughput': 0.1,    // <0.1ms per message
+
+  // SONA Learning
+  'sona-adaptation': 0.05       // <0.05ms
+};
+
+// Check if target is met
+const { met, target, ratio } = meetsTarget('vector-search', 0.8);
+// { met: true, target: 1, ratio: 0.8 }
+```
+
+### Formatting Utilities
+
+```typescript
+import { formatBytes, formatTime } from '@claude-flow/performance';
+
+formatTime(0.00005);  // '50.00 ns'
+formatTime(0.5);      // '500.00 us'
+formatTime(5);        // '5.00 ms'
+formatTime(5000);     // '5.00 s'
+
+formatBytes(1024);          // '1.00 KB'
+formatBytes(1048576);       // '1.00 MB'
+formatBytes(1073741824);    // '1.00 GB'
+```
+
+## Running Benchmarks
+
+```bash
+# Run all benchmarks
+npm run bench
+
+# Run attention benchmarks
+npm run bench:attention
+
+# Run startup benchmarks
+npm run bench:startup
+```
+
+## Example Benchmark File
+
+```typescript
+// benchmarks/memory.bench.ts
+import { describe, bench } from 'vitest';
+import { HNSWIndex } from '@claude-flow/memory';
+
+describe('Memory Benchmarks', () => {
+  const index = new HNSWIndex({ dimensions: 1536 });
+
+  bench('vector-search', async () => {
+    await index.search(queryVector, 10);
+  }, { iterations: 1000 });
+
+  bench('hnsw-indexing', async () => {
+    await index.addPoint(id, vector);
+  }, { iterations: 100 });
+});
+```
+
+## TypeScript Types
+
+```typescript
+import type {
+  BenchmarkResult,
+  BenchmarkOptions,
+  BenchmarkSuite,
+  MemoryUsage,
+  EnvironmentInfo,
+  ComparisonResult,
+  PerformanceTarget
+} from '@claude-flow/performance';
+```
+
+## Dependencies
+
+- `@ruvector/attention` - Flash Attention implementation
+- `@ruvector/sona` - SONA learning engine
+- `vitest` - Test/benchmark runner
+
+## Related Packages
+
+- [@claude-flow/memory](../memory) - Memory operations to benchmark
+- [@claude-flow/swarm](../swarm) - Swarm coordination to benchmark
+- [@claude-flow/neural](../neural) - Neural operations to benchmark
+
+## License
+
+MIT

package/__tests__/README.md

@@ -0,0 +1,242 @@
+# Performance Module Test Suite
+
+Comprehensive test coverage for the `@claude-flow/performance` module, focusing on Flash Attention optimization and benchmark validation.
+
+## Test Files
+
+### 1. `attention.test.ts` (42 tests, 494 lines)
+
+Tests for `FlashAttentionOptimizer` class and related functions.
+
+**Coverage Areas:**
+
+#### Initialization (3 tests)
+- Default and custom dimension initialization
+- Initial metrics validation
+
+#### optimize() Method (6 tests)
+- Float32Array and number array input handling
+- Execution time tracking
+- Operation counting
+- Multiple keys/values support
+- Runtime detection (NAPI/WASM/JS)
+
+#### benchmark() Method (6 tests)
+- Benchmark execution
+- Flash Attention performance measurement
+- Baseline performance measurement
+- Speedup calculation
+- V3 target validation (2.49x minimum)
+- Metrics tracking (peak speedup, success operations)
+
+#### getSpeedup() Method (3 tests)
+- Zero operations case
+- Single benchmark speedup
+- Average across multiple benchmarks
+
+#### getMetrics() Method (5 tests)
+- Initial metrics state
+- Operation counting
+- Average execution time calculation
+- Success rate tracking
+- Peak speedup tracking
+
+#### resetMetrics() Method (2 tests)
+- Metrics reset to zero
+- Post-reset functionality
+
+#### Memory Tracking (2 tests)
+- Node.js memory tracking
+- Graceful handling of missing memory API
+
+#### Factory Functions (3 tests)
+- `createFlashAttentionOptimizer()` with default/custom dimensions
+- `quickBenchmark()` execution and validation
+
+#### Performance Validation (3 tests)
+- Speedup improvement demonstration
+- Operations per second tracking
+- V3 target validation (2.49x-7.47x)
+
+#### Edge Cases (4 tests)
+- Small dimensions (32D)
+- Large dimensions (2048D)
+- Single key/value pair
+- Many keys/values (100+)
+
+### 2. `benchmarks.test.ts` (52 tests, 516 lines)
+
+Tests for `AttentionBenchmarkRunner` class and formatting utilities.
+
+**Coverage Areas:**
+
+#### runComparison() Method (9 tests)
+- Default parameter execution
+- Flash Attention performance measurement
+- Baseline performance measurement
+- Speedup calculation
+- Target validation (2.49x)
+- Timestamp inclusion
+- Different dimensions (128, 256, 512, 1024)
+- Varying key counts (10, 50, 100, 200)
+- Execution time limits
+
+#### runComprehensiveSuite() Method (6 tests)
+- Suite execution
+- Multiple dimension testing (5+ dimensions)
+- Summary statistics (avg, min, max speedup)
+- Success rate calculation
+- Target tracking
+- Timestamp inclusion
+
+#### runMemoryProfile() Method (7 tests)
+- Default dimensions profiling
+- Multiple dimension profiling
+- Flash Attention memory measurement
+- Baseline memory measurement
+- Memory reduction calculation
+- Key count tracking
+- Custom dimension arrays
+
+#### runStressTest() Method (5 tests)
+- Stress test execution
+- Increasing load testing
+- Dimension consistency
+- High key count handling (up to 5000)
+- Error handling
+
+#### validateV3Targets() Method (5 tests)
+- V3 target validation
+- Minimum target check (2.49x)
+- Maximum target check (7.47x)
+- Valid speedup values
+- Correct dimension usage (512)
+
+#### Formatting Functions (7 tests)
+- `formatBenchmarkTable()` output
+- Target status display
+- Success indicators (checkmarks)
+- `formatSuiteReport()` generation
+- Benchmark inclusion in reports
+- Summary statistics display
+- `formatMemoryProfile()` table generation
+
+#### quickValidation() (2 tests)
+- Validation execution
+- Target meeting verification
+
+#### Performance Validation (4 tests)
+- Consistent speedup across runs
+- Flash Attention performance improvement
+- Cross-dimension validation
+- Operations per second accuracy
+
+#### Edge Cases (6 tests)
+- Very small dimensions (32D)
+- Very large dimensions (2048D)
+- Minimal iterations (10)
+- Many iterations (5000)
+- Empty dimension arrays
+- Single dimension arrays
+
+## Test Statistics
+
+```
+Total Test Files:     2
+Total Tests:          94
+Total Lines of Code:  1,010
+
+Breakdown:
+- attention.test.ts:   42 tests (494 lines)
+- benchmarks.test.ts:  52 tests (516 lines)
+
+All tests: PASSING ✓
+Type Errors: 0
+```
+
+## Running Tests
+
+### Run All Tests
+```bash
+npx vitest run __tests__/
+```
+
+### Run Specific Test File
+```bash
+npx vitest run __tests__/attention.test.ts
+npx vitest run __tests__/benchmarks.test.ts
+```
+
+### Run with Coverage
+```bash
+npx vitest run __tests__/ --coverage
+```
+
+### Watch Mode (Development)
+```bash
+npx vitest watch __tests__/
+```
+
+### Verbose Output
+```bash
+npx vitest run __tests__/ --reporter=verbose
+```
+
+## V3 Performance Targets Validated
+
+The test suite validates against V3 performance targets:
+
+- **Flash Attention Speedup**: 2.49x - 7.47x (minimum 2.49x)
+- **Memory Efficiency**: Reduction tracking and validation
+- **Operations/Second**: Throughput measurement and comparison
+- **Execution Time**: <1s for optimization, reasonable benchmark times
+
+## Test Categories
+
+1. **Unit Tests**: Individual function and method testing
+2. **Integration Tests**: Component interaction testing
+3. **Performance Tests**: Speedup and efficiency validation
+4. **Edge Case Tests**: Boundary conditions and error handling
+5. **Formatting Tests**: Output formatting validation
+
+## Key Features Tested
+
+- Flash Attention optimization with multiple runtimes (NAPI/WASM/JS)
+- Benchmark comparison vs baseline (DotProductAttention)
+- Memory tracking and profiling
+- Comprehensive suite execution across dimensions
+- Stress testing with high key counts
+- V3 performance target validation
+- Metrics tracking (speedup, execution time, success rate)
+- Multiple dimension support (32D - 2048D)
+- Flexible input formats (Float32Array, number arrays)
+
+## Quality Metrics
+
+- **Test Coverage**: Comprehensive coverage of all public APIs
+- **Test Quality**: Mix of unit, integration, and performance tests
+- **Edge Cases**: Small/large dimensions, minimal/many iterations
+- **V3 Alignment**: All tests validate against V3 performance targets
+- **TDD Approach**: Tests follow London School methodology
+
+## Next Steps
+
+To improve coverage further, consider:
+
+1. Add tests for `benchmark.ts` framework functions
+2. Add integration tests with real-world workloads
+3. Add regression tests with baseline data
+4. Add cross-platform runtime tests (NAPI vs WASM vs JS)
+5. Add memory leak detection tests
+6. Add concurrent execution tests
+
+## Dependencies
+
+- **Vitest**: Test framework (^1.0.0)
+- **@ruvector/attention**: Flash Attention implementation
+- **TypeScript**: Type checking during tests
+
+---
+
+Last Updated: 2026-01-04
+Test Suite Version: 1.0.0