e11y 0.1.0 → 0.2.0

data/benchmarks/OPTIMIZATION.md

@@ -0,0 +1,246 @@
+# E11y Performance Optimization Strategies
+
+**Status:** Conditional (apply only if benchmarks fail)
+
+This document outlines optimization strategies to apply if performance benchmarks don't meet targets.
+
+## 📊 Benchmark First
+
+**ALWAYS run benchmarks before optimizing:**
+
+```bash
+bundle exec ruby benchmarks/e11y_benchmarks.rb
+```
+
+If all targets are met → **no optimization needed** ✅
+
+If some targets fail → apply strategies below ⚙️
+
+## 🎯 Optimization Strategies
+
+### 1. Memory Optimization
+
+**If memory usage exceeds targets:**
+
+#### 1.1 Reduce Object Allocations
+
+```ruby
+# BAD: Creates new hash on every call
+def event_data
+  { name: @name, value: @value, timestamp: Time.now }
+end
+
+# GOOD: Reuse hash
+def event_data
+  @event_data ||= {}
+  @event_data[:name] = @name
+  @event_data[:value] = @value
+  @event_data[:timestamp] = Time.now
+  @event_data
+end
+```
+
+#### 1.2 Optimize Buffer Size
+
+Current default: 10,000 events
+
+```ruby
+# Tune buffer size based on scale
+E11y.configure do |config|
+  config.buffer_size = case scale
+                       when :small then 1_000
+                       when :medium then 5_000
+                       when :large then 10_000
+                       end
+end
+```
+
+#### 1.3 Pool Reusable Objects
+
+```ruby
+# Event object pooling
+class EventPool
+  def initialize(size: 100)
+    @pool = Array.new(size) { E11y::Event::Base.new }
+    @mutex = Mutex.new
+  end
+
+  def checkout
+    @mutex.synchronize { @pool.pop || E11y::Event::Base.new }
+  end
+
+  def checkin(event)
+    @mutex.synchronize { @pool.push(event) if @pool.size < 100 }
+  end
+end
+```
+
+### 2. CPU Optimization
+
+**If CPU overhead exceeds targets:**
+
+#### 2.1 Reduce Regex Matches
+
+```ruby
+# BAD: Regex on hot path
+def matches_pattern?(name)
+  name =~ /^user\./
+end
+
+# GOOD: String prefix check
+def matches_pattern?(name)
+  name.start_with?("user.")
+end
+```
+
+#### 2.2 Cache Pattern Compilation
+
+```ruby
+# BAD: Compile regex every time
+def filter_events(events)
+  events.select { |e| e.name =~ /^(user|admin)\./ }
+end
+
+# GOOD: Compile once
+PATTERN = /^(user|admin)\./
+
+def filter_events(events)
+  events.select { |e| e.name =~ PATTERN }
+end
+```
+
+#### 2.3 Optimize JSON Serialization
+
+```ruby
+# Use Oj gem for faster JSON
+require "oj"
+
+module E11y
+  module Adapters
+    class File < Base
+      def format_event(event_data)
+        Oj.dump(event_data, mode: :compat)
+      end
+    end
+  end
+end
+
+# Add to gemspec:
+# spec.add_dependency "oj", "~> 3.16"
+```
+
+### 3. I/O Optimization
+
+**If adapter latency exceeds targets:**
+
+#### 3.1 Increase Batching
+
+```ruby
+E11y.configure do |config|
+  # Increase batch size for better I/O efficiency
+  config.flush_threshold = 1000  # Flush every 1000 events
+  config.flush_interval = 5.0    # Or every 5 seconds
+end
+```
+
+#### 3.2 Connection Pooling Tuning
+
+```ruby
+# For LokiAdapter
+E11y::Adapters::Loki.new(
+  url: "http://loki:3100",
+  connection_pool_size: 10,  # Increase for high throughput
+  timeout: 5
+)
+```
+
+#### 3.3 Compression Optimization
+
+```ruby
+# Use faster compression level
+E11y.configure do |config|
+  config.compression = {
+    enabled: true,
+    algorithm: :zstd,  # Faster than gzip
+    level: 1           # Fast compression (vs level 3)
+  }
+end
+```
+
+### 4. Profiling Tools
+
+**Use Ruby profilers to find bottlenecks:**
+
+#### 4.1 Memory Profiler
+
+```ruby
+require "memory_profiler"
+
+report = MemoryProfiler.report do
+  10_000.times { MyEvent.track(user_id: "123") }
+end
+
+report.pretty_print
+```
+
+#### 4.2 Stackprof (CPU profiler)
+
+```bash
+gem install stackprof
+
+# Add to benchmark:
+require "stackprof"
+
+StackProf.run(mode: :cpu, out: "tmp/stackprof.dump") do
+  # Your benchmark code
+end
+
+# Analyze:
+stackprof tmp/stackprof.dump --text
+```
+
+#### 4.3 RubyProf
+
+```ruby
+require "ruby-prof"
+
+result = RubyProf.profile do
+  # Your benchmark code
+end
+
+printer = RubyProf::FlatPrinter.new(result)
+printer.print(STDOUT)
+```
+
+## 🔧 Implementation Checklist
+
+If benchmarks fail, apply optimizations in this order:
+
+1. **Identify bottleneck** (use profilers)
+2. **Apply targeted fix** (don't optimize everything)
+3. **Re-run benchmarks** (verify improvement)
+4. **Repeat** (until targets met)
+
+## 📈 Performance Targets Reminder
+
+- **Small (1K/sec)**: track() <50μs p99, Buffer 10K/sec, Memory <100MB
+- **Medium (10K/sec)**: track() <1ms p99, Buffer 50K/sec, Memory <500MB
+- **Large (100K/sec)**: track() <5ms p99, Buffer 200K/sec, Memory <2GB
+
+## ⚠️ Anti-Patterns
+
+**Don't optimize prematurely:**
+- ❌ Don't apply optimizations "just in case"
+- ❌ Don't micro-optimize without profiling
+- ✅ Measure first, optimize second
+
+**Don't sacrifice readability:**
+- ❌ Don't make code unreadable for 1% gain
+- ✅ Keep code maintainable, profile-driven optimizations only
+
+## 📚 Additional Resources
+
+- [Ruby Performance Optimization (Book)](https://pragprog.com/titles/adrpo/ruby-performance-optimization/)
+- [benchmark-ips gem](https://github.com/evanphx/benchmark-ips)
+- [memory_profiler gem](https://github.com/SamSaffron/memory_profiler)
+- [stackprof gem](https://github.com/tmm1/stackprof)

data/benchmarks/README.md

@@ -0,0 +1,103 @@
+# E11y Performance Benchmarks
+
+Performance benchmark suite for E11y gem, testing at 3 scale levels.
+
+## 🎯 Performance Targets
+
+Based on **ADR-001 §5: Performance Requirements**
+
+### Small Scale (1K events/sec)
+- `track()` latency: **<50μs** (p99)
+- Buffer throughput: **10K events/sec**
+- Memory usage: **<100MB**
+- CPU overhead: **<5%**
+
+### Medium Scale (10K events/sec)
+- `track()` latency: **<1ms** (p99)
+- Buffer throughput: **50K events/sec**
+- Memory usage: **<500MB**
+- CPU overhead: **<10%**
+
+### Large Scale (100K events/sec)
+- `track()` latency: **<5ms** (p99)
+- Buffer throughput: **200K events/sec**
+- Memory usage: **<2GB**
+- CPU overhead: **<15%**
+
+## 🚀 Running Benchmarks
+
+### Run all scales
+
+```bash
+bundle exec ruby benchmarks/e11y_benchmarks.rb
+```
+
+### Run specific scale
+
+```bash
+# Small scale (1K events/sec)
+SCALE=small bundle exec ruby benchmarks/e11y_benchmarks.rb
+
+# Medium scale (10K events/sec)
+SCALE=medium bundle exec ruby benchmarks/e11y_benchmarks.rb
+
+# Large scale (100K events/sec)
+SCALE=large bundle exec ruby benchmarks/e11y_benchmarks.rb
+```
+
+### Run via runner
+
+```bash
+bundle exec ruby benchmarks/run_all.rb
+```
+
+## 📊 Metrics Collected
+
+1. **track() Latency**
+   - p50, p99, p999 percentiles (microseconds)
+   - Min, max, mean values
+
+2. **Buffer Throughput**
+   - Events per second
+   - Measured over sustained period (3-10 seconds)
+
+3. **Memory Usage**
+   - Total allocated memory (MB)
+   - Memory per event (KB)
+   - Object allocations and retentions
+
+4. **CPU Overhead**
+   - Informational only (manual profiling recommended)
+
+## ✅ Success Criteria
+
+Benchmarks **PASS** if all metrics meet targets for each scale.
+
+Exit codes:
+- `0` - All benchmarks passed ✅
+- `1` - Some benchmarks failed ❌
+
+## 🔧 Dependencies
+
+```ruby
+# Required gems (already in gemspec)
+gem "benchmark-ips", "~> 2.13"
+gem "memory_profiler", "~> 1.0"
+```
+
+## 📝 Notes
+
+- Benchmarks use `InMemory` adapter to eliminate I/O overhead
+- GC is triggered before memory profiling for clean measurements
+- CPU percentage is approximate (use external profilers for accuracy)
+- Results may vary based on Ruby version and hardware
+
+## 🎓 CI Integration
+
+In CI/CD:
+
+```yaml
+- name: Run performance benchmarks
+  run: bundle exec ruby benchmarks/e11y_benchmarks.rb
+  # Fails CI if benchmarks don't meet targets
+```

data/benchmarks/allocation_profiling.rb

@@ -0,0 +1,253 @@
+#!/usr/bin/env ruby
+# frozen_string_literal: true
+
+# E11y Allocation Profiling Script
+#
+# Purpose: Measure actual object allocations per event and verify
+#          we're meeting Ruby's best practices for minimal allocations.
+#
+# Usage:
+#   bundle exec ruby benchmarks/allocation_profiling.rb
+#
+# Expected results:
+# - Ruby minimum: 2 allocations per event (kwargs Hash + return Hash)
+# - E11y target: Close to theoretical minimum (2-5 allocations/event)
+# - For 1K events: 2K-5K allocations total
+
+require "bundler/setup"
+require "memory_profiler"
+require "e11y"
+
+# ============================================================================
+# Configuration
+# ============================================================================
+
+# Test with different event counts
+TEST_CASES = [
+  { count: 10, name: "10 events (warmup)" },
+  { count: 100, name: "100 events" },
+  { count: 1000, name: "1K events (DoD requirement)" },
+  { count: 10_000, name: "10K events" }
+].freeze
+
+# ============================================================================
+# Test Event Class
+# ============================================================================
+
+class AllocationTestEvent < E11y::Event::Base
+  # Minimal schema to isolate allocation sources
+  schema do
+    required(:value).filled(:integer)
+  end
+
+  # Skip validation for pure allocation measurement
+  validation_mode :never
+end
+
+# ============================================================================
+# Setup E11y (minimal config)
+# ============================================================================
+
+E11y.configure do |config|
+  config.enabled = true
+  config.adapters = [E11y::Adapters::InMemory.new]
+end
+
+# ============================================================================
+# Profiling Functions
+# ============================================================================
+
+def measure_allocations(event_count:)
+  # Warmup (avoid cold start allocations)
+  10.times { AllocationTestEvent.track(value: 1) }
+
+  # Force GC to start clean
+  GC.start
+
+  # Profile allocations
+  report = MemoryProfiler.report do
+    event_count.times do |i|
+      AllocationTestEvent.track(value: i)
+    end
+  end
+
+  {
+    total_allocated: report.total_allocated,
+    total_retained: report.total_retained,
+    total_allocated_memsize: report.total_allocated_memsize,
+    total_retained_memsize: report.total_retained_memsize,
+    per_event_allocated: (report.total_allocated.to_f / event_count).round(2),
+    per_event_retained: (report.total_retained.to_f / event_count).round(2),
+    per_event_memsize_bytes: (report.total_allocated_memsize.to_f / event_count).round(2)
+  }
+end
+
+def detailed_allocation_report(event_count:)
+  puts "\n#{'=' * 80}"
+  puts "  DETAILED ALLOCATION REPORT (#{event_count} events)"
+  puts "=" * 80
+
+  # Warmup
+  10.times { AllocationTestEvent.track(value: 1) }
+  GC.start
+
+  report = MemoryProfiler.report do
+    event_count.times do |i|
+      AllocationTestEvent.track(value: i)
+    end
+  end
+
+  puts "\n📊 Allocation Summary:"
+  puts "  Total allocated: #{report.total_allocated} objects"
+  puts "  Total retained:  #{report.total_retained} objects"
+  puts "  Memory size:     #{(report.total_allocated_memsize / 1024.0).round(2)} KB"
+  puts ""
+  puts "  Per-event allocated: #{(report.total_allocated.to_f / event_count).round(2)} objects"
+  puts "  Per-event retained:  #{(report.total_retained.to_f / event_count).round(2)} objects"
+  puts "  Per-event memory:    #{(report.total_allocated_memsize.to_f / event_count).round(2)} bytes"
+
+  puts "\n📍 Top Allocation Sources (by object count):"
+  report.allocated_memory_by_location.first(10).each_with_index do |(location, data), index|
+    puts "  #{index + 1}. #{location}"
+    puts "     Objects: #{data[:count]}"
+  end
+
+  puts "\n🔍 Allocations by Class:"
+  report.allocated_memory_by_class.first(10).each_with_index do |(klass, data), index|
+    puts "  #{index + 1}. #{klass}: #{data[:count]} objects"
+  end
+
+  # Leak detection
+  if report.total_retained.positive?
+    puts "\n⚠️  MEMORY LEAK WARNING:"
+    puts "  #{report.total_retained} objects retained (not garbage collected)"
+    puts ""
+    puts "  Retained objects by class:"
+    report.retained_memory_by_class.first(5).each_with_index do |(klass, data), index|
+      puts "    #{index + 1}. #{klass}: #{data[:count]} objects"
+    end
+  else
+    puts "\n✅ No memory leaks detected (0 retained objects)"
+  end
+
+  report
+end
+
+# ============================================================================
+# Verification Functions
+# ============================================================================
+
+def verify_allocation_target(result:, event_count:)
+  per_event = result[:per_event_allocated]
+  total = result[:total_allocated]
+
+  puts "\n🎯 Target Verification:"
+  puts ""
+
+  # Ruby theoretical minimum: 2 allocations per event
+  ruby_minimum = 2.0
+  is_near_minimum = per_event <= (ruby_minimum * 2.5) # 5 allocations = 2.5x minimum (reasonable overhead)
+
+  puts "  Ruby theoretical minimum:  #{ruby_minimum} allocations/event"
+  puts "  E11y actual:               #{per_event} allocations/event"
+  puts "  Overhead:                  #{(((per_event / ruby_minimum) * 100) - 100).round(1)}%"
+  puts ""
+
+  if is_near_minimum
+    puts "  ✅ GOOD: Near Ruby's theoretical minimum (< 2.5x)"
+  elsif per_event <= 10
+    puts "  ⚠️  ACCEPTABLE: Within reasonable range (< 10 allocations/event)"
+  else
+    puts "  ❌ CONCERN: High allocation count (> 10 allocations/event)"
+  end
+
+  # DoD requirement check (for context)
+  dod_target = 100 # <100 allocations per 1K events
+  dod_actual = (total.to_f / event_count * 1000).round
+
+  puts ""
+  puts "  DoD requirement (1K events): < #{dod_target} allocations"
+  puts "  E11y extrapolated (1K):      #{dod_actual} allocations"
+
+  if dod_actual <= dod_target
+    puts "  ✅ MEETS DoD requirement"
+  else
+    puts "  ❌ EXCEEDS DoD requirement (#{dod_actual - dod_target} over)"
+    puts "     Note: DoD target may be unrealistic for Ruby (see audit findings)"
+  end
+end
+
+def check_for_leaks(result:)
+  retained = result[:total_retained]
+
+  puts "\n🔍 Memory Leak Check:"
+  if retained.zero?
+    puts "  ✅ PASS: No memory leaks (0 retained objects)"
+  else
+    puts "  ❌ FAIL: #{retained} objects retained (potential memory leak)"
+  end
+end
+
+# ============================================================================
+# Main Execution
+# ============================================================================
+
+def main
+  puts "🚀 E11y Allocation Profiling"
+  puts "Audit: FEAT-4918 - Zero-Allocation Pattern Verification"
+  puts "Ruby: #{RUBY_VERSION}"
+  puts ""
+
+  results = {}
+
+  # Run all test cases
+  TEST_CASES.each do |test_case|
+    count = test_case[:count]
+    name = test_case[:name]
+
+    puts "\n#{'=' * 80}"
+    puts "  Testing: #{name}"
+    puts "=" * 80
+
+    result = measure_allocations(event_count: count)
+    results[count] = result
+
+    puts ""
+    puts "  Total allocated:   #{result[:total_allocated]} objects"
+    puts "  Total retained:    #{result[:total_retained]} objects"
+    puts "  Per-event alloc:   #{result[:per_event_allocated]} objects"
+    puts "  Per-event memory:  #{result[:per_event_memsize_bytes]} bytes"
+  end
+
+  # Detailed report for 1K events (DoD requirement)
+  detailed_allocation_report(event_count: 1000)
+
+  # Verification for 1K events
+  verify_allocation_target(result: results[1000], event_count: 1000)
+  check_for_leaks(result: results[1000])
+
+  # Summary
+  puts "\n#{'=' * 80}"
+  puts "  SUMMARY"
+  puts "=" * 80
+  puts ""
+
+  result_1k = results[1000]
+  per_event = result_1k[:per_event_allocated]
+
+  if per_event <= 5 && result_1k[:total_retained].zero?
+    puts "  ✅ EXCELLENT: Near-optimal allocations, no leaks"
+    exit 0
+  elsif per_event <= 10 && result_1k[:total_retained].zero?
+    puts "  ⚠️  ACCEPTABLE: Reasonable allocations, no leaks"
+    exit 0
+  elsif result_1k[:total_retained].positive?
+    puts "  ❌ FAIL: Memory leak detected"
+    exit 1
+  else
+    puts "  ❌ FAIL: High allocation count"
+    exit 1
+  end
+end
+
+main if __FILE__ == $PROGRAM_NAME