brainzlab-rails 0.1.1

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA256:
+  metadata.gz: 7ef611a0687f342dd6ae9cbf4b556ed91a8d8c66a0b38654896e841989ef1976
+  data.tar.gz: 3947b85d71ddfd34125ead8424b7ff15c97ca00c365204f3dd572aa072faee0c
+SHA512:
+  metadata.gz: 4d83e3d7ede72cd4e5fe53f3482c99f589b1e2125885dc3d5e1b23bce56a926c488a9f0daf9efa01cd27b1939392581954458b3da4c8832a0148f99d29bf07ef
+  data.tar.gz: 45d5f5a44d18119f68472dfe3f87d3b6665f6763a07f7aa94fd8fdc91e44e0cb3efe07540686d17d6e0f3b8004feb49b42af7e78433337c61098e6e7289b5856

data/CLAUDE.md

@@ -0,0 +1,144 @@
+# CLAUDE.md
+
+This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
+
+## Project: BrainzLab Rails Instrumentation
+
+Rails-native observability powered by ActiveSupport::Notifications. This gem hooks into ALL Rails instrumentation events and routes them intelligently to BrainzLab products.
+
+**Gem**: brainzlab-rails (on RubyGems.org)
+
+**GitHub**: brainz-lab/brainzlab-rails
+
+## Architecture
+
+```
+┌─────────────────────────────────────────────────────────────────┐
+│                    BRAINZLAB-RAILS GEM                          │
+│                                                                  │
+│  ┌──────────────────────────────────────────────────────────┐  │
+│  │              ActiveSupport::Notifications                 │  │
+│  │         (monotonic_subscribe for accurate timing)         │  │
+│  └──────────────────────────────────────────────────────────┘  │
+│                              │                                  │
+│                              ▼                                  │
+│  ┌──────────────────────────────────────────────────────────┐  │
+│  │                    Event Router                           │  │
+│  │           Routes events to appropriate products           │  │
+│  └──────────────────────────────────────────────────────────┘  │
+│                              │                                  │
+│        ┌─────────────────────┼─────────────────────┐           │
+│        │                     │                     │           │
+│        ▼                     ▼                     ▼           │
+│  ┌──────────┐         ┌──────────┐         ┌──────────┐       │
+│  │Collectors│         │Collectors│         │Analyzers │       │
+│  │  AC, AV  │         │  AR, AJ  │         │  N+1,    │       │
+│  │  Cable   │         │ Mailer   │         │SlowQuery │       │
+│  └──────────┘         └──────────┘         └──────────┘       │
+│                              │                                  │
+│                              ▼                                  │
+│  ┌──────────────────────────────────────────────────────────┐  │
+│  │                   BrainzLab SDK                           │  │
+│  │           Pulse • Recall • Reflex • Flux                  │  │
+│  └──────────────────────────────────────────────────────────┘  │
+└─────────────────────────────────────────────────────────────────┘
+```
+
+## Directory Structure
+
+```
+lib/
+├── brainzlab-rails.rb          # Main entry point
+└── brainzlab/
+    └── rails/
+        ├── version.rb
+        ├── configuration.rb    # Gem configuration
+        ├── subscriber.rb       # Event subscription (monotonic)
+        ├── event_router.rb     # Routes events to collectors
+        ├── railtie.rb          # Rails auto-initialization
+        ├── collectors/         # Event processors
+        │   ├── base.rb
+        │   ├── action_controller.rb
+        │   ├── action_view.rb
+        │   ├── active_record.rb
+        │   ├── active_job.rb
+        │   ├── action_cable.rb
+        │   ├── action_mailer.rb
+        │   ├── active_storage.rb
+        │   └── cache.rb
+        └── analyzers/          # Intelligent analysis
+            ├── n_plus_one_detector.rb
+            ├── slow_query_analyzer.rb
+            └── cache_efficiency.rb
+```
+
+## Key Features
+
+### 1. Monotonic Subscribe
+Uses `ActiveSupport::Notifications.monotonic_subscribe` for accurate timing instead of wall-clock time.
+
+### 2. Smart Event Routing
+Each Rails event is routed to appropriate products:
+- **Pulse**: APM spans for all performance-critical events
+- **Recall**: Structured logs for requests, jobs, emails
+- **Reflex**: Breadcrumbs and error context
+- **Flux**: Metrics (counters, histograms, timing)
+- **Nerve**: Job-specific monitoring
+
+### 3. Built-in Analyzers
+- **N+1 Detection**: Automatically detects repeated queries
+- **Slow Query Analyzer**: Identifies slow queries with suggestions
+- **Cache Efficiency**: Tracks hit rates and efficiency
+
+## Rails Events Covered
+
+| Component | Events |
+|-----------|--------|
+| Action Controller | 12 events |
+| Action View | 4 events |
+| Active Record | 5 events |
+| Active Job | 8 events |
+| Action Cable | 5 events |
+| Action Mailer | 3 events |
+| Active Storage | 12 events |
+| Cache | 15 events |
+| **Total** | **64+ events** |
+
+## Usage
+
+```ruby
+# Gemfile
+gem 'brainzlab-rails'
+
+# config/initializers/brainzlab.rb
+BrainzLab.configure do |config|
+  config.secret_key = ENV['BRAINZLAB_SECRET_KEY']
+end
+
+# That's it! Auto-starts via Railtie
+```
+
+## Configuration
+
+```ruby
+# config/application.rb
+config.brainzlab_rails.n_plus_one_detection = true
+config.brainzlab_rails.slow_query_threshold_ms = 100
+config.brainzlab_rails.sample_rate = 1.0
+config.brainzlab_rails.ignored_actions = ['HealthController#check']
+```
+
+## Common Commands
+
+```bash
+bundle install
+bundle exec rspec
+gem build brainzlab-rails.gemspec
+gem push brainzlab-rails-*.gem
+```
+
+## Dependencies
+
+- `brainzlab` gem (>= 0.1.4)
+- `rails` (>= 7.0)
+- `activesupport` (>= 7.0)

data/IMPLEMENTATION_PLAN.md

@@ -0,0 +1,370 @@
+# BrainzLab Rails Instrumentation - Implementation Plan
+
+## Overview
+
+This document outlines the complete implementation plan for the `brainzlab-rails` gem and required enhancements to the BrainzLab ecosystem.
+
+## Current State
+
+### Completed (Phase 1)
+
+- [x] **Gem Structure** - Created `brainzlab-rails` gem with proper gemspec
+- [x] **Subscriber** - Implemented `monotonic_subscribe` for accurate timing
+- [x] **Event Router** - Routes 64+ Rails events to appropriate products
+- [x] **Collectors** - All 8 collectors implemented:
+  - ActionController (12 events)
+  - ActionView (4 events)
+  - ActiveRecord (5 events)
+  - ActiveJob (8 events)
+  - ActionCable (5 events)
+  - ActionMailer (3 events)
+  - ActiveStorage (12 events)
+  - Cache (15 events)
+- [x] **Analyzers** - Three intelligent analyzers:
+  - N+1 Query Detector
+  - Slow Query Analyzer
+  - Cache Efficiency Tracker
+- [x] **Railtie** - Auto-initialization in Rails apps
+
+---
+
+## Phase 2: SDK Enhancements
+
+### 2.1 Enhanced Pulse Methods
+
+Add specialized APM methods to the BrainzLab SDK:
+
+```ruby
+# lib/brainzlab/pulse.rb
+module BrainzLab
+  module Pulse
+    class << self
+      # Record a span with structured attributes
+      def record_span(name:, duration_ms:, category:, attributes: {}, timestamp: nil)
+        # Implementation
+      end
+
+      # Start a trace context
+      def start_trace(name, attributes = {})
+        # Returns trace context
+      end
+
+      # End current trace
+      def end_trace(trace_context)
+        # Finalizes and sends trace
+      end
+    end
+  end
+end
+```
+
+**Files to modify:**
+- `lib/brainzlab/pulse.rb` (add `record_span`, `start_trace`, `end_trace`)
+- `lib/brainzlab/pulse/span.rb` (new - span data structure)
+- `lib/brainzlab/pulse/trace.rb` (new - trace context)
+
+### 2.2 Enhanced Flux Methods
+
+Add metric methods to the SDK:
+
+```ruby
+# lib/brainzlab/flux.rb
+module BrainzLab
+  module Flux
+    class << self
+      def increment(metric, value = 1, tags: {})
+      def gauge(metric, value, tags: {})
+      def histogram(metric, value, tags: {})
+      def timing(metric, value_ms, tags: {})
+    end
+  end
+end
+```
+
+**Files to modify:**
+- `lib/brainzlab/flux.rb` (new module)
+- `lib/brainzlab/flux/client.rb` (metrics client)
+- `lib/brainzlab/flux/buffer.rb` (metric batching)
+
+### 2.3 Configuration Updates
+
+```ruby
+# lib/brainzlab/configuration.rb
+# Add:
+attr_accessor :flux_enabled
+attr_accessor :flux_url
+attr_accessor :flux_api_key
+
+def flux_effectively_enabled?
+  flux_enabled && flux_url.present?
+end
+```
+
+---
+
+## Phase 3: Server-Side Enhancements
+
+### 3.1 Pulse Dashboard Updates
+
+The Pulse server needs to handle structured span data:
+
+**New API Endpoints:**
+```
+POST /api/v1/spans         - Ingest spans with attributes
+GET  /api/v1/traces/:id    - Get trace with spans
+GET  /api/v1/requests      - Request list with breakdown
+```
+
+**Database Schema:**
+```ruby
+create_table :spans do |t|
+  t.references :trace, null: false
+  t.string :name, null: false
+  t.string :category
+  t.float :duration_ms
+  t.jsonb :attributes, default: {}
+  t.datetime :started_at
+  t.datetime :finished_at
+  t.timestamps
+end
+
+create_table :traces do |t|
+  t.references :project, null: false
+  t.string :trace_id, null: false
+  t.string :name
+  t.float :total_duration_ms
+  t.jsonb :metadata, default: {}
+  t.timestamps
+end
+```
+
+**Dashboard Features:**
+- Request waterfall (like Chrome DevTools)
+- SQL query breakdown with timing
+- View rendering breakdown
+- N+1 alerts
+- Slow query highlighting
+
+### 3.2 Flux Product (New)
+
+If Flux doesn't exist, create it:
+
+**Key Features:**
+- Time-series metrics storage (TimescaleDB)
+- Real-time dashboards
+- Custom metric definitions
+- Anomaly detection
+- Alerting integration
+
+**Models:**
+```ruby
+# Metric definition
+class Metric < ApplicationRecord
+  belongs_to :project
+  has_many :data_points
+end
+
+# Time-series data (TimescaleDB hypertable)
+class DataPoint < ApplicationRecord
+  belongs_to :metric
+  # time, value, tags (jsonb)
+end
+```
+
+### 3.3 Recall Enhancements
+
+**Structured Log Filtering:**
+- Filter by Rails event type
+- Filter by controller/action
+- Filter by job class
+- Search by payload attributes
+
+**UI Updates:**
+- Event type badges
+- Expandable payload viewer
+- Request correlation view
+
+### 3.4 Reflex Enhancements
+
+**Breadcrumb Improvements:**
+- Group by category (db, http, cache, job)
+- Timeline visualization
+- SQL queries before error
+- Cache state at error time
+
+**Error Context:**
+- Attach request breakdown
+- Show N+1 warnings
+- Include slow queries
+
+---
+
+## Phase 4: Advanced Features
+
+### 4.1 N+1 Query Dashboard
+
+Dedicated view for N+1 detection:
+
+```
+/dashboard/performance/n-plus-one
+
+Features:
+- List of detected N+1 patterns
+- Query frequency
+- Affected controllers/actions
+- Fix suggestions (eager loading)
+- Track resolution status
+```
+
+### 4.2 Slow Query Analysis
+
+```
+/dashboard/performance/slow-queries
+
+Features:
+- Slow query log with EXPLAIN
+- Index recommendations
+- Query patterns
+- Performance trends
+```
+
+### 4.3 Cache Efficiency Dashboard
+
+```
+/dashboard/performance/cache
+
+Features:
+- Overall hit rate
+- Per-key statistics
+- Miss patterns
+- TTL analysis
+- Size tracking
+```
+
+### 4.4 Action Cable Monitor (Unique Differentiator)
+
+```
+/dashboard/websockets
+
+Features:
+- Active connections
+- Subscription health
+- Broadcast latency
+- Message throughput
+- Channel performance
+```
+
+---
+
+## Phase 5: Testing & Documentation
+
+### 5.1 Test Suite
+
+```ruby
+# spec/brainzlab/rails/subscriber_spec.rb
+# spec/brainzlab/rails/event_router_spec.rb
+# spec/brainzlab/rails/collectors/*_spec.rb
+# spec/brainzlab/rails/analyzers/*_spec.rb
+```
+
+### 5.2 Integration Tests
+
+```ruby
+# Test with real Rails app
+# Verify all 64+ events are captured
+# Verify accurate timing with monotonic_subscribe
+# Test sampling
+# Test filtering
+```
+
+### 5.3 Documentation
+
+- README with quick start
+- Configuration reference
+- Event reference (all 64+ events)
+- Dashboard usage guide
+- Troubleshooting guide
+
+---
+
+## Implementation Priority
+
+### Week 1: Core SDK
+1. [ ] Add `Pulse.record_span` to SDK
+2. [ ] Add `Flux` module to SDK
+3. [ ] Update SDK configuration
+4. [ ] Publish SDK v0.1.5
+
+### Week 2: Gem Polish
+1. [ ] Add comprehensive tests
+2. [ ] Validate all event handlers
+3. [ ] Performance benchmarking
+4. [ ] Publish gem v0.1.0
+
+### Week 3: Pulse Updates
+1. [ ] Span ingestion API
+2. [ ] Trace aggregation
+3. [ ] Request waterfall UI
+4. [ ] N+1 detection UI
+
+### Week 4: Additional Products
+1. [ ] Flux metric ingestion
+2. [ ] Recall structured logs UI
+3. [ ] Reflex breadcrumb improvements
+4. [ ] Action Cable dashboard
+
+---
+
+## Event → Product Matrix
+
+| Rails Event | Pulse | Recall | Reflex | Flux | Nerve |
+|-------------|-------|--------|--------|------|-------|
+| process_action.action_controller | ✓ | ✓ | ✓* | ✓ | |
+| sql.active_record | ✓ | | | ✓ | |
+| render_*.action_view | ✓ | | | ✓ | |
+| *.active_job | ✓ | ✓ | ✓* | ✓ | ✓ |
+| *.action_cable | ✓ | ✓ | ✓* | ✓ | |
+| cache_*.active_support | ✓ | | | ✓ | |
+| deliver.action_mailer | ✓ | ✓ | ✓* | ✓ | |
+| service_*.active_storage | ✓ | ✓ | | ✓ | |
+
+*✓ = Only when exception in payload*
+
+---
+
+## Success Metrics
+
+1. **Coverage**: 100% of Rails instrumentation events captured
+2. **Accuracy**: < 1ms timing variance vs wall-clock
+3. **Performance**: < 5% overhead in production
+4. **Adoption**: Zero-config installation via Railtie
+5. **Insights**: Automatic N+1 detection, slow query analysis
+
+---
+
+## Competitive Advantage
+
+```
+┌─────────────────────────────────────────────────────────────────┐
+│                                                                 │
+│   THEM (Datadog/New Relic):                                    │
+│   - Generic agents that monkey-patch                           │
+│   - "HTTP 200 in 234ms"                                        │
+│   - Separate products, separate billing                        │
+│                                                                 │
+│   US (BrainzLab):                                              │
+│   - Native Rails via ActiveSupport::Notifications             │
+│   - "PostsController#index: 4 queries (N+1!), 3 cache hits"   │
+│   - One gem, all products, Rails-optimized                    │
+│                                                                 │
+└─────────────────────────────────────────────────────────────────┘
+```
+
+---
+
+## Next Steps
+
+1. **Immediate**: Add Pulse.record_span to SDK
+2. **This Week**: Publish brainzlab-rails gem v0.1.0
+3. **Next Week**: Update Pulse dashboard for spans
+4. **Future**: Build Action Cable dashboard (differentiator)

data/Rakefile

@@ -0,0 +1,8 @@
+# frozen_string_literal: true
+
+require 'bundler/gem_tasks'
+require 'rspec/core/rake_task'
+
+RSpec::Core::RakeTask.new(:spec)
+
+task default: :spec

data/brainzlab-rails.gemspec

@@ -0,0 +1,42 @@
+# frozen_string_literal: true
+
+require_relative 'lib/brainzlab/rails/version'
+
+Gem::Specification.new do |spec|
+  spec.name          = 'brainzlab-rails'
+  spec.version       = BrainzLab::Rails::VERSION
+  spec.authors       = ['Brainz Lab']
+  spec.email         = ['support@brainzlab.ai']
+
+  spec.summary       = 'Rails-native observability powered by ActiveSupport::Notifications'
+  spec.description   = 'Deep Rails instrumentation that routes events to Pulse (APM), Recall (Logs), Reflex (Errors), Flux (Metrics), and Nerve (Jobs). One gem, full Rails observability.'
+  spec.homepage      = 'https://brainzlab.ai'
+  spec.license       = 'MIT'
+  spec.required_ruby_version = '>= 3.1.0'
+
+  spec.metadata['homepage_uri'] = spec.homepage
+  spec.metadata['source_code_uri'] = 'https://github.com/brainz-lab/brainzlab-rails'
+  spec.metadata['changelog_uri'] = 'https://github.com/brainz-lab/brainzlab-rails/blob/main/CHANGELOG.md'
+
+  spec.files = Dir.chdir(__dir__) do
+    `git ls-files -z`.split("\x0").reject do |f|
+      (File.expand_path(f) == __FILE__) ||
+        f.start_with?(*%w[bin/ test/ spec/ features/ .git .github appveyor Gemfile])
+    end
+  end
+  spec.bindir = 'exe'
+  spec.executables = spec.files.grep(%r{\Aexe/}) { |f| File.basename(f) }
+  spec.require_paths = ['lib']
+
+  # Dependencies
+  spec.add_dependency 'brainzlab', '>= 0.1.6'
+  spec.add_dependency 'rails', '>= 7.0'
+  spec.add_dependency 'activesupport', '>= 7.0'
+
+  # Development dependencies
+  spec.add_development_dependency 'bundler', '~> 2.0'
+  spec.add_development_dependency 'rake', '~> 13.0'
+  spec.add_development_dependency 'rspec', '~> 3.0'
+  spec.add_development_dependency 'rspec-rails', '~> 6.0'
+  spec.add_development_dependency 'rubocop', '~> 1.21'
+end

data/lib/brainzlab/rails/analyzers/cache_efficiency.rb

@@ -0,0 +1,123 @@
+# frozen_string_literal: true
+
+module BrainzLab
+  module Rails
+    module Analyzers
+      # Tracks cache efficiency metrics and provides insights
+      class CacheEfficiency
+        WINDOW_SIZE = 1000 # Rolling window for efficiency calculation
+
+        def initialize
+          @hits = 0
+          @misses = 0
+          @reads = []
+          @writes = 0
+          @generates = 0
+        end
+
+        def track(event_data)
+          case event_data[:name]
+          when 'cache_read.active_support'
+            track_read(event_data)
+          when 'cache_read_multi.active_support'
+            track_read_multi(event_data)
+          when 'cache_write.active_support', 'cache_write_multi.active_support'
+            @writes += 1
+          when 'cache_generate.active_support'
+            @generates += 1
+          when 'cache_fetch_hit.active_support'
+            @hits += 1
+          end
+
+          # Trim old reads to maintain window
+          trim_reads if @reads.size > WINDOW_SIZE * 2
+        end
+
+        def hit_rate
+          total = @hits + @misses
+          return 0.0 if total == 0
+
+          (@hits.to_f / total * 100).round(2)
+        end
+
+        def efficiency_report
+          {
+            hit_rate: hit_rate,
+            total_hits: @hits,
+            total_misses: @misses,
+            total_writes: @writes,
+            total_generates: @generates,
+            recent_reads: recent_reads_stats
+          }
+        end
+
+        def reset!
+          @hits = 0
+          @misses = 0
+          @reads = []
+          @writes = 0
+          @generates = 0
+        end
+
+        private
+
+        def track_read(event_data)
+          payload = event_data[:payload]
+          hit = payload[:hit]
+
+          if hit
+            @hits += 1
+          else
+            @misses += 1
+          end
+
+          @reads << {
+            key: payload[:key],
+            hit: hit,
+            duration_ms: event_data[:duration_ms],
+            timestamp: Time.now
+          }
+        end
+
+        def track_read_multi(event_data)
+          payload = event_data[:payload]
+          keys = payload[:key] || []
+          hits = payload[:hits] || []
+
+          @hits += hits.size
+          @misses += (keys.size - hits.size)
+
+          @reads << {
+            key: "multi:#{keys.size}",
+            hit: hits.size == keys.size,
+            duration_ms: event_data[:duration_ms],
+            timestamp: Time.now,
+            multi: true,
+            hit_count: hits.size,
+            total_count: keys.size
+          }
+        end
+
+        def recent_reads_stats
+          return {} if @reads.empty?
+
+          recent = @reads.last(100)
+          hits = recent.count { |r| r[:hit] }
+          misses = recent.size - hits
+
+          {
+            count: recent.size,
+            hits: hits,
+            misses: misses,
+            hit_rate: (hits.to_f / recent.size * 100).round(2),
+            avg_duration_ms: (recent.sum { |r| r[:duration_ms] } / recent.size).round(3)
+          }
+        end
+
+        def trim_reads
+          @reads = @reads.last(WINDOW_SIZE)
+        end
+      end
+    end
+  end
+end