rails_benchmark_suite 0.3.2 → 0.3.3

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 52350ff4b60a92bb839f96eb8d5032df181fb8578c12ea72afd42ede82f89a43
-  data.tar.gz: 6191dfad41045b1b3035a4482a32aa1fc18b560d1b1903282081e28e400fe7c8
+  metadata.gz: 836276b8a2872af0084df8bbad548c23cd1102aefb9d8e8399107c79a154a237
+  data.tar.gz: 174cc4856a798619687f55ceb182d03f423d33ccd830a90ec5c9d538a0411f2a
 SHA512:
-  metadata.gz: 1a5965daecea7f9044a77e7232a2d838a1b1aca81a96ef27043f1238cc1f946e7b2abca48537668bcc4dd743861f9985592a000c1230fa284c3627659d373906
-  data.tar.gz: b191d19713a1ae9821bac379007284c84de10c2994e1165ef648f188c363b2167f89105917911f0d2ca6ca4d716f44150b156aa5ff50c76e969aca55897ab1b6
+  metadata.gz: 626e81a963ca2a345a51236c24188caaec3d8b871392262cd6189551ed66997cd7722573327645445cd5a068be7f32b3e918fe4fb8e4d0f0518cf65094fbe3ef
+  data.tar.gz: f9893a9176305a878e9a2be2f0bb80e9fb4f4613837e98c74042f74917fa79e7498618cc1c75de4ac026f9ce4ab9b7152c7721d8db5dae89b5b3ea0dd232c9fb

data/CHANGELOG.md

@@ -1,4 +1,18 @@
 # Changelog
+## [0.3.3] - 2026-01-06
+### Added
+- **Request Heft Workload**: A new benchmark measuring the full Rails stack overhead (Middleware + Router + Controller + View).
+- **Boot Structure Analysis**: Auto-Boot Profiler detects which `app/` folders are slowing down startup time.
+- **Security**: Uses ephemeral in-memory route injection (zero production footprint/risk).
+- **Docs**: Updated weights and workload descriptions in README.
+
+### Fixed
+- **Boot Profiler Accuracy**: Now fully initializes the Rails environment (instead of just application config) to ensure autoloaders/Zeitwerk function correctly during file profiling.
+- **Boot Profiler Stability**: The boot profiler now suppresses Rails logs and warnings to prevent JSON corruption.
+- **Infinite Score Bug**: Workloads now strictly check dependencies before registering. Missing dependencies (e.g. `--skip-rails`) correctly trigger weight redistribution instead of reporting infinite IPS.
+- **JSON Output**: Boot Structure Analysis is now included in the JSON report (`--json`) for CI/CD visibility.
+- **UI**: Fixed character encoding issues in terminal reporter emojis.
+
 ## [0.3.2] - 2026-01-05
 ### Refactoring & Polish
 - **DRY Logic**: Centralized "Efficiency" calculation; removed duplicate math from HTML templates.

data/Gemfile.lock

@@ -1,7 +1,7 @@
 PATH
   remote: .
   specs:
-    rails_benchmark_suite (0.3.2)
+    rails_benchmark_suite (0.3.3)
       actionview (~> 8.1)
       activerecord (~> 8.1)
       activestorage (~> 8.1)

data/README.md

@@ -8,6 +8,10 @@ A standardized performance suite designed to measure the "Heft" of a machine usi
 
 Think of this as a **"Test Track" for Rails servers**. Unlike profilers that measure your specific application code, this gem runs a **fixed, standardized set of Rails operations** (Active Record object allocation, SQL query complexity, ActionView rendering, and background job throughput) to measure the raw performance of your server and Ruby configuration.
 
+**Key Features:**
+- **Auto-Boot Profiler (Default):** Automatically detects slow folders (Models, Controllers, etc.) during startup—runs on every benchmark.
+- **Isolated Environment:** Uses in-memory SQLite—never touches your production data.
+
 To ensure a level playing field, the gem boots an **isolated, in-memory SQLite environment**. It creates its own schema and records, meaning it **never touches your production data** and returns comparable results across any machine.
 
 ## 📊 The "Heft" Score
@@ -100,7 +104,7 @@ Enable YJIT for maximum performance measurement accuracy. This is the recommende
 ```bash
 bundle exec rails_benchmark_suite --json > report.json
 ```
-Perfect for CI/CD pipelines and programmatic analysis. Outputs clean JSON without any UI elements.
+Perfect for CI/CD pipelines and programmatic analysis. Outputs clean JSON including Boot Structure Analysis and all workload metrics.
 
 ### 📊 Visual Diagnostics (HTML Report)
 
@@ -156,9 +160,10 @@ RHI Score = Σ (4-Thread IPS × Weight)
 
 | Workload | Weight | Rationale |
 |----------|--------|-----------|  
-| **Active Record** | 40% | Database operations are the core of most Rails apps |
-| **View Rendering** | 20% | ERB/ActionView processing |
-| **Solid Queue** | 20% | Background job throughput |
+| **Active Record** | 30% | Database operations are the core of most Rails apps |
+| **Request Heft** | 30% | Full Stack Overhead (Middleware → Router → Controller) |
+| **View Rendering** | 10% | ERB/ActionView processing |
+| **Solid Queue** | 10% | Background job throughput |
 | **Cache Operations** | 10% | Memory store performance |
 | **Image Processing** | 10% | Optional - requires libvips |
 
@@ -181,6 +186,7 @@ Your RHI score maps to these performance tiers:
 The gem measures performance across critical Rails subsystems using a dedicated, isolated schema:
 
 * **Active Record Heft:** Standardized CRUD: Creation, indexing, and complex querying.
+* **Request Heft:** Full-stack request throughput (Middleware → Router → Controller).
 * **Cache Heft:** High-frequency read/writes to the Rails memory store.
 * **Solid Queue Heft:** Background job enqueuing and database-backed polling stress.
 * **View Heft:** Partial rendering overhead and ActionView throughput.

data/lib/rails_benchmark_suite/boot_profiler_script.rb

@@ -0,0 +1,85 @@
+#!/usr/bin/env ruby
+# frozen_string_literal: true
+
+# Boot Structure Analysis - Standalone Profiler Script
+# Executed as subprocess to measure cold boot times per app/ directory
+# Outputs JSON to STDOUT
+# 
+# IMPORTANT: Uses stdout silencing to handle noisy Rails boots that may
+# output deprecation warnings or logs during require statements.
+
+require "json"
+require "benchmark"
+
+# Output Silencer - swallows all puts/print from Rails boot
+def silence_stdout
+  original_stdout = $stdout
+  original_stderr = $stderr
+  $stdout = File.open(File::NULL, "w")
+  $stderr = File.open(File::NULL, "w")
+  yield
+ensure
+  $stdout = original_stdout
+  $stderr = original_stderr
+end
+
+results = []
+
+begin
+  # Boot Phase: Minimal Rails setup (silenced to avoid noisy output)
+  silence_stdout do
+    require "bundler/setup"
+    
+    # Check if we're in a Rails app
+    env_file = File.join(Dir.pwd, "config", "environment.rb")
+    unless File.exist?(env_file)
+      # Not in a Rails app - will output empty results after restore
+      break
+    end
+
+    # Load full Rails environment to initialize Zeitwerk autoloading
+    ENV["RAILS_ENV"] ||= "development"
+    require File.join(Dir.pwd, "config", "environment")
+  end
+  
+  # Check if we're in a Rails app (outside silence block)
+  env_file = File.join(Dir.pwd, "config", "environment.rb")
+  unless File.exist?(env_file)
+    puts "[]"
+    exit 0
+  end
+  
+  # Profiling Phase: Measure each app/ subdirectory
+  target_dirs = Dir.glob("app/*").select { |f| File.directory?(f) }
+
+  target_dirs.each do |dir|
+    files = Dir.glob("#{dir}/**/*.rb")
+    next if files.empty?
+
+    time_ms = Benchmark.realtime do
+      # Silence loading phase - models/controllers often trigger logs
+      silence_stdout do
+        files.each do |file|
+          begin
+            require file
+          rescue LoadError, NameError, StandardError
+            # Silently skip files with missing dependencies
+          end
+        end
+      end
+    end * 1000 # Convert to milliseconds
+
+    results << {
+      path: dir,
+      time_ms: time_ms.round(2),
+      file_count: files.size
+    }
+  end
+
+rescue => e
+  # If anything fails, return error info for debugging
+  results = [{ error: e.message, backtrace: e.backtrace&.first(3) }]
+end
+
+# Output JSON to STDOUT (only after restoring stdout)
+puts results.to_json

data/lib/rails_benchmark_suite/reporter.rb

@@ -116,18 +116,56 @@ module RailsBenchmarkSuite
         renderer.border.separator = :each_row
         renderer.border.style = :blue
       end
+
+      # 2. Boot Structure Analysis
+      render_boot_analysis(payload[:boot_analysis])
       
-      # 2. Insights List
+      # 3. Insights List
       puts ""
       check_scaling_insights(results)
       check_yjit_insight
       check_memory_insights(results)
       
-      # 3. Final Score Dashboard
+      # 4. Final Score Dashboard
       render_final_score(total_score)
       show_hardware_tier(tier)
     end
 
+    def render_boot_analysis(boot_data)
+      return unless boot_data&.any?
+
+      puts ""
+      puts pastel.bold("\u{1F4C2} Boot Structure Analysis")
+      puts ""
+
+      rows = boot_data.map do |entry|
+        time_ms = entry[:time_ms] || 0
+        
+        # Color coding: Red > 500ms, Yellow > 200ms, Green otherwise
+        time_color = if time_ms > 500
+          :red
+        elsif time_ms > 200
+          :yellow
+        else
+          :green
+        end
+        
+        time_str = pastel.decorate("#{time_ms.round(0)}ms", time_color)
+        
+        [entry[:path], time_str, entry[:file_count]]
+      end
+
+      table = TTY::Table.new(
+        header: ["Directory", "Load Time", "Files"],
+        rows: rows
+      )
+
+      puts table.render(:unicode, padding: [0, 1]) do |renderer|
+        renderer.border.separator = :each_row
+        renderer.border.style = :cyan
+      end
+    end
+
     def check_scaling_insights(results)
       poor_scaling = results.select do |r|
         entries = r[:report].entries
@@ -141,7 +179,7 @@ module RailsBenchmarkSuite
       end
       
       if poor_scaling.any?
-        puts pastel.yellow.bold("💡 Insight (Scaling):") + " Scaling below 1.0x detected."
+        puts pastel.yellow.bold("\u{1F4A1} Insight (Scaling):") + " Scaling below 1.0x detected."
         puts "   This indicates SQLite lock contention or Ruby GIL saturation."
       end
     end
@@ -149,7 +187,7 @@ module RailsBenchmarkSuite
     def check_yjit_insight
       unless yjit_enabled?
         puts ""
-        puts pastel.yellow.bold("💡 Insight (YJIT):") + " YJIT is OFF."
+        puts pastel.yellow.bold("\u{1F4A1} Insight (YJIT):") + " YJIT is OFF."
         puts "   Run with RUBY_OPT=\"--yjit\" for ~20% boost."
       end
     end
@@ -157,7 +195,7 @@ module RailsBenchmarkSuite
     def check_memory_insights(results)
       results.select { |r| r[:memory_delta_mb] > 20 }.each do |r|
         puts ""
-        puts pastel.yellow.bold("💡 Insight (Memory):") + " High growth in #{r[:name]} (#{r[:memory_delta_mb].round(1)}MB)"
+        puts pastel.yellow.bold("\u{1F4A1} Insight (Memory):") + " High growth in #{r[:name]} (#{r[:memory_delta_mb].round(1)}MB)"
         puts "   Suggests heavy object allocation."
       end
     end
@@ -173,7 +211,7 @@ module RailsBenchmarkSuite
       end
       
       puts ""
-      puts pastel.bold("📊 Performance Tier: ") + comparison
+      puts pastel.bold("\u{1F4CA} Performance Tier: ") + comparison
       puts ""
       print cursor.show
     end
@@ -201,6 +239,7 @@ module RailsBenchmarkSuite
         system: system_info,
         total_score: payload[:total_score].round(0),
         tier: payload[:tier],
+        boot_analysis: payload[:boot_analysis] || [],
         workloads: []
       }
       

data/lib/rails_benchmark_suite/runner.rb

@@ -5,6 +5,8 @@ require_relative "schema"
 require_relative "../dummy/app/models/benchmark_user"
 require_relative "../dummy/app/models/benchmark_post"
 require_relative "../dummy/app/models/benchmark_job"
+require "open3"
+require "json"
 
 module RailsBenchmarkSuite
   class Runner
@@ -49,19 +51,54 @@ module RailsBenchmarkSuite
         options: runner_options,
         show_progress: !@config.json
       ).execute
+
+      # 4. Boot Structure Analysis (run for all modes including JSON)
+      payload[:boot_analysis] = run_boot_analysis
       
-      # 4. Report Results
+      # 5. Report Results
       if @config.json
         Reporter.as_json(payload)
       else
         Reporter.render(payload)
       end
 
-      # 5. HTML Report Generation
+      # 6. HTML Report Generation
       if @config.html
         require_relative "reporters/html_reporter"
         Reporters::HtmlReporter.new(payload).generate
       end
     end
+
+    private
+
+    def run_boot_analysis
+      script_path = File.join(__dir__, "boot_profiler_script.rb")
+      return nil unless File.exist?(script_path)
+
+      begin
+        stdout, status = Open3.capture2("bundle", "exec", "ruby", script_path)
+        return nil unless status.success?
+
+        # Try to parse the whole output first
+        results = begin
+          JSON.parse(stdout, symbolize_names: true)
+        rescue JSON::ParserError
+          # Fallback: Try last line (in case silence failed and noise leaked)
+          last_line = stdout.strip.lines.last
+          begin
+            JSON.parse(last_line, symbolize_names: true)
+          rescue JSON::ParserError
+            nil
+          end
+        end
+
+        return nil if results.nil? || results.empty? || results.first&.dig(:error)
+
+        # Sort by load time descending (slowest first)
+        results.sort_by { |r| -(r[:time_ms] || 0) }
+      rescue StandardError
+        nil
+      end
+    end
   end
 end

data/lib/rails_benchmark_suite/templates/report.html.erb

@@ -121,6 +121,46 @@
         </div>
     </div>
 
+    <% if @payload[:boot_analysis]&.any? %>
+    <!-- Boot Structure Analysis -->
+    <div class="card mt-5">
+        <div class="card-header bg-transparent border-bottom border-secondary">
+            <h5 class="mb-0">📂 Boot Structure Analysis</h5>
+        </div>
+        <div class="card-body">
+            <div class="row">
+                <div class="col-md-6">
+                    <canvas id="bootChart" style="max-height: 300px;"></canvas>
+                </div>
+                <div class="col-md-6">
+                    <table class="table table-dark table-striped table-hover mb-0 align-middle">
+                        <thead>
+                            <tr>
+                                <th>Directory</th>
+                                <th class="text-end">Load Time</th>
+                                <th class="text-end">Files</th>
+                            </tr>
+                        </thead>
+                        <tbody>
+                            <% @payload[:boot_analysis].each do |entry| %>
+                                <% 
+                                   time_ms = entry[:time_ms] || 0
+                                   time_class = time_ms > 500 ? 'text-danger' : (time_ms > 200 ? 'text-warning' : 'text-success')
+                                %>
+                                <tr>
+                                    <td class="font-monospace"><%= entry[:path] %></td>
+                                    <td class="text-end font-monospace <%= time_class %>"><%= time_ms.round(0) %>ms</td>
+                                    <td class="text-end text-secondary"><%= entry[:file_count] %></td>
+                                </tr>
+                            <% end %>
+                        </tbody>
+                    </table>
+                </div>
+            </div>
+        </div>
+    </div>
+    <% end %>
+
     <!-- JavaScript Injection -->
     <script>
         const CHART_DATA = <%= @chart_payload %>;
@@ -181,6 +221,53 @@
                     }
                 }
             });
+
+            <% if @payload[:boot_analysis]&.any? %>
+            // Boot Structure Analysis Chart
+            const bootCtx = document.getElementById('bootChart');
+            const bootLabels = <%= @payload[:boot_analysis].map { |e| e[:path] }.to_json %>;
+            const bootData = <%= @payload[:boot_analysis].map { |e| e[:time_ms] }.to_json %>;
+            
+            const bootColors = bootData.map(ms => {
+                if (ms > 500) return 'rgba(244, 67, 54, 0.8)';
+                if (ms > 200) return 'rgba(255, 193, 7, 0.8)';
+                return 'rgba(76, 175, 80, 0.8)';
+            });
+
+            new Chart(bootCtx, {
+                type: 'bar',
+                data: {
+                    labels: bootLabels,
+                    datasets: [{
+                        label: 'Load Time (ms)',
+                        data: bootData,
+                        backgroundColor: bootColors,
+                        borderColor: bootColors.map(c => c.replace('0.8)', '1)')),
+                        borderWidth: 1
+                    }]
+                },
+                options: {
+                    indexAxis: 'y',
+                    responsive: true,
+                    maintainAspectRatio: false,
+                    scales: {
+                        x: {
+                            beginAtZero: true,
+                            grid: { color: '#333' },
+                            ticks: { color: '#aaa' },
+                            title: { display: true, text: 'Milliseconds', color: '#aaa' }
+                        },
+                        y: {
+                            grid: { display: false },
+                            ticks: { color: '#fff' }
+                        }
+                    },
+                    plugins: {
+                        legend: { display: false }
+                    }
+                }
+            });
+            <% end %>
         });
     </script>
 </body>

data/lib/rails_benchmark_suite/version.rb

@@ -1,3 +1,3 @@
 module RailsBenchmarkSuite
-  VERSION = "0.3.2"
+  VERSION = "0.3.3"
 end

data/lib/rails_benchmark_suite/workload_runner.rb

@@ -6,13 +6,14 @@ require "tty-spinner"
 
 module RailsBenchmarkSuite
   class WorkloadRunner
-    # Base weights for each workload
+    # Base weights for each workload (must sum to 1.0)
     BASE_WEIGHTS = {
-      "Active Record Heft" => 0.4,
-      "View Heft" => 0.2,
-      "Solid Queue Heft" => 0.2,
-      "Cache Heft" => 0.1,
-      "Image Heft" => 0.1
+      "Active Record Heft" => 0.3,  # Reduced from 0.4
+      "Request Heft"       => 0.3,  # NEW: Full Stack Overhead
+      "View Heft"          => 0.1,  # Reduced from 0.2
+      "Solid Queue Heft"   => 0.1,  # Reduced from 0.2
+      "Cache Heft"         => 0.1,
+      "Image Heft"         => 0.1
     }.freeze
 
     def initialize(workloads, options: {}, show_progress: true)

data/lib/rails_benchmark_suite/workloads/image_heft_workload.rb

@@ -10,18 +10,16 @@ begin
   FileUtils.mkdir_p(ASSET_DIR)
   SAMPLE_IMAGE = File.join(ASSET_DIR, "sample.jpg")
 
-  # Only register if vips is actually available
-  RailsBenchmarkSuite::Runner.register_workload("Image Heft", weight: 0.1) do
-    # Gracefully handle missing dependencies
-    if File.exist?(SAMPLE_IMAGE)
+  # Only register if vips is available AND sample image exists
+  if File.exist?(SAMPLE_IMAGE)
+    RailsBenchmarkSuite::Runner.register_workload("Image Heft", weight: 0.1) do
       ImageProcessing::Vips
         .source(SAMPLE_IMAGE)
         .resize_to_limit(800, 800)
         .call
-    else
-      # Maintain benchmark stability if asset is missing
-      true
     end
+  else
+    puts "\n⚠️  Skipping Image Workload: sample.jpg not found in assets/\n\n"
   end
 
 rescue LoadError, StandardError

data/lib/rails_benchmark_suite/workloads/request_heft_workload.rb

@@ -0,0 +1,51 @@
+# frozen_string_literal: true
+
+# Request Heft Workload
+# Benchmarks the full Rails request lifecycle (Middleware → Router → Controller → View)
+# Uses ephemeral in-memory route injection - zero production footprint
+
+# Check dependencies BEFORE registering to prevent infinite IPS bug
+has_rails = begin
+  require "rails"
+  require "action_controller/railtie"
+  defined?(Rails.application) && Rails.application
+rescue LoadError
+  false
+end
+
+# Only register if Rails is present
+if has_rails
+  RailsBenchmarkSuite::Runner.register_workload("Request Heft", weight: 0.3) do
+    begin
+      # Step A: Define Ephemeral Controller
+      # Anonymous class avoids constant pollution
+      controller_class = Class.new(ActionController::Base) do
+        layout false
+
+        def index
+          render plain: "OK"
+        end
+      end
+
+      # Step B: Inject Ephemeral Route
+      # Obscure path to avoid collisions with host app routes
+      Rails.application.routes.draw do
+        get "/_rbs_benchmark/heft", to: controller_class.action(:index)
+      end
+
+      # Step C: Execute Full-Stack Request
+      env = Rack::MockRequest.env_for("/_rbs_benchmark/heft")
+      status, _headers, _body = Rails.application.call(env)
+
+      # Step D: Validation
+      # Don't read/close body to avoid I/O overhead skewing benchmark
+      raise RuntimeError, "Request Heft failed: status=#{status}" unless status == 200
+
+      true
+    rescue => e
+      # Error handling - don't crash the entire suite
+      warn "[Request Heft] Error: #{e.message}" if ENV["DEBUG"]
+      false
+    end
+  end
+end

metadata

@@ -1,13 +1,13 @@
 --- !ruby/object:Gem::Specification
 name: rails_benchmark_suite
 version: !ruby/object:Gem::Version
-  version: 0.3.2
+  version: 0.3.3
 platform: ruby
 authors:
 - RailsBenchmarkSuite Contributors
 bindir: bin
 cert_chain: []
-date: 2026-01-06 00:00:00.000000000 Z
+date: 2026-01-24 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: benchmark-ips
@@ -259,6 +259,7 @@ files:
 - lib/dummy/app/views/rails_benchmark_suite/heft_view.html.erb
 - lib/dummy/config/benchmark_database.yml
 - lib/rails_benchmark_suite.rb
+- lib/rails_benchmark_suite/boot_profiler_script.rb
 - lib/rails_benchmark_suite/configuration.rb
 - lib/rails_benchmark_suite/database_manager.rb
 - lib/rails_benchmark_suite/db_setup.rb
@@ -276,6 +277,7 @@ files:
 - lib/rails_benchmark_suite/workloads/cache_heft_workload.rb
 - lib/rails_benchmark_suite/workloads/image_heft_workload.rb
 - lib/rails_benchmark_suite/workloads/job_heft_workload.rb
+- lib/rails_benchmark_suite/workloads/request_heft_workload.rb
 - lib/rails_benchmark_suite/workloads/view_heft_workload.rb
 - rails_benchmark_suite.gemspec
 homepage: https://github.com/overnet/rails_benchmark_suite