rails_benchmark_suite 0.2.9 → 0.3.1

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: c48ce5faaa5348bcd47cdd9c88ba9b1441d0f7bcd0a6fa601d84b36014673d1a
-  data.tar.gz: f39f188c55fe6749a5b9ba4ae9c5c44e926bd5d6588737f689192ab841deda92
+  metadata.gz: 473aa21869380dae81d4f9ee8668ccf161721923fd6c453cc711caf9d6303fd1
+  data.tar.gz: 5101bb8d63885263dc3fcbb8851d52f3ed48a1f919b0abfab38d43dc3f3df2a2
 SHA512:
-  metadata.gz: 28f32ff03be6cbd83aabb1c3f8750316f5c3e2cb69de127885f96a4ef27841b12ee916999c2dea59c314605a6755f0aee34d0c610338c596bbff2789120a7ce0
-  data.tar.gz: 2e4f5a17c66e0caed9a95a46cdc3e0049783eea603f9cd263d699f492d49440f7ceaab45bf4c469b8b05fc316ca999052d12705dfe29b0edf54b032160fc2966
+  metadata.gz: 11311aa76df4c103c8b3ea2e12a803ddeb339cd454127b246557341ddd5c7872fa01c4ff00f4c14ae711d95b48f0d84b148ec7f8b0a39fda7ab70cac0d070bad
+  data.tar.gz: 3f141277e2daae966aa4d7d7be684e112889fa757282e36a4f35ac34d60ac91ca76903281d9e091b9549ef567d6e8b7a26b13221326bf8273c3c33fe7e6082fc

data/.gitignore

@@ -1,5 +1,7 @@
-*.gem
+
 .bundle/
 vendor/bundle/
 Gemfile.lock
 .ruby-lsp/
+*.gem
+rails_benchmark_report.html

data/CHANGELOG.md

@@ -1,4 +1,69 @@
 # Changelog
+## [0.3.1] - 2026-01-04
+*Major Architectural Repair & TTY Overhaul*
+
+### Added
+- **--html**: Static HTML Reporter with Chart.js visualization.
+- **--profile**: Automated "Scaling Efficiency" calculation (1T vs MaxT).
+- **--db**: Real database integration (Postgres/MySQL support).
+- **Hardware Awareness**: Auto-detection of CPU cores for thread defaults.
+- **UI**: Rich terminal output using `tty-spinner`, `tty-table`, and `tty-box` for a dashboard-style report.
+- **Architecture**: Implemented the standard `lib/dummy` Rails Engine pattern for internal tests.
+
+### Changed
+- **Refactor**: Complete structural overhaul. Moved monolithic logic from `lib/rails_benchmark_suite.rb` into a proper namespace (`lib/rails_benchmark_suite/`).
+- **Concurrency**: Default thread count is now dynamic (`Etc.nprocessors`) instead of hardcoded to 4.
+- **Safety**: Renamed internal test model from `User` to `BenchmarkUser` to prevent collisions when running inside host apps.
+
+### Fixed
+- **CLI**: Fixed the non-functional `--help` flag (now implemented via `OptionParser`).
+- **Reporting**: Restored the "Scaling (x)" column in the final report to correctly visualize performance degradation.
+
+## [0.3.0] - 2025-01-03
+
+### Major Architectural Refactor (SRP)
+- **Runner Split**: Dismantled the monolithic `Runner` class (260+ lines) into three specialized modules:
+  - `DatabaseManager`: Handles ActiveRecord connection, schema loading, and SQLite PRAGMA optimizations
+  - `WorkloadRunner`: Manages benchmark execution engine with BASE_WEIGHTS normalization and complete payload generation
+  - `Formatter`: Centralized UI rendering, ANSI colors, insights engine, and output formatting
+- **Runner Coordinator**: `Runner` is now a minimal 23-line coordinator delegating to the three modules
+
+### Normalized RHI Math Engine
+- **BASE_WEIGHTS**: Defined workload weights (Active Record: 0.4, View: 0.2, Solid Queue: 0.2, Cache: 0.1, Image: 0.1)
+- **Dynamic Weight Redistribution**: When workloads are skipped (e.g., missing libvips), weights are normalized proportionally to maintain 100% scale
+- **Formula**: `RHI Score = Σ (4T_IPS × Adjusted_Weight)` where adjusted weights always sum to 1.0
+
+### Performance Insights Engine
+- **Scaling Analysis**: Warns when multi-threading scaling < 0.8x, indicating SQLite lock contention or Ruby GIL saturation
+- **YJIT Detection**: Displays hint to enable YJIT when disabled (typical 15-25% boost)
+- **Memory Monitoring**: Alerts when workload memory growth exceeds 20MB, suggesting heavy object allocation
+- **Hardware Tiering**: Provides comparison labels (Entry/Dev < 50, Production-Ready 50-200, High-Performance > 200)
+
+### UI/UX Enhancements
+- **Box Alignment**: Fixed header and final score boxes with proper text length calculation (60-char width)
+- **Table Spacing**: Added separator line between progress logs and results table for better readability
+- **Insights Display**: Integrated insights below summary table with emoji indicators (💡, 📊)
+- **Enhanced Number Formatting**: Smart k/M suffixes for readability (e.g., "15.3k", "1.2M")
+- **YJIT Hints**: Helpful reminder `(run with RUBY_OPT="--yjit" for max perf)` when YJIT is disabled
+- Silent migrations: Added `ActiveRecord::Migration.verbose = false` to reduce noise
+- Cross-platform install instructions for libvips (macOS and Linux)
+
+### 🐛 Fixes
+- Fixed `.gitignore` to properly track `gemspec` file for gem distribution
+- **JSON Guard**: Ensures clean JSON output without any UI noise when `--json` flag is used
+- Improved CLI output suppression in JSON mode
+
+### 📖 Documentation
+- **Calculation Formula**: Added "How It's Calculated" section with RHI formula: `Σ (4-Thread IPS × Weight)`
+- **Workload Weights Table**: Documented weights (Active Record 40%, View 20%, Jobs 20%, Cache 10%, Image 10%)
+- **Hardware Tiers**: Explained tier classification system
+- Complete README rewrite with four execution methods:
+  - Standard: `bundle exec rails_benchmark_suite`
+  - High Performance: `RUBY_OPT="--yjit" bundle exec rails_benchmark_suite` 
+  - JSON Export: `bundle exec rails_benchmark_suite --json > report.json`
+  - Standalone: `bin/rails_benchmark_suite`
+- Added comprehensive System Requirements section
+- Updated all terminology from "Suite" to "Workload"
 
 ## [0.2.0] - 2025-12-31
 

data/Gemfile.lock

@@ -1,7 +1,7 @@
 PATH
   remote: .
   specs:
-    rails_benchmark_suite (0.2.6)
+    rails_benchmark_suite (0.3.1)
       actionview (~> 8.1)
       activerecord (~> 8.1)
       activestorage (~> 8.1)
@@ -9,7 +9,11 @@ PATH
       concurrent-ruby (~> 1.3)
       get_process_mem (~> 1.0)
       image_processing (~> 1.14)
+      pastel (~> 0.8)
       sqlite3 (~> 2.8)
+      tty-box (~> 0.7)
+      tty-spinner (~> 0.9)
+      tty-table (~> 0.12)
 
 GEM
   remote: https://rubygems.org/
@@ -116,6 +120,9 @@ GEM
       racc (~> 1.4)
     nokogiri (1.19.0-x86_64-linux-musl)
       racc (~> 1.4)
+    ostruct (0.6.3)
+    pastel (0.8.0)
+      tty-color (~> 0.5)
     racc (1.8.1)
     rack (3.2.4)
     rack-session (2.1.1)
@@ -145,9 +152,29 @@ GEM
     sqlite3 (2.9.0-x86_64-darwin)
     sqlite3 (2.9.0-x86_64-linux-gnu)
     sqlite3 (2.9.0-x86_64-linux-musl)
+    strings (0.2.1)
+      strings-ansi (~> 0.2)
+      unicode-display_width (>= 1.5, < 3.0)
+      unicode_utils (~> 1.4)
+    strings-ansi (0.2.0)
     timeout (0.6.0)
+    tty-box (0.7.0)
+      pastel (~> 0.8)
+      strings (~> 0.2.0)
+      tty-cursor (~> 0.7)
+    tty-color (0.6.0)
+    tty-cursor (0.7.1)
+    tty-screen (0.8.2)
+    tty-spinner (0.9.3)
+      tty-cursor (~> 0.7)
+    tty-table (0.12.0)
+      pastel (~> 0.8)
+      strings (~> 0.2.0)
+      tty-screen (~> 0.8)
     tzinfo (2.0.6)
       concurrent-ruby (~> 1.0)
+    unicode-display_width (2.6.0)
+    unicode_utils (1.4.0)
     uri (1.1.1)
     useragent (0.16.11)
 
@@ -166,6 +193,7 @@ PLATFORMS
 DEPENDENCIES
   bundler (~> 2.5)
   minitest (~> 5.0)
+  ostruct (~> 0.6)
   rails_benchmark_suite!
   rake (~> 13.0)
 

data/README.md

@@ -26,6 +26,19 @@ The Heft Score is a weighted metric representing a machine's ability to handle R
 | **150+** | 🚀 High Performance | Apple M-series Pro/Max, Ryzen 5000+ |
 | **300+** | ⚡ Blazing | Server-grade Metal, M3 Ultra |
 
+### 📊 Visual Reports (HTML)
+
+Visualize your threading efficiency and bottlenecks with a self-contained dashboard:
+
+```bash
+bundle exec rails_benchmark_suite -t 8 --html
+```
+
+This generates `rails_benchmark_report.html` containing:
+- **Scaling Curve**: Interactive bar chart comparing 1-thread vs Max-thread throughput.
+- **Efficiency Heatmap**: Score cards identifying which workloads are Gil-bound vs CPU-bound.
+- **Detailed Metrics**: Raw IPS and detailed scaling factors.
+
 ---
 
 ## 🚀 Quick Start
@@ -56,19 +69,67 @@ Rails Benchmark Suite prioritizes **Benchmarking** (via `benchmark-ips`) over **
 * **Rails:** 8.1+
 * **Database:** SQLite3
 
-### Use within a Rails Application
+## 📋 System Requirements
+
+### Required
+- **Ruby**: 3.3+ (3.4+ recommended for YJIT)
+- **Rails**: 8.0+  
+- **Database**: SQLite3
+
+### Optional (for Image Heft workload)
+- **macOS**: `brew install vips`
+- **Linux (Ubuntu/Debian)**: `sudo apt install libvips-dev`
+
+---
+
+## 🚀 Usage
+
+### Standard Execution
+```bash
+bundle exec rails_benchmark_suite
+```
+The easy way - run the benchmark with your current Ruby configuration.
+
+### High Performance (Recommended)
+```bash
+RUBY_OPT="--yjit" bundle exec rails_benchmark_suite
+```
+Enable YJIT for maximum performance measurement accuracy. This is the recommended method for Rails 8+ benchmarking.
+
+### JSON Export (Automation)
+```bash
+bundle exec rails_benchmark_suite --json > report.json
+```
+Perfect for CI/CD pipelines and programmatic analysis. Outputs clean JSON without any UI elements.
+
+### 📊 Visual Diagnostics (HTML Report)
+
+![Rails Benchmark Suite Report](docs/images/report_v0_3_1.png)
 
-Add to your Gemfile:
+To diagnose scaling bottlenecks, run the tool in Profile Mode (`--profile`) and generate the HTML Report (`--html`). This compares Single-Thread vs Multi-Thread performance side-by-side.
 
-```ruby
-gem "rails_benchmark_suite", group: :development
+```bash
+bundle exec rails_benchmark_suite --profile --html
 ```
 
-### Usage Flags
-* `--yjit`: Enables the Ruby JIT compiler (significant for Rails 8+ performance).
-* `-S`: Corrects the path to look for the executable in your current bundle.
-* `--json`: For programmatic consumption of results.
-* `--skip-rails`: To ignore the host application and run in isolated mode.
+**Opening the Report**:
+- **Local:** `open tmp/rails_benchmark_report.html`
+- **Remote:** `scp user@server:/current/path/tmp/rails_benchmark_report.html .`
+
+### Command Line Options
+
+| Flag | Description |
+| :--- | :--- |
+| `--html` | Generates a visual dashboard (`tmp/rails_benchmark_report.html`). **Best used with `--profile`.** |
+| `--profile` | **Diagnostic Mode.** Runs the benchmark twice (1 Thread vs Max Threads) to calculate "Scaling Efficiency." Required to populate the "Scaling Curve" charts. |
+| `--db` | Uses your local `config/database.yml` (Postgres/MySQL) instead of the synthetic in-memory SQLite. |
+| `-t [N]` | Manually set the thread count (Default: Auto-detects CPU cores). |
+
+### Configuration Flags
+- `--json`: Output results in JSON format
+- `--skip-rails`: Run in isolated mode without loading Rails environment
+- `--version`: Display gem version
+- `-h` / `--help`: Show usage help
 
 ### Standalone Usage
 
@@ -83,7 +144,39 @@ bin/rails_benchmark_suite
 
 ---
 
-## 🧪 The "Heft" Suites
+## 📐 How It's Calculated
+
+The **Rails Heft Index (RHI)** measures your hardware's ability to handle Rails workloads using this formula:
+
+```
+RHI Score = Σ (4-Thread IPS × Weight)
+```
+
+### Workload Weights
+
+| Workload | Weight | Rationale |
+|----------|--------|-----------|  
+| **Active Record** | 40% | Database operations are the core of most Rails apps |
+| **View Rendering** | 20% | ERB/ActionView processing |
+| **Solid Queue** | 20% | Background job throughput |
+| **Cache Operations** | 10% | Memory store performance |
+| **Image Processing** | 10% | Optional - requires libvips |
+
+**Why 4-Thread IPS?** We use 4-thread performance to simulate production concurrency where multiple requests are handled simultaneously.
+
+**Dynamic Weight Redistribution:** If a workload is skipped (e.g., Image Processing without libvips), its weight is redistributed proportionally among remaining workloads to maintain a 100% scale.
+
+### Hardware Tiers
+
+Your RHI score maps to these performance tiers:
+
+- **< 50**: Entry/Dev - Suitable for local development
+- **50-200**: Production-Ready - Handles moderate production traffic
+- **> 200**: High-Performance - Optimized for high-traffic applications
+
+---
+
+## 🧪 The "Heft" Workloads
 
 The gem measures performance across critical Rails subsystems using a dedicated, isolated schema:
 

data/bin/rails_benchmark_suite

@@ -2,11 +2,13 @@
 # frozen_string_literal: true
 
 require "optparse"
+require "etc"
 $LOAD_PATH.unshift File.expand_path("../lib", __dir__)
 require "rails_benchmark_suite"
 require "rails_benchmark_suite/version"
 
-options = {}
+config = RailsBenchmarkSuite::Configuration.new
+
 OptionParser.new do |opts|
   opts.banner = "Usage: rails_benchmark_suite [options]"
 
@@ -15,12 +17,28 @@ OptionParser.new do |opts|
     exit 0
   end
 
+  opts.on("-t", "--threads N", Integer, "Number of threads (default: auto set to #{Etc.nprocessors} cores)") do |n|
+    config.threads = n
+  end
+
+  opts.on("-p", "--profile", "Enable scaling efficiency profile mode") do
+    config.profile = true
+  end
+
+  opts.on("-d", "--db", "Use real database from config/database.yml instead of in-memory SQLite") do
+    config.db = true
+  end
+
   opts.on("--skip-rails", "Skip Rails environment loading") do
-    options[:skip_rails] = true
+    config.skip_rails = true
   end
 
   opts.on("-j", "--json", "Output results as JSON") do
-    options[:json] = true
+    config.json = true
+  end
+
+  opts.on("--html", "Generate a visual HTML report (rails_benchmark_report.html)") do
+    config.html = true
   end
 
   opts.on("-h", "--help", "Prints this help") do
@@ -29,21 +47,26 @@ OptionParser.new do |opts|
   end
 end.parse!
 
-puts "RailsBenchmarkSuite v#{RailsBenchmarkSuite::VERSION}"
+unless config.json
+  puts "RailsBenchmarkSuite v#{RailsBenchmarkSuite::VERSION}"
+  puts "Configuration: Threads=#{config.threads} | Profile=#{config.profile} | Local DB=#{config.db}"
+end
 
 # Rails Detection
 rails_env_path = File.join(Dir.pwd, "config", "environment.rb")
 
-if !options[:skip_rails] && File.exist?(rails_env_path)
-  puts "Rails environment detected at #{rails_env_path}. Loading..."
+if !config.skip_rails && File.exist?(rails_env_path)
+  puts "Rails environment detected at #{rails_env_path}. Loading..." unless config.json
   require rails_env_path
-  puts "Rails loaded successfully."
+  puts "Rails loaded successfully." unless config.json
 else
-  if options[:skip_rails]
-    puts "Skipping Rails loading (requested via --skip-rails)."
-  else
-    puts "No Rails environment detected (config/environment.rb not found)."
+  unless config.json
+    if config.skip_rails
+      puts "Skipping Rails loading (requested via --skip-rails)."
+    else
+      puts "No Rails environment detected (config/environment.rb not found)."
+    end
   end
 end
 
-RailsBenchmarkSuite.run(json: options[:json])
+RailsBenchmarkSuite::Runner.new(config).run

data/lib/dummy/app/models/benchmark_job.rb

@@ -0,0 +1,7 @@
+module RailsBenchmarkSuite
+  module Dummy
+    class BenchmarkJob < ActiveRecord::Base
+      self.table_name = "simulated_jobs" # Keep table name consistent or rename? Let's use simulated_jobs as it's descriptive
+    end
+  end
+end

data/lib/dummy/app/models/benchmark_post.rb

@@ -0,0 +1,9 @@
+module RailsBenchmarkSuite
+  module Dummy
+    class BenchmarkPost < ActiveRecord::Base
+      self.table_name = "benchmark_posts"
+
+      belongs_to :user, class_name: "RailsBenchmarkSuite::Dummy::BenchmarkUser", foreign_key: "benchmark_user_id"
+    end
+  end
+end

data/lib/dummy/app/models/benchmark_user.rb

@@ -0,0 +1,9 @@
+module RailsBenchmarkSuite
+  module Dummy
+    class BenchmarkUser < ActiveRecord::Base
+      self.table_name = "benchmark_users"
+
+      has_many :posts, class_name: "RailsBenchmarkSuite::Dummy::BenchmarkPost", foreign_key: "benchmark_user_id"
+    end
+  end
+end

data/lib/dummy/app/views/rails_benchmark_suite/heft_view.html.erb

@@ -0,0 +1,11 @@
+<h1>Dashboard for <%= user.name %></h1>
+<ul>
+  <% posts.each do |post| %>
+    <li>
+      <strong><%= post.title %></strong>
+      <p><%= post.body.truncate(50) %></p>
+      <small>Views: <%= number_with_delimiter(post.views) %></small>
+    </li>
+  <% end %>
+</ul>
+<footer>Generated at <%= Time.now.to_s %></footer>

data/lib/dummy/config/benchmark_database.yml

@@ -0,0 +1,16 @@
+# Dummy DB Config for isolated benchmarking
+test:
+  adapter: sqlite3
+  database: ":memory:"
+  pool: 50
+  timeout: 5000
+
+development:
+  adapter: sqlite3
+  database: "file:dummy_bench.sqlite3?mode=memory&cache=shared"
+  pool: 50
+  timeout: 30000 
+  # High performance settings
+  variables:
+    journal_mode: WAL
+    synchronous: NORMAL

data/lib/rails_benchmark_suite/configuration.rb

@@ -0,0 +1,22 @@
+# frozen_string_literal: true
+
+require "etc"
+
+module RailsBenchmarkSuite
+  class Configuration
+    attr_accessor :threads, :profile, :db, :skip_rails, :json, :html
+
+    def initialize
+      @threads = Etc.nprocessors
+      @profile = false
+      @db = false
+      @skip_rails = false
+      @json = false
+      @html = false
+    end
+
+    def db_mode
+      @db ? "Local DB" : "SQLite (Memory)"
+    end
+  end
+end

data/lib/rails_benchmark_suite/database_manager.rb

@@ -0,0 +1,56 @@
+# frozen_string_literal: true
+
+require "active_record"
+require "yaml"
+
+module RailsBenchmarkSuite
+  class DatabaseManager
+    SETUP_MUTEX = Mutex.new
+
+    def setup(use_local_db: false)
+      # Silence migrations
+      ActiveRecord::Migration.verbose = false
+
+      if use_local_db
+        setup_real_database
+      else
+        setup_dummy_database
+      end
+    end
+
+    private
+
+    def setup_real_database
+      config_path = File.join(Dir.pwd, "config", "database.yml")
+      unless File.exist?(config_path)
+        raise "Database config not found at #{config_path} (required for --db option)"
+      end
+
+      db_config = YAML.load_file(config_path)
+      env = defined?(Rails) ? Rails.env : "development"
+      
+      ActiveRecord::Base.establish_connection(db_config[env])
+      puts "Connected to local database (#{env})"
+    end
+
+    def setup_dummy_database
+      # Load internal dummy config
+      config_path = File.expand_path("../dummy/config/benchmark_database.yml", __dir__)
+      db_config = YAML.load_file(config_path)
+      
+      # Use "development" profile which has the PRAGMA optimizations
+      ActiveRecord::Base.establish_connection(db_config["development"])
+
+      # Apply manual optimizations if needed (though config should handle it)
+      conn = ActiveRecord::Base.connection.raw_connection
+      conn.busy_timeout = 10000
+      
+      # Setup Schema once safely
+      SETUP_MUTEX.synchronize do
+        unless ActiveRecord::Base.connection.table_exists?(:benchmark_users)
+          RailsBenchmarkSuite::Schema.load
+        end
+      end
+    end
+  end
+end

data/lib/rails_benchmark_suite/db_setup.rb

@@ -5,3 +5,6 @@ require "sqlite3"
 
 # Silence ActiveRecord logs during benchmarks to avoid IO bottlenecks
 ActiveRecord::Base.logger = nil
+
+# Silence migration output
+ActiveRecord::Migration.verbose = false

data/lib/rails_benchmark_suite/models/user.rb

@@ -1,9 +1,10 @@
 # frozen_string_literal: true
 
+require "rails_benchmark_suite/dummy/app/models/benchmark_user"
+require "rails_benchmark_suite/dummy/app/models/benchmark_post"
+
 module RailsBenchmarkSuite
   module Models
-    class User < ActiveRecord::Base
-      has_many :posts, class_name: "RailsBenchmarkSuite::Models::Post", dependent: :destroy
-    end
+    # Proxy or Legacy support if needed, but primary logic is now in Dummy namespace
   end
 end