rails_benchmark_suite 0.2.7 → 0.2.9

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: b72123961fe15e37146749930e0bd09d4d52a821c739a7ec662bc84976f9ac3b
-  data.tar.gz: 8fb581b9a516e6eaec6b664d19bb9354f115bfb61282c1a3cb289d1d72ac203c
+  metadata.gz: c48ce5faaa5348bcd47cdd9c88ba9b1441d0f7bcd0a6fa601d84b36014673d1a
+  data.tar.gz: f39f188c55fe6749a5b9ba4ae9c5c44e926bd5d6588737f689192ab841deda92
 SHA512:
-  metadata.gz: 63b8fdab356d3d60e2aa05f4e99df82da0617a988da5c92bf8ec430ed3016fa4e594c856ecfc0f03d9aee42bc63a365e54dc254014614124bd851339e6962ac9
-  data.tar.gz: 96d310ed31f7ce0476a824c6a086ec5acbe4fa4cdfa5e9282b285a4dec9133f41997e5f534e07eb537a2d67b5c1a46b4e258aba2c97f491ac0088b131a407477
+  metadata.gz: 28f32ff03be6cbd83aabb1c3f8750316f5c3e2cb69de127885f96a4ef27841b12ee916999c2dea59c314605a6755f0aee34d0c610338c596bbff2789120a7ce0
+  data.tar.gz: 2e4f5a17c66e0caed9a95a46cdc3e0049783eea603f9cd263d699f492d49440f7ceaab45bf4c469b8b05fc316ca999052d12705dfe29b0edf54b032160fc2966

data/README.md

@@ -1,23 +1,44 @@
-# Rails Benchmark Suite
+# Rails Benchmark Suite 🚀
+
+**Standardized Hardware Benchmarking for Rails 8.1+**
 
 A standardized performance suite designed to measure the "Heft" of a machine using realistic, high-throughput Rails 8+ workloads.
 
-Unlike synthetic CPU benchmarks, **Rails Benchmark Suite** simulates Active Record object allocation, SQL query complexity, ActionView rendering, and background job throughput.
+## 🛠 What is this?
+
+Think of this as a **"Test Track" for Rails servers**. Unlike profilers that measure your specific application code, this gem runs a **fixed, standardized set of Rails operations** (Active Record object allocation, SQL query complexity, ActionView rendering, and background job throughput) to measure the raw performance of your server and Ruby configuration.
+
+To ensure a level playing field, the gem boots an **isolated, in-memory SQLite environment**. It creates its own schema and records, meaning it **never touches your production data** and returns comparable results across any machine.
 
 ## 📊 The "Heft" Score
 
-The Heft Score is a weighted metric representing a machine's ability to handle Rails tasks. 
-- **Baseline:** A score of **100** is calibrated to represent an **AWS c6g.large** (ARM) instance.
-- **Objective:** To provide a simple, comparable number for evaluating different computing platforms (Cloud VMs, bare-metal, or local dev rigs).
+The Heft Score is a weighted metric representing a machine's ability to handle Rails tasks.
+* **Baseline:** A score of **100** is calibrated to represent an **AWS c6g.large** (ARM) instance.
+* **Objective:** To provide a simple, comparable number for evaluating different computing platforms (Cloud VMs, bare-metal, or local dev rigs).
 
 ### Baseline Comparisons
+
 | Score | Classification | Comparable Hardware |
 | :--- | :--- | :--- |
-| < 40 | 🐢 Sluggish | Older Intel Macs, Entry-level VPS |
-| 60 | 🚙 Capable | Standard Cloud VM (c5.large/standard) |
-| **100** | **🏎️ Baseline** | **AWS c6g.large (2 vCPU ARM)** |
-| 150+ | 🚀 High Performance | Apple M-series Pro/Max, Ryzen 5000+ |
-| 300+ | ⚡ Blazing | Server-grade Metal, M3 Ultra |
+| **< 40** | 🐢 Sluggish | Older Intel Macs, Entry-level VPS |
+| **60** | 🚙 Capable | Standard Cloud VM (c5.large/standard) |
+| **100** | 🏎️ Baseline | AWS c6g.large (2 vCPU ARM) |
+| **150+** | 🚀 High Performance | Apple M-series Pro/Max, Ryzen 5000+ |
+| **300+** | ⚡ Blazing | Server-grade Metal, M3 Ultra |
+
+---
+
+## 🚀 Quick Start
+
+Ensure you are in your Rails root directory and run:
+
+```bash
+ruby --yjit -S bundle exec rails_benchmark_suite
+```
+
+**Note:** `bundle exec` is mandatory for Rails environment stability and to prevent Minitest version conflicts.
+
+---
 
 ## 🛠 Technical Philosophy
 
@@ -26,28 +47,16 @@ Rails Benchmark Suite prioritizes **Benchmarking** (via `benchmark-ips`) over **
 * **Benchmarking:** Focuses on macro-throughput—"How many iterations can the hardware handle?" This provides the final Heft Score.
 * **Why no Profiling?** Profiling tools (like `StackProf` or `Vernier`) introduce instrumentation overhead that skews hardware metrics. We aim for "Conceptual Compression"—one clear number to inform infrastructure decisions.
 
+---
+
 ## 🚀 Installation & Usage
 
 ### Requirements
-- **Ruby**: 3.3+ (Ruby with YJIT support highly recommended for accurate performance measurement)
-- **Database**: SQLite3
-
-### Prerequisites
-* **Ruby:** 3.4.1+ (Recommended for latest YJIT/Prism performance)
+* **Ruby:** 3.3+ (Ruby with YJIT support highly recommended)
+* **Rails:** 8.1+
 * **Database:** SQLite3
 
-### Standalone Usage
-If you want to test hardware performance without an existing application:
-
-```bash
-git clone https://github.com/overnet/rails_benchmark_suite.git
-cd rails_benchmark_suite
-bundle install
-bin/rails_benchmark_suite
-```
-
 ### Use within a Rails Application
-Rails Benchmark Suite is "Rails-aware." Adding it to your app allows you to benchmark your specific configuration and custom suites.
 
 Add to your Gemfile:
 
@@ -55,84 +64,69 @@ Add to your Gemfile:
 gem "rails_benchmark_suite", group: :development
 ```
 
-## Usage
+### Usage Flags
+* `--yjit`: Enables the Ruby JIT compiler (significant for Rails 8+ performance).
+* `-S`: Corrects the path to look for the executable in your current bundle.
+* `--json`: For programmatic consumption of results.
+* `--skip-rails`: To ignore the host application and run in isolated mode.
 
-To get the most accurate 'Heft' score, run with YJIT enabled:
-
-```bash
-ruby --yjit -S bundle exec rails_benchmark_suite
-```
-
-**Why these flags?**
-- `--yjit`: Enables the Ruby JIT compiler (significant for Rails 8+ performance).
-- `-S`: Corrects the path to look for the executable in your current bundle.
-- `bundle exec`: Prevents version conflicts (e.g., Minitest) between the gem and your host application.
+### Standalone Usage
 
-**Standalone Usage:**
-If running outside a Rails project:
+If you want to test hardware performance without an existing application:
 
 ```bash
+git clone https://github.com/overnet/rails_benchmark_suite.git
+cd rails_benchmark_suite
+bundle install
 bin/rails_benchmark_suite
 ```
 
-**JSON Output:**
-For programmatic consumption:
-
-```bash
-ruby --yjit -S bundle exec rails_benchmark_suite --json
-```
-
-> **Note:** Use `--skip-rails` to ignore the host application and run in isolated mode.
+---
 
-## Performance
+## 🧪 The "Heft" Suites
 
-**Understanding YJIT Status:**
+The gem measures performance across critical Rails subsystems using a dedicated, isolated schema:
 
-If you see `YJIT: Disabled`, this is a result of your Ruby binary build, not the gem itself. The gem will work perfectly fine without YJIT, but performance measurements will be more accurate with YJIT enabled.
+* **Active Record Heft:** Standardized CRUD: Creation, indexing, and complex querying.
+* **Cache Heft:** High-frequency read/writes to the Rails memory store.
+* **Solid Queue Heft:** Background job enqueuing and database-backed polling stress.
+* **View Heft:** Partial rendering overhead and ActionView throughput.
+* **Image Heft:** Image processing performance (requires libvips).
 
-To enable YJIT, you need Ruby compiled with YJIT support (requires Rust compiler during Ruby installation). See the Troubleshooting section below for installation instructions.
+---
 
-## Troubleshooting
+## ⚠️ Troubleshooting
 
 ### YJIT Shows "Disabled"
 
-If you see `YJIT: Disabled (Requires Ruby with YJIT support for best results)`, it means your Ruby was not compiled with YJIT support. To get the best performance:
+If you see `YJIT: Disabled`, it means your Ruby was not compiled with YJIT support.
 
-1. Ensure `rustc` (Rust compiler) is installed on your system
-2. Reinstall Ruby with YJIT support:
-   ```bash
-   # Using rbenv
-   RUBY_CONFIGURE_OPTS="--enable-yjit" rbenv install 3.4.1
-   
-   # Using rvm
-   rvm install 3.4.1 --enable-yjit
-   ```
-3. Verify YJIT is available: `ruby --yjit -e "puts RubyVM::YJIT.enabled?"`
+* **Fix (rbenv):** `RUBY_CONFIGURE_OPTS="--enable-yjit" rbenv install 3.4.1`
+* **Fix (rvm):** `rvm install 3.4.1 --enable-yjit`
 
 ### SQLite Lock Errors
 
-If you encounter `SQLite3::BusyException` or "database table is locked" errors, ensure you're running the latest version of the gem (v0.2.7+) which includes automatic concurrency retries with smart backoff.
+Version 0.2.9+ includes surgical connection resets and randomized backoffs to handle SQLite concurrency. If issues persist, ensure no other processes are accessing the benchmark database.
 
-**v0.2.7+** includes:
-- Automatic retry logic with sleep backoff
-- Per-thread unique identifiers to prevent conflicts
-- Optimized busy timeout settings (10 seconds)
-
-If issues persist, try reducing concurrency or ensuring no other processes are accessing the benchmark database.
+---
 
 ## 🏗 Architecture
+
 * **Engine:** Built on `benchmark-ips`.
-* **Database:** Uses In-Memory SQLite with `cache=shared` for multi-threaded accuracy.
-* **Isolation:** Uses transactional rollbacks (`ActiveRecord::Rollback`) to ensure test isolation without the overhead of row deletion.
-* **Threading:** Supports 1-thread and 4-thread scaling tests to measure vertical efficiency.
-* **Modern Stack:** Optimized for Rails 8+ defaults, including Solid Queue simulation and YJIT detection.
+* **Database:** Uses In-Memory SQLite with `cache=shared` and a 50-connection pool for multi-threaded accuracy.
+* **Isolation:** Uses transactional rollbacks and Mutex-wrapped schema creation.
+* **Threading:** Supports 1-thread and 4-thread scaling tests.
+
+---
 
 ## 📜 Credits
-This project is a functional implementation of the performance benchmark vision discussed in the Rails community.
 
 * **Vision:** Inspired by @dhh in [rails/rails#50451](https://github.com/rails/rails/issues/50451).
 * **Initial Roadmap:** Based on suggestions by @JoeDupuis.
 * **Implementation:** The Rails Community.
 
+---
+
 ## 📄 License
+
 The gem is available as open source under the terms of the MIT License.

data/lib/rails_benchmark_suite/runner.rb

@@ -17,12 +17,12 @@ module RailsBenchmarkSuite
     SETUP_MUTEX = Mutex.new
 
     def run
-      # Hardened Isolation: Shared Cache URI for multi-threading (v0.2.7)
+      # Ultimate Hardening: Massive pool and timeout for zero lock contention (v0.2.9)
       ActiveRecord::Base.establish_connection(
         adapter: "sqlite3",
         database: "file:heft_db?mode=memory&cache=shared",
-        pool: 16,
-        timeout: 10000
+        pool: 50,
+        timeout: 30000
       )
 
       # The 'Busy Timeout' Hammer - force it directly on the raw connection
@@ -36,8 +36,10 @@ module RailsBenchmarkSuite
         end
       end
 
-      # High-Performance Pragmas
-      ActiveRecord::Base.connection.raw_connection.execute("PRAGMA synchronous = OFF")
+      # High-Performance Pragmas for WAL + NORMAL sync
+      ActiveRecord::Base.connection.raw_connection.execute("PRAGMA journal_mode = WAL")
+      ActiveRecord::Base.connection.raw_connection.execute("PRAGMA synchronous = NORMAL")
+      ActiveRecord::Base.connection.raw_connection.execute("PRAGMA mmap_size = 268435456") # 256MB - reduce disk I/O
 
       puts "Running RailsBenchmarkSuite Benchmarks..." unless @json_output
       puts system_report unless @json_output
@@ -57,7 +59,7 @@ module RailsBenchmarkSuite
           
           # Single Threaded
           x.report("#{suite[:name]} (1 thread)") do
-            suite[:block].call
+            with_retries { suite[:block].call }
           end
 
           # Multi Threaded (4 threads)
@@ -66,7 +68,7 @@ module RailsBenchmarkSuite
               Thread.new do
                 # Ensure each thread gets a dedicated connection
                 ActiveRecord::Base.connection_pool.with_connection do
-                  suite[:block].call
+                  with_retries { suite[:block].call }
                 end
               end
             end
@@ -95,6 +97,19 @@ module RailsBenchmarkSuite
 
     private
 
+    def with_retries
+      yield
+    rescue ActiveRecord::StatementInvalid => e
+      if e.message =~ /locked/i
+        # Specifically drop the lock for THIS connection only
+        ActiveRecord::Base.connection.reset!
+        sleep(rand(0.01..0.05))
+        retry
+      else
+        raise e
+      end
+    end
+
     def system_report
       info = RailsBenchmarkSuite::Reporter.system_info
       yjit_status = if defined?(RubyVM::YJIT) && RubyVM::YJIT.enabled?

data/lib/rails_benchmark_suite/suites/active_record_suite.rb

@@ -6,39 +6,30 @@ require "active_record"
 RailsBenchmarkSuite.register_suite("Active Record Heft", weight: 0.4) do
   # Workload: Create User with Posts, Join Query, Update
   # Use transaction rollback to keep the DB clean and avoid costly destroy callbacks
-  begin
-    ActiveRecord::Base.transaction do
-      # 1. Create - with unique email per thread
-      user = RailsBenchmarkSuite::Models::User.create!(
-        name: "Benchmark User", 
-        email: "test-#{Thread.current.object_id}@example.com"
-      )
-      
-      # 2. Create associated records (simulate some weight)
-      10.times do |i|
-        user.posts.create!(title: "Post #{i}", body: "Content " * 50)
-      end
-      
-      # 3. Complex Query (Join + Order)
-      # Unloading the relation to force execution
-      RailsBenchmarkSuite::Models::User.joins(:posts)
-                  .where(users: { id: user.id })
-                  .where("posts.views >= ?", 0)
-                  .order("posts.created_at DESC")
-                  .to_a
-      
-      # 4. Update
-      user.update!(name: "Updated User")
-      
-      # Rollback everything to leave the DB clean for next iteration
-      raise ActiveRecord::Rollback
-    end
-  rescue ActiveRecord::StatementInvalid => e
-    if e.message =~ /locked/i
-      sleep(0.005)
-      retry
-    else
-      raise e
+  ActiveRecord::Base.transaction do
+    # 1. Create - with unique email per thread
+    user = RailsBenchmarkSuite::Models::User.create!(
+      name: "Benchmark User", 
+      email: "test-#{Thread.current.object_id}@example.com"
+    )
+    
+    # 2. Create associated records (simulate some weight)
+    10.times do |i|
+      user.posts.create!(title: "Post #{i}", body: "Content " * 50)
     end
+    
+    # 3. Complex Query (Join + Order)
+    # Unloading the relation to force execution
+    RailsBenchmarkSuite::Models::User.joins(:posts)
+                .where(users: { id: user.id })
+                .where("posts.views >= ?", 0)
+                .order("posts.created_at DESC")
+                .to_a
+    
+    # 4. Update
+    user.update!(name: "Updated User")
+    
+    # Rollback everything to leave the DB clean for next iteration
+    raise ActiveRecord::Rollback
   end
 end

data/lib/rails_benchmark_suite/suites/image_heft_suite.rb

@@ -1,5 +1,6 @@
 # frozen_string_literal: true
 
+# Check if vips is available at registration time
 begin
   require "image_processing/vips"
   require "fileutils"
@@ -9,6 +10,7 @@ begin
   FileUtils.mkdir_p(ASSET_DIR)
   SAMPLE_IMAGE = File.join(ASSET_DIR, "sample.jpg")
 
+  # Only register if vips is actually available
   RailsBenchmarkSuite.register_suite("Image Heft", weight: 0.1) do
     # Gracefully handle missing dependencies
     if File.exist?(SAMPLE_IMAGE)
@@ -22,13 +24,8 @@ begin
     end
   end
 
-rescue LoadError, StandardError => e
-  # Register a skipped suite if Libvips is unavailable
-  RailsBenchmarkSuite.register_suite("Image Heft (Skipped)", weight: 0.0) do
-    @warned ||= begin
-      warn "⚠️  [RailsBenchmarkSuite] ImageHeft skipped: #{e.message}. Install libvips to enable."
-      true
-    end
-  end
+rescue LoadError, StandardError
+  # Don't register the suite at all if vips is unavailable
+  puts "⚠️  Skipping Image Heft: libvips not available. Install with: brew install vips (macOS)"
 end
 

data/lib/rails_benchmark_suite/version.rb

@@ -1,3 +1,3 @@
 module RailsBenchmarkSuite
-  VERSION = "0.2.7"
+  VERSION = "0.2.9"
 end

metadata

@@ -1,13 +1,13 @@
 --- !ruby/object:Gem::Specification
 name: rails_benchmark_suite
 version: !ruby/object:Gem::Version
-  version: 0.2.7
+  version: 0.2.9
 platform: ruby
 authors:
 - RailsBenchmarkSuite Contributors
 bindir: bin
 cert_chain: []
-date: 2026-01-02 00:00:00.000000000 Z
+date: 2026-01-03 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: benchmark-ips