rails_benchmark_suite 0.2.9 → 0.3.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: c48ce5faaa5348bcd47cdd9c88ba9b1441d0f7bcd0a6fa601d84b36014673d1a
4
- data.tar.gz: f39f188c55fe6749a5b9ba4ae9c5c44e926bd5d6588737f689192ab841deda92
3
+ metadata.gz: 150b032cfebbd12d60c2a407a292cf04367c4b821d44541aa21c7b01243b5288
4
+ data.tar.gz: 5fcffc2550db42c299f3c417abb11c9b2d8c3bdea7a27ecf3f8fd7c1d624bf31
5
5
  SHA512:
6
- metadata.gz: 28f32ff03be6cbd83aabb1c3f8750316f5c3e2cb69de127885f96a4ef27841b12ee916999c2dea59c314605a6755f0aee34d0c610338c596bbff2789120a7ce0
7
- data.tar.gz: 2e4f5a17c66e0caed9a95a46cdc3e0049783eea603f9cd263d699f492d49440f7ceaab45bf4c469b8b05fc316ca999052d12705dfe29b0edf54b032160fc2966
6
+ metadata.gz: 785219aa58ac8b7353bf67920c4f419bf628d92f2fc539f02bd09408fb7aac13a6352d5aa68a09f0c60a7f1d95c34e016d12bb4ab015544ecf6962ca0c72aa54
7
+ data.tar.gz: 81706842da33247fbace2b62c4dee9f42332ff83a9ed1ab905960296f4954f6915c4055abe97910d9a2e572472d9454ebe7e01fce17dba0b604cd4b43f2f4988
data/.gitignore CHANGED
@@ -1,4 +1,4 @@
1
- *.gem
1
+
2
2
  .bundle/
3
3
  vendor/bundle/
4
4
  Gemfile.lock
data/CHANGELOG.md CHANGED
@@ -1,5 +1,51 @@
1
1
  # Changelog
2
2
 
3
+ ## [0.3.0] - 2025-01-03
4
+
5
+ ### Major Architectural Refactor (SRP)
6
+ - **Runner Split**: Dismantled the monolithic `Runner` class (260+ lines) into three specialized modules:
7
+ - `DatabaseManager`: Handles ActiveRecord connection, schema loading, and SQLite PRAGMA optimizations
8
+ - `WorkloadRunner`: Manages benchmark execution engine with BASE_WEIGHTS normalization and complete payload generation
9
+ - `Formatter`: Centralized UI rendering, ANSI colors, insights engine, and output formatting
10
+ - **Runner Coordinator**: `Runner` is now a minimal 23-line coordinator delegating to the three modules
11
+
12
+ ### Normalized RHI Math Engine
13
+ - **BASE_WEIGHTS**: Defined workload weights (Active Record: 0.4, View: 0.2, Solid Queue: 0.2, Cache: 0.1, Image: 0.1)
14
+ - **Dynamic Weight Redistribution**: When workloads are skipped (e.g., missing libvips), weights are normalized proportionally to maintain 100% scale
15
+ - **Formula**: `RHI Score = Σ (4T_IPS × Adjusted_Weight)` where adjusted weights always sum to 1.0
16
+
17
+ ### Performance Insights Engine
18
+ - **Scaling Analysis**: Warns when multi-threading scaling < 0.8x, indicating SQLite lock contention or Ruby GIL saturation
19
+ - **YJIT Detection**: Displays hint to enable YJIT when disabled (typical 15-25% boost)
20
+ - **Memory Monitoring**: Alerts when workload memory growth exceeds 20MB, suggesting heavy object allocation
21
+ - **Hardware Tiering**: Provides comparison labels (Entry/Dev < 50, Production-Ready 50-200, High-Performance > 200)
22
+
23
+ ### UI/UX Enhancements
24
+ - **Box Alignment**: Fixed header and final score boxes with proper text length calculation (60-char width)
25
+ - **Table Spacing**: Added separator line between progress logs and results table for better readability
26
+ - **Insights Display**: Integrated insights below summary table with emoji indicators (💡, 📊)
27
+ - **Enhanced Number Formatting**: Smart k/M suffixes for readability (e.g., "15.3k", "1.2M")
28
+ - **YJIT Hints**: Helpful reminder `(run with RUBY_OPT="--yjit" for max perf)` when YJIT is disabled
29
+ - Silent migrations: Added `ActiveRecord::Migration.verbose = false` to reduce noise
30
+ - Cross-platform install instructions for libvips (macOS and Linux)
31
+
32
+ ### 🐛 Fixes
33
+ - Fixed `.gitignore` to properly track `gemspec` file for gem distribution
34
+ - **JSON Guard**: Ensures clean JSON output without any UI noise when `--json` flag is used
35
+ - Improved CLI output suppression in JSON mode
36
+
37
+ ### 📖 Documentation
38
+ - **Calculation Formula**: Added "How It's Calculated" section with RHI formula: `Σ (4-Thread IPS × Weight)`
39
+ - **Workload Weights Table**: Documented weights (Active Record 40%, View 20%, Jobs 20%, Cache 10%, Image 10%)
40
+ - **Hardware Tiers**: Explained tier classification system
41
+ - Complete README rewrite with four execution methods:
42
+ - Standard: `bundle exec rails_benchmark_suite`
43
+ - High Performance: `RUBY_OPT="--yjit" bundle exec rails_benchmark_suite`
44
+ - JSON Export: `bundle exec rails_benchmark_suite --json > report.json`
45
+ - Standalone: `bin/rails_benchmark_suite`
46
+ - Added comprehensive System Requirements section
47
+ - Updated all terminology from "Suite" to "Workload"
48
+
3
49
  ## [0.2.0] - 2025-12-31
4
50
 
5
51
  ### Added
data/Gemfile.lock CHANGED
@@ -1,7 +1,7 @@
1
1
  PATH
2
2
  remote: .
3
3
  specs:
4
- rails_benchmark_suite (0.2.6)
4
+ rails_benchmark_suite (0.3.0)
5
5
  actionview (~> 8.1)
6
6
  activerecord (~> 8.1)
7
7
  activestorage (~> 8.1)
@@ -116,6 +116,7 @@ GEM
116
116
  racc (~> 1.4)
117
117
  nokogiri (1.19.0-x86_64-linux-musl)
118
118
  racc (~> 1.4)
119
+ ostruct (0.6.3)
119
120
  racc (1.8.1)
120
121
  rack (3.2.4)
121
122
  rack-session (2.1.1)
@@ -166,6 +167,7 @@ PLATFORMS
166
167
  DEPENDENCIES
167
168
  bundler (~> 2.5)
168
169
  minitest (~> 5.0)
170
+ ostruct (~> 0.6)
169
171
  rails_benchmark_suite!
170
172
  rake (~> 13.0)
171
173
 
data/README.md CHANGED
@@ -56,19 +56,43 @@ Rails Benchmark Suite prioritizes **Benchmarking** (via `benchmark-ips`) over **
56
56
  * **Rails:** 8.1+
57
57
  * **Database:** SQLite3
58
58
 
59
- ### Use within a Rails Application
59
+ ## 📋 System Requirements
60
60
 
61
- Add to your Gemfile:
61
+ ### Required
62
+ - **Ruby**: 3.3+ (3.4+ recommended for YJIT)
63
+ - **Rails**: 8.0+
64
+ - **Database**: SQLite3
62
65
 
63
- ```ruby
64
- gem "rails_benchmark_suite", group: :development
66
+ ### Optional (for Image Heft workload)
67
+ - **macOS**: `brew install vips`
68
+ - **Linux (Ubuntu/Debian)**: `sudo apt install libvips-dev`
69
+
70
+ ---
71
+
72
+ ## 🚀 Usage
73
+
74
+ ### Standard Execution
75
+ ```bash
76
+ bundle exec rails_benchmark_suite
65
77
  ```
78
+ The easy way - run the benchmark with your current Ruby configuration.
66
79
 
67
- ### Usage Flags
68
- * `--yjit`: Enables the Ruby JIT compiler (significant for Rails 8+ performance).
69
- * `-S`: Corrects the path to look for the executable in your current bundle.
70
- * `--json`: For programmatic consumption of results.
71
- * `--skip-rails`: To ignore the host application and run in isolated mode.
80
+ ### High Performance (Recommended)
81
+ ```bash
82
+ RUBY_OPT="--yjit" bundle exec rails_benchmark_suite
83
+ ```
84
+ Enable YJIT for maximum performance measurement accuracy. This is the recommended method for Rails 8+ benchmarking.
85
+
86
+ ### JSON Export (Automation)
87
+ ```bash
88
+ bundle exec rails_benchmark_suite --json > report.json
89
+ ```
90
+ Perfect for CI/CD pipelines and programmatic analysis. Outputs clean JSON without any UI elements.
91
+
92
+ ### Additional Options
93
+ - `--skip-rails`: Run in isolated mode without loading Rails environment
94
+ - `--version`: Display gem version
95
+ - `--help`: Show all available options
72
96
 
73
97
  ### Standalone Usage
74
98
 
@@ -83,7 +107,39 @@ bin/rails_benchmark_suite
83
107
 
84
108
  ---
85
109
 
86
- ## 🧪 The "Heft" Suites
110
+ ## 📐 How It's Calculated
111
+
112
+ The **Rails Heft Index (RHI)** measures your hardware's ability to handle Rails workloads using this formula:
113
+
114
+ ```
115
+ RHI Score = Σ (4-Thread IPS × Weight)
116
+ ```
117
+
118
+ ### Workload Weights
119
+
120
+ | Workload | Weight | Rationale |
121
+ |----------|--------|-----------|
122
+ | **Active Record** | 40% | Database operations are the core of most Rails apps |
123
+ | **View Rendering** | 20% | ERB/ActionView processing |
124
+ | **Solid Queue** | 20% | Background job throughput |
125
+ | **Cache Operations** | 10% | Memory store performance |
126
+ | **Image Processing** | 10% | Optional - requires libvips |
127
+
128
+ **Why 4-Thread IPS?** We use 4-thread performance to simulate production concurrency where multiple requests are handled simultaneously.
129
+
130
+ **Dynamic Weight Redistribution:** If a workload is skipped (e.g., Image Processing without libvips), its weight is redistributed proportionally among remaining workloads to maintain a 100% scale.
131
+
132
+ ### Hardware Tiers
133
+
134
+ Your RHI score maps to these performance tiers:
135
+
136
+ - **< 50**: Entry/Dev - Suitable for local development
137
+ - **50-200**: Production-Ready - Handles moderate production traffic
138
+ - **> 200**: High-Performance - Optimized for high-traffic applications
139
+
140
+ ---
141
+
142
+ ## 🧪 The "Heft" Workloads
87
143
 
88
144
  The gem measures performance across critical Rails subsystems using a dedicated, isolated schema:
89
145
 
@@ -29,20 +29,24 @@ OptionParser.new do |opts|
29
29
  end
30
30
  end.parse!
31
31
 
32
- puts "RailsBenchmarkSuite v#{RailsBenchmarkSuite::VERSION}"
32
+ unless options[:json]
33
+ puts "RailsBenchmarkSuite v#{RailsBenchmarkSuite::VERSION}"
34
+ end
33
35
 
34
36
  # Rails Detection
35
37
  rails_env_path = File.join(Dir.pwd, "config", "environment.rb")
36
38
 
37
39
  if !options[:skip_rails] && File.exist?(rails_env_path)
38
- puts "Rails environment detected at #{rails_env_path}. Loading..."
40
+ puts "Rails environment detected at #{rails_env_path}. Loading..." unless options[:json]
39
41
  require rails_env_path
40
- puts "Rails loaded successfully."
42
+ puts "Rails loaded successfully." unless options[:json]
41
43
  else
42
- if options[:skip_rails]
43
- puts "Skipping Rails loading (requested via --skip-rails)."
44
- else
45
- puts "No Rails environment detected (config/environment.rb not found)."
44
+ unless options[:json]
45
+ if options[:skip_rails]
46
+ puts "Skipping Rails loading (requested via --skip-rails)."
47
+ else
48
+ puts "No Rails environment detected (config/environment.rb not found)."
49
+ end
46
50
  end
47
51
  end
48
52
 
@@ -0,0 +1,38 @@
1
+ # frozen_string_literal: true
2
+
3
+ require "active_record"
4
+
5
+ module RailsBenchmarkSuite
6
+ class DatabaseManager
7
+ SETUP_MUTEX = Mutex.new
8
+
9
+ def setup
10
+ # Silence migrations
11
+ ActiveRecord::Migration.verbose = false
12
+
13
+ # Ultimate Hardening: Massive pool and timeout for zero lock contention (v0.3.0)
14
+ ActiveRecord::Base.establish_connection(
15
+ adapter: "sqlite3",
16
+ database: "file:heft_db?mode=memory&cache=shared",
17
+ pool: 50,
18
+ timeout: 30000
19
+ )
20
+
21
+ # The 'Busy Timeout' Hammer - force it directly on the raw connection
22
+ ActiveRecord::Base.connection.raw_connection.busy_timeout = 10000
23
+
24
+ # Setup Schema once safely with Mutex
25
+ SETUP_MUTEX.synchronize do
26
+ # Verify if schema already loaded by checking for a table
27
+ unless ActiveRecord::Base.connection.table_exists?(:users)
28
+ RailsBenchmarkSuite::Schema.load
29
+ end
30
+ end
31
+
32
+ # High-Performance Pragmas for WAL + NORMAL sync
33
+ ActiveRecord::Base.connection.raw_connection.execute("PRAGMA journal_mode = WAL")
34
+ ActiveRecord::Base.connection.raw_connection.execute("PRAGMA synchronous = NORMAL")
35
+ ActiveRecord::Base.connection.raw_connection.execute("PRAGMA mmap_size = 268435456") # 256MB - reduce disk I/O
36
+ end
37
+ end
38
+ end
@@ -5,3 +5,6 @@ require "sqlite3"
5
5
 
6
6
  # Silence ActiveRecord logs during benchmarks to avoid IO bottlenecks
7
7
  ActiveRecord::Base.logger = nil
8
+
9
+ # Silence migration output
10
+ ActiveRecord::Migration.verbose = false
@@ -0,0 +1,206 @@
1
+ # frozen_string_literal: true
2
+
3
+ require "json"
4
+
5
+ module RailsBenchmarkSuite
6
+ module Formatter
7
+ # ANSI Color Codes
8
+ RED = "\e[31m"
9
+ YELLOW = "\e[33m"
10
+ GREEN = "\e[32m"
11
+ BLUE = "\e[34m"
12
+ BOLD = "\e[1m"
13
+ RESET = "\e[0m"
14
+
15
+ module_function
16
+
17
+ def header(info)
18
+ box_width = 60 # Internal width
19
+
20
+ # Line 1: Simple text
21
+ line1 = "Rails Heft Index (RHI) v0.3.0"
22
+
23
+ # Line 2: Build without colors first to measure
24
+ yjit_status = info[:yjit] ? 'ON' : 'OFF'
25
+ yjit_hint_text = info[:yjit] ? "" : " (use RUBY_OPT=\"--yjit\")"
26
+ line2_plain = "Ruby #{info[:ruby_version]} • #{info[:processors]} Cores • YJIT: #{yjit_status}#{yjit_hint_text}"
27
+
28
+ # Now build with colors
29
+ yjit_color = info[:yjit] ? GREEN : RED
30
+ yjit_hint_colored = info[:yjit] ? "" : " #{YELLOW}(use RUBY_OPT=\"--yjit\")#{RESET}"
31
+ line2 = "Ruby #{info[:ruby_version]} • #{info[:processors]} Cores • YJIT: #{yjit_color}#{yjit_status}#{RESET}#{yjit_hint_colored}"
32
+
33
+ puts "\n"
34
+ puts "#{BLUE}┌#{'─' * box_width}┐#{RESET}"
35
+ puts "#{BLUE}│#{RESET} #{BOLD}#{line1}#{RESET}#{' ' * (box_width - 2 - line1.length)}#{BLUE}│#{RESET}"
36
+ puts "#{BLUE}│#{RESET} #{line2}#{' ' * (box_width - 2 - line2_plain.length)}#{BLUE}│#{RESET}"
37
+ puts "#{BLUE}└#{'─' * box_width}┘#{RESET}"
38
+ puts ""
39
+ end
40
+
41
+ def render_progress(num, total, name, state)
42
+ if state == "Running"
43
+ print "[#{num}/#{total}] Running #{name}... "
44
+ else
45
+ puts state
46
+ end
47
+ end
48
+
49
+ def summary_with_insights(payload)
50
+ results = payload[:results]
51
+ total_score = payload[:total_score]
52
+ tier = payload[:tier]
53
+
54
+ # Add spacing and separator before table
55
+ puts "\n"
56
+ puts "─" * 72
57
+ puts ""
58
+
59
+ # Table header
60
+ printf "#{BOLD}%-28s %10s %10s %10s %7s#{RESET}\n", "Workload", "1T IPS", "4T IPS", "Scaling", "Weight"
61
+ puts "─" * 72
62
+
63
+ # Table rows
64
+ results.each do |data|
65
+ report = data[:report]
66
+ entries = report.entries
67
+
68
+ entry_1t = entries.find { |e| e.label.include?("(1 thread)") }
69
+ entry_4t = entries.find { |e| e.label.include?("(4 threads)") }
70
+
71
+ ips_1t = entry_1t ? entry_1t.ips : 0
72
+ ips_4t = entry_4t ? entry_4t.ips : 0
73
+
74
+ scaling = ips_1t > 0 ? (ips_4t / ips_1t) : 0
75
+
76
+ # Color scaling based on performance
77
+ scaling_color = if scaling >= 0.6
78
+ GREEN
79
+ elsif scaling >= 0.3
80
+ YELLOW
81
+ else
82
+ RED
83
+ end
84
+
85
+ printf "%-28s %10s %10s #{scaling_color}%9.2fx#{RESET} %7.1f\n",
86
+ data[:name],
87
+ humanize(ips_1t),
88
+ humanize(ips_4t),
89
+ scaling,
90
+ data[:adjusted_weight]
91
+ end
92
+
93
+ # Display insights
94
+ check_scaling_insights(results)
95
+ check_yjit_insight
96
+ check_memory_insights(results)
97
+
98
+ # Display final score
99
+ render_final_score(total_score)
100
+
101
+ # Display tier comparison
102
+ show_hardware_tier(tier)
103
+ end
104
+
105
+ def check_scaling_insights(results)
106
+ #Extract scaling from results
107
+ poor_scaling = results.select do |r|
108
+ entries = r[:report].entries
109
+ entry_1t = entries.find { |e| e.label.include?("(1 thread)") }
110
+ entry_4t = entries.find { |e| e.label.include?("(4 threads)") }
111
+ ips_1t = entry_1t ? entry_1t.ips : 0
112
+ ips_4t = entry_4t ? entry_4t.ips : 0
113
+ scaling = ips_1t > 0 ? (ips_4t / ips_1t) : 0
114
+ scaling < 0.8
115
+ end
116
+
117
+ if poor_scaling.any?
118
+ puts "\n💡 Insight (Scaling): Scaling below 1.0x detected."
119
+ puts " This indicates SQLite lock contention or Ruby GIL saturation."
120
+ end
121
+ end
122
+
123
+ def check_yjit_insight
124
+ unless defined?(RubyVM::YJIT) && RubyVM::YJIT.enabled?
125
+ puts "\n💡 Insight (YJIT): YJIT is OFF."
126
+ puts " Run with RUBY_OPT=\"--yjit\" for ~20% boost."
127
+ end
128
+ end
129
+
130
+ def check_memory_insights(results)
131
+ high_memory = results.select { |r| r[:memory_delta_mb] > 20 }
132
+ high_memory.each do |r|
133
+ puts "\n💡 Insight (Memory): High growth in #{r[:name]} (#{r[:memory_delta_mb].round(1)}MB)"
134
+ puts " Suggests heavy object allocation."
135
+ end
136
+ end
137
+
138
+ def show_hardware_tier(tier)
139
+ comparison = case tier
140
+ when "Entry/Dev"
141
+ "📊 Performance Tier: Entry-Level (Suitable for dev/testing, may struggle with high production traffic)"
142
+ when "Production-Ready"
143
+ "📊 Performance Tier: Professional-Grade (Matches the throughput of dedicated production cloud instances)"
144
+ else
145
+ "📊 Performance Tier: High-Performance (Exceptional throughput, comparable to bare-metal or high-end workstations)"
146
+ end
147
+
148
+ puts "\n#{comparison}\n"
149
+ end
150
+
151
+ def render_final_score(score)
152
+ box_width = 60 # Same as header
153
+
154
+ # Build text without colors to measure
155
+ score_text = "RAILS HEFT INDEX (RHI): #{score.round(0)}"
156
+
157
+ # Build with colors
158
+ score_colored = "#{GREEN}#{BOLD}RAILS HEFT INDEX (RHI): #{score.round(0)}#{RESET}"
159
+
160
+ puts ""
161
+ puts "#{BLUE}┌#{'─' * box_width}┐#{RESET}"
162
+ puts "#{BLUE}│#{RESET} #{score_colored}#{' ' * (box_width - 2 - score_text.length)}#{BLUE}│#{RESET}"
163
+ puts "#{BLUE}└#{'─' * box_width}┘#{RESET}"
164
+ puts ""
165
+ end
166
+
167
+ def as_json(payload)
168
+ out = {
169
+ system: RailsBenchmarkSuite::Reporter.system_info,
170
+ total_score: payload[:total_score].round(0),
171
+ tier: payload[:tier],
172
+ workloads: []
173
+ }
174
+
175
+ payload[:results].each do |data|
176
+ entries = data[:report].entries
177
+ entry_1t = entries.find { |e| e.label.include?("(1 thread)") }
178
+ entry_4t = entries.find { |e| e.label.include?("(4 threads)") }
179
+ ips_1t = entry_1t ? entry_1t.ips : 0
180
+ ips_4t = entry_4t ? entry_4t.ips : 0
181
+
182
+ out[:workloads] << {
183
+ name: data[:name],
184
+ adjusted_weight: data[:adjusted_weight],
185
+ ips_1t: ips_1t,
186
+ ips_4t: ips_4t,
187
+ scaling: ips_1t > 0 ? (ips_4t / ips_1t) : 0,
188
+ memory_delta_mb: data[:memory_delta_mb]
189
+ }
190
+ end
191
+
192
+ puts out.to_json
193
+ end
194
+
195
+ def humanize(ips)
196
+ return "0" if ips.nil? || ips == 0
197
+ if ips >= 1_000_000
198
+ "#{(ips / 1_000_000.0).round(1)}M"
199
+ elsif ips >= 1_000
200
+ "#{(ips / 1_000.0).round(1)}k"
201
+ else
202
+ ips.round(1).to_s
203
+ end
204
+ end
205
+ end
206
+ end
@@ -1,211 +1,23 @@
1
1
  # frozen_string_literal: true
2
2
 
3
- require "benchmark/ips"
4
- require "get_process_mem"
5
-
6
3
  module RailsBenchmarkSuite
7
4
  class Runner
8
- def initialize(suites, json: false)
9
- @suites = suites
5
+ def initialize(workloads, json: false)
6
+ @workloads = workloads
10
7
  @json_output = json
11
8
  end
12
9
 
13
- def register(name, &block)
14
- @suites << { name: name, block: block }
15
- end
16
-
17
- SETUP_MUTEX = Mutex.new
18
-
19
10
  def run
20
- # Ultimate Hardening: Massive pool and timeout for zero lock contention (v0.2.9)
21
- ActiveRecord::Base.establish_connection(
22
- adapter: "sqlite3",
23
- database: "file:heft_db?mode=memory&cache=shared",
24
- pool: 50,
25
- timeout: 30000
26
- )
27
-
28
- # The 'Busy Timeout' Hammer - force it directly on the raw connection
29
- ActiveRecord::Base.connection.raw_connection.busy_timeout = 10000
11
+ DatabaseManager.new.setup
12
+ Formatter.header(Reporter.system_info) unless @json_output
30
13
 
31
- # Setup Schema once safely with Mutex
32
- SETUP_MUTEX.synchronize do
33
- # Verify if schema already loaded by checking for a table
34
- unless ActiveRecord::Base.connection.table_exists?(:users)
35
- RailsBenchmarkSuite::Schema.load
36
- end
37
- end
38
-
39
- # High-Performance Pragmas for WAL + NORMAL sync
40
- ActiveRecord::Base.connection.raw_connection.execute("PRAGMA journal_mode = WAL")
41
- ActiveRecord::Base.connection.raw_connection.execute("PRAGMA synchronous = NORMAL")
42
- ActiveRecord::Base.connection.raw_connection.execute("PRAGMA mmap_size = 268435456") # 256MB - reduce disk I/O
43
-
44
- puts "Running RailsBenchmarkSuite Benchmarks..." unless @json_output
45
- puts system_report unless @json_output
46
- puts "\n" unless @json_output
47
-
48
- results = {}
49
-
50
- @suites.each do |suite|
51
- puts "== Running Suite: #{suite[:name]} ==" unless @json_output
52
-
53
- # Capture memory before
54
- mem_before = GetProcessMem.new.mb
55
-
56
- # Run benchmark
57
- report = Benchmark.ips do |x|
58
- x.config(:time => 5, :warmup => 2)
59
-
60
- # Single Threaded
61
- x.report("#{suite[:name]} (1 thread)") do
62
- with_retries { suite[:block].call }
63
- end
64
-
65
- # Multi Threaded (4 threads)
66
- x.report("#{suite[:name]} (4 threads)") do
67
- threads = 4.times.map do
68
- Thread.new do
69
- # Ensure each thread gets a dedicated connection
70
- ActiveRecord::Base.connection_pool.with_connection do
71
- with_retries { suite[:block].call }
72
- end
73
- end
74
- end
75
- threads.each(&:join)
76
- end
77
-
78
- x.compare!
79
- end
80
-
81
- # Capture memory after
82
- mem_after = GetProcessMem.new.mb
83
-
84
- results[suite[:name]] = {
85
- report: report,
86
- memory_delta_mb: mem_after - mem_before,
87
- weight: suite[:weight]
88
- }
89
-
90
- puts "Memory Footprint: #{mem_after.round(2)} MB (+#{(mem_after - mem_before).round(2)} MB)" unless @json_output
91
- puts "\n" unless @json_output
92
- end
14
+ # Delegate ALL math and execution to the WorkloadRunner
15
+ payload = WorkloadRunner.new(@workloads, show_progress: !@json_output).execute
93
16
 
94
- print_summary(results)
95
- results
96
- end
97
-
98
- private
99
-
100
- def with_retries
101
- yield
102
- rescue ActiveRecord::StatementInvalid => e
103
- if e.message =~ /locked/i
104
- # Specifically drop the lock for THIS connection only
105
- ActiveRecord::Base.connection.reset!
106
- sleep(rand(0.01..0.05))
107
- retry
108
- else
109
- raise e
110
- end
111
- end
112
-
113
- def system_report
114
- info = RailsBenchmarkSuite::Reporter.system_info
115
- yjit_status = if defined?(RubyVM::YJIT) && RubyVM::YJIT.enabled?
116
- "Enabled"
117
- else
118
- "Disabled (Requires Ruby with YJIT support for best results)"
119
- end
120
- "System: Ruby #{info[:ruby_version]} (#{info[:platform]}), #{info[:processors]} Cores. YJIT: #{yjit_status}. Libvips: #{info[:libvips]}"
121
- end
122
-
123
- def print_summary(results)
124
17
  if @json_output
125
- print_json(results)
126
- return
127
- end
128
-
129
- puts "\n"
130
- puts "=========================================================================================="
131
- puts "| %-25s | %-25s | %-12s | %-15s |" % ["Suite", "IPS (1t / 4t)", "Scaling", "Mem Delta"]
132
- puts "=========================================================================================="
133
-
134
- total_score = 0
135
-
136
- results.each do |name, data|
137
- report = data[:report]
138
- entries = report.entries
139
-
140
- entry_1t = entries.find { |e| e.label.include?("(1 thread)") }
141
- entry_4t = entries.find { |e| e.label.include?("(4 threads)") }
142
-
143
- ips_1t = entry_1t ? entry_1t.ips : 0
144
- ips_4t = entry_4t ? entry_4t.ips : 0
145
-
146
- scaling = ips_1t > 0 ? (ips_4t / ips_1t) : 0
147
- mem = data[:memory_delta_mb]
148
-
149
- # Heft Score: Weighted Sum of 4t IPS
150
- weight = data[:weight] || 1.0
151
- weighted_score = ips_4t * weight
152
- total_score += weighted_score
153
-
154
- puts "| %-25s | %-25s | x%-11.2f | +%-14.2fMB |" % [
155
- name + " (w: #{weight})",
156
- "#{humanize(ips_1t)} / #{humanize(ips_4t)}",
157
- scaling,
158
- mem
159
- ]
160
- end
161
- puts "=========================================================================================="
162
- puts "\n"
163
- puts " >>> FINAL HEFT SCORE: #{total_score.round(0)} <<<"
164
- puts "\n"
165
- end
166
-
167
- def print_json(results)
168
- require "json"
169
-
170
- out = {
171
- system: RailsBenchmarkSuite::Reporter.system_info,
172
- total_score: 0,
173
- suites: []
174
- }
175
-
176
- total_score = 0
177
-
178
- results.each do |name, data|
179
- weight = data[:weight] || 1.0
180
-
181
- # Parse reports
182
- ips_1t = data[:report].entries.find { |e| e.label.include?("(1 thread)") }&.ips || 0
183
- ips_4t = data[:report].entries.find { |e| e.label.include?("(4 threads)") }&.ips || 0
184
-
185
- weighted_score = ips_4t * weight
186
- total_score += weighted_score
187
-
188
- out[:suites] << {
189
- name: name,
190
- weight: weight,
191
- ips_1t: ips_1t,
192
- ips_4t: ips_4t,
193
- scaling: ips_1t > 0 ? (ips_4t / ips_1t) : 0,
194
- memory_delta_mb: data[:memory_delta_mb],
195
- score: weighted_score
196
- }
197
- end
198
-
199
- out[:total_score] = total_score.round(0)
200
- puts out.to_json
201
- end
202
-
203
- def humanize(ips)
204
- return "0" if ips.nil?
205
- if ips > 1000
206
- "%.1fk" % (ips / 1000.0)
18
+ Formatter.as_json(payload)
207
19
  else
208
- "%.1f" % ips
20
+ Formatter.summary_with_insights(payload)
209
21
  end
210
22
  end
211
23
  end
@@ -1,3 +1,3 @@
1
1
  module RailsBenchmarkSuite
2
- VERSION = "0.2.9"
2
+ VERSION = "0.3.0"
3
3
  end
@@ -0,0 +1,121 @@
1
+ # frozen_string_literal: true
2
+
3
+ require "benchmark/ips"
4
+ require "get_process_mem"
5
+
6
+ module RailsBenchmarkSuite
7
+ class WorkloadRunner
8
+ # Base weights for each workload
9
+ BASE_WEIGHTS = {
10
+ "Active Record Heft" => 0.4,
11
+ "View Heft" => 0.2,
12
+ "Solid Queue Heft" => 0.2,
13
+ "Cache Heft" => 0.1,
14
+ "Image Heft" => 0.1
15
+ }.freeze
16
+
17
+ def initialize(workloads, show_progress: true)
18
+ @workloads = workloads
19
+ @show_progress = show_progress
20
+ end
21
+
22
+ def execute
23
+ # Run all workloads and collect results
24
+ results = @workloads.map.with_index do |w, index|
25
+ if @show_progress
26
+ Formatter.render_progress(index + 1, @workloads.size, w[:name], "Running")
27
+ end
28
+
29
+ result = run_single_workload(w)
30
+
31
+ if @show_progress
32
+ Formatter.render_progress(index + 1, @workloads.size, w[:name], "Done ✓")
33
+ end
34
+
35
+ result
36
+ end
37
+
38
+ # Calculate normalized weights
39
+ weight_pool = results.sum { |r| BASE_WEIGHTS[r[:name]] || 0 }
40
+
41
+ results.each do |r|
42
+ base_weight = BASE_WEIGHTS[r[:name]] || 1.0
43
+ r[:adjusted_weight] = base_weight / weight_pool
44
+ end
45
+
46
+ # Calculate total score
47
+ total_score = results.sum do |r|
48
+ entries = r[:report].entries
49
+ entry_4t = entries.find { |e| e.label.include?("(4 threads)") }
50
+ ips_4t = entry_4t ? entry_4t.ips : 0
51
+ ips_4t * r[:adjusted_weight]
52
+ end
53
+
54
+ # Determine tier
55
+ tier = if total_score < 50
56
+ "Entry/Dev"
57
+ elsif total_score < 200
58
+ "Production-Ready"
59
+ else
60
+ "High-Performance"
61
+ end
62
+
63
+ # Return complete payload
64
+ {
65
+ results: results,
66
+ total_score: total_score,
67
+ tier: tier
68
+ }
69
+ end
70
+
71
+ private
72
+
73
+ def run_single_workload(workload)
74
+ mem_before = GetProcessMem.new.mb
75
+
76
+ # Run benchmark
77
+ report = Benchmark.ips do |x|
78
+ x.config(:time => 5, :warmup => 2)
79
+
80
+ # Single Threaded
81
+ x.report("#{workload[:name]} (1 thread)") do
82
+ with_retries { workload[:block].call }
83
+ end
84
+
85
+ # Multi Threaded (4 threads)
86
+ x.report("#{workload[:name]} (4 threads)") do
87
+ threads = 4.times.map do
88
+ Thread.new do
89
+ ActiveRecord::Base.connection_pool.with_connection do
90
+ with_retries { workload[:block].call }
91
+ end
92
+ end
93
+ end
94
+ threads.each(&:join)
95
+ end
96
+
97
+ x.compare!
98
+ end
99
+
100
+ mem_after = GetProcessMem.new.mb
101
+
102
+ {
103
+ name: workload[:name],
104
+ report: report,
105
+ memory_delta_mb: mem_after - mem_before
106
+ }
107
+ end
108
+
109
+ def with_retries
110
+ yield
111
+ rescue ActiveRecord::StatementInvalid => e
112
+ if e.message =~ /locked/i
113
+ ActiveRecord::Base.connection.reset!
114
+ sleep(rand(0.01..0.05))
115
+ retry
116
+ else
117
+ raise e
118
+ end
119
+ end
120
+ end
121
+ end
@@ -2,8 +2,8 @@
2
2
 
3
3
  require "active_record"
4
4
 
5
- # Benchmark Suite
6
- RailsBenchmarkSuite.register_suite("Active Record Heft", weight: 0.4) do
5
+ # Benchmark Workload
6
+ RailsBenchmarkSuite.register_workload("Active Record Heft", weight: 0.4) do
7
7
  # Workload: Create User with Posts, Join Query, Update
8
8
  # Use transaction rollback to keep the DB clean and avoid costly destroy callbacks
9
9
  ActiveRecord::Base.transaction do
@@ -3,8 +3,8 @@
3
3
  require "active_support/cache"
4
4
  require "securerandom"
5
5
 
6
- # Benchmark Suite
7
- RailsBenchmarkSuite.register_suite("Cache Heft", weight: 0.1) do
6
+ # Benchmark Workload
7
+ RailsBenchmarkSuite.register_workload("Cache Heft", weight: 0.1) do
8
8
  # Simulate SolidCache using MemoryStore
9
9
  @cache ||= ActiveSupport::Cache::MemoryStore.new
10
10
 
@@ -11,7 +11,7 @@ begin
11
11
  SAMPLE_IMAGE = File.join(ASSET_DIR, "sample.jpg")
12
12
 
13
13
  # Only register if vips is actually available
14
- RailsBenchmarkSuite.register_suite("Image Heft", weight: 0.1) do
14
+ RailsBenchmarkSuite.register_workload("Image Heft", weight: 0.1) do
15
15
  # Gracefully handle missing dependencies
16
16
  if File.exist?(SAMPLE_IMAGE)
17
17
  ImageProcessing::Vips
@@ -25,7 +25,6 @@ begin
25
25
  end
26
26
 
27
27
  rescue LoadError, StandardError
28
- # Don't register the suite at all if vips is unavailable
29
- puts "⚠️ Skipping Image Heft: libvips not available. Install with: brew install vips (macOS)"
28
+ # Don't register the workload at all if vips is unavailable
29
+ puts "⚠️ Skipping Image Workload: libvips not available. Install with: 'brew install vips' (macOS) or 'sudo apt install libvips-dev' (Linux)"
30
30
  end
31
-
@@ -4,7 +4,7 @@ require "active_record"
4
4
  require "json"
5
5
 
6
6
 
7
- RailsBenchmarkSuite.register_suite("Solid Queue Heft", weight: 0.2) do
7
+ RailsBenchmarkSuite.register_workload("Solid Queue Heft", weight: 0.2) do
8
8
  # Simulation: Enqueue 100 jobs, then work them off
9
9
 
10
10
  # 1. Enqueue Loop
@@ -3,17 +3,17 @@
3
3
  require "action_view"
4
4
  require "ostruct"
5
5
 
6
- # Benchmark Suite
6
+ # Benchmark Workload
7
7
 
8
8
 
9
- # Helper for the suite
9
+ # Helper for the workload
10
10
  module RailsBenchmarkSuiteNumberHelper
11
11
  def self.number_with_delimiter(number)
12
12
  number.to_s.gsub(/(\d)(?=(\d\d\d)+(?!\d))/, "\\1,")
13
13
  end
14
14
  end
15
15
 
16
- RailsBenchmarkSuite.register_suite("View Heft", weight: 0.2) do
16
+ RailsBenchmarkSuite.register_workload("View Heft", weight: 0.2) do
17
17
  # Setup context once
18
18
  @view_renderer ||= begin
19
19
  lookup_context = ActionView::LookupContext.new([File.expand_path(__dir__)])
@@ -3,6 +3,9 @@
3
3
  require "concurrent"
4
4
  require "rails_benchmark_suite/version"
5
5
  require "rails_benchmark_suite/reporter"
6
+ require "rails_benchmark_suite/database_manager"
7
+ require "rails_benchmark_suite/workload_runner"
8
+ require "rails_benchmark_suite/formatter"
6
9
  require "rails_benchmark_suite/runner"
7
10
  require "rails_benchmark_suite/db_setup"
8
11
  require "rails_benchmark_suite/schema"
@@ -11,17 +14,17 @@ require "rails_benchmark_suite/models/post"
11
14
  require "rails_benchmark_suite/models/simulated_job"
12
15
 
13
16
  module RailsBenchmarkSuite
14
- @suites = []
17
+ @workloads = []
15
18
 
16
- def self.register_suite(name, weight: 1.0, &block)
17
- @suites << { name: name, weight: weight, block: block }
19
+ def self.register_workload(name, weight: 1.0, &block)
20
+ @workloads << { name: name, weight: weight, block: block }
18
21
  end
19
22
 
20
23
  def self.run(json: false)
21
- # Load suites
22
- Dir[File.join(__dir__, "rails_benchmark_suite", "suites", "*.rb")].each { |f| require f }
24
+ # Load workloads
25
+ Dir[File.join(__dir__, "rails_benchmark_suite", "workloads", "*.rb")].each { |f| require f }
23
26
 
24
- runner = Runner.new(@suites, json: json)
27
+ runner = Runner.new(@workloads, json: json)
25
28
  runner.run
26
29
  end
27
30
  end
@@ -8,8 +8,8 @@ Gem::Specification.new do |spec|
8
8
  spec.authors = ["RailsBenchmarkSuite Contributors"]
9
9
  spec.email = ["team@rails.org"]
10
10
 
11
- spec.summary = "Rails-style functionality & performance benchmark tool"
12
- spec.description = "Measures the 'Heft' (processing power) of a machine using realistic Rails workloads."
11
+ spec.summary = "Rails Heft Index (RHI) - Hardware benchmarking using realistic workloads"
12
+ spec.description = "Measures the Rails Heft Index (RHI), a weighted performance score based on realistic Rails 8+ workloads across Active Record, caching, views, jobs, and image processing."
13
13
  spec.homepage = "https://github.com/overnet/rails_benchmark_suite"
14
14
  spec.license = "MIT"
15
15
 
@@ -36,6 +36,7 @@ Gem::Specification.new do |spec|
36
36
  spec.add_development_dependency "bundler", "~> 2.5"
37
37
  spec.add_development_dependency "rake", "~> 13.0"
38
38
  spec.add_development_dependency "minitest", "~> 5.0"
39
+ spec.add_development_dependency "ostruct", "~> 0.6"
39
40
 
40
41
  spec.required_ruby_version = ">= 3.4.0"
41
42
  end
metadata CHANGED
@@ -1,7 +1,7 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: rails_benchmark_suite
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.2.9
4
+ version: 0.3.0
5
5
  platform: ruby
6
6
  authors:
7
7
  - RailsBenchmarkSuite Contributors
@@ -163,8 +163,23 @@ dependencies:
163
163
  - - "~>"
164
164
  - !ruby/object:Gem::Version
165
165
  version: '5.0'
166
- description: Measures the 'Heft' (processing power) of a machine using realistic Rails
167
- workloads.
166
+ - !ruby/object:Gem::Dependency
167
+ name: ostruct
168
+ requirement: !ruby/object:Gem::Requirement
169
+ requirements:
170
+ - - "~>"
171
+ - !ruby/object:Gem::Version
172
+ version: '0.6'
173
+ type: :development
174
+ prerelease: false
175
+ version_requirements: !ruby/object:Gem::Requirement
176
+ requirements:
177
+ - - "~>"
178
+ - !ruby/object:Gem::Version
179
+ version: '0.6'
180
+ description: Measures the Rails Heft Index (RHI), a weighted performance score based
181
+ on realistic Rails 8+ workloads across Active Record, caching, views, jobs, and
182
+ image processing.
168
183
  email:
169
184
  - team@rails.org
170
185
  executables:
@@ -182,19 +197,22 @@ files:
182
197
  - Rakefile
183
198
  - bin/rails_benchmark_suite
184
199
  - lib/rails_benchmark_suite.rb
200
+ - lib/rails_benchmark_suite/database_manager.rb
185
201
  - lib/rails_benchmark_suite/db_setup.rb
202
+ - lib/rails_benchmark_suite/formatter.rb
186
203
  - lib/rails_benchmark_suite/models/post.rb
187
204
  - lib/rails_benchmark_suite/models/simulated_job.rb
188
205
  - lib/rails_benchmark_suite/models/user.rb
189
206
  - lib/rails_benchmark_suite/reporter.rb
190
207
  - lib/rails_benchmark_suite/runner.rb
191
208
  - lib/rails_benchmark_suite/schema.rb
192
- - lib/rails_benchmark_suite/suites/active_record_suite.rb
193
- - lib/rails_benchmark_suite/suites/cache_heft_suite.rb
194
- - lib/rails_benchmark_suite/suites/image_heft_suite.rb
195
- - lib/rails_benchmark_suite/suites/job_heft_suite.rb
196
- - lib/rails_benchmark_suite/suites/view_heft_suite.rb
197
209
  - lib/rails_benchmark_suite/version.rb
210
+ - lib/rails_benchmark_suite/workload_runner.rb
211
+ - lib/rails_benchmark_suite/workloads/active_record_workload.rb
212
+ - lib/rails_benchmark_suite/workloads/cache_heft_workload.rb
213
+ - lib/rails_benchmark_suite/workloads/image_heft_workload.rb
214
+ - lib/rails_benchmark_suite/workloads/job_heft_workload.rb
215
+ - lib/rails_benchmark_suite/workloads/view_heft_workload.rb
198
216
  - rails_benchmark_suite.gemspec
199
217
  homepage: https://github.com/overnet/rails_benchmark_suite
200
218
  licenses:
@@ -219,5 +237,5 @@ required_rubygems_version: !ruby/object:Gem::Requirement
219
237
  requirements: []
220
238
  rubygems_version: 3.6.2
221
239
  specification_version: 4
222
- summary: Rails-style functionality & performance benchmark tool
240
+ summary: Rails Heft Index (RHI) - Hardware benchmarking using realistic workloads
223
241
  test_files: []