rails_benchmark_suite 0.2.8 β†’ 0.2.9

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (5) hide show
  1. checksums.yaml +4 -4
  2. data/README.md +65 -77
  3. data/lib/rails_benchmark_suite/runner.rb +1 -1
  4. data/lib/rails_benchmark_suite/version.rb +1 -1
  5. metadata +2 -2
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: b4dba673f286e0a526b602537257364dda4b7477eed8698d7f69a0993b3b0ff5
4
- data.tar.gz: 344f0dd405e3249068746359edff5d9b1578bdf2bb736861f27548283f64d91d
3
+ metadata.gz: c48ce5faaa5348bcd47cdd9c88ba9b1441d0f7bcd0a6fa601d84b36014673d1a
4
+ data.tar.gz: f39f188c55fe6749a5b9ba4ae9c5c44e926bd5d6588737f689192ab841deda92
5
5
  SHA512:
6
- metadata.gz: 65ae4cd0975954f2d7ccb3e5b60d25c587eb76d7ea3d3bfc771b1e879c6ccf5b6749b823323506973c500c49c415b2050b12bacdc7b497fc4d3e9791c1ac9943
7
- data.tar.gz: d1af906dc1e84f78d92b74c27f74156aa8fd25d92016a317d342a084f5657bc72d2eb2f6b6e10a822e46073c71f886a1563e8f52d2ee5f68764b5e68c41a4a35
6
+ metadata.gz: 28f32ff03be6cbd83aabb1c3f8750316f5c3e2cb69de127885f96a4ef27841b12ee916999c2dea59c314605a6755f0aee34d0c610338c596bbff2789120a7ce0
7
+ data.tar.gz: 2e4f5a17c66e0caed9a95a46cdc3e0049783eea603f9cd263d699f492d49440f7ceaab45bf4c469b8b05fc316ca999052d12705dfe29b0edf54b032160fc2966
data/README.md CHANGED
@@ -1,29 +1,44 @@
1
- # Rails Benchmark Suite
1
+ # Rails Benchmark Suite πŸš€
2
+
3
+ **Standardized Hardware Benchmarking for Rails 8.1+**
2
4
 
3
5
  A standardized performance suite designed to measure the "Heft" of a machine using realistic, high-throughput Rails 8+ workloads.
4
6
 
5
- Unlike synthetic CPU benchmarks, **Rails Benchmark Suite** simulates Active Record object allocation, SQL query complexity, ActionView rendering, and background job throughput.
7
+ ## πŸ›  What is this?
8
+
9
+ Think of this as a **"Test Track" for Rails servers**. Unlike profilers that measure your specific application code, this gem runs a **fixed, standardized set of Rails operations** (Active Record object allocation, SQL query complexity, ActionView rendering, and background job throughput) to measure the raw performance of your server and Ruby configuration.
10
+
11
+ To ensure a level playing field, the gem boots an **isolated, in-memory SQLite environment**. It creates its own schema and records, meaning it **never touches your production data** and returns comparable results across any machine.
6
12
 
7
13
  ## πŸ“Š The "Heft" Score
8
14
 
9
- The Heft Score is a weighted metric representing a machine's ability to handle Rails tasks.
10
- - **Baseline:** A score of **100** is calibrated to represent an **AWS c6g.large** (ARM) instance.
11
- - **Objective:** To provide a simple, comparable number for evaluating different computing platforms (Cloud VMs, bare-metal, or local dev rigs).
15
+ The Heft Score is a weighted metric representing a machine's ability to handle Rails tasks.
16
+ * **Baseline:** A score of **100** is calibrated to represent an **AWS c6g.large** (ARM) instance.
17
+ * **Objective:** To provide a simple, comparable number for evaluating different computing platforms (Cloud VMs, bare-metal, or local dev rigs).
12
18
 
13
19
  ### Baseline Comparisons
20
+
14
21
  | Score | Classification | Comparable Hardware |
15
22
  | :--- | :--- | :--- |
16
- | < 40 | 🐒 Sluggish | Older Intel Macs, Entry-level VPS |
17
- | 60 | πŸš™ Capable | Standard Cloud VM (c5.large/standard) |
18
- | **100** | **🏎️ Baseline** | **AWS c6g.large (2 vCPU ARM)** |
19
- | 150+ | πŸš€ High Performance | Apple M-series Pro/Max, Ryzen 5000+ |
20
- | 300+ | ⚑ Blazing | Server-grade Metal, M3 Ultra |
23
+ | **< 40** | 🐒 Sluggish | Older Intel Macs, Entry-level VPS |
24
+ | **60** | πŸš™ Capable | Standard Cloud VM (c5.large/standard) |
25
+ | **100** | 🏎️ Baseline | AWS c6g.large (2 vCPU ARM) |
26
+ | **150+** | πŸš€ High Performance | Apple M-series Pro/Max, Ryzen 5000+ |
27
+ | **300+** | ⚑ Blazing | Server-grade Metal, M3 Ultra |
28
+
29
+ ---
30
+
31
+ ## πŸš€ Quick Start
32
+
33
+ Ensure you are in your Rails root directory and run:
21
34
 
22
- ### Quick Start
23
35
  ```bash
24
36
  ruby --yjit -S bundle exec rails_benchmark_suite
25
37
  ```
26
- **Note:** `bundle exec` is required for Rails environment stability and to prevent Minitest version conflicts.
38
+
39
+ **Note:** `bundle exec` is mandatory for Rails environment stability and to prevent Minitest version conflicts.
40
+
41
+ ---
27
42
 
28
43
  ## πŸ›  Technical Philosophy
29
44
 
@@ -32,28 +47,16 @@ Rails Benchmark Suite prioritizes **Benchmarking** (via `benchmark-ips`) over **
32
47
  * **Benchmarking:** Focuses on macro-throughputβ€”"How many iterations can the hardware handle?" This provides the final Heft Score.
33
48
  * **Why no Profiling?** Profiling tools (like `StackProf` or `Vernier`) introduce instrumentation overhead that skews hardware metrics. We aim for "Conceptual Compression"β€”one clear number to inform infrastructure decisions.
34
49
 
50
+ ---
51
+
35
52
  ## πŸš€ Installation & Usage
36
53
 
37
54
  ### Requirements
38
- - **Ruby**: 3.3+ (Ruby with YJIT support highly recommended for accurate performance measurement)
39
- - **Database**: SQLite3
40
-
41
- ### Prerequisites
42
- * **Ruby:** 3.4.1+ (Recommended for latest YJIT/Prism performance)
55
+ * **Ruby:** 3.3+ (Ruby with YJIT support highly recommended)
56
+ * **Rails:** 8.1+
43
57
  * **Database:** SQLite3
44
58
 
45
- ### Standalone Usage
46
- If you want to test hardware performance without an existing application:
47
-
48
- ```bash
49
- git clone https://github.com/overnet/rails_benchmark_suite.git
50
- cd rails_benchmark_suite
51
- bundle install
52
- bin/rails_benchmark_suite
53
- ```
54
-
55
59
  ### Use within a Rails Application
56
- Rails Benchmark Suite is "Rails-aware." Adding it to your app allows you to benchmark your specific configuration and custom suites.
57
60
 
58
61
  Add to your Gemfile:
59
62
 
@@ -61,84 +64,69 @@ Add to your Gemfile:
61
64
  gem "rails_benchmark_suite", group: :development
62
65
  ```
63
66
 
64
- ## Usage
65
-
66
- To get the most accurate 'Heft' score, run with YJIT enabled:
67
-
68
- ```bash
69
- ruby --yjit -S bundle exec rails_benchmark_suite
70
- ```
67
+ ### Usage Flags
68
+ * `--yjit`: Enables the Ruby JIT compiler (significant for Rails 8+ performance).
69
+ * `-S`: Corrects the path to look for the executable in your current bundle.
70
+ * `--json`: For programmatic consumption of results.
71
+ * `--skip-rails`: To ignore the host application and run in isolated mode.
71
72
 
72
- **Why these flags?**
73
- - `--yjit`: Enables the Ruby JIT compiler (significant for Rails 8+ performance).
74
- - `-S`: Corrects the path to look for the executable in your current bundle.
75
- - `bundle exec`: Prevents version conflicts (e.g., Minitest) between the gem and your host application.
73
+ ### Standalone Usage
76
74
 
77
- **Standalone Usage:**
78
- If running outside a Rails project:
75
+ If you want to test hardware performance without an existing application:
79
76
 
80
77
  ```bash
78
+ git clone https://github.com/overnet/rails_benchmark_suite.git
79
+ cd rails_benchmark_suite
80
+ bundle install
81
81
  bin/rails_benchmark_suite
82
82
  ```
83
83
 
84
- **JSON Output:**
85
- For programmatic consumption:
86
-
87
- ```bash
88
- ruby --yjit -S bundle exec rails_benchmark_suite --json
89
- ```
90
-
91
- > **Note:** Use `--skip-rails` to ignore the host application and run in isolated mode.
84
+ ---
92
85
 
93
- ## Performance
86
+ ## πŸ§ͺ The "Heft" Suites
94
87
 
95
- **Understanding YJIT Status:**
88
+ The gem measures performance across critical Rails subsystems using a dedicated, isolated schema:
96
89
 
97
- If you see `YJIT: Disabled`, this is a result of your Ruby binary build, not the gem itself. The gem will work perfectly fine without YJIT, but performance measurements will be more accurate with YJIT enabled.
90
+ * **Active Record Heft:** Standardized CRUD: Creation, indexing, and complex querying.
91
+ * **Cache Heft:** High-frequency read/writes to the Rails memory store.
92
+ * **Solid Queue Heft:** Background job enqueuing and database-backed polling stress.
93
+ * **View Heft:** Partial rendering overhead and ActionView throughput.
94
+ * **Image Heft:** Image processing performance (requires libvips).
98
95
 
99
- To enable YJIT, you need Ruby compiled with YJIT support (requires Rust compiler during Ruby installation). See the Troubleshooting section below for installation instructions.
96
+ ---
100
97
 
101
- ## Troubleshooting
98
+ ## ⚠️ Troubleshooting
102
99
 
103
100
  ### YJIT Shows "Disabled"
104
101
 
105
- If you see `YJIT: Disabled (Requires Ruby with YJIT support for best results)`, it means your Ruby was not compiled with YJIT support. To get the best performance:
102
+ If you see `YJIT: Disabled`, it means your Ruby was not compiled with YJIT support.
106
103
 
107
- 1. Ensure `rustc` (Rust compiler) is installed on your system
108
- 2. Reinstall Ruby with YJIT support:
109
- ```bash
110
- # Using rbenv
111
- RUBY_CONFIGURE_OPTS="--enable-yjit" rbenv install 3.4.1
112
-
113
- # Using rvm
114
- rvm install 3.4.1 --enable-yjit
115
- ```
116
- 3. Verify YJIT is available: `ruby --yjit -e "puts RubyVM::YJIT.enabled?"`
104
+ * **Fix (rbenv):** `RUBY_CONFIGURE_OPTS="--enable-yjit" rbenv install 3.4.1`
105
+ * **Fix (rvm):** `rvm install 3.4.1 --enable-yjit`
117
106
 
118
107
  ### SQLite Lock Errors
119
108
 
120
- If you encounter `SQLite3::BusyException` or "database table is locked" errors, ensure you're running the latest version of the gem (v0.2.7+) which includes automatic concurrency retries with smart backoff.
109
+ Version 0.2.9+ includes surgical connection resets and randomized backoffs to handle SQLite concurrency. If issues persist, ensure no other processes are accessing the benchmark database.
121
110
 
122
- **v0.2.7+** includes:
123
- - Automatic retry logic with sleep backoff
124
- - Per-thread unique identifiers to prevent conflicts
125
- - Optimized busy timeout settings (10 seconds)
126
-
127
- If issues persist, try reducing concurrency or ensuring no other processes are accessing the benchmark database.
111
+ ---
128
112
 
129
113
  ## πŸ— Architecture
114
+
130
115
  * **Engine:** Built on `benchmark-ips`.
131
- * **Database:** Uses In-Memory SQLite with `cache=shared` for multi-threaded accuracy.
132
- * **Isolation:** Uses transactional rollbacks (`ActiveRecord::Rollback`) to ensure test isolation without the overhead of row deletion.
133
- * **Threading:** Supports 1-thread and 4-thread scaling tests to measure vertical efficiency.
134
- * **Modern Stack:** Optimized for Rails 8+ defaults, including Solid Queue simulation and YJIT detection.
116
+ * **Database:** Uses In-Memory SQLite with `cache=shared` and a 50-connection pool for multi-threaded accuracy.
117
+ * **Isolation:** Uses transactional rollbacks and Mutex-wrapped schema creation.
118
+ * **Threading:** Supports 1-thread and 4-thread scaling tests.
119
+
120
+ ---
135
121
 
136
122
  ## πŸ“œ Credits
137
- This project is a functional implementation of the performance benchmark vision discussed in the Rails community.
138
123
 
139
124
  * **Vision:** Inspired by @dhh in [rails/rails#50451](https://github.com/rails/rails/issues/50451).
140
125
  * **Initial Roadmap:** Based on suggestions by @JoeDupuis.
141
126
  * **Implementation:** The Rails Community.
142
127
 
128
+ ---
129
+
143
130
  ## πŸ“„ License
131
+
144
132
  The gem is available as open source under the terms of the MIT License.
data/lib/rails_benchmark_suite/runner.rb CHANGED
@@ -17,7 +17,7 @@ module RailsBenchmarkSuite
17
17
  SETUP_MUTEX = Mutex.new
18
18
 
19
19
  def run
20
- # Ultimate Hardening: Massive pool and timeout for zero lock contention (v0.2.8)
20
+ # Ultimate Hardening: Massive pool and timeout for zero lock contention (v0.2.9)
21
21
  ActiveRecord::Base.establish_connection(
22
22
  adapter: "sqlite3",
23
23
  database: "file:heft_db?mode=memory&cache=shared",
data/lib/rails_benchmark_suite/version.rb CHANGED
@@ -1,3 +1,3 @@
1
1
  module RailsBenchmarkSuite
2
- VERSION = "0.2.8"
2
+ VERSION = "0.2.9"
3
3
  end
metadata CHANGED
@@ -1,13 +1,13 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: rails_benchmark_suite
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.2.8
4
+ version: 0.2.9
5
5
  platform: ruby
6
6
  authors:
7
7
  - RailsBenchmarkSuite Contributors
8
8
  bindir: bin
9
9
  cert_chain: []
10
- date: 2026-01-02 00:00:00.000000000 Z
10
+ date: 2026-01-03 00:00:00.000000000 Z
11
11
  dependencies:
12
12
  - !ruby/object:Gem::Dependency
13
13
  name: benchmark-ips