rspec-turbo 0.1.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml ADDED
@@ -0,0 +1,7 @@
1
+ ---
2
+ SHA256:
3
+ metadata.gz: 6e75805ec30e8c00d22ab5db21967fb3ddfdeb0ab8f6edabad1047f6b8d3d895
4
+ data.tar.gz: 3081d8c6f070131b441e4d00c368396b3e19cd48a2d79ed13b60a427a30c0033
5
+ SHA512:
6
+ metadata.gz: 6f29717c3d1da16330b2019c4ff0cd7043f846ac0bc803716ddd86bdc338b382c41037c58ed68b9182a9fcad66ac0406d1ca994f76c3c3e2402e2533bff767b0
7
+ data.tar.gz: e66d30be4d8d35ba555cd4da8b5d950f19bf88fcdbd5df1c9feff10c14aab6956dbbd244d83cb82e3783264cfaedb58fd2ea80d05a7536f94691e684d808f78e
data/CHANGELOG.md ADDED
@@ -0,0 +1,27 @@
1
+ # Changelog
2
+
3
+ ## [0.1.0] - 2026-06-12
4
+
5
+ Initial extraction from the single-file `turbo.rb` runner into a gem.
6
+
7
+ ### Added
8
+ - Parallel RSpec runner with dry-run-based example counting and LPT bin-packing.
9
+ - Splitting of oversized spec files across workers by example ID.
10
+ - Schema-fingerprinted test DB setup caching (Rails).
11
+ - Live TTY dashboard and CI-friendly periodic progress mode.
12
+ - Slowest folders/files report fed by the bundled `slow_profile` profiler,
13
+ enabled by default (disable with `RSPEC_TURBO_NO_PROFILE=1`) and safe outside
14
+ Rails (times examples without counting SQL when ActiveSupport is absent).
15
+ - JUnit XML output (`JUNIT_DIR`) and SimpleCov coverage merging (`COVERAGE=1`).
16
+ - Three entry points: the `rspec-turbo` binary plus a `spec:turbo` Rake task
17
+ (reachable as both `rake spec:turbo` and `rails spec:turbo`), registered in
18
+ Rails apps through a Railtie.
19
+ - `coverage:merge` Rake task that collates per-worker SimpleCov result files
20
+ with `SimpleCov.collate`, emitting JSON on CI (`JSONFormatter`) and HTML
21
+ locally (`HTMLFormatter`); glob overridable via `RSPEC_TURBO_COVERAGE_GLOB`.
22
+
23
+ ### Fixed (versus the original script)
24
+ - `DbSetup#show_log` referenced an undefined `w` variable on failure.
25
+ - Missing `require "set"` for `FileDiscovery`.
26
+ - A dead `Process.clock_gettime` call in `DbSetup#run!`.
27
+ - `"\nTop \d"` string literal that was meant to be a regular expression.
data/LICENSE.txt ADDED
@@ -0,0 +1,21 @@
1
+ The MIT License (MIT)
2
+
3
+ Copyright (c) 2026 thadeu
4
+
5
+ Permission is hereby granted, free of charge, to any person obtaining a copy
6
+ of this software and associated documentation files (the "Software"), to deal
7
+ in the Software without restriction, including without limitation the rights
8
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9
+ copies of the Software, and to permit persons to whom the Software is
10
+ furnished to do so, subject to the following conditions:
11
+
12
+ The above copyright notice and this permission notice shall be included in
13
+ all copies or substantial portions of the Software.
14
+
15
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21
+ SOFTWARE.
data/README.md ADDED
@@ -0,0 +1,287 @@
1
+ # โšก rspec-turbo
2
+
3
+ [![Gem Version](https://img.shields.io/gem/v/rspec-turbo)](https://rubygems.org/gems/rspec-turbo)
4
+ [![Ruby](https://img.shields.io/badge/ruby-%3E%3D%203.0-CC342D)](https://www.ruby-lang.org)
5
+ [![Style](https://img.shields.io/badge/code_style-standard-brightgreen)](https://github.com/standardrb/standard)
6
+ [![License: MIT](https://img.shields.io/badge/license-MIT-blue)](LICENSE.txt)
7
+
8
+ **Run your whole RSpec suite in parallel โ€” with zero config.**
9
+
10
+ `rspec-turbo` spreads your specs across every core, balancing the load by the
11
+ **actual number of examples** (not file size, not a stale timing log) and even
12
+ splitting a single oversized file across workers. One command, a live progress
13
+ dashboard, and a report that tells you exactly which folders are slowing you
14
+ down.
15
+
16
+ ```sh
17
+ bundle add rspec-turbo --group test
18
+ bundle exec rspec-turbo
19
+ ```
20
+
21
+ That's it. No runtime logs to maintain, no grouping flags to tune.
22
+
23
+ ---
24
+
25
+ ## ๐ŸŽ๏ธ See it run
26
+
27
+ ```text
28
+ ====================================================================
29
+ RSpec Turbo - Parallel
30
+ ====================================================================
31
+
32
+ โœ“ 8 DB(s) ready (0s)
33
+ โœ“ 4210 examples ยท 312 files ยท 8 batches (~526 each) (3s)
34
+
35
+ โœ“ worker/01 1m02s PASS requests/v1
36
+ โœ“ worker/02 58s PASS models ยท services
37
+ โ น worker/03 ~520 ex 46s jobs ยท mailers
38
+ ...
39
+
40
+ โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘ 2731/4210 65%
41
+
42
+ ====================================================================
43
+ RSpec Turbo Report
44
+ ====================================================================
45
+
46
+ Slowest folders โ†ณ optimize these first
47
+
48
+ requests/v1 1m12s โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“
49
+ models 48s โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘
50
+ services 31s โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘
51
+
52
+ โœ“ All passed ยท 4210 examples ยท 8 workers ยท wall 1m04s sum 7m58s 7.4x
53
+ ```
54
+
55
+ *(Illustrative output โ€” your speedup scales with your cores.)*
56
+
57
+ ---
58
+
59
+ ## Why not just use `parallel_tests`?
60
+
61
+ [`parallel_tests`](https://github.com/grosser/parallel_tests) is a great,
62
+ battle-tested tool โ€” and if you run Minitest, Cucumber, Test::Unit **and**
63
+ RSpec, its multi-framework reach is exactly what you want.
64
+
65
+ But if your project is **RSpec-only**, that generality costs you. `rspec-turbo`
66
+ does one thing and tunes hard for it:
67
+
68
+ | | `parallel_tests` | **`rspec-turbo`** |
69
+ |---|---|---|
70
+ | **Scope** | Multi-framework (RSpec, Minitest, Cucumberโ€ฆ) | RSpec only โ€” focused and lean |
71
+ | **Default balancing** | By file **size** (bytes) โ€” a rough proxy for time | By **actual example count** from one `rspec --dry-run` |
72
+ | **Best-case balancing** | `--group-by runtime`, needs a runtime log you record and keep fresh | Recomputed every run from the dry-run โ€” **always current, nothing to maintain** |
73
+ | **Unit of work** | A whole file โ€” one giant `*_spec.rb` stalls a process | A file **or** example-ID slices โ€” **splits big files across workers** |
74
+ | **Config to balance well** | Generate + commit `tmp/parallel_runtime_rspec.log` | **None** โ€” good distribution out of the box |
75
+ | **Live output** | Per-process stdout, interleaved | Live TTY dashboard (spinner per worker + global bar) / clean CI progress |
76
+ | **Final report** | Concatenated process outputs | **One consolidated report**: failures per worker, speedup, slowest folders/files |
77
+ | **Slow-test insight** | DIY (`--profile` per process, aggregate yourself) | **Built in, on by default** โ€” per-file time + SQL query counts, aggregated |
78
+ | **Test-DB setup** | `rake parallel:prepare` (you decide when to re-run) | Automatic, **schema-fingerprint cached** โ€” skipped when the schema hasn't changed |
79
+ | **JUnit / coverage merge** | Extra wiring | Built in (`JUNIT_DIR`, `COVERAGE=1`) |
80
+
81
+ ### What that means in practice
82
+
83
+ - **Better balance, no homework.** `parallel_tests`' file-size grouping puts a
84
+ 500-line file with 3 slow examples in the same weight class as a 500-line file
85
+ with 80 fast ones. `rspec-turbo` counts the *examples* (via a fast dry-run) and
86
+ packs them with a longest-processing-time-first heuristic โ€” and it does this
87
+ every run, so it never goes stale and there's no runtime log to commit.
88
+ - **No single-file bottleneck.** When one mega `*_spec.rb` holds 20% of your
89
+ suite, a file-based splitter leaves one process grinding while the rest idle.
90
+ `rspec-turbo` slices that file by example ID across workers.
91
+ - **Answers, not just speed.** Every run ends with a ranked "slowest folders /
92
+ files" report (and SQL query counts under Rails), so you know *what* to
93
+ optimize next โ€” not just that the suite is slow.
94
+
95
+ ---
96
+
97
+ ## Install
98
+
99
+ Add it to the `:test` group of your Gemfile:
100
+
101
+ ```ruby
102
+ group :test do
103
+ gem "rspec-turbo"
104
+ end
105
+ ```
106
+
107
+ ```sh
108
+ bundle install
109
+ ```
110
+
111
+ ## Usage
112
+
113
+ ```sh
114
+ bundle exec rspec-turbo # all of spec/
115
+ bundle exec rspec-turbo spec/models lib # specific folders
116
+ bundle exec rspec-turbo spec/models/project_spec.rb # a single file
117
+ bundle exec rspec-turbo --exclude-pattern "spec/requests/**/*"
118
+ bundle exec rspec-turbo --fail-fast spec/models
119
+
120
+ RSPEC_TURBO_MAX=6 bundle exec rspec-turbo # cap workers
121
+ RSPEC_TURBO_FORCE_SETUP=1 bundle exec rspec-turbo # recreate test DBs
122
+ ```
123
+
124
+ Any RSpec flag you pass through (`--tag`, `--seed`, `--order`, โ€ฆ) is forwarded
125
+ to every worker.
126
+
127
+ ### Three ways to launch it
128
+
129
+ ```sh
130
+ bundle exec rspec-turbo # the binary โ€” full control (paths, flags)
131
+ bundle exec rails spec:turbo # the same task, via the Rails CLI
132
+ bundle exec rake spec:turbo # Rake task โ€” runs the whole suite
133
+ ```
134
+
135
+ In a Rails app the `spec:turbo` task is registered automatically (via a
136
+ Railtie), and `rails spec:turbo` works because Rails routes unknown commands to
137
+ Rake. The `rake`/`rails` forms run the **entire** suite โ€” ideal for CI; for
138
+ specific folders or RSpec flags, reach for the `rspec-turbo` binary.
139
+
140
+ ## How it works
141
+
142
+ ```
143
+ parse argv โ†’ DbSetup โ†’ FileDiscovery โ†’ BatchPlanner โ†’ Executor (pool) โ†’ Report
144
+ ```
145
+
146
+ 1. **DbSetup** โ€” spawns one `rails db:drop db:create db:schema:load db:seed` per
147
+ worker slot (each with its own `TEST_ENV_NUMBER`). Cached by a fingerprint of
148
+ `db/schema.rb` + `db/seeds.rb` and the worker count, so repeat runs skip it.
149
+ 2. **FileDiscovery** โ€” globs `*_spec.rb`, applies `--exclude-pattern`.
150
+ 3. **BatchPlanner** โ€” one `rspec --dry-run --format json` counts examples per
151
+ file, then bin-packs files into balanced batches (Longest-Processing-Time
152
+ first). Files heavier than a batch's fair share are split into example-ID
153
+ slices so no single file bottlenecks a worker.
154
+ 4. **Executor** โ€” a fixed pool of slots; each finished slot is recycled until
155
+ the queue drains. Live dashboard on a TTY, periodic `[progress]` lines on CI.
156
+ 5. **Report** โ€” failures, slowest folders/files, and a one-line summary with
157
+ wall time, summed CPU time and the resulting speedup.
158
+
159
+ ## Requirements
160
+
161
+ - **Rails** โ€” `DbSetup` uses `rails db:*`. (Non-Rails projects can still run if
162
+ the databases already exist; set `RSPEC_TURBO_FORCE_SETUP=0`, the default, and
163
+ the setup is skipped on a cache hit.)
164
+ - **`rspec_junit_formatter`** โ€” only when `JUNIT_DIR` is set.
165
+ - **`simplecov` + `simplecov_json_formatter`** โ€” only when `COVERAGE=1`
166
+ (see [Coverage](#coverage-optional) below).
167
+
168
+ ## Environment variables
169
+
170
+ | Variable | Default | Purpose |
171
+ |---|---|---|
172
+ | `RSPEC_TURBO_MAX` | nproc | Number of parallel workers |
173
+ | `RSPEC_TURBO_LOG_DIR` | `tmp/rspec-turbo` | Where per-worker logs live |
174
+ | `RSPEC_TURBO_FORCE_SETUP` | off | `1` recreates the test DBs even if cached |
175
+ | `RSPEC_TURBO_PROGRESS_INTERVAL` | `30` | Seconds between CI progress lines |
176
+ | `COVERAGE` | `0` | `1` merges SimpleCov results after the run |
177
+ | `JUNIT_DIR` | โ€” | Emit one JUnit XML per worker into this dir |
178
+ | `CI` | โ€” | Forces the plain (non-TTY) progress mode |
179
+
180
+ ### Slowest-files report (on by default)
181
+
182
+ The "Slowest folders / Slowest files" section is fed by the bundled
183
+ `slow_profile` hook, loaded into every worker. It is **on by default**: each
184
+ worker times every example, and under Rails it also counts SQL queries via
185
+ `ActiveSupport::Notifications`. Outside Rails it degrades gracefully โ€” it just
186
+ times examples and reports zero queries.
187
+
188
+ Turn it off with the master kill switch:
189
+
190
+ ```sh
191
+ RSPEC_TURBO_NO_PROFILE=1 bundle exec rspec-turbo
192
+ ```
193
+
194
+ | Variable | Default | Purpose |
195
+ |---|---|---|
196
+ | `RSPEC_TURBO_NO_PROFILE` | off | `1` disables profiling entirely (master kill switch) |
197
+ | `RSPEC_PROFILE_THRESHOLD_TIME` | `0.2` | Seconds an example must exceed to make the "slow examples" list |
198
+ | `RSPEC_PROFILE_THRESHOLD_QUERIES` | `30` | Query count an example must exceed to make that list |
199
+ | `RSPEC_PROFILE_GROUP_BY` | โ€” | `1`/`auto`, a base path, or a comma list of folders to bucket by |
200
+
201
+ ### Coverage (optional)
202
+
203
+ With `COVERAGE=1`, each worker records coverage under its own `TEST_ENV_NUMBER`
204
+ and rspec-turbo merges the results into a single report when the run ends, via
205
+ the bundled `coverage:merge` task โ€” **JSON on CI**
206
+ (`SimpleCov::Formatter::JSONFormatter`), **HTML locally**
207
+ (`SimpleCov::Formatter::HTMLFormatter`).
208
+
209
+ 1. Add the formatters to your Gemfile:
210
+
211
+ ```ruby
212
+ group :test do
213
+ gem "simplecov", require: false
214
+ gem "simplecov_json_formatter", require: false
215
+ end
216
+ ```
217
+
218
+ 2. Have each worker write its **own** result file (so parallel workers don't
219
+ clobber each other), keyed by `TEST_ENV_NUMBER` โ€” at the very top of
220
+ `spec/spec_helper.rb`, before your app is required:
221
+
222
+ ```ruby
223
+ if ENV["COVERAGE"] == "1"
224
+ require "simplecov"
225
+ SimpleCov.command_name "worker_#{ENV["TEST_ENV_NUMBER"]}"
226
+ SimpleCov.coverage_dir "coverage/#{ENV["TEST_ENV_NUMBER"]}"
227
+ SimpleCov.start "rails"
228
+ end
229
+ ```
230
+
231
+ 3. Run it:
232
+
233
+ ```sh
234
+ COVERAGE=1 bundle exec rspec-turbo
235
+ ```
236
+
237
+ The merge collates `coverage/**/.resultset.json` (override with
238
+ `RSPEC_TURBO_COVERAGE_GLOB`) and writes the combined report to `coverage/`. You
239
+ can also run it on its own: `bundle exec rake coverage:merge`.
240
+
241
+ ## Architecture
242
+
243
+ ```
244
+ lib/rspec_turbo/
245
+ โ”œโ”€โ”€ config.rb # env-driven settings + derived log paths
246
+ โ”œโ”€โ”€ terminal.rb # colour, duration formatting, spinner, separators
247
+ โ”œโ”€โ”€ options.rb # split ARGV into rspec flags vs folders
248
+ โ”œโ”€โ”€ db_setup.rb # cached parallel test-DB creation (Rails)
249
+ โ”œโ”€โ”€ file_discovery.rb # find + filter *_spec.rb files
250
+ โ”œโ”€โ”€ batch_planner.rb # dry-run counting + LPT bin-packing
251
+ โ”œโ”€โ”€ display.rb # live spinner + final report + log parsing
252
+ โ”œโ”€โ”€ worker.rb # spawn one rspec process per batch
253
+ โ”œโ”€โ”€ executor.rb # the slot pool + TTY/CI run loops
254
+ โ”œโ”€โ”€ runner.rb # top-level orchestration
255
+ โ”œโ”€โ”€ progress_reporter.rb # formatter injected into workers (progress bar)
256
+ โ”œโ”€โ”€ slow_profile.rb # profiler injected into workers (slow report)
257
+ โ”œโ”€โ”€ railtie.rb # registers the spec:turbo task in Rails apps
258
+ โ””โ”€โ”€ tasks.rake # the rake spec:turbo / rails spec:turbo task
259
+ ```
260
+
261
+ ## Development
262
+
263
+ ```sh
264
+ bundle install
265
+ bundle exec rake # runs the specs + Standard
266
+ bundle exec rspec # specs only
267
+ bundle exec standardrb # lint
268
+ bundle exec standardrb --fix
269
+ ```
270
+
271
+ Style is enforced by [Standard Ruby](https://github.com/standardrb/standard).
272
+ The `.rubocop.yml` simply loads Standard's ruleset so editors and tooling that
273
+ speak RuboCop pick up the same rules; the canonical runner is `standardrb`.
274
+ VS Code is pre-wired (`.vscode/settings.json`) to format on save with Standard
275
+ via the Ruby LSP extension.
276
+
277
+ ## Contributing
278
+
279
+ Issues and pull requests are welcome. Run `bundle exec rake` before opening a
280
+ PR โ€” it must be green (specs + Standard).
281
+
282
+ **If `rspec-turbo` shaves minutes off your CI, drop a โญ on the repo** โ€” it helps
283
+ other RSpec teams find it.
284
+
285
+ ## License
286
+
287
+ MIT. See [LICENSE.txt](LICENSE.txt).
data/exe/rspec-turbo ADDED
@@ -0,0 +1,6 @@
1
+ #!/usr/bin/env ruby
2
+ # frozen_string_literal: true
3
+
4
+ require "rspec_turbo"
5
+
6
+ RSpecTurbo::Runner.new(ARGV).run
@@ -0,0 +1,4 @@
1
+ # frozen_string_literal: true
2
+
3
+ # Alias so `require "rspec-turbo"` (matching the gem name) works too.
4
+ require_relative "rspec_turbo"
@@ -0,0 +1,127 @@
1
+ # frozen_string_literal: true
2
+
3
+ require "json"
4
+
5
+ module RSpecTurbo
6
+ # Runs `rspec --dry-run --format json` to count examples per file, then packs
7
+ # the files into N balanced batches using the Longest-Processing-Time first
8
+ # (LPT) greedy heuristic. Files heavier than a single batch's fair share are
9
+ # split into slices of individual example IDs so one huge file can't bottle-
10
+ # neck a worker.
11
+ #
12
+ # If the dry-run fails for any reason it falls back to equal-weight packing.
13
+ class BatchPlanner
14
+ attr_reader :counts, :batches, :pending_count, :dry_run_elapsed
15
+
16
+ def initialize(files, num_workers:, rspec_options: [])
17
+ @files = files
18
+ @n = num_workers
19
+ @rspec_options = rspec_options
20
+ @counts = {}
21
+ @batches = []
22
+ @pending_count = 0
23
+ @dry_run_elapsed = 0
24
+ end
25
+
26
+ def plan!
27
+ result = dry_run
28
+ @counts = result[:counts]
29
+ @pending_count = result[:pending_count]
30
+ units = build_units(@files, @counts, result[:ids])
31
+ @batches = bin_pack(units)
32
+
33
+ self
34
+ end
35
+
36
+ def example_count(units) = units.sum { |unit| unit_weight(unit) }
37
+
38
+ private
39
+
40
+ def dry_run
41
+ return empty_result if @files.empty?
42
+
43
+ t0 = Process.clock_gettime(Process::CLOCK_MONOTONIC)
44
+ raw = capture_dry_run
45
+ @dry_run_elapsed = (Process.clock_gettime(Process::CLOCK_MONOTONIC) - t0).round
46
+
47
+ json_start = raw.index("{")
48
+ raise "No JSON in dry-run output" unless json_start
49
+
50
+ parse_examples(JSON.parse(raw[json_start..]))
51
+ rescue => e
52
+ warn " โš  dry-run failed (#{e.message}) โ€” using equal-weight distribution"
53
+ log_dry_run_error
54
+
55
+ empty_result
56
+ end
57
+
58
+ def capture_dry_run
59
+ File.open(Config.dry_run_log, "w") do |err_file|
60
+ IO.popen(
61
+ # COVERAGE=0 keeps SimpleCov from contaminating the JSON on stdout.
62
+ [{"COVERAGE" => "0", "TEST_ENV_NUMBER" => "1"},
63
+ "bundle", "exec", "rspec", "--dry-run", "--format", "json",
64
+ *@rspec_options, *@files.map { |f| "spec/#{f}" }],
65
+ err: err_file, &:read
66
+ )
67
+ end
68
+ end
69
+
70
+ def parse_examples(parsed)
71
+ counts = Hash.new(0)
72
+ ids = Hash.new { |hash, key| hash[key] = [] }
73
+
74
+ parsed["examples"].each do |example|
75
+ file = example["file_path"].delete_prefix("./spec/")
76
+ counts[file] += 1
77
+ ids[file] << example["id"]
78
+ end
79
+
80
+ {counts: counts, ids: ids, pending_count: parsed.dig("summary", "pending_count").to_i}
81
+ end
82
+
83
+ def empty_result = {counts: {}, ids: {}, pending_count: 0}
84
+
85
+ def log_dry_run_error
86
+ return unless File.exist?(Config.dry_run_log)
87
+
88
+ last_err = File.readlines(Config.dry_run_log).last(10).join.strip
89
+ warn " Dry-run stderr:\n#{last_err}" unless last_err.empty?
90
+ end
91
+
92
+ def unit_weight(unit) = unit.is_a?(Array) ? unit.size : (@counts[unit] || 1)
93
+
94
+ # A "unit" is either a whole file (a String) or a slice of example IDs
95
+ # (an Array) carved out of a file too heavy to fit in one batch.
96
+ def build_units(files, counts, ids)
97
+ total = files.sum { |f| counts[f] || 1 }
98
+ threshold = [(total.to_f / @n).ceil, 1].max
99
+
100
+ files.flat_map do |file|
101
+ file_ids = ids[file].to_a
102
+
103
+ if (counts[file] || 1) > threshold && file_ids.size > 1
104
+ file_ids.each_slice(threshold).to_a
105
+ else
106
+ [file]
107
+ end
108
+ end
109
+ end
110
+
111
+ def bin_pack(units)
112
+ n = [@n, units.size].min
113
+
114
+ return [units] if n <= 1
115
+
116
+ buckets = Array.new(n) { [0, []] }
117
+
118
+ units.sort_by { |unit| -unit_weight(unit) }.each do |unit|
119
+ bucket = buckets.min_by { |total, _| total }
120
+ bucket[1] << unit
121
+ bucket[0] += unit_weight(unit)
122
+ end
123
+
124
+ buckets.reject { |_, packed| packed.empty? }.map { |_, packed| packed }
125
+ end
126
+ end
127
+ end
@@ -0,0 +1,63 @@
1
+ # frozen_string_literal: true
2
+
3
+ require "etc"
4
+
5
+ module RSpecTurbo
6
+ # Central place for environment-driven settings and derived log paths.
7
+ #
8
+ # All tuning happens through environment variables so the runner stays a
9
+ # single zero-config binary:
10
+ #
11
+ # RSPEC_TURBO_MAX number of parallel workers (default: nproc)
12
+ # RSPEC_TURBO_LOG_DIR where per-worker logs live
13
+ # RSPEC_TURBO_FORCE_SETUP=1 recreate test DBs even if cached
14
+ # RSPEC_TURBO_PROGRESS_INTERVAL seconds between CI progress lines
15
+ # COVERAGE=1 merge SimpleCov results after the run
16
+ # JUNIT_DIR=path emit JUnit XML per worker into this dir
17
+ #
18
+ # Slow-test profiling is opt-in and feeds the "Slowest folders/files" report
19
+ # (see slow_profile.rb): RSPEC_PROFILE_SLOW=1, RSPEC_PROFILE_GROUP_BY, etc.
20
+ module Config
21
+ module_function
22
+
23
+ TTY = $stdout.tty? && !ENV["CI"]
24
+
25
+ def tty? = TTY
26
+
27
+ def workers
28
+ Integer(ENV.fetch("RSPEC_TURBO_MAX") { ENV.fetch("RSPEC_PARALLEL_MAX", Etc.nprocessors) })
29
+ end
30
+
31
+ def force_setup?
32
+ flag = ENV["RSPEC_TURBO_FORCE_SETUP"] || ENV["RSPEC_PARALLEL_FORCE_SETUP"]
33
+
34
+ %w[1 true yes].include?(flag.to_s.downcase)
35
+ end
36
+
37
+ def progress_interval = Integer(ENV.fetch("RSPEC_TURBO_PROGRESS_INTERVAL", "30"))
38
+
39
+ def coverage? = %w[1 true].include?(ENV.fetch("COVERAGE", "0").downcase)
40
+
41
+ def junit_dir = ENV["JUNIT_DIR"]
42
+
43
+ # Slow-test profiling is on by default; RSPEC_TURBO_NO_PROFILE=1 is the
44
+ # master kill switch. See slow_profile.rb and Worker.profile_env.
45
+ def profile? = !%w[1 true yes].include?(ENV["RSPEC_TURBO_NO_PROFILE"].to_s.downcase)
46
+
47
+ # โ”€โ”€ Derived paths โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€
48
+
49
+ def log_dir = ENV.fetch("RSPEC_TURBO_LOG_DIR") { ENV.fetch("RSPEC_LOG_DIR", "tmp/rspec-turbo") }
50
+
51
+ def setup_marker_dir = File.join(log_dir, "setup")
52
+
53
+ def log_path(label) = File.join(log_dir, "#{label.tr("/", "_")}.log")
54
+
55
+ def progress_path(slot) = File.join(log_dir, "progress_#{slot}.txt")
56
+
57
+ def setup_log_path(slot) = File.join(log_dir, "setup_slot#{slot}.log")
58
+
59
+ def dry_run_log = File.join(log_dir, "dry_run_stderr.log")
60
+
61
+ def coverage_merge_log = File.join(log_dir, "coverage_merge.log")
62
+ end
63
+ end
@@ -0,0 +1,86 @@
1
+ # frozen_string_literal: true
2
+
3
+ require "digest"
4
+ require "fileutils"
5
+
6
+ module RSpecTurbo
7
+ # Ensures N test databases exist, one per worker slot, by spawning a Rails
8
+ # db setup process per slot (each with its own TEST_ENV_NUMBER).
9
+ #
10
+ # The result is cached by a fingerprint of db/schema.rb + db/seeds.rb plus
11
+ # the worker count, so repeat runs skip setup entirely. Set
12
+ # RSPEC_TURBO_FORCE_SETUP=1 to force recreation.
13
+ class DbSetup
14
+ FINGERPRINT_FILES = ["db/schema.rb", "db/seeds.rb"].freeze
15
+ SETUP_COMMAND = ["bundle", "exec", "rails", "db:drop", "db:create", "db:schema:load", "db:seed"].freeze
16
+
17
+ def initialize(num_workers, force: Config.force_setup?)
18
+ @n = num_workers
19
+ @force = force
20
+ end
21
+
22
+ def run!
23
+ return true if !@force && cached?
24
+
25
+ FileUtils.mkdir_p(Config.log_dir)
26
+ failed = wait_all(spawn_all)
27
+
28
+ return write_marker && true if failed.empty?
29
+
30
+ failed.each { |worker| show_log(worker) }
31
+ false
32
+ end
33
+
34
+ def cached? = File.exist?(marker_path)
35
+
36
+ private
37
+
38
+ def spawn_all
39
+ (1..@n).map do |slot|
40
+ log = Config.setup_log_path(slot)
41
+ pid = Process.spawn(
42
+ {"TEST_ENV_NUMBER" => slot.to_s, "RAILS_ENV" => "test"},
43
+ *SETUP_COMMAND,
44
+ out: log, err: [:child, :out]
45
+ )
46
+
47
+ {pid: pid, slot: slot, log: log}
48
+ end
49
+ end
50
+
51
+ def wait_all(workers)
52
+ workers.filter_map do |worker|
53
+ _, status = Process.waitpid2(worker[:pid])
54
+
55
+ status.success? ? nil : worker
56
+ end
57
+ end
58
+
59
+ def show_log(worker)
60
+ return unless File.exist?(worker[:log])
61
+
62
+ warn "\nโ”€โ”€ slot #{worker[:slot]} output โ”€โ”€"
63
+ warn File.read(worker[:log]).lines.last(15).join
64
+ end
65
+
66
+ def write_marker
67
+ FileUtils.mkdir_p(Config.setup_marker_dir)
68
+ FileUtils.touch(marker_path)
69
+ end
70
+
71
+ def marker_path
72
+ File.join(Config.setup_marker_dir, "slots-#{@n}-schema-#{schema_fingerprint}")
73
+ end
74
+
75
+ def schema_fingerprint
76
+ digest = Digest::SHA256.new
77
+
78
+ FINGERPRINT_FILES.each do |path|
79
+ digest.update(path)
80
+ digest.update(File.exist?(path) ? File.read(path) : "")
81
+ end
82
+
83
+ digest.hexdigest[0, 12]
84
+ end
85
+ end
86
+ end