rspec-parallelize 0.1.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml ADDED
@@ -0,0 +1,7 @@
1
+ ---
2
+ SHA256:
3
+ metadata.gz: d8d7264478426a91fda4840abeaa5880f023c51a35010cca70f06005358d1b3b
4
+ data.tar.gz: b344bb00b3462817fe9a3430be2e847cd1372a8d8175bfaa16534dc832840607
5
+ SHA512:
6
+ metadata.gz: 91448ed11ff3995f703dcd495796232656973a3adba1623bdca500edea2d56147499b5acd2e94e684ad99bedb22ab3c8aaa2ab46b4d21920b18e304e3bd4f047
7
+ data.tar.gz: 02704b59a0eacbb439b89d98fe508794a7ff9867325e3d4d4394e8ecb1c16feddae5abbacf81e7c4109eb5cfc76c3d433d6175454df0495faf859e101ea89a78
data/CHANGELOG.md ADDED
@@ -0,0 +1,5 @@
1
+ # rspec-parallelize
2
+
3
+ ## 0.1.0
4
+
5
+ - First release: plain `rspec` runs in parallel like Rails' minitest `parallelize`. Workers are forked after boot and pull top-level example groups from a shared queue, results show in RSpec's normal output, and with ActiveRecord each worker gets its own test database. Shows a live progress line in the terminal.
data/LICENSE.txt ADDED
@@ -0,0 +1,21 @@
1
+ MIT License
2
+
3
+ Copyright (c) 2026 Alpha
4
+
5
+ Permission is hereby granted, free of charge, to any person obtaining a copy
6
+ of this software and associated documentation files (the "Software"), to deal
7
+ in the Software without restriction, including without limitation the rights
8
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9
+ copies of the Software, and to permit persons to whom the Software is
10
+ furnished to do so, subject to the following conditions:
11
+
12
+ The above copyright notice and this permission notice shall be included in all
13
+ copies or substantial portions of the Software.
14
+
15
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21
+ SOFTWARE.
data/README.md ADDED
@@ -0,0 +1,93 @@
1
+ # rspec-parallelize
2
+
3
+ Run plain `rspec` in parallel, the way Rails' minitest `parallelize` does.
4
+
5
+ Your suite boots **once**, then workers are forked from it (sharing memory
6
+ copy-on-write) and pull top-level example groups from a shared queue. Results
7
+ are replayed into RSpec's own reporter, so the output looks like a normal run
8
+ and every formatter, `--seed`, `--only-failures` and CI reporter keeps working.
9
+ No extra binary, no `rake parallel:setup`.
10
+
11
+ ## Usage
12
+
13
+ ```ruby
14
+ # Gemfile
15
+ gem "rspec-parallelize", group: :test
16
+
17
+ # spec/spec_helper.rb (or spec/rails_helper.rb) — once, for the whole suite
18
+ require "rspec/parallelize"
19
+ RSpec::Parallelize.configure
20
+ ```
21
+
22
+ Then run `rspec` as usual.
23
+
24
+ | Option / env | Effect |
25
+ |---|---|
26
+ | `workers:` (default `:half_of_processors`) | Integer, `:half_of_processors` (half the CPU cores, min 2 — keeps your laptop usable) or `:number_of_processors` |
27
+ | `threshold:` (default 50) | Runs with fewer examples stay serial, so a single spec is still instant |
28
+ | `RSPEC_PARALLELIZE_PROGRESS=0` | Hides the live progress line (it is already off in CI and when output is piped) |
29
+ | `PARALLEL_WORKERS=8` | Overrides `workers:` (e.g. more workers on CI); `PARALLEL_WORKERS=1` disables parallelism |
30
+
31
+ ### Per-worker setup
32
+
33
+ ```ruby
34
+ RSpec::Parallelize.after_fork do |worker_number| # 0-based
35
+ Redis.current.select(worker_number)
36
+ end
37
+
38
+ RSpec::Parallelize.before_exit do |worker_number|
39
+ # clean up per-worker resources
40
+ end
41
+ ```
42
+
43
+ Each worker also gets `TEST_ENV_NUMBER` (`""`, `"2"`, `"3"`, … like
44
+ parallel_tests), so existing `database.yml` templates keep working.
45
+
46
+ ## Rails / ActiveRecord
47
+
48
+ If ActiveRecord is loaded, every worker automatically gets its own test
49
+ database, just like Rails' minitest `parallelize`: `myapp_test` becomes
50
+ `myapp_test_0`, `myapp_test_1`, ..., each rebuilt from `db/schema.rb` (or
51
+ `structure.sql`) at the start of the run. Nothing to create by hand, no
52
+ `TEST_ENV_NUMBER` needed in `database.yml`. It uses Rails' own
53
+ `ActiveRecord::TestDatabases`, and respects
54
+ `config.active_support.parallelize_test_databases = false`.
55
+
56
+ In a plain Ruby project none of this loads; no Rails code is required.
57
+
58
+ ## How it works
59
+
60
+ - A **unit of work** is a top-level example group, so `before(:all)` hooks run
61
+ exactly once, as they do serially.
62
+ - `before(:suite)` / `after(:suite)` hooks run in **each worker**, so per-worker
63
+ resources (databases etc.) get set up and cleaned up.
64
+ - If a worker crashes, its group is reported as an error, a replacement worker
65
+ is forked, and the run exits non-zero.
66
+ - Requires `fork` (MRI on Linux/macOS). Elsewhere it warns and runs serially.
67
+
68
+ ## Status
69
+
70
+ Early. Planned next: slowest-groups-first scheduling from recorded runtimes,
71
+ and SimpleCov merging.
72
+
73
+ ## Development
74
+
75
+ ```sh
76
+ bundle install
77
+ bundle exec rake # specs + RuboCop
78
+ bundle exec rspec # specs only
79
+ bundle exec rubocop -a # lint, applying safe fixes
80
+ ```
81
+
82
+ ### Releasing
83
+
84
+ Versions are managed with [Changesets](https://github.com/changesets/changesets);
85
+ `package.json` holds the version and `scripts/sync_version.rb` copies it into
86
+ `lib/rspec/parallelize/version.rb` and `Gemfile.lock`.
87
+
88
+ 1. `pnpm install`, then `pnpm set-release` in your branch to describe the
89
+ change (patch / minor / major). Commit the generated `.changeset/*.md`.
90
+ 2. Once merged to `main`, the Release workflow opens a "Version Packages" PR
91
+ (version bump + `CHANGELOG.md`).
92
+ 3. Merging that PR tags `vX.Y.Z`, creates the GitHub release and publishes the
93
+ gem to RubyGems via trusted publishing.
@@ -0,0 +1,39 @@
1
+ # frozen_string_literal: true
2
+
3
+ module RSpec
4
+ module Parallelize
5
+ # Gives every worker its own test database(s), the way Rails' minitest
6
+ # `parallelize` does: `myapp_test` becomes `myapp_test_0`, `myapp_test_1`,
7
+ # ... each rebuilt from db/schema.rb (or structure.sql) at the start of the
8
+ # run. Delegates to Rails' own ActiveRecord::TestDatabases.
9
+ #
10
+ # Only active when ActiveRecord is loaded, and honours
11
+ # `config.active_support.parallelize_test_databases = false`.
12
+ module ActiveRecordIntegration
13
+ module_function
14
+
15
+ def enabled?
16
+ return false unless defined?(::ActiveRecord::Base)
17
+ return true unless ::ActiveSupport.respond_to?(:parallelize_test_databases)
18
+
19
+ ::ActiveSupport.parallelize_test_databases != false
20
+ end
21
+
22
+ # Forked children must not share the parent's database sockets.
23
+ def before_fork
24
+ return unless enabled?
25
+
26
+ require 'active_record/test_databases'
27
+ ::ActiveRecord::Base.connection_handler.clear_all_connections!
28
+ end
29
+
30
+ def after_fork(worker_number)
31
+ return unless enabled?
32
+
33
+ ::ActiveRecord::TestDatabases.create_and_load_schema(
34
+ worker_number, env_name: ::ActiveRecord::ConnectionHandling::DEFAULT_ENV.call
35
+ )
36
+ end
37
+ end
38
+ end
39
+ end
@@ -0,0 +1,50 @@
1
+ # frozen_string_literal: true
2
+
3
+ module RSpec
4
+ module Parallelize
5
+ # Settings from RSpec::Parallelize.configure and the PARALLEL_WORKERS env var.
6
+ class Configuration
7
+ DEFAULT_THRESHOLD = 50
8
+ # Leaves half the cores free so the machine stays usable during a local
9
+ # run; CI can ask for more with PARALLEL_WORKERS or `workers:`.
10
+ DEFAULT_WORKERS = :half_of_processors
11
+
12
+ attr_reader :after_fork_hooks, :before_exit_hooks, :threshold
13
+
14
+ def initialize
15
+ @enabled = false
16
+ @workers = DEFAULT_WORKERS
17
+ @threshold = DEFAULT_THRESHOLD
18
+ @after_fork_hooks = []
19
+ @before_exit_hooks = []
20
+ end
21
+
22
+ def enable(workers:, threshold:)
23
+ @enabled = true
24
+ @workers = workers
25
+ @threshold = Integer(threshold)
26
+ end
27
+
28
+ def enabled?
29
+ @enabled
30
+ end
31
+
32
+ # PARALLEL_WORKERS wins over the configured value, like Rails.
33
+ def worker_count
34
+ value = ENV.fetch('PARALLEL_WORKERS', '').strip
35
+
36
+ value = @workers if value.empty?
37
+
38
+ case value.to_s
39
+ when 'number_of_processors' then Etc.nprocessors
40
+ when 'half_of_processors' then [Etc.nprocessors / 2, 2].max
41
+ else Integer(value)
42
+ end
43
+ end
44
+
45
+ def fork_supported?
46
+ Process.respond_to?(:fork) && RUBY_ENGINE == 'ruby'
47
+ end
48
+ end
49
+ end
50
+ end
@@ -0,0 +1,225 @@
1
+ # frozen_string_literal: true
2
+
3
+ module RSpec
4
+ module Parallelize
5
+ # The parent side. Forks workers, hands out top-level groups from a queue
6
+ # as workers become free, and replays the events they send back into
7
+ # RSpec's real reporter so every formatter sees a normal run.
8
+ class Orchestrator
9
+ WorkerState = Struct.new(:number, :pid, :commands, :results, :group_id, :done, keyword_init: true)
10
+
11
+ # How often the live progress line is redrawn while nothing arrives.
12
+ PROGRESS_INTERVAL = 1
13
+
14
+ def initialize(configuration, example_groups, worker_count)
15
+ @configuration = configuration
16
+ @example_groups = example_groups
17
+ @worker_count = worker_count
18
+ @queue = example_groups.map(&:id)
19
+ @workers = {}
20
+ @stats = Hash.new { |h, number| h[number] = { pids: [], groups: 0, examples: 0 } }
21
+ @progress = ProgressLine.new(configuration.output_stream)
22
+ index_suite
23
+ end
24
+
25
+ # Returns true when every example passed.
26
+ def run
27
+ @example_count = RSpec.world.example_count(@example_groups)
28
+ @configuration.reporter.report(@example_count) do |reporter|
29
+ @reporter = reporter
30
+ @started_at = Process.clock_gettime(Process::CLOCK_MONOTONIC)
31
+ announce
32
+ @worker_count.times { |number| spawn_worker(number) }
33
+ process_messages until @workers.empty?
34
+ @progress.stop
35
+ report_stats
36
+ reporter.failed_examples.empty?
37
+ end
38
+ ensure
39
+ @progress.stop
40
+ shutdown
41
+ end
42
+
43
+ private
44
+
45
+ def index_suite
46
+ @top_groups = {}
47
+ @groups = {}
48
+ @examples = {}
49
+ @example_groups.each do |top|
50
+ @top_groups[top.id] = top
51
+ top.descendants.each do |group|
52
+ @groups[group.id] = group
53
+ group.examples.each { |example| @examples[example.id] = example }
54
+ end
55
+ end
56
+ end
57
+
58
+ def spawn_worker(number)
59
+ commands_r, commands_w = IO.pipe
60
+ results_r, results_w = IO.pipe
61
+ [commands_r, commands_w, results_r, results_w].each(&:binmode)
62
+ parent_ends = @workers.values.flat_map { |w| [w.commands, w.results] }
63
+ ActiveRecordIntegration.before_fork
64
+
65
+ pid = Process.fork do
66
+ # Drop every parent-side pipe end so a worker's EOF is detected even
67
+ # while siblings are still running.
68
+ (parent_ends + [commands_w, results_r]).each(&:close)
69
+ Worker.new(number, @configuration, @top_groups, commands_r, results_w).run
70
+ end
71
+
72
+ commands_r.close
73
+ results_w.close
74
+ @stats[number][:pids] << pid
75
+ @workers[results_r] = WorkerState.new(number: number, pid: pid, commands: commands_w, results: results_r)
76
+ end
77
+
78
+ def announce
79
+ databases = ActiveRecordIntegration.enabled? ? ', each with its own test database' : ''
80
+ @reporter.message("rspec-parallelize: running #{@example_count} examples in #{@example_groups.size} groups " \
81
+ "using #{@worker_count} processes#{databases} (parent pid #{Process.pid})")
82
+ @progress.start
83
+ draw_progress
84
+ end
85
+
86
+ def draw_progress
87
+ finished = @stats.each_value.sum { |stat| stat[:examples] }
88
+ busy = @workers.each_value.count(&:group_id)
89
+ failures = @reporter.failed_examples.size
90
+ elapsed = (Process.clock_gettime(Process::CLOCK_MONOTONIC) - @started_at).round
91
+
92
+ @progress.update(
93
+ "rspec-parallelize: #{finished}/#{@example_count} examples · " \
94
+ "#{failures} #{failures == 1 ? 'failure' : 'failures'} · " \
95
+ "#{busy}/#{@worker_count} workers busy · #{elapsed}s"
96
+ )
97
+ end
98
+
99
+ def report_stats
100
+ lines = @stats.sort.map do |number, stat|
101
+ format(' worker %<n>d (pid %<pids>s): %<groups>d groups, %<examples>d examples',
102
+ n: number, pids: stat[:pids].join(', '), groups: stat[:groups], examples: stat[:examples])
103
+ end
104
+ @reporter.message("\nrspec-parallelize: work per process\n#{lines.join("\n")}")
105
+ end
106
+
107
+ def process_messages
108
+ ready, = IO.select(@workers.keys, nil, nil, PROGRESS_INTERVAL)
109
+ Array(ready).each do |io|
110
+ worker = @workers.fetch(io)
111
+ message = begin
112
+ Marshal.load(io) # rubocop:disable Security/MarshalLoad
113
+ rescue EOFError
114
+ nil
115
+ end
116
+
117
+ message ? handle(worker, message) : worker_exited(worker)
118
+ end
119
+ draw_progress
120
+ end
121
+
122
+ def handle(worker, message)
123
+ case message.first
124
+ when :ready
125
+ replay(message[1])
126
+ dispatch(worker)
127
+ when :result
128
+ replay(message[2])
129
+ @stats[worker.number][:groups] += 1
130
+ @stats[worker.number][:examples] += message[2].count { |event| event.first == :example_finished }
131
+ worker.group_id = nil
132
+ dispatch(worker)
133
+ when :done
134
+ replay(message[1])
135
+ worker.done = true
136
+ end
137
+ end
138
+
139
+ def dispatch(worker)
140
+ @queue.clear if RSpec.world.wants_to_quit || @reporter.fail_fast_limit_met?
141
+ worker.group_id = @queue.shift
142
+ Marshal.dump(worker.group_id, worker.commands)
143
+ worker.commands.flush
144
+ rescue Errno::EPIPE
145
+ nil # the worker died; its EOF is handled in #worker_exited
146
+ end
147
+
148
+ def worker_exited(worker)
149
+ @workers.delete(worker.results)
150
+ worker.results.close
151
+ worker.commands.close unless worker.commands.closed?
152
+ _, status = Process.wait2(worker.pid)
153
+ return if worker.done
154
+
155
+ report_crash(worker, status)
156
+ # Only replace workers that got as far as running a group; one dying
157
+ # during boot (e.g. a failing after_fork hook) would just die again.
158
+ if worker.group_id && !@queue.empty?
159
+ spawn_worker(worker.number)
160
+ elsif @workers.empty? && !@queue.empty?
161
+ report_unrun_groups
162
+ end
163
+ end
164
+
165
+ def report_unrun_groups
166
+ error = RuntimeError.new("#{@queue.size} example group(s) were not run because every worker crashed")
167
+ error.set_backtrace([])
168
+ @reporter.notify_non_example_exception(error, 'rspec-parallelize')
169
+ @queue.clear
170
+ end
171
+
172
+ def report_crash(worker, status)
173
+ what = worker.group_id ? "while running #{describe_group(worker.group_id)}" : 'before finishing'
174
+ error = RuntimeError.new("Worker #{worker.number} crashed #{what} (#{status.inspect})")
175
+ # Point at the crashed group's `describe` line so RSpec shows it instead
176
+ # of "Unable to find matching line from backtrace".
177
+ location = worker.group_id && @top_groups.fetch(worker.group_id).metadata[:location]
178
+ error.set_backtrace(location ? ["#{location}:in '<top (required)>'"] : [])
179
+ @reporter.notify_non_example_exception(error, "rspec-parallelize worker #{worker.number}")
180
+ end
181
+
182
+ def describe_group(group_id)
183
+ group = @top_groups.fetch(group_id)
184
+ "#{group.metadata[:rerun_file_path] || group.file_path} (#{group.description})"
185
+ end
186
+
187
+ def replay(events)
188
+ events.each { |name, *args| replay_event(name, *args) }
189
+ end
190
+
191
+ def replay_event(name, *args)
192
+ case name
193
+ when :example_group_started, :example_group_finished
194
+ @reporter.public_send(name, @groups.fetch(args[0]))
195
+ when :example_started, :example_passed, :example_failed, :example_pending
196
+ @reporter.public_send(name, @examples.fetch(args[0]))
197
+ when :example_finished
198
+ example = @examples.fetch(args[0])
199
+ Serializer.load_result(example, args[1])
200
+ @reporter.example_finished(example)
201
+ when :non_example_exception
202
+ @reporter.notify_non_example_exception(Serializer.load_exception(args[0]), args[1])
203
+ else # :message, :deprecation, :publish
204
+ @reporter.public_send(name, *args)
205
+ end
206
+ end
207
+
208
+ # Reached on normal completion and on interrupts: never leave orphans.
209
+ # Signal every worker before waiting on any, so they stop in parallel.
210
+ def shutdown
211
+ @workers.each_value do |worker|
212
+ Process.kill(:TERM, worker.pid)
213
+ rescue Errno::ESRCH
214
+ nil
215
+ end
216
+ @workers.each_value do |worker| # rubocop:disable Style/CombinableLoops
217
+ Process.wait(worker.pid)
218
+ rescue Errno::ECHILD
219
+ nil
220
+ end
221
+ @workers.clear
222
+ end
223
+ end
224
+ end
225
+ end
@@ -0,0 +1,82 @@
1
+ # frozen_string_literal: true
2
+
3
+ require 'io/console'
4
+
5
+ module RSpec
6
+ module Parallelize
7
+ # A live status line pinned to the bottom row of the terminal:
8
+ #
9
+ # rspec-parallelize: 1234/6288 examples · 2 failures · 12/12 workers busy · 41s
10
+ #
11
+ # The rows above it become a scroll region, so formatter output (dots,
12
+ # failures) scrolls as usual without overwriting the line. Only shown on
13
+ # an interactive terminal; never in CI, when piped, or with
14
+ # RSPEC_PARALLELIZE_PROGRESS=0.
15
+ class ProgressLine
16
+ MIN_ROWS = 3
17
+
18
+ def initialize(output)
19
+ @output = output
20
+ @active = false
21
+ end
22
+
23
+ def enabled?
24
+ @output.respond_to?(:tty?) && @output.tty? &&
25
+ !ENV.key?('CI') && ENV['RSPEC_PARALLELIZE_PROGRESS'] != '0' &&
26
+ rows >= MIN_ROWS
27
+ end
28
+
29
+ def start
30
+ return unless enabled?
31
+
32
+ @rows = rows
33
+ # Make room for the status line, then confine scrolling to the rows above it.
34
+ write("\n\e[1A")
35
+ reserve_bottom_row
36
+ @active = true
37
+ end
38
+
39
+ def update(text)
40
+ return unless @active
41
+
42
+ if rows != @rows # terminal was resized
43
+ @rows = rows
44
+ reserve_bottom_row
45
+ end
46
+
47
+ write("\e7\e[#{@rows};1H\e[2K#{text[0, columns - 1]}\e8")
48
+ end
49
+
50
+ def stop
51
+ return unless @active
52
+
53
+ @active = false
54
+ write("\e7\e[#{@rows};1H\e[2K\e[r\e8")
55
+ end
56
+
57
+ private
58
+
59
+ # DECSTBM moves the cursor home, hence the save/restore around it.
60
+ def reserve_bottom_row
61
+ write("\e7\e[1;#{@rows - 1}r\e8")
62
+ end
63
+
64
+ def rows
65
+ @output.winsize[0]
66
+ rescue StandardError
67
+ 0
68
+ end
69
+
70
+ def columns
71
+ @output.winsize[1]
72
+ rescue StandardError
73
+ 80
74
+ end
75
+
76
+ def write(sequence)
77
+ @output.write(sequence)
78
+ @output.flush
79
+ end
80
+ end
81
+ end
82
+ end
@@ -0,0 +1,40 @@
1
+ # frozen_string_literal: true
2
+
3
+ module RSpec
4
+ module Parallelize
5
+ # Prepended onto RSpec::Core::Runner. RSpec has already loaded, filtered
6
+ # and ordered the groups by the time `run_specs` is called; we only change
7
+ # who executes them.
8
+ module RunnerExtension
9
+ def run_specs(example_groups)
10
+ config = Parallelize.configuration
11
+ return super unless config.enabled? && !@configuration.dry_run?
12
+
13
+ workers = config.worker_count
14
+ count = @world.example_count(example_groups)
15
+ return super if count.zero?
16
+
17
+ reason = serial_reason(config, workers, count)
18
+ if reason
19
+ @configuration.reporter.message("rspec-parallelize: running serially (#{reason})")
20
+ return super
21
+ end
22
+
23
+ passed = Orchestrator.new(@configuration, example_groups, [workers, example_groups.size].min).run
24
+ exit_code(passed)
25
+ end
26
+
27
+ private
28
+
29
+ def serial_reason(config, workers, count)
30
+ if workers <= 1
31
+ "workers = #{workers}"
32
+ elsif count < config.threshold
33
+ "#{count} examples is below the threshold of #{config.threshold}"
34
+ elsif !config.fork_supported?
35
+ "fork is not available on #{RUBY_ENGINE}/#{RUBY_PLATFORM}"
36
+ end
37
+ end
38
+ end
39
+ end
40
+ end
@@ -0,0 +1,90 @@
1
+ # frozen_string_literal: true
2
+
3
+ module RSpec
4
+ module Parallelize
5
+ # Stand-in for an exception whose class can't be found in the parent.
6
+ class RemoteException < StandardError; end
7
+
8
+ # Converts example results and exceptions into plain data that can cross
9
+ # the worker -> parent pipe, and back again.
10
+ module Serializer
11
+ module_function
12
+
13
+ def dump_result(example)
14
+ result = example.execution_result
15
+ {
16
+ status: result.status,
17
+ started_at: result.started_at,
18
+ finished_at: result.finished_at,
19
+ run_time: result.run_time,
20
+ pending_message: result.pending_message,
21
+ pending_fixed: result.pending_fixed,
22
+ exception: dump_exception(result.exception),
23
+ pending_exception: dump_exception(result.pending_exception),
24
+ # Descriptions of `it { is_expected.to ... }` are generated at run time.
25
+ description: example.metadata[:description],
26
+ full_description: example.metadata[:full_description]
27
+ }
28
+ end
29
+
30
+ def load_result(example, data)
31
+ result = example.execution_result
32
+
33
+ %i[status started_at finished_at run_time pending_message pending_fixed].each do |key|
34
+ result.public_send(:"#{key}=", data[key])
35
+ end
36
+
37
+ result.exception = load_exception(data[:exception])
38
+ result.pending_exception = load_exception(data[:pending_exception])
39
+ example.instance_variable_set(:@exception, result.exception)
40
+ example.metadata[:description] = data[:description]
41
+ example.metadata[:full_description] = data[:full_description]
42
+ end
43
+
44
+ def dump_exception(exception)
45
+ return nil if exception.nil?
46
+
47
+ {
48
+ marshaled: Marshal.dump(exception)
49
+ }
50
+ rescue TypeError, ArgumentError
51
+ {
52
+ class_name: exception.class.name,
53
+ message: safe_message(exception),
54
+ backtrace: exception.backtrace,
55
+ cause: dump_exception(exception.cause)
56
+ }
57
+ end
58
+
59
+ def load_exception(data)
60
+ return nil if data.nil?
61
+ return Marshal.load(data[:marshaled]) if data[:marshaled] # rubocop:disable Security/MarshalLoad
62
+
63
+ rebuild_exception(data)
64
+ end
65
+
66
+ # Recreates an exception of the original class without calling its
67
+ # (possibly picky) initializer, so formatters print the real class name.
68
+ def rebuild_exception(data)
69
+ klass = Object.const_get(data[:class_name].to_s) rescue nil # rubocop:disable Style/RescueModifier
70
+ klass = RemoteException unless klass.is_a?(Class) && klass <= Exception
71
+
72
+ message = klass == RemoteException ? "#{data[:class_name]}: #{data[:message]}" : data[:message]
73
+ exception = klass.allocate
74
+ Exception.instance_method(:initialize).bind_call(exception, message)
75
+ exception.define_singleton_method(:message) { message }
76
+ exception.define_singleton_method(:to_s) { message }
77
+ exception.set_backtrace(data[:backtrace]) if data[:backtrace]
78
+ cause = load_exception(data[:cause])
79
+ exception.define_singleton_method(:cause) { cause }
80
+ exception
81
+ end
82
+
83
+ def safe_message(exception)
84
+ exception.message.to_s
85
+ rescue Exception => e # rubocop:disable Lint/RescueException
86
+ "#{exception.class} (message raised #{e.class})"
87
+ end
88
+ end
89
+ end
90
+ end
@@ -0,0 +1,7 @@
1
+ # frozen_string_literal: true
2
+
3
+ module RSpec
4
+ module Parallelize
5
+ VERSION = '0.1.0'
6
+ end
7
+ end
@@ -0,0 +1,81 @@
1
+ # frozen_string_literal: true
2
+
3
+ module RSpec
4
+ module Parallelize
5
+ # The forked side. Pulls top-level group ids from the parent, runs them
6
+ # against its own (copy-on-write) copy of the loaded suite, and sends the
7
+ # recorded reporter events back.
8
+ #
9
+ # Protocol, one Marshal'd value per message:
10
+ # parent -> worker: group id to run, or nil to stop
11
+ # worker -> parent: [:result, group_id, events] after each group,
12
+ # [:done, events] once suite hooks have finished
13
+ class Worker
14
+ def initialize(number, configuration, groups_by_id, commands, results)
15
+ @number = number
16
+ @configuration = configuration
17
+ @groups_by_id = groups_by_id
18
+ @commands = commands
19
+ @results = results
20
+ end
21
+
22
+ def run
23
+ setup_environment
24
+ @reporter = install_reporter
25
+ run_after_fork_hooks
26
+ @configuration.with_suite_hooks { serve }
27
+ Parallelize.configuration.before_exit_hooks.each { |hook| hook.call(@number) }
28
+ send_message([:done, @reporter.drain])
29
+ finish(0)
30
+ rescue EOFError, Errno::EPIPE
31
+ finish(0) # parent went away
32
+ rescue Exception => e # rubocop:disable Lint/RescueException
33
+ warn "rspec-parallelize: worker #{@number} failed: #{e.class}: #{e.message}\n#{Array(e.backtrace).join("\n")}"
34
+ finish(1)
35
+ end
36
+
37
+ private
38
+
39
+ # Databases first, so user hooks and before(:suite) can already use them.
40
+ def run_after_fork_hooks
41
+ ActiveRecordIntegration.after_fork(@number)
42
+ Parallelize.configuration.after_fork_hooks.each { |hook| hook.call(@number) }
43
+ end
44
+
45
+ def serve
46
+ send_message([:ready, @reporter.drain])
47
+
48
+ while (group_id = Marshal.load(@commands)) # rubocop:disable Security/MarshalLoad
49
+ @groups_by_id.fetch(group_id).run(@reporter)
50
+ send_message([:result, group_id, @reporter.drain])
51
+ end
52
+ end
53
+
54
+ # TEST_ENV_NUMBER follows parallel_tests: "" for the first worker, then
55
+ # "2", "3", ... so existing database.yml templates keep working.
56
+ def setup_environment
57
+ ENV['TEST_ENV_NUMBER'] = @number.zero? ? '' : (@number + 1).to_s
58
+ ENV['PARALLEL_WORKER_NUMBER'] = @number.to_s
59
+ RSpec.world.wants_to_quit = false
60
+ end
61
+
62
+ def install_reporter
63
+ reporter = WorkerReporter.new(@configuration)
64
+ @configuration.instance_variable_set(:@reporter, reporter)
65
+ reporter
66
+ end
67
+
68
+ # Skips at_exit handlers inherited from the parent (RSpec autorun etc.).
69
+ def finish(status)
70
+ $stdout.flush
71
+ $stderr.flush
72
+ exit!(status)
73
+ end
74
+
75
+ def send_message(message)
76
+ Marshal.dump(message, @results)
77
+ @results.flush
78
+ end
79
+ end
80
+ end
81
+ end
@@ -0,0 +1,77 @@
1
+ # frozen_string_literal: true
2
+
3
+ module RSpec
4
+ module Parallelize
5
+ # Installed as the reporter inside a worker. Instead of notifying
6
+ # formatters it records each event as plain data; the parent replays the
7
+ # events into the real reporter (see Orchestrator#replay).
8
+ class WorkerReporter < RSpec::Core::Reporter
9
+ attr_reader :events
10
+
11
+ def initialize(configuration)
12
+ super
13
+
14
+ @events = []
15
+ end
16
+
17
+ def drain
18
+ drained = @events
19
+ @events = []
20
+ drained
21
+ end
22
+
23
+ def example_group_started(group)
24
+ @events << [:example_group_started, group.id]
25
+ end
26
+
27
+ def example_group_finished(group)
28
+ @events << [:example_group_finished, group.id]
29
+ end
30
+
31
+ def example_started(example)
32
+ @examples << example
33
+ @events << [:example_started, example.id]
34
+ end
35
+
36
+ # Called once the status is recorded, before passed/failed/pending.
37
+ def example_finished(example)
38
+ @events << [:example_finished, example.id, Serializer.dump_result(example)]
39
+ end
40
+
41
+ def example_passed(example)
42
+ @events << [:example_passed, example.id]
43
+ end
44
+
45
+ def example_failed(example)
46
+ @failed_examples << example
47
+ @events << [:example_failed, example.id]
48
+ end
49
+
50
+ def example_pending(example)
51
+ @pending_examples << example
52
+ @events << [:example_pending, example.id]
53
+ end
54
+
55
+ def message(message)
56
+ @events << [:message, message.to_s]
57
+ end
58
+
59
+ def deprecation(hash)
60
+ @events << [:deprecation, hash.transform_values { |v| v.is_a?(String) || v.nil? ? v : v.to_s }]
61
+ end
62
+
63
+ # Custom events whose payload can't cross the pipe are dropped.
64
+ def publish(event, options = {})
65
+ Marshal.dump(options)
66
+ @events << [:publish, event, options]
67
+ rescue TypeError
68
+ nil
69
+ end
70
+
71
+ def notify_non_example_exception(exception, context_description)
72
+ @configuration.world.non_example_failure = true
73
+ @events << [:non_example_exception, Serializer.dump_exception(exception), context_description]
74
+ end
75
+ end
76
+ end
77
+ end
@@ -0,0 +1,51 @@
1
+ # frozen_string_literal: true
2
+
3
+ require 'etc'
4
+ require 'rspec/core'
5
+ require 'rspec/parallelize/version'
6
+ require 'rspec/parallelize/configuration'
7
+ require 'rspec/parallelize/active_record'
8
+ require 'rspec/parallelize/progress_line'
9
+ require 'rspec/parallelize/serializer'
10
+ require 'rspec/parallelize/worker_reporter'
11
+ require 'rspec/parallelize/worker'
12
+ require 'rspec/parallelize/orchestrator'
13
+ require 'rspec/parallelize/runner_extension'
14
+
15
+ module RSpec
16
+ # Runs plain `rspec` in parallel: the suite boots once, workers are forked
17
+ # from it, and top-level example groups are handed out from a shared queue.
18
+ #
19
+ # # spec/spec_helper.rb
20
+ # RSpec::Parallelize.configure
21
+ module Parallelize
22
+ class << self
23
+ def configuration
24
+ @configuration ||= Configuration.new
25
+ end
26
+
27
+ # Enables parallel runs. `workers` is an Integer, :half_of_processors
28
+ # (the default) or :number_of_processors; runs with fewer than
29
+ # `threshold` examples stay serial.
30
+ def configure(workers: Configuration::DEFAULT_WORKERS, threshold: Configuration::DEFAULT_THRESHOLD)
31
+ configuration.enable(workers: workers, threshold: threshold)
32
+ end
33
+
34
+ # Runs in each worker right after it is forked, before any `before(:suite)`
35
+ # hook. Receives the 0-based worker number.
36
+ def after_fork(&block)
37
+ configuration.after_fork_hooks << block
38
+ end
39
+
40
+ # Runs in each worker after its last group and `after(:suite)` hooks.
41
+ def before_exit(&block)
42
+ configuration.before_exit_hooks << block
43
+ end
44
+ end
45
+ end
46
+ end
47
+
48
+ RSpec::Core::Runner.prepend(RSpec::Parallelize::RunnerExtension)
49
+
50
+ # Keep our frames out of failure backtraces, like RSpec does for its own.
51
+ RSpec.configuration.backtrace_exclusion_patterns << %r{/lib/rspec/parallelize(\.rb|/)}
metadata ADDED
@@ -0,0 +1,74 @@
1
+ --- !ruby/object:Gem::Specification
2
+ name: rspec-parallelize
3
+ version: !ruby/object:Gem::Version
4
+ version: 0.1.0
5
+ platform: ruby
6
+ authors:
7
+ - Alpha
8
+ bindir: bin
9
+ cert_chain: []
10
+ date: 1980-01-02 00:00:00.000000000 Z
11
+ dependencies:
12
+ - !ruby/object:Gem::Dependency
13
+ name: rspec-core
14
+ requirement: !ruby/object:Gem::Requirement
15
+ requirements:
16
+ - - ">="
17
+ - !ruby/object:Gem::Version
18
+ version: '3.12'
19
+ - - "<"
20
+ - !ruby/object:Gem::Version
21
+ version: '4'
22
+ type: :runtime
23
+ prerelease: false
24
+ version_requirements: !ruby/object:Gem::Requirement
25
+ requirements:
26
+ - - ">="
27
+ - !ruby/object:Gem::Version
28
+ version: '3.12'
29
+ - - "<"
30
+ - !ruby/object:Gem::Version
31
+ version: '4'
32
+ description: |
33
+ Boots your suite once, forks workers, and feeds them top-level example groups
34
+ from a shared queue. Results are replayed into RSpec's own reporter, so every
35
+ formatter, --only-failures and --seed keep working.
36
+ executables: []
37
+ extensions: []
38
+ extra_rdoc_files: []
39
+ files:
40
+ - CHANGELOG.md
41
+ - LICENSE.txt
42
+ - README.md
43
+ - lib/rspec/parallelize.rb
44
+ - lib/rspec/parallelize/active_record.rb
45
+ - lib/rspec/parallelize/configuration.rb
46
+ - lib/rspec/parallelize/orchestrator.rb
47
+ - lib/rspec/parallelize/progress_line.rb
48
+ - lib/rspec/parallelize/runner_extension.rb
49
+ - lib/rspec/parallelize/serializer.rb
50
+ - lib/rspec/parallelize/version.rb
51
+ - lib/rspec/parallelize/worker.rb
52
+ - lib/rspec/parallelize/worker_reporter.rb
53
+ licenses:
54
+ - MIT
55
+ metadata:
56
+ rubygems_mfa_required: 'true'
57
+ rdoc_options: []
58
+ require_paths:
59
+ - lib
60
+ required_ruby_version: !ruby/object:Gem::Requirement
61
+ requirements:
62
+ - - ">="
63
+ - !ruby/object:Gem::Version
64
+ version: '3.0'
65
+ required_rubygems_version: !ruby/object:Gem::Requirement
66
+ requirements:
67
+ - - ">="
68
+ - !ruby/object:Gem::Version
69
+ version: '0'
70
+ requirements: []
71
+ rubygems_version: 4.0.20
72
+ specification_version: 4
73
+ summary: Run plain `rspec` in parallel, the way Rails' minitest `parallelize` does.
74
+ test_files: []