parallel_tests 2.28.0 → 3.4.0

Sign up to get free protection for your applications and to get access to all the features.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: a6a3302bc606b106b21ad18230f4c44c0981890cf5e603ecd7a85f4a800766f6
4
- data.tar.gz: c1269f9f6a3faba3d8220a360d2b050d49ccef83c14c83cf6a243a4dca7d245c
3
+ metadata.gz: fd8878310c757d2a906bbf5ca62b68e21941cc79472a9833944f88087449c229
4
+ data.tar.gz: d55a5ac186f1c16d9568b8f6bb145e0016bafc1c2fb9748593e96997eaf2034c
5
5
  SHA512:
6
- metadata.gz: a1e9ea15748222c5acc25f8a2fee413bed29e7dff207ff968b1ffe2cbe54bb221acca26c5c83abe5f0188a8f41226392fdf356be75a2e47573c18cc0c477b406
7
- data.tar.gz: ea67c57a7037e57936ca818e79239bb42a8c18cdecd26cb6ac7496672073c3cca652e9f814227139a816676d721e5cbf8fb68fbb77da49138a5ce8426cbffce0
6
+ metadata.gz: f01821e38506525feaafb8a9ff5a480ab1f6f2b0bed849088d72682a699f82ae50bb2a78b94526f243a65c2474ff8f2d3a06c204b44431a7687d877cbbc91dd8
7
+ data.tar.gz: 2823702fc7e5d047a69c57d3ee4c0b72afbb51603d58e34a3656899d0026a4b749e94494b9edf234fdcc5fd81bae59dfba55f920d777f5f7d1f7c21fb5219552
data/Readme.md CHANGED
@@ -2,7 +2,7 @@
2
2
 
3
3
  [![Gem Version](https://badge.fury.io/rb/parallel_tests.svg)](https://rubygems.org/gems/parallel_tests)
4
4
  [![Build Status](https://travis-ci.org/grosser/parallel_tests.svg)](https://travis-ci.org/grosser/parallel_tests/builds)
5
- [![Build status](https://ci.appveyor.com/api/projects/status/708b1up4pqc34x3y?svg=true)](https://ci.appveyor.com/project/grosser/parallel-tests)
5
+ [![Build status](https://github.com/grosser/parallel_tests/workflows/windows/badge.svg)](https://github.com/grosser/parallel_tests/actions?query=workflow%3Awindows)
6
6
 
7
7
  Speedup Test::Unit + RSpec + Cucumber + Spinach by running parallel on multiple CPU cores.<br/>
8
8
  ParallelTests splits tests into even groups (by number of lines or runtime) and runs each group in a single process with its own database.
@@ -82,6 +82,8 @@ Running things once
82
82
  ===================
83
83
 
84
84
  ```Ruby
85
+ require "parallel_tests"
86
+
85
87
  # preparation:
86
88
  # affected by race-condition: first process may boot slower than the second
87
89
  # either sleep a bit or use a lock for example File.lock
@@ -97,7 +99,6 @@ at_exit do
97
99
  undo_something
98
100
  end
99
101
  end
100
-
101
102
  ```
102
103
 
103
104
  Even test group run-times
@@ -192,7 +193,6 @@ Setup for non-rails
192
193
 
193
194
  Options are:
194
195
  <!-- copy output from bundle exec ./bin/parallel_test -h -->
195
-
196
196
  -n [PROCESSES] How many processes to use, default: available CPUs
197
197
  -p, --pattern [PATTERN] run tests matching this regex pattern
198
198
  --exclude-pattern [PATTERN] exclude tests matching this regex pattern
@@ -205,7 +205,9 @@ Options are:
205
205
  default - runtime when runtime log is filled otherwise filesize
206
206
  -m, --multiply-processes [FLOAT] use given number as a multiplier of processes to run
207
207
  -s, --single [PATTERN] Run all matching files in the same process
208
- -i, --isolate Do not run any other tests in the group used by --single(-s)
208
+ -i, --isolate Do not run any other tests in the group used by --single(-s).
209
+ Automatically turned on if --isolate-n is set above 0.
210
+ --isolate-n Number of processes for isolated groups. Default to 1 when --isolate is on.
209
211
  --only-group INT[, INT]
210
212
  -e, --exec [COMMAND] execute this code parallel and with ENV['TEST_ENV_NUMBER']
211
213
  -o, --test-options '[OPTIONS]' execute test commands with those options
@@ -222,10 +224,14 @@ Options are:
222
224
  --ignore-tags [PATTERN] When counting steps ignore scenarios with tags that match this pattern
223
225
  --nice execute test commands with low priority.
224
226
  --runtime-log [PATH] Location of previously recorded test runtimes
225
- --allowed-missing Allowed percentage of missing runtimes (default = 50)
227
+ --allowed-missing [INT] Allowed percentage of missing runtimes (default = 50)
226
228
  --unknown-runtime [FLOAT] Use given number as unknown runtime (otherwise use average time)
227
- --verbose Print more output
228
- --quiet Do not print anything, apart from test output
229
+ --first-is-1 Use "1" as TEST_ENV_NUMBER to not reuse the default test environment
230
+ --fail-fast Stop all groups when one group fails (best used with --test-options '--fail-fast' if supported
231
+ --verbose Print debug output
232
+ --verbose-process-command Displays only the command that will be executed by each process
233
+ --verbose-rerun-command When there are failures, displays the command executed by each process that failed
234
+ --quiet Print only tests output
229
235
  -v, --version Show Version
230
236
  -h, --help Show this.
231
237
 
@@ -254,6 +260,7 @@ TIPS
254
260
  - Instantly see failures (instead of just a red F) with [rspec-instafail](https://github.com/grosser/rspec-instafail)
255
261
  - Use [rspec-retry](https://github.com/NoRedInk/rspec-retry) (not rspec-rerun) to rerun failed tests.
256
262
  - [JUnit formatter configuration](https://github.com/grosser/parallel_tests/wiki#with-rspec_junit_formatter----by-jgarber)
263
+ - Use [parallel_split_test](https://github.com/grosser/parallel_split_test) to run multiple scenarios in a single spec file, concurrently. (`parallel_tests` [works at the file-level and intends to stay that way](https://github.com/grosser/parallel_tests/issues/747#issuecomment-580216980))
257
264
 
258
265
  ### Cucumber
259
266
 
@@ -264,14 +271,13 @@ TIPS
264
271
  - Builds a HTML report from JSON with support for debug msgs & embedded Base64 images.
265
272
 
266
273
  ### General
267
- - [SQL schema format] use :ruby schema format to get faster parallel:prepare`
268
274
  - [ZSH] use quotes to use rake arguments `rake "parallel:prepare[3]"`
269
275
  - [Memcached] use different namespaces<br/>
270
276
  e.g. `config.cache_store = ..., namespace: "test_#{ENV['TEST_ENV_NUMBER']}"`
271
277
  - Debug errors that only happen with multiple files using `--verbose` and [cleanser](https://github.com/grosser/cleanser)
272
278
  - `export PARALLEL_TEST_PROCESSORS=13` to override default processor count
273
279
  - Shell alias: `alias prspec='parallel_rspec -m 2 --'`
274
- - [Spring] to use spring you have to [patch it](https://github.com/grosser/parallel_tests/wiki/Spring)
280
+ - [Spring] Add the [spring-commands-parallel-tests](https://github.com/DocSpring/spring-commands-parallel-tests) gem to your `Gemfile` to get `parallel_tests` working with Spring.
275
281
  - `--first-is-1` will make the first environment be `1`, so you can test while running your full suite.<br/>
276
282
  `export PARALLEL_TEST_FIRST_IS_1=true` will provide the same result
277
283
  - [email_spec and/or action_mailer_cache_delivery](https://github.com/grosser/parallel_tests/wiki)
@@ -283,11 +289,6 @@ TIPS
283
289
 
284
290
  Contribute your own gotchas to the [Wiki](https://github.com/grosser/parallel_tests/wiki) or even better open a PR :)
285
291
 
286
- TODO
287
- ====
288
- - fix tests vs cucumber >= 1.2 `unknown option --format`
289
- - add unit tests for cucumber runtime formatter
290
-
291
292
  Authors
292
293
  ====
293
294
  inspired by [pivotal labs](https://blog.pivotal.io/labs/labs/parallelize-your-rspec-suite)
@@ -375,6 +376,9 @@ inspired by [pivotal labs](https://blog.pivotal.io/labs/labs/parallelize-your-rs
375
376
  - [Justin Doody](https://github.com/justindoody)
376
377
  - [Sandeep Singh](https://github.com/sandeepnagra)
377
378
  - [Calaway](https://github.com/calaway)
379
+ - [alboyadjian](https://github.com/alboyadjian)
380
+ - [Nathan Broadbent](https://github.com/ndbroadbent)
381
+ - [Vikram B Kumar](https://github.com/v-kumar)
378
382
 
379
383
  [Michael Grosser](http://grosser.it)<br/>
380
384
  michael@grosser.it<br/>
@@ -2,6 +2,7 @@ require 'optparse'
2
2
  require 'tempfile'
3
3
  require 'parallel_tests'
4
4
  require 'shellwords'
5
+ require 'pathname'
5
6
 
6
7
  module ParallelTests
7
8
  class CLI
@@ -41,9 +42,10 @@ module ParallelTests
41
42
  Tempfile.open 'parallel_tests-lock' do |lock|
42
43
  ParallelTests.with_pid_file do
43
44
  simulate_output_for_ci options[:serialize_stdout] do
44
- Parallel.map(items, :in_threads => num_processes) do |item|
45
+ Parallel.map(items, in_threads: num_processes) do |item|
45
46
  result = yield(item)
46
47
  reprint_output(result, lock.path) if options[:serialize_stdout]
48
+ ParallelTests.stop_all_processes if options[:fail_fast] && result[:exit_status] != 0
47
49
  result
48
50
  end
49
51
  end
@@ -124,7 +126,7 @@ module ParallelTests
124
126
  failing_sets = test_results.reject { |r| r[:exit_status] == 0 }
125
127
  return if failing_sets.none?
126
128
 
127
- if options[:verbose]
129
+ if options[:verbose] || options[:verbose_rerun_command]
128
130
  puts "\n\nTests have failed for a parallel_test group. Use the following command to run the group again:\n\n"
129
131
  failing_sets.each do |failing_set|
130
132
  command = failing_set[:command]
@@ -190,6 +192,12 @@ module ParallelTests
190
192
  options[:isolate] = true
191
193
  end
192
194
 
195
+ opts.on("--isolate-n [PROCESSES]",
196
+ Integer,
197
+ "Use 'isolate' singles with number of processes, default: 1.") do |n|
198
+ options[:isolate_count] = n
199
+ end
200
+
193
201
  opts.on("--only-group INT[, INT]", Array) { |groups| options[:only_group] = groups.map(&:to_i) }
194
202
 
195
203
  opts.on("-e", "--exec [COMMAND]", "execute this code parallel and with ENV['TEST_ENV_NUMBER']") { |path| options[:execute] = path }
@@ -219,8 +227,11 @@ module ParallelTests
219
227
  opts.on("--allowed-missing [INT]", Integer, "Allowed percentage of missing runtimes (default = 50)") { |percent| options[:allowed_missing_percent] = percent }
220
228
  opts.on("--unknown-runtime [FLOAT]", Float, "Use given number as unknown runtime (otherwise use average time)") { |time| options[:unknown_runtime] = time }
221
229
  opts.on("--first-is-1", "Use \"1\" as TEST_ENV_NUMBER to not reuse the default test environment") { options[:first_is_1] = true }
222
- opts.on("--verbose", "Print more output (mutually exclusive with quiet)") { options[:verbose] = true }
223
- opts.on("--quiet", "Print tests output only (mutually exclusive with verbose)") { options[:quiet] = true }
230
+ opts.on("--fail-fast", "Stop all groups when one group fails (best used with --test-options '--fail-fast' if supported") { options[:fail_fast] = true }
231
+ opts.on("--verbose", "Print debug output") { options[:verbose] = true }
232
+ opts.on("--verbose-process-command", "Displays only the command that will be executed by each process") { options[:verbose_process_command] = true }
233
+ opts.on("--verbose-rerun-command", "When there are failures, displays the command executed by each process that failed") { options[:verbose_rerun_command] = true }
234
+ opts.on("--quiet", "Print only tests output") { options[:quiet] = true }
224
235
  opts.on("-v", "--version", "Show Version") { puts ParallelTests::VERSION; exit }
225
236
  opts.on("-h", "--help", "Show this.") { puts opts; exit }
226
237
  end.parse!(argv)
@@ -237,7 +248,7 @@ module ParallelTests
237
248
  files, remaining = extract_file_paths(argv)
238
249
  unless options[:execute]
239
250
  abort "Pass files or folders to run" unless files.any?
240
- options[:files] = files
251
+ options[:files] = files.map { |file_path| Pathname.new(file_path).cleanpath.to_s }
241
252
  end
242
253
 
243
254
  append_test_options(options, remaining)
@@ -313,9 +324,8 @@ module ParallelTests
313
324
  end
314
325
 
315
326
  def final_fail_message
316
- fail_message = "#{@runner.name}s Failed"
327
+ fail_message = "Tests Failed"
317
328
  fail_message = "\e[31m#{fail_message}\e[0m" if use_colors?
318
-
319
329
  fail_message
320
330
  end
321
331
 
@@ -0,0 +1,31 @@
1
+ begin
2
+ gem "cuke_modeler", "~> 3.0"
3
+ require 'cuke_modeler'
4
+ rescue LoadError
5
+ raise 'Grouping by number of cucumber steps requires the `cuke_modeler` modeler gem with requirement `~> 3.0`. Add `gem "cuke_modeler", "~> 3.0"` to your `Gemfile`, run `bundle install` and try again.'
6
+ end
7
+
8
+ module ParallelTests
9
+ module Cucumber
10
+ class FeaturesWithSteps
11
+ class << self
12
+ def all(tests, options)
13
+ ignore_tag_pattern = options[:ignore_tag_pattern].nil? ? nil : Regexp.compile(options[:ignore_tag_pattern])
14
+ # format of hash will be FILENAME => NUM_STEPS
15
+ steps_per_file = tests.each_with_object({}) do |file,steps|
16
+ feature = ::CukeModeler::FeatureFile.new(file).feature
17
+
18
+ # skip feature if it matches tag regex
19
+ next if feature.tags.grep(ignore_tag_pattern).any?
20
+
21
+ # count the number of steps in the file
22
+ # will only include a feature if the regex does not match
23
+ all_steps = feature.scenarios.map{|a| a.steps.count if a.tags.grep(ignore_tag_pattern).empty? }.compact
24
+ steps[file] = all_steps.inject(0,:+)
25
+ end
26
+ steps_per_file.sort_by { |_, value| -value }
27
+ end
28
+ end
29
+ end
30
+ end
31
+ end
@@ -1,37 +1,33 @@
1
- require 'cucumber/tag_expressions/parser'
2
- require 'cucumber/core/gherkin/tag_expression'
3
-
4
1
  module ParallelTests
5
2
  module Cucumber
6
3
  module Formatters
7
4
  class ScenarioLineLogger
8
5
  attr_reader :scenarios
9
6
 
10
- def initialize(tag_expression = ::Cucumber::Core::Gherkin::TagExpression.new([]))
7
+ def initialize(tag_expression = nil)
11
8
  @scenarios = []
12
9
  @tag_expression = tag_expression
13
10
  end
14
11
 
15
12
  def visit_feature_element(uri, feature_element, feature_tags, line_numbers: [])
16
- scenario_tags = feature_element[:tags].map { |tag| tag[:name] }
13
+ scenario_tags = feature_element.tags.map { |tag| tag.name }
17
14
  scenario_tags = feature_tags + scenario_tags
18
- if feature_element[:examples].nil? # :Scenario
19
- test_line = feature_element[:location][:line]
15
+ if feature_element.is_a?(CukeModeler::Scenario) # :Scenario
16
+ test_line = feature_element.source_line
20
17
 
21
18
  # We don't accept the feature_element if the current tags are not valid
22
- return unless @tag_expression.evaluate(scenario_tags)
19
+ return unless matches_tags?(scenario_tags)
23
20
  # or if it is not at the correct location
24
21
  return if line_numbers.any? && !line_numbers.include?(test_line)
25
22
 
26
- @scenarios << [uri, feature_element[:location][:line]].join(":")
23
+ @scenarios << [uri, feature_element.source_line].join(":")
27
24
  else # :ScenarioOutline
28
- feature_element[:examples].each do |example|
29
- example_tags = example[:tags].map { |tag| tag[:name] }
25
+ feature_element.examples.each do |example|
26
+ example_tags = example.tags.map(&:name)
30
27
  example_tags = scenario_tags + example_tags
31
- next unless @tag_expression.evaluate(example_tags)
32
- rows = example[:tableBody].select { |body| body[:type] == :TableRow }
33
- rows.each do |row|
34
- test_line = row[:location][:line]
28
+ next unless matches_tags?(example_tags)
29
+ example.rows[1..-1].each do |row|
30
+ test_line = row.source_line
35
31
  next if line_numbers.any? && !line_numbers.include?(test_line)
36
32
 
37
33
  @scenarios << [uri, test_line].join(':')
@@ -42,6 +38,12 @@ module ParallelTests
42
38
 
43
39
  def method_missing(*args)
44
40
  end
41
+
42
+ private
43
+
44
+ def matches_tags?(tags)
45
+ @tag_expression.nil? || @tag_expression.evaluate(tags)
46
+ end
45
47
  end
46
48
  end
47
49
  end
@@ -1,12 +1,17 @@
1
1
  require 'cucumber/tag_expressions/parser'
2
- require 'cucumber/core/gherkin/tag_expression'
3
2
  require 'cucumber/runtime'
4
3
  require 'cucumber'
5
4
  require 'parallel_tests/cucumber/scenario_line_logger'
6
5
  require 'parallel_tests/gherkin/listener'
7
- require 'gherkin/errors'
8
6
  require 'shellwords'
9
7
 
8
+ begin
9
+ gem "cuke_modeler", "~> 3.0"
10
+ require 'cuke_modeler'
11
+ rescue LoadError
12
+ raise 'Grouping by individual cucumber scenarios requires the `cuke_modeler` modeler gem with requirement `~> 3.0`. Add `gem "cuke_modeler", "~> 3.0"` to your `Gemfile`, run `bundle install` and try again.'
13
+ end
14
+
10
15
  module ParallelTests
11
16
  module Cucumber
12
17
  class Scenarios
@@ -40,32 +45,17 @@ module ParallelTests
40
45
  path, *test_lines = path.split(/:(?=\d+)/)
41
46
  test_lines.map!(&:to_i)
42
47
 
43
- # We encode the file and get the content of it
44
- source = ::Cucumber::Runtime::NormalisedEncodingFile.read(path)
45
48
  # We create a Gherkin document, this will be used to decode the details of each scenario
46
- document = ::Cucumber::Core::Gherkin::Document.new(path, source)
47
-
48
- # We create a parser for the gherkin document
49
- parser = ::Gherkin::Parser.new()
50
- scanner = ::Gherkin::TokenScanner.new(document.body)
51
-
52
- begin
53
- # We make an attempt to parse the gherkin document, this could be failed if the document is not well formatted
54
- result = parser.parse(scanner)
55
- feature_tags = result[:feature][:tags].map { |tag| tag[:name] }
56
-
57
- # We loop on each children of the feature
58
- result[:feature][:children].each do |feature_element|
59
- # If the type of the child is not a scenario or scenario outline, we continue, we are only interested by the name of the scenario here
60
- next unless /Scenario/.match(feature_element[:type])
49
+ document = ::CukeModeler::FeatureFile.new(path)
50
+ feature = document.feature
61
51
 
62
- # It's a scenario, we add it to the scenario_line_logger
63
- scenario_line_logger.visit_feature_element(document.uri, feature_element, feature_tags, line_numbers: test_lines)
64
- end
52
+ # We make an attempt to parse the gherkin document, this could be failed if the document is not well formatted
53
+ feature_tags = feature.tags.map(&:name)
65
54
 
66
- rescue StandardError => e
67
- # Exception if the document is no well formated or error in the tags
68
- raise ::Cucumber::Core::Gherkin::ParseError.new("#{document.uri}: #{e.message}")
55
+ # We loop on each children of the feature
56
+ feature.tests.each do |test|
57
+ # It's a scenario, we add it to the scenario_line_logger
58
+ scenario_line_logger.visit_feature_element(document.path, test, feature_tags, line_numbers: test_lines)
69
59
  end
70
60
  end
71
61
 
@@ -1,5 +1,3 @@
1
- require 'gherkin/parser'
2
-
3
1
  module ParallelTests
4
2
  module Gherkin
5
3
  class Listener
@@ -10,6 +8,7 @@ module ParallelTests
10
8
  def initialize
11
9
  @steps, @uris = [], []
12
10
  @collect = {}
11
+ @feature, @ignore_tag_pattern = nil
13
12
  reset_counters!
14
13
  end
15
14
 
@@ -14,7 +14,7 @@ module ParallelTests
14
14
  end
15
15
 
16
16
  config.on_event :test_case_finished do |event|
17
- @example_times[event.test_case.feature.file] += ParallelTests.now.to_f - @start_at
17
+ @example_times[event.test_case.location.file] += ParallelTests.now.to_f - @start_at
18
18
  end
19
19
 
20
20
  config.on_event :test_run_finished do |_|
@@ -2,7 +2,7 @@ module ParallelTests
2
2
  class Grouper
3
3
  class << self
4
4
  def by_steps(tests, num_groups, options)
5
- features_with_steps = build_features_with_steps(tests, options)
5
+ features_with_steps = group_by_features_with_steps(tests, options)
6
6
  in_even_groups_by_size(features_with_steps, num_groups)
7
7
  end
8
8
 
@@ -15,19 +15,46 @@ module ParallelTests
15
15
  groups = Array.new(num_groups) { {:items => [], :size => 0} }
16
16
 
17
17
  # add all files that should run in a single process to one group
18
- (options[:single_process] || []).each do |pattern|
19
- matched, items = items.partition { |item, _size| item =~ pattern }
20
- matched.each { |item, size| add_to_group(groups.first, item, size) }
18
+ single_process_patterns = options[:single_process] || []
19
+
20
+ single_items, items = items.partition do |item, _size|
21
+ single_process_patterns.any? { |pattern| item =~ pattern }
21
22
  end
22
23
 
23
- groups_to_fill = (options[:isolate] ? groups[1..-1] : groups)
24
- group_features_by_size(items_to_group(items), groups_to_fill)
24
+ isolate_count = isolate_count(options)
25
+
26
+ if isolate_count >= num_groups
27
+ raise 'Number of isolated processes must be less than total the number of processes'
28
+ end
29
+
30
+ if isolate_count >= 1
31
+ # add all files that should run in a multiple isolated processes to their own groups
32
+ group_features_by_size(items_to_group(single_items), groups[0..(isolate_count - 1)])
33
+ # group the non-isolated by size
34
+ group_features_by_size(items_to_group(items), groups[isolate_count..-1])
35
+ else
36
+ # add all files that should run in a single non-isolated process to first group
37
+ single_items.each { |item, size| add_to_group(groups.first, item, size) }
38
+
39
+ # group all by size
40
+ group_features_by_size(items_to_group(items), groups)
41
+ end
25
42
 
26
43
  groups.map! { |g| g[:items].sort }
27
44
  end
28
45
 
29
46
  private
30
47
 
48
+ def isolate_count(options)
49
+ if options[:isolate_count] && options[:isolate_count] > 1
50
+ options[:isolate_count]
51
+ elsif options[:isolate]
52
+ 1
53
+ else
54
+ 0
55
+ end
56
+ end
57
+
31
58
  def largest_first(files)
32
59
  files.sort_by{|_item, size| size }.reverse
33
60
  end
@@ -41,23 +68,9 @@ module ParallelTests
41
68
  group[:size] += size
42
69
  end
43
70
 
44
- def build_features_with_steps(tests, options)
45
- require 'gherkin/parser'
46
- ignore_tag_pattern = options[:ignore_tag_pattern].nil? ? nil : Regexp.compile(options[:ignore_tag_pattern])
47
- parser = ::Gherkin::Parser.new
48
- # format of hash will be FILENAME => NUM_STEPS
49
- steps_per_file = tests.each_with_object({}) do |file,steps|
50
- feature = parser.parse(File.read(file)).fetch(:feature)
51
-
52
- # skip feature if it matches tag regex
53
- next if feature[:tags].grep(ignore_tag_pattern).any?
54
-
55
- # count the number of steps in the file
56
- # will only include a feature if the regex does not match
57
- all_steps = feature[:children].map{|a| a[:steps].count if a[:tags].grep(ignore_tag_pattern).empty? }.compact
58
- steps[file] = all_steps.inject(0,:+)
59
- end
60
- steps_per_file.sort_by { |_, value| -value }
71
+ def group_by_features_with_steps(tests, options)
72
+ require 'parallel_tests/cucumber/features_with_steps'
73
+ ParallelTests::Cucumber::FeaturesWithSteps.all(tests, options)
61
74
  end
62
75
 
63
76
  def group_by_scenarios(tests, options={})
@@ -2,7 +2,7 @@ require 'json'
2
2
 
3
3
  module ParallelTests
4
4
  class Pids
5
- attr_reader :pids, :file_path, :mutex
5
+ attr_reader :file_path, :mutex
6
6
 
7
7
  def initialize(file_path)
8
8
  @file_path = file_path
@@ -4,8 +4,6 @@ module ParallelTests
4
4
  module RSpec
5
5
  class Runner < ParallelTests::Test::Runner
6
6
  DEV_NULL = (WINDOWS ? "NUL" : "/dev/null")
7
- NAME = 'RSpec'
8
-
9
7
  class << self
10
8
  def run_tests(test_files, process_number, num_processes, options)
11
9
  exe = executable # expensive, so we cache
@@ -14,17 +12,14 @@ module ParallelTests
14
12
  end
15
13
 
16
14
  def determine_executable
17
- cmd = case
15
+ case
18
16
  when File.exist?("bin/rspec")
19
17
  ParallelTests.with_ruby_binary("bin/rspec")
20
18
  when ParallelTests.bundler_enabled?
21
- cmd = (run("bundle show rspec-core") =~ %r{Could not find gem.*} ? "spec" : "rspec")
22
- "bundle exec #{cmd}"
19
+ "bundle exec rspec"
23
20
  else
24
- %w[spec rspec].detect{|cmd| system "#{cmd} --version > #{DEV_NULL} 2>&1" }
21
+ "rspec"
25
22
  end
26
-
27
- cmd or raise("Can't find executables rspec or spec")
28
23
  end
29
24
 
30
25
  def runtime_log
@@ -53,6 +48,22 @@ module ParallelTests
53
48
  "#{clean} --seed #{seed}"
54
49
  end
55
50
 
51
+ # Summarize results from threads and colorize results based on failure and pending counts.
52
+ #
53
+ def summarize_results(results)
54
+ text = super
55
+ return text unless $stdout.tty?
56
+ sums = sum_up_results(results)
57
+ color =
58
+ if sums['failure'].positive?
59
+ 31 # red
60
+ elsif sums['pending'].positive?
61
+ 33 # yellow
62
+ else
63
+ 32 # green
64
+ end
65
+ "\e[#{color}m#{text}\e[0m"
66
+ end
56
67
 
57
68
  private
58
69
 
@@ -1,10 +1,19 @@
1
1
  require 'rake'
2
+ require 'shellwords'
2
3
 
3
4
  module ParallelTests
4
5
  module Tasks
5
6
  class << self
6
7
  def rails_env
7
- ENV['RAILS_ENV'] || 'test'
8
+ 'test'
9
+ end
10
+
11
+ def rake_bin
12
+ # Prevent 'Exec format error' Errno::ENOEXEC on Windows
13
+ return "rake" if RUBY_PLATFORM =~ /mswin|mingw|cygwin/
14
+ binstub_path = File.join('bin', 'rake')
15
+ return binstub_path if File.exist?(binstub_path)
16
+ "rake"
8
17
  end
9
18
 
10
19
  def load_lib
@@ -88,67 +97,84 @@ end
88
97
  namespace :parallel do
89
98
  desc "Setup test databases via db:setup --> parallel:setup[num_cpus]"
90
99
  task :setup, :count do |_,args|
91
- command = "rake db:setup RAILS_ENV=#{ParallelTests::Tasks.rails_env}"
100
+ command = "#{ParallelTests::Tasks.rake_bin} db:setup RAILS_ENV=#{ParallelTests::Tasks.rails_env}"
92
101
  ParallelTests::Tasks.run_in_parallel(ParallelTests::Tasks.suppress_schema_load_output(command), args)
93
102
  end
94
103
 
95
104
  desc "Create test databases via db:create --> parallel:create[num_cpus]"
96
105
  task :create, :count do |_,args|
97
- ParallelTests::Tasks.run_in_parallel("rake db:create RAILS_ENV=#{ParallelTests::Tasks.rails_env}", args)
106
+ ParallelTests::Tasks.run_in_parallel(
107
+ "#{ParallelTests::Tasks.rake_bin} db:create RAILS_ENV=#{ParallelTests::Tasks.rails_env}", args)
98
108
  end
99
109
 
100
110
  desc "Drop test databases via db:drop --> parallel:drop[num_cpus]"
101
111
  task :drop, :count do |_,args|
102
- ParallelTests::Tasks.run_in_parallel("rake db:drop RAILS_ENV=#{ParallelTests::Tasks.rails_env} DISABLE_DATABASE_ENVIRONMENT_CHECK=1", args)
112
+ ParallelTests::Tasks.run_in_parallel(
113
+ "#{ParallelTests::Tasks.rake_bin} db:drop RAILS_ENV=#{ParallelTests::Tasks.rails_env} " \
114
+ "DISABLE_DATABASE_ENVIRONMENT_CHECK=1", args)
103
115
  end
104
116
 
105
117
  desc "Update test databases by dumping and loading --> parallel:prepare[num_cpus]"
106
118
  task(:prepare, [:count]) do |_,args|
107
119
  ParallelTests::Tasks.check_for_pending_migrations
108
- if defined?(ActiveRecord) && ActiveRecord::Base.schema_format == :ruby
109
- # dump then load in parallel
110
- Rake::Task['db:schema:dump'].invoke
111
- Rake::Task['parallel:load_schema'].invoke(args[:count])
120
+ if defined?(ActiveRecord::Base) && [:ruby, :sql].include?(ActiveRecord::Base.schema_format)
121
+ # fast: dump once, load in parallel
122
+ type = (ActiveRecord::Base.schema_format == :ruby ? "schema" : "structure")
123
+ Rake::Task["db:#{type}:dump"].invoke
124
+
125
+ # remove database connection to prevent "database is being accessed by other users"
126
+ ActiveRecord::Base.remove_connection if ActiveRecord::Base.configurations.any?
127
+
128
+ Rake::Task["parallel:load_#{type}"].invoke(args[:count])
112
129
  else
113
- # there is no separate dump / load for schema_format :sql -> do it safe and slow
130
+ # slow: dump and load in in serial
114
131
  args = args.to_hash.merge(:non_parallel => true) # normal merge returns nil
115
- taskname = Rake::Task.task_defined?('db:test:prepare') ? 'db:test:prepare' : 'app:db:test:prepare'
116
- ParallelTests::Tasks.run_in_parallel("rake #{taskname}", args)
132
+ task_name = Rake::Task.task_defined?('db:test:prepare') ? 'db:test:prepare' : 'app:db:test:prepare'
133
+ ParallelTests::Tasks.run_in_parallel("#{ParallelTests::Tasks.rake_bin} #{task_name}", args)
134
+ next
117
135
  end
118
136
  end
119
137
 
120
138
  # when dumping/resetting takes too long
121
139
  desc "Update test databases via db:migrate --> parallel:migrate[num_cpus]"
122
140
  task :migrate, :count do |_,args|
123
- ParallelTests::Tasks.run_in_parallel("rake db:migrate RAILS_ENV=#{ParallelTests::Tasks.rails_env}", args)
141
+ ParallelTests::Tasks.run_in_parallel(
142
+ "#{ParallelTests::Tasks.rake_bin} db:migrate RAILS_ENV=#{ParallelTests::Tasks.rails_env}", args)
124
143
  end
125
144
 
126
145
  desc "Rollback test databases via db:rollback --> parallel:rollback[num_cpus]"
127
146
  task :rollback, :count do |_,args|
128
- ParallelTests::Tasks.run_in_parallel("rake db:rollback RAILS_ENV=#{ParallelTests::Tasks.rails_env}", args)
147
+ ParallelTests::Tasks.run_in_parallel(
148
+ "#{ParallelTests::Tasks.rake_bin} db:rollback RAILS_ENV=#{ParallelTests::Tasks.rails_env}", args)
129
149
  end
130
150
 
131
151
  # just load the schema (good for integration server <-> no development db)
132
152
  desc "Load dumped schema for test databases via db:schema:load --> parallel:load_schema[num_cpus]"
133
153
  task :load_schema, :count do |_,args|
134
- command = "rake #{ParallelTests::Tasks.purge_before_load} db:schema:load RAILS_ENV=#{ParallelTests::Tasks.rails_env} DISABLE_DATABASE_ENVIRONMENT_CHECK=1"
154
+ command = "#{ParallelTests::Tasks.rake_bin} #{ParallelTests::Tasks.purge_before_load} " \
155
+ "db:schema:load RAILS_ENV=#{ParallelTests::Tasks.rails_env} DISABLE_DATABASE_ENVIRONMENT_CHECK=1"
135
156
  ParallelTests::Tasks.run_in_parallel(ParallelTests::Tasks.suppress_schema_load_output(command), args)
136
157
  end
137
158
 
138
159
  # load the structure from the structure.sql file
139
160
  desc "Load structure for test databases via db:structure:load --> parallel:load_structure[num_cpus]"
140
161
  task :load_structure, :count do |_,args|
141
- ParallelTests::Tasks.run_in_parallel("rake #{ParallelTests::Tasks.purge_before_load} db:structure:load RAILS_ENV=#{ParallelTests::Tasks.rails_env} DISABLE_DATABASE_ENVIRONMENT_CHECK=1", args)
162
+ ParallelTests::Tasks.run_in_parallel(
163
+ "#{ParallelTests::Tasks.rake_bin} #{ParallelTests::Tasks.purge_before_load} " \
164
+ "db:structure:load RAILS_ENV=#{ParallelTests::Tasks.rails_env} DISABLE_DATABASE_ENVIRONMENT_CHECK=1", args)
142
165
  end
143
166
 
144
167
  desc "Load the seed data from db/seeds.rb via db:seed --> parallel:seed[num_cpus]"
145
168
  task :seed, :count do |_,args|
146
- ParallelTests::Tasks.run_in_parallel("rake db:seed RAILS_ENV=#{ParallelTests::Tasks.rails_env}", args)
169
+ ParallelTests::Tasks.run_in_parallel(
170
+ "#{ParallelTests::Tasks.rake_bin} db:seed RAILS_ENV=#{ParallelTests::Tasks.rails_env}", args)
147
171
  end
148
172
 
149
173
  desc "Launch given rake command in parallel"
150
174
  task :rake, :command, :count do |_, args|
151
- ParallelTests::Tasks.run_in_parallel("RAILS_ENV=#{ParallelTests::Tasks.rails_env} rake #{args.command}", args)
175
+ ParallelTests::Tasks.run_in_parallel(
176
+ "RAILS_ENV=#{ParallelTests::Tasks.rails_env} #{ParallelTests::Tasks.rake_bin} " \
177
+ "#{args.command}", args)
152
178
  end
153
179
 
154
180
  ['test', 'spec', 'features', 'features-spinach'].each do |type|
@@ -171,7 +197,8 @@ namespace :parallel do
171
197
  # Using the relative path to find the binary allow to run a specific version of it
172
198
  executable = File.join(File.dirname(__FILE__), '..', '..', 'bin', 'parallel_test')
173
199
 
174
- command = "#{ParallelTests.with_ruby_binary(Shellwords.escape(executable))} #{type} --type #{test_framework} " \
200
+ command = "#{ParallelTests.with_ruby_binary(Shellwords.escape(executable))} #{type} " \
201
+ "--type #{test_framework} " \
175
202
  "-n #{count} " \
176
203
  "--pattern '#{pattern}' " \
177
204
  "--test-options '#{options}' " \
@@ -3,15 +3,9 @@ require 'parallel_tests'
3
3
  module ParallelTests
4
4
  module Test
5
5
  class Runner
6
- NAME = 'Test'
7
-
8
6
  class << self
9
7
  # --- usually overwritten by other runners
10
8
 
11
- def name
12
- NAME
13
- end
14
-
15
9
  def runtime_log
16
10
  'tmp/parallel_runtime_test.log'
17
11
  end
@@ -78,7 +72,7 @@ module ParallelTests
78
72
  cmd = "nice #{cmd}" if options[:nice]
79
73
  cmd = "#{cmd} 2>&1" if options[:combine_stderr]
80
74
 
81
- puts cmd if options[:verbose] && !options[:serialize_stdout]
75
+ puts cmd if report_process_command?(options) && !options[:serialize_stdout]
82
76
 
83
77
  execute_command_and_capture_output(env, cmd, options)
84
78
  end
@@ -94,7 +88,9 @@ module ParallelTests
94
88
  exitstatus = $?.exitstatus
95
89
  seed = output[/seed (\d+)/,1]
96
90
 
97
- output = [cmd, output].join("\n") if options[:verbose] && options[:serialize_stdout]
91
+ if report_process_command?(options) && options[:serialize_stdout]
92
+ output = [cmd, output].join("\n")
93
+ end
98
94
 
99
95
  {:stdout => output, :exit_status => exitstatus, :command => cmd, :seed => seed}
100
96
  end
@@ -177,7 +173,7 @@ module ParallelTests
177
173
  allowed_missing -= 1 unless time = runtimes[test]
178
174
  if allowed_missing < 0
179
175
  log = options[:runtime_log] || runtime_log
180
- raise "Runtime log file '#{log}' does not contain sufficient data to sort #{tests.size} test files, please update it."
176
+ raise "Runtime log file '#{log}' does not contain sufficient data to sort #{tests.size} test files, please update or remove it."
181
177
  end
182
178
  [test, time]
183
179
  end
@@ -186,11 +182,7 @@ module ParallelTests
186
182
  puts "Runtime found for #{tests.count(&:last)} of #{tests.size} tests"
187
183
  end
188
184
 
189
- # fill gaps with unknown-runtime if given, average otherwise
190
- known, unknown = tests.partition(&:last)
191
- average = (known.any? ? known.map!(&:last).inject(:+) / known.size : 1)
192
- unknown_runtime = options[:unknown_runtime] || average
193
- unknown.each { |set| set[1] = unknown_runtime }
185
+ set_unknown_runtime tests, options
194
186
  end
195
187
 
196
188
  def runtimes(tests, options)
@@ -235,6 +227,22 @@ module ParallelTests
235
227
  end
236
228
  Dir[File.join(folder, pattern)].uniq
237
229
  end
230
+
231
+ private
232
+
233
+ # fill gaps with unknown-runtime if given, average otherwise
234
+ # NOTE: an optimization could be doing runtime by average runtime per file size, but would need file checks
235
+ def set_unknown_runtime(tests, options)
236
+ known, unknown = tests.partition(&:last)
237
+ return if unknown.empty?
238
+ unknown_runtime = options[:unknown_runtime] ||
239
+ (known.empty? ? 1 : known.map!(&:last).inject(:+) / known.size) # average
240
+ unknown.each { |set| set[1] = unknown_runtime }
241
+ end
242
+
243
+ def report_process_command?(options)
244
+ options[:verbose] || options[:verbose_process_command]
245
+ end
238
246
  end
239
247
  end
240
248
  end
@@ -1,3 +1,3 @@
1
1
  module ParallelTests
2
- VERSION = Version = '2.28.0'
2
+ VERSION = Version = '3.4.0'
3
3
  end
metadata CHANGED
@@ -1,14 +1,14 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: parallel_tests
3
3
  version: !ruby/object:Gem::Version
4
- version: 2.28.0
4
+ version: 3.4.0
5
5
  platform: ruby
6
6
  authors:
7
7
  - Michael Grosser
8
8
  autorequire:
9
9
  bindir: bin
10
10
  cert_chain: []
11
- date: 2019-02-07 00:00:00.000000000 Z
11
+ date: 2020-11-23 00:00:00.000000000 Z
12
12
  dependencies:
13
13
  - !ruby/object:Gem::Dependency
14
14
  name: parallel
@@ -42,6 +42,7 @@ files:
42
42
  - lib/parallel_tests.rb
43
43
  - lib/parallel_tests/cli.rb
44
44
  - lib/parallel_tests/cucumber/failures_logger.rb
45
+ - lib/parallel_tests/cucumber/features_with_steps.rb
45
46
  - lib/parallel_tests/cucumber/runner.rb
46
47
  - lib/parallel_tests/cucumber/scenario_line_logger.rb
47
48
  - lib/parallel_tests/cucumber/scenarios.rb
@@ -62,10 +63,14 @@ files:
62
63
  - lib/parallel_tests/test/runner.rb
63
64
  - lib/parallel_tests/test/runtime_logger.rb
64
65
  - lib/parallel_tests/version.rb
65
- homepage: http://github.com/grosser/parallel_tests
66
+ homepage: https://github.com/grosser/parallel_tests
66
67
  licenses:
67
68
  - MIT
68
- metadata: {}
69
+ metadata:
70
+ bug_tracker_uri: https://github.com/grosser/parallel_tests/issues
71
+ documentation_uri: https://github.com/grosser/parallel_tests/blob/v3.4.0/Readme.md
72
+ source_code_uri: https://github.com/grosser/parallel_tests/tree/v3.4.0
73
+ wiki_uri: https://github.com/grosser/parallel_tests/wiki
69
74
  post_install_message:
70
75
  rdoc_options: []
71
76
  require_paths:
@@ -74,15 +79,14 @@ required_ruby_version: !ruby/object:Gem::Requirement
74
79
  requirements:
75
80
  - - ">="
76
81
  - !ruby/object:Gem::Version
77
- version: 2.2.0
82
+ version: 2.4.0
78
83
  required_rubygems_version: !ruby/object:Gem::Requirement
79
84
  requirements:
80
85
  - - ">="
81
86
  - !ruby/object:Gem::Version
82
87
  version: '0'
83
88
  requirements: []
84
- rubyforge_project:
85
- rubygems_version: 2.7.6
89
+ rubygems_version: 3.1.3
86
90
  signing_key:
87
91
  specification_version: 4
88
92
  summary: Run Test::Unit / RSpec / Cucumber / Spinach in parallel