parallel_tests 1.3.7 → 3.7.3

checksums.yaml

@@ -1,7 +1,7 @@
 ---
-SHA1:
-  metadata.gz: 1646c6d92a16b6c2f141d2cfd20af1dcdf491141
-  data.tar.gz: 2265ae4f1843166e30f11b7b4855b6dea98a58bf
+SHA256:
+  metadata.gz: 042cf590688332180e07fcf6472cc9b88b740276df0511677cf77f7fe649fa2e
+  data.tar.gz: cb4fd166b030e16574bf5d4790e5d1e0a3d57001d5474ab81c3d54613af976e2
 SHA512:
-  metadata.gz: d303d05c5dd6a5ad7985b18aca984c07cb501f4c07b5b0ca75fd944cbf8cd20ca89535a90d4ef1b0e6259fadcb206ea0325a8dcec5b830cf71569f6ae218d419
-  data.tar.gz: 516172942dd242cc622c4ac013fe7a74091c05c588b1d20b81282aadfe65153008dcf59e367d465d711ea049efb84978194c7f36a4e51ca473e9f104980e797f
+  metadata.gz: 98e679733a74273ac77e71db47bc3af3f0f5a928dab351be9419644a0e7fe2bc004a81c929928ef0bd98ea2b8e5b63d1ab9fadcf75e22d098a65c5164343fe0b
+  data.tar.gz: 2ce2dd070733cd0bb61d0cb610f886a5590d5e267cc43613ab7d08fc251d989ebd4a58ad2f0a9c88478499fe55ff4cb7386fabc2eba8ee7fb3440d4899594654

data/Readme.md

@@ -1,3 +1,9 @@
+# parallel_tests
+
+[![Gem Version](https://badge.fury.io/rb/parallel_tests.svg)](https://rubygems.org/gems/parallel_tests)
+[![Build Status](https://travis-ci.org/grosser/parallel_tests.svg)](https://travis-ci.org/grosser/parallel_tests/builds)
+[![Build status](https://github.com/grosser/parallel_tests/workflows/windows/badge.svg)](https://github.com/grosser/parallel_tests/actions?query=workflow%3Awindows)
+
 Speedup Test::Unit + RSpec + Cucumber + Spinach by running parallel on multiple CPU cores.<br/>
 ParallelTests splits tests into even groups (by number of lines or runtime) and runs each group in a single process with its own database.
 
@@ -6,10 +12,10 @@ Setup for Rails
 [RailsCasts episode #413 Fast Tests](http://railscasts.com/episodes/413-fast-tests)
 
 ### Install
+`Gemfile`:
 
 ```ruby
-# Gemfile
-gem "parallel_tests", :group => :development
+gem 'parallel_tests', group: [:development, :test]
 ```
 
 ### Add to `config/database.yml`
@@ -17,7 +23,7 @@ gem "parallel_tests", :group => :development
 ParallelTests uses 1 database per test-process.
 <table>
   <tr><td>Process number</td><td>1</td><td>2</td><td>3</td></tr>
-  <tr><td>`ENV['TEST_ENV_NUMBER']`</td><td>''</td><td>'2'</td><td>'3'</td></tr>
+  <tr><td>ENV['TEST_ENV_NUMBER']</td><td>''</td><td>'2'</td><td>'3'</td></tr>
 </table>
 
 ```yaml
@@ -31,6 +37,15 @@ test:
 ### Copy development schema (repeat after migrations)
     rake parallel:prepare
 
+### Run migrations in additional database(s) (repeat after migrations)
+    rake parallel:migrate
+
+### Setup environment from scratch (create db and loads schema, useful for CI)
+    rake parallel:setup
+    
+### Drop all test databases
+    rake parallel:drop
+
 ### Run!
     rake parallel:test          # Test::Unit
     rake parallel:spec          # RSpec
@@ -42,11 +57,12 @@ test:
     rake parallel:test    --> got 4 CPUs? --> 26 seconds
     ...
 
-Test by pattern (e.g. use one integration server per subfolder / see if you broke any 'user'-related tests)
+Test by pattern with Regex (e.g. use one integration server per subfolder / see if you broke any 'user'-related tests)
 
     rake parallel:test[^test/unit] # every test file in test/unit folder
     rake parallel:test[user]  # run users_controller + user_helper + user tests
     rake parallel:test['user|product']  # run user and product related tests
+    rake parallel:spec['spec\/(?!features)'] # run RSpec tests except the tests in spec/features
 
 
 ### Example output
@@ -63,6 +79,8 @@ Test by pattern (e.g. use one integration server per subfolder / see if you brok
 RAILS_ENV=test parallel_test -e "rake my:custom:task"
 # or
 rake parallel:rake[my:custom:task]
+# limited parallelism
+rake parallel:rake[my:custom:task,2]
 ```
 
 
@@ -70,10 +88,17 @@ Running things once
 ===================
 
 ```Ruby
-# effected by race-condition: first process may boot slower the second
+require "parallel_tests"
+
+# preparation:
+# affected by race-condition: first process may boot slower than the second
 # either sleep a bit or use a lock for example File.lock
 ParallelTests.first_process? ? do_something : sleep(1)
 
+# cleanup:
+# last_process? does NOT mean last finished process, just last started
+ParallelTests.last_process? ? do_something : sleep(1)
+
 at_exit do
   if ParallelTests.first_process?
     ParallelTests.wait_for_other_processes_to_finish
@@ -82,24 +107,22 @@ at_exit do
 end
 ```
 
-Loggers
-===================
-
 Even test group run-times
--------------------------
+=========================
 
-### RSpec
+Test groups are often not balanced and will run for different times, making everything wait for the slowest group.
+Use these loggers to record test runtime and then use the recorded runtime to balance test groups more evenly.
 
-Add the `RuntimeLogger` to log how long each test takes to run.
-This log file will be loaded on the next test run, and the tests will be grouped
-so that each process should finish around the same time.
+### RSpec
 
 Rspec: Add to your `.rspec_parallel` (or `.rspec`) :
 
     --format progress
     --format ParallelTests::RSpec::RuntimeLogger --out tmp/parallel_runtime_rspec.log
 
-### Test::Unit & Minitest 4/5
+To use a custom logfile location (default: `tmp/parallel_runtime_rspec.log`), use the CLI: `parallel_test spec -t rspec --runtime-log my.log`
+
+### Minitest
 
 Add to your `test_helper.rb`:
 ```ruby
@@ -109,6 +132,9 @@ require 'parallel_tests/test/runtime_logger' if ENV['RECORD_RUNTIME']
 results will be logged to tmp/parallel_runtime_test.log when `RECORD_RUNTIME` is set,
 so it is not always required or overwritten.
 
+Loggers
+=======
+
 RSpec: SummaryLogger
 --------------------
 
@@ -122,17 +148,19 @@ Add the following to your `.rspec_parallel` (or `.rspec`) :
 RSpec: FailuresLogger
 -----------------------
 
-Produce pasteable command-line snippets for each failed example.
+Produce pastable command-line snippets for each failed example. For example:
 
-E.g.
-
-    rspec /path/to/my_spec.rb:123 # should do something
+```bash
+rspec /path/to/my_spec.rb:123 # should do something
+```
 
-Add the following to your `.rspec_parallel` (or `.rspec`) :
+Add to `.rspec_parallel` or use as CLI flag:
 
     --format progress
     --format ParallelTests::RSpec::FailuresLogger --out tmp/failing_specs.log
 
+(Not needed to retry failures, for that pass [--only-failures](https://relishapp.com/rspec/rspec-core/docs/command-line/only-failures) to rspec)
+
 Cucumber: FailuresLogger
 -----------------------
 
@@ -157,46 +185,78 @@ Setup for non-rails
 
     gem install parallel_tests
     # go to your project dir
-    parallel_test test/
-    parallel_rspec spec/
-    parallel_cucumber features/
-    parallel_spinach features/
+    parallel_test
+    parallel_rspec
+    parallel_cucumber
+    parallel_spinach
 
- - use ENV['TEST_ENV_NUMBER'] inside your tests to select separate db/memcache/etc.
- - Only run selected files & folders:
+ - use `ENV['TEST_ENV_NUMBER']` inside your tests to select separate db/memcache/etc. (docker compose: expose it)
 
-    parallel_test test/bar test/baz/foo_text.rb
+ - Only run a subset of files / folders:
+ 
+    `parallel_test test/bar test/baz/foo_text.rb`
 
-Options are:
+ - Pass test-options and files via `--`:
+ 
+    `parallel_rspec -- -t acceptance -f progress -- spec/foo_spec.rb spec/acceptance`
+    
+ - Pass in test options, by using the -o flag (wrap everything in quotes):
+ 
+    `parallel_cucumber -n 2 -o '-p foo_profile --tags @only_this_tag or @only_that_tag --format summary'`
 
+Options are:
+<!-- copy output from bundle exec ./bin/parallel_test -h -->
     -n [PROCESSES]                   How many processes to use, default: available CPUs
-    -p, --pattern [PATTERN]          run tests matching this pattern
+    -p, --pattern [PATTERN]          run tests matching this regex pattern
+        --exclude-pattern [PATTERN]  exclude tests matching this regex pattern
         --group-by [TYPE]            group tests by:
-          found - order of finding files
-          steps - number of cucumber/spinach steps
-          scenarios - individual cucumber scenarios
-          filesize - by size of the file
-          runtime - info from runtime log
-          default - runtime when runtime log is filled otherwise filesize
+                                     found - order of finding files
+                                     steps - number of cucumber/spinach steps
+                                     scenarios - individual cucumber scenarios
+                                     filesize - by size of the file
+                                     runtime - info from runtime log
+                                     default - runtime when runtime log is filled otherwise filesize
     -m, --multiply-processes [FLOAT] use given number as a multiplier of processes to run
     -s, --single [PATTERN]           Run all matching files in the same process
     -i, --isolate                    Do not run any other tests in the group used by --single(-s)
-        --only-group INT[, INT]
-    -e, --exec [COMMAND]             execute this code parallel and with ENV['TEST_ENV_NUM']
+        --isolate-n [PROCESSES]      Use 'isolate'  singles with number of processes, default: 1.
+        --highest-exit-status        Exit with the highest exit status provided by test run(s)
+        --specify-groups [SPECS]     Use 'specify-groups' if you want to specify multiple specs running in multiple
+                                     processes in a specific formation. Commas indicate specs in the same process,
+                                     pipes indicate specs in a new process. Cannot use with --single, --isolate, or
+                                     --isolate-n.  Ex.
+                                     $ parallel_tests -n 3 . --specify-groups '1_spec.rb,2_spec.rb|3_spec.rb'
+                                       Process 1 will contain 1_spec.rb and 2_spec.rb
+                                       Process 2 will contain 3_spec.rb
+                                       Process 3 will contain all other specs
+        --only-group INT[,INT]
+    -e, --exec [COMMAND]             execute this code parallel and with ENV['TEST_ENV_NUMBER']
     -o, --test-options '[OPTIONS]'   execute test commands with those options
     -t, --type [TYPE]                test(default) / rspec / cucumber / spinach
+        --suffix [PATTERN]           override built in test file pattern (should match suffix):
+                                     '_spec.rb$' - matches rspec files
+                                     '_(test|spec).rb$' - matches test or spec files
         --serialize-stdout           Serialize stdout output, nothing will be written until everything is done
+        --prefix-output-with-test-env-number
+                                     Prefixes test env number to the output when not using --serialize-stdout
         --combine-stderr             Combine stderr into stdout, useful in conjunction with --serialize-stdout
         --non-parallel               execute same commands but do not in parallel, needs --exec
         --no-symlinks                Do not traverse symbolic links to find test files
         --ignore-tags [PATTERN]      When counting steps ignore scenarios with tags that match this pattern
         --nice                       execute test commands with low priority.
         --runtime-log [PATH]         Location of previously recorded test runtimes
-        --verbose                    Print more output
+        --allowed-missing [INT]      Allowed percentage of missing runtimes (default = 50)
+        --unknown-runtime [FLOAT]    Use given number as unknown runtime (otherwise use average time)
+        --first-is-1                 Use "1" as TEST_ENV_NUMBER to not reuse the default test environment
+        --fail-fast                  Stop all groups when one group fails (best used with --test-options '--fail-fast' if supported
+        --verbose                    Print debug output
+        --verbose-process-command    Displays only the command that will be executed by each process
+        --verbose-rerun-command      When there are failures, displays the command executed by each process that failed
+        --quiet                      Print only tests output
     -v, --version                    Show Version
     -h, --help                       Show this.
 
-You can run any kind of code in parallel with -e / --execute
+You can run any kind of code in parallel with -e / --exec
 
     parallel_test -n 5 -e 'ruby -e "puts %[hello from process #{ENV[:TEST_ENV_NUMBER.to_s].inspect}]"'
     hello from process "2"
@@ -213,34 +273,46 @@ You can run any kind of code in parallel with -e / --execute
 
 TIPS
 ====
- - [RSpec] add a `.rspec_parallel` to use different options, e.g. **no --drb**
- - Spring does not work with parallel_tests, use `DISABLE_SPRING=1 rake parallel:spec` if you have spring hardcoded in your binaries
- - [RSpec] remove `--loadby` from `.rspec`
- - [RSpec] Instantly see failures (instead of just a red F) with [rspec-instafail](https://github.com/grosser/rspec-instafail)
- - [Cucumber] add a `parallel: foo` profile to your `config/cucumber.yml` and it will be used to run parallel tests
- - [Capybara setup](https://github.com/grosser/parallel_tests/wiki)
- - [Sphinx setup](https://github.com/grosser/parallel_tests/wiki)
- - [Capistrano setup](https://github.com/grosser/parallel_tests/wiki/Remotely-with-capistrano) let your tests run on a big box instead of your laptop
- - [SQL schema format] use :ruby schema format to get faster parallel:prepare`
- - `export PARALLEL_TEST_PROCESSORS=X` in your environment and parallel_tests will use this number of processors by default
+
+### RSpec
+
+ - Add a `.rspec_parallel` to use different options, e.g. **no --drb**
+ - Remove `--loadby` from `.rspec`
+ - Instantly see failures (instead of just a red F) with [rspec-instafail](https://github.com/grosser/rspec-instafail)
+ - Use [rspec-retry](https://github.com/NoRedInk/rspec-retry) (not rspec-rerun) to rerun failed tests.
+ - [JUnit formatter configuration](https://github.com/grosser/parallel_tests/wiki#with-rspec_junit_formatter----by-jgarber)
+ - Use [parallel_split_test](https://github.com/grosser/parallel_split_test) to run multiple scenarios in a single spec file, concurrently. (`parallel_tests` [works at the file-level and intends to stay that way](https://github.com/grosser/parallel_tests/issues/747#issuecomment-580216980))
+
+### Cucumber
+
+ - Add a `parallel: foo` profile to your `config/cucumber.yml` and it will be used to run parallel tests
+ - [ReportBuilder](https://github.com/rajatthareja/ReportBuilder) can help with combining parallel test results
+   - Supports Cucumber 2.0+ and is actively maintained
+   - Combines many JSON files into a single file
+   - Builds a HTML report from JSON with support for debug msgs & embedded Base64 images.
+
+### General
  - [ZSH] use quotes to use rake arguments `rake "parallel:prepare[3]"`
+ - [Memcached] use different namespaces<br/>
+   e.g. `config.cache_store = ..., namespace: "test_#{ENV['TEST_ENV_NUMBER']}"`
+ - Debug errors that only happen with multiple files using `--verbose` and [cleanser](https://github.com/grosser/cleanser)
+ - `export PARALLEL_TEST_PROCESSORS=13` to override default processor count
+ - Shell alias: `alias prspec='parallel_rspec -m 2 --'`
+ - [Spring] Add the [spring-commands-parallel-tests](https://github.com/DocSpring/spring-commands-parallel-tests) gem to your `Gemfile` to get `parallel_tests` working with Spring.
+ - `--first-is-1` will make the first environment be `1`, so you can test while running your full suite.<br/>
+   `export PARALLEL_TEST_FIRST_IS_1=true` will provide the same result
  - [email_spec and/or action_mailer_cache_delivery](https://github.com/grosser/parallel_tests/wiki)
- - [Memcached] use different namespaces e.g. `config.cache_store = ..., namespace: "test_#{ENV['TEST_ENV_NUMBER']}"`
  - [zeus-parallel_tests](https://github.com/sevos/zeus-parallel_tests)
- - [Distributed parallel test (e.g. Travis Support)](https://github.com/grosser/parallel_tests/wiki/Distributed-Parallel-Tests-and-Travis-Support)
- - Debug errors that only happen with multiple files using `--verbose` and [cleanser](https://github.com/grosser/cleanser)
- - Contribute your own gotaches to the [Wiki](https://github.com/grosser/parallel_tests/wiki) or even better open a PR :)
+ - [Distributed Parallel Tests on CI systems)](https://github.com/grosser/parallel_tests/wiki/Distributed-Parallel-Tests-on-CI-systems) learn how `parallel_tests` can run on distributed servers such as Travis and GitLab-CI. Also shows you how to use parallel_tests without adding `TEST_ENV_NUMBER`-backends
+ - [Capybara setup](https://github.com/grosser/parallel_tests/wiki)
+ - [Sphinx setup](https://github.com/grosser/parallel_tests/wiki)
+ - [Capistrano setup](https://github.com/grosser/parallel_tests/wiki/Remotely-with-capistrano) let your tests run on a big box instead of your laptop
 
-TODO
-====
- - fix tests vs cucumber >= 1.2 `unknown option --format`
- - add integration tests for the rake tasks, maybe generate a rails project ...
- - add unit tests for cucumber runtime formatter
- - make windows compatible
+Contribute your own gotchas to the [Wiki](https://github.com/grosser/parallel_tests/wiki) or even better open a PR :)
 
 Authors
 ====
-inspired by [pivotal labs](http://pivotallabs.com/users/miked/blog/articles/849-parallelize-your-rspec-suite)
+inspired by [pivotal labs](https://blog.pivotal.io/labs/labs/parallelize-your-rspec-suite)
 
 ### [Contributors](https://github.com/grosser/parallel_tests/contributors)
  - [Charles Finkel](http://charlesfinkel.com/)
@@ -309,8 +381,28 @@ inspired by [pivotal labs](http://pivotallabs.com/users/miked/blog/articles/849-
  - [Mike Mueller](https://github.com/mmueller)
  - [Aaron Jensen](https://github.com/aaronjensen)
  - [Ed Slocomb](https://github.com/edslocomb)
+ - [Cezary Baginski](https://github.com/e2)
+ - [Marius Ioana](https://github.com/mariusioana)
+ - [Lukas Oberhuber](https://github.com/lukaso)
+ - [Ryan Zhang](https://github.com/ryanus)
+ - [Rhett Sutphin](https://github.com/rsutphin)
+ - [Doc Ritezel](https://github.com/ohrite)
+ - [Alexandre Wilhelm](https://github.com/dogild)
+ - [Jerry](https://github.com/boblington)
+ - [Aleksei Gusev](https://github.com/hron)
+ - [Scott Olsen](https://github.com/scottolsen)
+ - [Andrei Botalov](https://github.com/abotalov)
+ - [Zachary Attas](https://github.com/snackattas)
+ - [David Rodríguez](https://github.com/deivid-rodriguez)
+ - [Justin Doody](https://github.com/justindoody)
+ - [Sandeep Singh](https://github.com/sandeepnagra)
+ - [Calaway](https://github.com/calaway)
+ - [alboyadjian](https://github.com/alboyadjian)
+ - [Nathan Broadbent](https://github.com/ndbroadbent)
+ - [Vikram B Kumar](https://github.com/v-kumar)
+ - [Joshua Pinter](https://github.com/joshuapinter)
+ - [Zach Dennis](https://github.com/zdennis)
 
 [Michael Grosser](http://grosser.it)<br/>
 michael@grosser.it<br/>
-License: MIT<br/>
-[![Build Status](https://travis-ci.org/grosser/parallel_tests.png)](https://travis-ci.org/grosser/parallel_tests)
+License: MIT

data/bin/parallel_cucumber

@@ -1,7 +1,8 @@
 #!/usr/bin/env ruby
+# frozen_string_literal: true
 
 # enable local usage from cloned repo
-root = File.expand_path("../..", __FILE__)
+root = File.expand_path('..', __dir__)
 $LOAD_PATH << "#{root}/lib" if File.exist?("#{root}/Gemfile")
 
 require "parallel_tests"

data/bin/parallel_rspec

@@ -1,7 +1,8 @@
 #!/usr/bin/env ruby
+# frozen_string_literal: true
 
 # enable local usage from cloned repo
-root = File.expand_path("../..", __FILE__)
+root = File.expand_path('..', __dir__)
 $LOAD_PATH << "#{root}/lib" if File.exist?("#{root}/Gemfile")
 
 require "parallel_tests"

data/bin/parallel_spinach

@@ -1,7 +1,8 @@
 #!/usr/bin/env ruby
+# frozen_string_literal: true
 
 # enable local usage from cloned repo
-root = File.expand_path("../..", __FILE__)
+root = File.expand_path('..', __dir__)
 $LOAD_PATH << "#{root}/lib" if File.exist?("#{root}/Gemfile")
 
 require "parallel_tests"

data/bin/parallel_test

@@ -1,7 +1,8 @@
 #!/usr/bin/env ruby
+# frozen_string_literal: true
 
 # enable local usage from cloned repo
-root = File.expand_path("../..", __FILE__)
+root = File.expand_path('..', __dir__)
 $LOAD_PATH << "#{root}/lib" if File.exist?("#{root}/Gemfile")
 
 require "parallel_tests"