semaphore_test_boosters 1.7.3 → 1.8.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA1:
-  metadata.gz: 55ecf357e070533847ba7fc22764a026d0ec2918
-  data.tar.gz: 52e0d4afb19f129f079378600b871bb0c47e9d4f
+  metadata.gz: 9403e453151e8b33705cd21a2d2421efd6160e30
+  data.tar.gz: be15846080e83674e3cae45e34a8969646993228
 SHA512:
-  metadata.gz: ecb7d0fb08e4a94de4bc17d9adefe2b2694185666b0c16fa0b4c0fcee4483c5f2de5b484646ea8f83448a5647b38b89e074648080fba0bec43e2160630541e31
-  data.tar.gz: 866a3b4b308094b0f3ffb1d9593f6595ed74ba5683c191178f8738915f00fddccd21e41c30fcfa7f55955a35e0de38cead7218eb14420d5d886d9d1eb9bd3270
+  metadata.gz: c1383be5aba30566585ed5568ccda59ed28ad9097ee2f8f5999c614ae66133b7d145b3d69a61c1a6b10d06bfd68ec381e60caf93fa3acf93653b2d187accdab3
+  data.tar.gz: 80d349f51e9618bc7966fb4806b4346ef4465bd075a973e6e84bd759b07e404ac7929798eab8e8ced990f835628f588f0e3035ea86f9b1b1634dbe05653cf54f

data/README.md

@@ -1,8 +1,23 @@
 # Test Boosters
 
 [![Gem Version](https://badge.fury.io/rb/semaphore_test_boosters.svg)](https://badge.fury.io/rb/semaphore_test_boosters)
+[![Build Status](https://semaphoreci.com/api/v1/renderedtext/test-boosters/branches/master/badge.svg)](https://semaphoreci.com/renderedtext/test-boosters)
 
-Auto Parallelization — runs test files in multiple threads.
+Auto Parallelization — runs test files in multiple jobs
+
+- [Installation](#installation)
+
+- RSpec Booster
+  - [Running RSpec jobs](#rspec-booster)
+  - [RSpec Split Configuration](#rspec-split-configuration)
+  - [Leftover RSpec specs](#leftover-rspec-specs)
+  - [Passing custom options to RSpec](#custom-rspec-options)
+
+- Cucumber Booster
+  - [Running Cucumber jobs](#cucumber-booster)
+  - [Cucumber Split Configuration](#cucumber-split-configuration)
+  - [Leftover Cucumber specs](#leftover-rspec-specs)
+  - [Passing custom options to Cucumber](#custom-cucumber-options)
 
 ## Installation
 
@@ -10,25 +25,39 @@ Auto Parallelization — runs test files in multiple threads.
 gem install semaphore_test_boosters
 ````
 
-## Usage
+## RSpec Booster
 
-### RSpec Booster
+The RSpec Booster splits your RSpec test suite to multiple jobs.
 
-The RSpec booster command allows you to run one out of several parallel RSpec
-threads.
+For example, if you want to split your RSpec test suite into 21 jobs, and run
+then executed the 3rd job, use the following command:
 
 ``` bash
-rspec_booster --thread 3
+rspec_booster --thread 3/21
 ```
 
-#### RSpec Split Configuration
+By default, RSpec Booster will distribute your spec files based on their size
+into multiple jobs. This is OK for your first run, but the distribution is
+usually suboptimal.
+
+If you want to achieve better build times, and split your test files more
+evenly, you need to provide a split configuration file for RSpec Booster.
+
+On Semaphore, this file is generated before every build based on the duration's
+of your previous builds.
+
+### RSpec Split Configuration
+
+The `rspec_split_configuration.json` should be placed in your home directory and
+should contain the list of files for each RSpec Booster job.
 
-The `rspec_split_configuration.json` located in your home directory is used to
-pass test files to the booster. On Semaphore, this file contains a list of
-specs files distributed across threads based on their estimated durations.
+For example, if you have 3 RSpec Booster jobs, and you want to run:
 
-For example, if you have a `rspec_split_configuration.json` located in your home
-directory with the following content:
+- `spec/a_spec.rb` and `spec/b_spec.rb` in the first job
+- `spec/c_spec.rb` and `spec/d_spec.rb` in the second job
+- `spec/e_spec.rb` in the third job
+
+you should put the following in your split configuration file:
 
 ``` json
 [
@@ -38,19 +67,13 @@ directory with the following content:
 ]
 ```
 
-The `rspec_booster` command will deduce that you have 3 threads in total and
-that you want to run `spec/a_spec.rb` and `spec/b_spec.rb` on the first thread,
-`spec/c_spec.rb` and `spec/d_spec.rb` on the second thread, and `spec/e_spec.rb`
-on the third thread.
-
-#### Leftover files
+### Leftover RSpec specs
 
-The RSpec Split Configuration contains only those spec files that have an
-estimated duration recorded on Semaphore. New files, whose estimated duration
-is not yet stored on Semaphore will be distributed across threads in based on
-their file size in a round robin fashion.
+Files that are part of your RSpec test suite, but are not in the split
+configuration file, are called "leftover files". These files will be distributed
+based on their file size in a round robin fashion across your jobs.
 
-For example, if you have the following split configuration:
+For example, if you have the following in your split configuration:
 
 ``` json
 [
@@ -70,16 +93,16 @@ spec/d_spec.rb
 spec/e_spec.rb
 ```
 
-When you run the `rspec_booster --thread 1` command, the files from the
-configuration's first thread and some leftover files will be executed.
+When you run the `rspec_booster --job 1/3` command, the files from the
+configuration's first job and some leftover files will be executed.
 
 ``` bash
-rspec_booster --thread 1
+rspec_booster --job 1/3
 
 # => runs: bundle exec rspec spec/a_spec.rb spec/d_spec.rb
 ```
 
-#### Custom RSpec options
+### Custom RSpec options
 
 By default, `rspec_booster` passes the following options to RSpec:
 
@@ -96,17 +119,97 @@ like this:
 ``` bash
 export TB_RSPEC_OPTIONS = '--fail-fast'
 
-rspec_booster --thread 2
+rspec_booster --job 2/3
 
 # => runs: bundle exec rspec --fail-fast --format documentation --format json --out ~/rspec_report.json <file_list>
 ```
 
+## Cucumber Booster
+
+The Cucumber Booster splits your Cucumber test suite to multiple jobs.
+
+For example, if you want to split your Cucumber test suite into 21 jobs, and run
+then executed the 3rd job, use the following command:
+
+``` bash
+cucumber_booster --thread 3/21
+```
+
+By default, Cucumber Booster will distribute your spec files based on their size
+into multiple jobs. This is OK for your first run, but the distribution is
+usually suboptimal.
+
+If you want to achieve better build times, and split your test files more
+evenly, you need to provide a split configuration file for Cucumber Booster.
+
+On Semaphore, this file is generated before every build based on the duration's
+of your previous builds.
+
+### Cucumber Split Configuration
+
+The `cucumber_split_configuration.json` should be placed in your home directory
+and should contain the list of files for each Cucumber Booster job.
+
+For example, if you have 3 Cucumber Booster jobs, and you want to run:
+
+- `features/a.feature` and `spec/b.feature` in the first job
+- `features/c.feature` and `spec/d.feature` in the second job
+- `features/e.feature` in the third job
+
+you should put the following in your split configuration file:
+
+``` json
+[
+  { "files": ["features/a.feature", "features/b.feature"] },
+  { "files": ["features/c.feature", "features/d.feature"] },
+  { "files": ["features/e.feature"] }
+]
+```
+
+### Leftover files
+
+Files that are part of your Cucumber test suite, but are not in the split
+configuration file, are called "leftover files". These files will be distributed
+based on their file size in a round robin fashion across your jobs.
+
+For example, if you have the following in your split configuration:
+
+``` json
+[
+  { "files": ["features/a.feature"] }
+  { "files": ["features/b.feature"] }
+  { "files": ["features/c.feature"] }
+]
+```
+
+and the following files in your spec directory:
+
+``` bash
+features/a.feature
+features/b.feature
+features/c.feature
+features/d.feature
+features/e.feature
+```
+
+When you run the `cucumber_booster --job 1/3` command, the files from the
+configuration's first job and some leftover files will be executed.
+
+``` bash
+cucumber_booster --job 1/3
+
+# => runs: bundle exec cucumber features/a.feature features/d.feature
+```
+
+### Custom Cucumber options
+
+Currently, you can't pass custom options to Cucumber.
+
 ## Contributing
 
 Bug reports and pull requests are welcome on GitHub at
 https://github.com/renderedtext/test-boosters.
 
-
 ## License
 
 The gem is available as open source under the terms of the

data/exe/cucumber_booster

@@ -4,9 +4,9 @@ require "test_boosters"
 
 cli_options = TestBoosters::CliParser.parse
 
-thread_index = cli_options[:thread_index] - 1
-thread_count = cli_options[:thread_count]
+job_index = cli_options[:job_index] - 1
+job_count = cli_options[:job_count]
 
-cucumber_booster = TestBoosters::Cucumber::Booster.new(thread_index, thread_count)
+cucumber_booster = TestBoosters::Cucumber::Booster.new(job_index, job_count)
 
 exit(cucumber_booster.run)

data/exe/rspec_booster

@@ -4,9 +4,9 @@ require "test_boosters"
 
 cli_options = TestBoosters::CliParser.parse
 
-thread_index = cli_options[:thread_index] - 1
-thread_count = cli_options[:thread_count]
+job_index = cli_options[:job_index] - 1
+job_count = cli_options[:job_count]
 
-rspec_booster = TestBoosters::Rspec::Booster.new(thread_index, thread_count)
+rspec_booster = TestBoosters::Rspec::Booster.new(job_index, job_count)
 
 exit(rspec_booster.run)

data/lib/test_boosters.rb

@@ -15,8 +15,8 @@ module TestBoosters
   require "test_boosters/project_info"
 
   require "test_boosters/rspec/booster"
-  require "test_boosters/rspec/thread"
+  require "test_boosters/rspec/job"
 
   require "test_boosters/cucumber/booster"
-  require "test_boosters/cucumber/thread"
+  require "test_boosters/cucumber/job"
 end

data/lib/test_boosters/cli_parser.rb

@@ -4,15 +4,19 @@ module TestBoosters
 
     # :reek:TooManyStatements
     # :reek:NestedIterators
+    # :reek:DuplicateMethodCall
     def parse
       options = {}
 
       parser = OptionParser.new do |opts|
         opts.on("--thread INDEX") do |parameter|
-          thread_index, thread_count, _rest = parameter.split("/")
+          puts "[DEPRECATION WARNING] The '--thread' parameter is deprecated. Please use '--job' instead."
 
-          options[:thread_index] = thread_index.to_i
-          options[:thread_count] = thread_count.to_i
+          options.merge!(parse_job_params(parameter))
+        end
+
+        opts.on("--job INDEX") do |parameter|
+          options.merge!(parse_job_params(parameter))
         end
       end
 
@@ -20,5 +24,13 @@ module TestBoosters
 
       options
     end
+
+    # parses input like '1/32' and ouputs { :job_index => 1, :job_count => 32 }
+    def parse_job_params(input_parameter)
+      job_index, job_count, _rest = input_parameter.split("/")
+
+      { :job_index => job_index.to_i, :job_count => job_count.to_i }
+    end
+
   end
 end

data/lib/test_boosters/cucumber/booster.rb

@@ -2,19 +2,19 @@ module TestBoosters
   module Cucumber
     class Booster
 
-      attr_reader :thread_index
-      attr_reader :thread_count
+      attr_reader :job_index
+      attr_reader :job_count
 
-      def initialize(thread_index, thread_count)
-        @thread_index = thread_index
-        @thread_count = thread_count
+      def initialize(job_index, job_count)
+        @job_index = job_index
+        @job_count = job_count
       end
 
       def run
         TestBoosters::Shell.display_title("Cucumber Booster v#{TestBoosters::VERSION}")
         display_system_info
 
-        threads[@thread_index].run
+        jobs[@job_index].run
       end
 
       def display_system_info
@@ -25,12 +25,12 @@ module TestBoosters
         puts
       end
 
-      def threads
-        @threads ||= Array.new(@thread_count) do |thread_index|
-          known    = all_specs & split_configuration.files_for_thread(thread_index)
-          leftover = leftover_specs.select(:index => thread_index, :total => thread_count)
+      def jobs
+        @jobs ||= Array.new(job_count) do |job_index|
+          known    = all_specs & split_configuration.files_for_job(job_index)
+          leftover = leftover_specs.select(:index => job_index, :total => job_count)
 
-          TestBoosters::Cucumber::Thread.new(known, leftover)
+          TestBoosters::Cucumber::Job.new(known, leftover)
         end
       end
 

data/lib/test_boosters/cucumber/{thread.rb → job.rb}

@@ -1,6 +1,6 @@
 module TestBoosters
   module Cucumber
-    class Thread
+    class Job
 
       attr_reader :files_from_split_configuration
       attr_reader :leftover_files
@@ -13,14 +13,14 @@ module TestBoosters
       # :reek:TooManyStatements { max_statements: 10 }
       def run
         if all_files.empty?
-          puts("No files to run in this thread!")
+          puts("No files to run in this job!")
 
           return 0
         end
 
         run_cucumber_config
 
-        display_thread_info
+        display_job_info
 
         exit_status = run_cucumber
 
@@ -29,13 +29,13 @@ module TestBoosters
         exit_status
       end
 
-      def display_thread_info
+      def display_job_info
         TestBoosters::Shell.display_files(
-          "Known specs for this thread",
+          "Known specs for this job",
           files_from_split_configuration)
 
         TestBoosters::Shell.display_files(
-          "Leftover specs for this thread",
+          "Leftover specs for this job",
           leftover_files)
       end
 

data/lib/test_boosters/leftover_files.rb

@@ -16,16 +16,16 @@ module TestBoosters
 
     private
 
-    def file_distribution(thread_count)
+    def file_distribution(job_count)
       # create N empty boxes
-      threads = Array.new(thread_count) { [] }
+      jobs = Array.new(job_count) { [] }
 
       # distribute files in Round Robin fashion
       sorted_files_by_file_size.each.with_index do |file, index|
-        threads[index % thread_count] << file
+        jobs[index % job_count] << file
       end
 
-      threads
+      jobs
     end
 
     def sorted_files_by_file_size

data/lib/test_boosters/rspec/booster.rb

@@ -2,19 +2,19 @@ module TestBoosters
   module Rspec
     class Booster
 
-      attr_reader :thread_index
-      attr_reader :thread_count
+      attr_reader :job_index
+      attr_reader :job_count
 
-      def initialize(thread_index, thread_count)
-        @thread_index = thread_index
-        @thread_count = thread_count
+      def initialize(job_index, job_count)
+        @job_index = job_index
+        @job_count = job_count
       end
 
       def run
         TestBoosters::Shell.display_title("RSpec Booster v#{TestBoosters::VERSION}")
         display_system_info
 
-        threads[@thread_index].run
+        jobs[@job_index].run
       end
 
       def display_system_info
@@ -25,12 +25,12 @@ module TestBoosters
         puts
       end
 
-      def threads
-        @threads ||= Array.new(@thread_count) do |thread_index|
-          known    = all_specs & split_configuration.files_for_thread(thread_index)
-          leftover = leftover_specs.select(:index => thread_index, :total => thread_count)
+      def jobs
+        @jobs ||= Array.new(job_count) do |job_index|
+          known    = all_specs & split_configuration.files_for_job(job_index)
+          leftover = leftover_specs.select(:index => job_index, :total => job_count)
 
-          TestBoosters::Rspec::Thread.new(known, leftover)
+          TestBoosters::Rspec::Job.new(known, leftover)
         end
       end
 

data/lib/test_boosters/rspec/{thread.rb → job.rb}

@@ -1,6 +1,6 @@
 module TestBoosters
   module Rspec
-    class Thread
+    class Job
 
       attr_reader :files_from_split_configuration
       attr_reader :leftover_files
@@ -13,12 +13,12 @@ module TestBoosters
       # :reek:TooManyStatements { max_statements: 10 }
       def run
         if all_files.empty?
-          puts("No files to run in this thread!")
+          puts("No files to run in this job!")
 
           return 0
         end
 
-        display_thread_info
+        display_job_info
 
         exit_status = run_rspec
 
@@ -27,14 +27,9 @@ module TestBoosters
         exit_status
       end
 
-      def display_thread_info
-        TestBoosters::Shell.display_files(
-          "Known specs for this thread",
-          files_from_split_configuration)
-
-        TestBoosters::Shell.display_files(
-          "Leftover specs for this thread",
-          leftover_files)
+      def display_job_info
+        TestBoosters::Shell.display_files("Known specs for this job", files_from_split_configuration)
+        TestBoosters::Shell.display_files("Leftover specs for this job", leftover_files)
 
         puts "RSpec options: #{rspec_options}"
       end

data/lib/test_boosters/split_configuration.rb

@@ -1,7 +1,7 @@
 module TestBoosters
   class SplitConfiguration
 
-    Thread = Struct.new(:files, :thread_index)
+    Job = Struct.new(:files)
 
     def initialize(path)
       @path = path
@@ -13,23 +13,23 @@ module TestBoosters
     end
 
     def valid?
-      threads # try to load data into memory
+      jobs # try to load data into memory
 
       @valid
     end
 
     def all_files
-      @all_files ||= threads.map(&:files).flatten.sort
+      @all_files ||= jobs.map(&:files).flatten.sort
     end
 
-    def files_for_thread(thread_index)
-      thread = threads[thread_index]
+    def files_for_job(job_index)
+      job = jobs[job_index]
 
-      thread ? thread.files : []
+      job ? job.files : []
     end
 
-    def threads
-      @threads ||= present? ? load_data : []
+    def jobs
+      @jobs ||= present? ? load_data : []
     end
 
     private
@@ -38,8 +38,10 @@ module TestBoosters
     def load_data
       @valid = false
 
-      content = JSON.parse(File.read(@path)).map.with_index do |raw_thread, index|
-        TestBoosters::SplitConfiguration::Thread.new(raw_thread.fetch("files").sort, index)
+      content = JSON.parse(File.read(@path)).map do |raw_job|
+        files = raw_job.fetch("files").sort
+
+        TestBoosters::SplitConfiguration::Job.new(files)
       end
 
       @valid = true

data/lib/test_boosters/version.rb

@@ -1,3 +1,3 @@
 module TestBoosters
-  VERSION = "1.7.3".freeze
+  VERSION = "1.8.0".freeze
 end

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: semaphore_test_boosters
 version: !ruby/object:Gem::Version
-  version: 1.7.3
+  version: 1.8.0
 platform: ruby
 authors:
 - Developers at Rendered Text
@@ -160,13 +160,13 @@ files:
 - lib/test_boosters.rb
 - lib/test_boosters/cli_parser.rb
 - lib/test_boosters/cucumber/booster.rb
-- lib/test_boosters/cucumber/thread.rb
+- lib/test_boosters/cucumber/job.rb
 - lib/test_boosters/insights_uploader.rb
 - lib/test_boosters/leftover_files.rb
 - lib/test_boosters/logger.rb
 - lib/test_boosters/project_info.rb
 - lib/test_boosters/rspec/booster.rb
-- lib/test_boosters/rspec/thread.rb
+- lib/test_boosters/rspec/job.rb
 - lib/test_boosters/shell.rb
 - lib/test_boosters/split_configuration.rb
 - lib/test_boosters/version.rb