knapsack_pro 1.18.2 → 1.19.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: c57b67afbab19446717852ab88147438653644d7953ada292a41f36c681a520d
-  data.tar.gz: eef083501ca2a40b09cf189e27ea8ecf7aa94f7813649b6404cc08e3cb20f0cc
+  metadata.gz: 824a069d97efb456cec01426d85ebee23d4eba646620483a09def750fba06007
+  data.tar.gz: 4c0bd1456766af5889562764590862d64626399e12d9af184343c1c6eabd4cb7
 SHA512:
-  metadata.gz: e8e3d39c85e16d78e09c3131b1e07803bc115ba587c948d7643b6785eac90d58ffb3e7ef64ccdd7049ed59cf3def65a772316ee12babb4dc272c7d4279f50bf4
-  data.tar.gz: fd81ab872ece61ceff06015e4070f71bf951d29a5cdbe584da8e0f3a7cee0973ad53c0695ea51c6d8c5a428e1f4944027565aa5b2898a05e9b95d84703252813
+  metadata.gz: 3b557f2efc472c09108e33c7c9ef612a141986a79d7e1969a5bbea198d57341b3ac2c8a9e76a1ab8767993922405a813eac545fcaf23d1e28986ff3c9e5d586e
+  data.tar.gz: ae5853cec21a9453fa290c26ac90eebce49231848edd5b638a5cdea58a9f897f9f2bcac82c26ac953b57f53ef327e66bceebe2c67e79e36cc0a6f6d8b96d0ff4

data/CHANGELOG.md

@@ -1,5 +1,29 @@
 # Change Log
 
+### 1.19.0
+
+* RSpec split test files by test examples (by individual `it`s)
+
+    https://github.com/KnapsackPro/knapsack_pro-ruby/pull/102
+
+    __Note:__ See [PR](https://github.com/KnapsackPro/knapsack_pro-ruby/pull/102) for more details. This is an experimental feature and it may not work for very large test suite.
+
+    __How to use it__: In order to split RSpec test files by test examples across parallel CI nodes you need to set flag:
+
+    ```
+    KNAPSACK_PRO_RSPEC_SPLIT_BY_TEST_EXAMPLES=true
+    ```
+
+    Thanks to that your CI build speed can be faster. We recommend using this feature with Queue Mode to ensure parallel CI nodes finish work at a similar time which gives you the shortest CI build time.
+
+    Doing tests split by test examples can generate a lot of logs by `knapsack_pro` gem in Queue Mode. We recommend to set log level to:
+
+    ```
+    KNAPSACK_PRO_LOG_LEVEL=warn
+    ```
+
+https://github.com/KnapsackPro/knapsack_pro-ruby/compare/v1.18.2...v1.19.0
+
 ### 1.18.2
 
 * If `KnapsackPro::Hooks::Queue.before_queue` hook has block of code that raises an exception then ensure the hook was called only once.

data/README.md

@@ -82,6 +82,8 @@ We keep this old FAQ in README to not break old links spread across the web. You
     - [KNAPSACK_PRO_FIXED_QUEUE_SPLIT (remember queue split on retry CI node)](#knapsack_pro_fixed_queue_split-remember-queue-split-on-retry-ci-node)
     - [KNAPSACK_PRO_MODIFY_DEFAULT_RSPEC_FORMATTERS (hide duplicated summary of pending and failed tests)](#knapsack_pro_modify_default_rspec_formatters-hide-duplicated-summary-of-pending-and-failed-tests)
   - [Supported test runners in queue mode](#supported-test-runners-in-queue-mode)
+- [Split test files by test cases](#split-test-files-by-test-cases)
+  - [RSpec split test files by test examples (individual `it`)](#rspec-split-test-files-by-test-examples-individual-it)
 - [Extra configuration for CI server](#extra-configuration-for-ci-server)
   - [Info about ENV variables](#info-about-env-variables)
     - [KNAPSACK_PRO_FIXED_TEST_SUITE_SPLIT (test suite split based on seed)](#knapsack_pro_fixed_test_suite_split-test-suite-split-based-on-seed)
@@ -97,6 +99,7 @@ We keep this old FAQ in README to not break old links spread across the web. You
   - [Test file names encryption](#test-file-names-encryption)
     - [How to enable test file names encryption?](#how-to-enable-test-file-names-encryption)
     - [How to debug test file names?](#how-to-debug-test-file-names)
+      - [Preview encrypted RSpec test example paths?](#preview-encrypted-rspec-test-example-paths)
     - [How to enable branch names encryption?](#how-to-enable-branch-names-encryption)
     - [How to debug branch names?](#how-to-debug-branch-names)
   - [Supported CI providers](#supported-ci-providers)
@@ -592,6 +595,33 @@ At this moment the queue mode works for:
 * Minitest
 * Cucumber
 
+## Split test files by test cases
+
+__Note:__ this is an experimental feature. It works for Regular Mode and Queue Mode. For large test suite with a few thousand test files, it may generate too many RSpec test example paths that may lead to too large JSON payload in request to Knapsack Pro API and this could trigger the API timeout.
+
+Please give us feedback so we could improve the feature.
+https://knapsackpro.com/contact
+
+__How it works__: You can split slow test file by test cases. Thanks to that the test file can be split across parallel CI nodes because test cases from the test file will run on different CI nodes.
+
+This is helpful when you have one or a few very slow test files that are a bottleneck for CI build speed and you don't want to manually create a few smaller test files from the slow test files. Instead, you can tell `knapsack_pro` gem to split your test files by test cases across parallel CI nodes.
+
+### RSpec split test files by test examples (by individual `it`s)
+
+In order to split RSpec test files by test examples across parallel CI nodes you need to set flag:
+
+```
+KNAPSACK_PRO_RSPEC_SPLIT_BY_TEST_EXAMPLES=true
+```
+
+Thanks to that your CI build speed can be faster. We recommend using this feature with Queue Mode to ensure parallel CI nodes finish work at a similar time which gives you the shortest CI build time.
+
+Doing tests split by test examples can generate a lot of logs by `knapsack_pro` gem in Queue Mode. We recommend to set log level to:
+
+```
+KNAPSACK_PRO_LOG_LEVEL=warn
+```
+
 ## Extra configuration for CI server
 
 ### Info about ENV variables
@@ -781,6 +811,16 @@ KNAPSACK_PRO_SALT=xxx bundle exec rake knapsack_pro:encrypted_test_file_names[rs
 
 You can pass the name of test runner like `rspec`, `minitest`, `test_unit`, `cucumber`, `spinach` as argument to rake task.
 
+##### Preview encrypted RSpec test example paths?
+
+If you split RSpec tests by test examples (by individual `it`) you can preview encrypted test example paths this way:
+
+```bash
+KNAPSACK_PRO_RSPEC_SPLIT_BY_TEST_EXAMPLES=true \
+KNAPSACK_PRO_SALT=xxx \
+bundle exec rake knapsack_pro:encrypted_test_file_names[rspec]
+```
+
 #### How to enable branch names encryption?
 
 You need to add environment variable `KNAPSACK_PRO_BRANCH_ENCRYPTED=true` to your CI server.
@@ -2316,7 +2356,7 @@ Now you can preview logs in `Artifacts` tab in the Circle CI build view.
 
 #### How to split tests based on test level instead of test file level?
 
-If you want to split one big test file (test file with long time execution) across multiple CI nodes then you can:
+If you want to split one big test file (test file with long time execution) across multiple CI nodes then you can [check this tip](#split-test-files-by-test-cases) or use other methods like:
 
 ##### A. Create multiple small test files
 

data/lib/knapsack_pro.rb

@@ -68,6 +68,7 @@ require_relative 'knapsack_pro/runners/queue/base_runner'
 require_relative 'knapsack_pro/runners/queue/rspec_runner'
 require_relative 'knapsack_pro/runners/queue/cucumber_runner'
 require_relative 'knapsack_pro/runners/queue/minitest_runner'
+require_relative 'knapsack_pro/test_case_detectors/rspec_test_example_detector'
 require_relative 'knapsack_pro/crypto/encryptor'
 require_relative 'knapsack_pro/crypto/branch_encryptor'
 require_relative 'knapsack_pro/crypto/decryptor'

data/lib/knapsack_pro/adapters/rspec_adapter.rb

@@ -22,14 +22,20 @@ module KnapsackPro
       def bind_time_tracker
         ::RSpec.configure do |config|
           config.around(:each) do |example|
-            current_example_group =
-              if ::RSpec.respond_to?(:current_example)
-                ::RSpec.current_example.metadata[:example_group]
+            KnapsackPro.tracker.current_test_path =
+              if KnapsackPro::Config::Env.rspec_split_by_test_examples?
+                example.id
               else
-                example.metadata
+                current_example_group =
+                  if ::RSpec.respond_to?(:current_example)
+                    ::RSpec.current_example.metadata[:example_group]
+                  else
+                    example.metadata
+                  end
+
+                KnapsackPro::Adapters::RSpecAdapter.test_path(current_example_group)
               end
 
-            KnapsackPro.tracker.current_test_path = KnapsackPro::Adapters::RSpecAdapter.test_path(current_example_group)
             KnapsackPro.tracker.start_timer
 
             example.run

data/lib/knapsack_pro/base_allocator_builder.rb

@@ -18,7 +18,33 @@ module KnapsackPro
     end
 
     def test_dir
-      KnapsackPro::Config::Env.test_dir || test_file_pattern.split('/').first.gsub(/({)/, '')
+      KnapsackPro::Config::Env.test_dir || TestFilePattern.test_dir(adapter_class)
+    end
+
+    def test_files
+      found_test_files = KnapsackPro::TestFileFinder.call(test_file_pattern)
+
+      if adapter_class == KnapsackPro::Adapters::RSpecAdapter && KnapsackPro::Config::Env.rspec_split_by_test_examples?
+        test_files_count = found_test_files.size
+
+        KnapsackPro.logger.warn("Generating RSpec test examples JSON report to prepare your test suite to be split by test examples (by individual 'it's. Thanks to that a single test file can be split across parallel CI nodes). Analyzing #{test_files_count} test files.")
+
+        if test_files_count > 1000
+          KnapsackPro.logger.warn("You have more than 1000 test files, it may take longer to generate test examples. Please wait...")
+        end
+
+        # generate RSpec JSON report in separate process to not pollute RSpec state
+        cmd = 'bundle exec rake knapsack_pro:rspec_test_example_detector'
+        unless Kernel.system(cmd)
+          raise "Could not generate JSON report for RSpec. Rake task failed when running #{cmd}"
+        end
+
+        # read JSON report
+        detector = KnapsackPro::TestCaseDetectors::RSpecTestExampleDetector.new
+        detector.test_file_example_paths
+      else
+        found_test_files
+      end
     end
 
     private
@@ -36,9 +62,5 @@ module KnapsackPro
     def test_file_pattern
       TestFilePattern.call(adapter_class)
     end
-
-    def test_files
-      KnapsackPro::TestFileFinder.call(test_file_pattern)
-    end
   end
 end

data/lib/knapsack_pro/config/env.rb

@@ -167,6 +167,14 @@ module KnapsackPro
           ENV.fetch('KNAPSACK_PRO_CUCUMBER_QUEUE_PREFIX', 'bundle exec')
         end
 
+        def rspec_split_by_test_examples
+          ENV.fetch('KNAPSACK_PRO_RSPEC_SPLIT_BY_TEST_EXAMPLES', false)
+        end
+
+        def rspec_split_by_test_examples?
+          rspec_split_by_test_examples.to_s == 'true'
+        end
+
         def test_suite_token
           env_name = 'KNAPSACK_PRO_TEST_SUITE_TOKEN'
           ENV[env_name] || raise("Missing environment variable #{env_name}. You should set environment variable like #{env_name}_RSPEC (note there is suffix _RSPEC at the end). knapsack_pro gem will set #{env_name} based on #{env_name}_RSPEC value. If you use other test runner than RSpec then use proper suffix.")

data/lib/knapsack_pro/runners/queue/rspec_runner.rb

@@ -16,7 +16,9 @@ module KnapsackPro
           cli_args = (args || '').split
           # if user didn't provide the format then use explicitly default progress formatter
           # in order to avoid KnapsackPro::Formatters::RSpecQueueSummaryFormatter being the only default formatter
-          cli_args += ['--format', 'progress'] unless cli_args.include?('--format')
+          if !cli_args.include?('--format') && !cli_args.include?('-f')
+            cli_args += ['--format', 'progress']
+          end
           cli_args += [
             '--format', KnapsackPro::Formatters::RSpecQueueSummaryFormatter.to_s,
             '--default-path', runner.test_dir,

data/lib/knapsack_pro/test_case_detectors/rspec_test_example_detector.rb

@@ -0,0 +1,68 @@
+module KnapsackPro
+  module TestCaseDetectors
+    class RSpecTestExampleDetector
+      REPORT_DIR = 'tmp/knapsack_pro/test_case_detectors/rspec'
+      REPORT_PATH = "#{REPORT_DIR}/rspec_dry_run_json_report.json"
+
+      def generate_json_report
+        require 'rspec/core'
+
+        ensure_report_dir_exists
+        remove_old_json_report
+
+        test_file_paths = KnapsackPro::TestFileFinder.call(test_file_pattern)
+
+        cli_args = [
+          '--dry-run',
+          '--format', 'json',
+          '--out', REPORT_PATH,
+          '--default-path', test_dir,
+        ] + test_file_paths.map { |t| t.fetch('path') }
+        options = RSpec::Core::ConfigurationOptions.new(cli_args)
+        exit_code = RSpec::Core::Runner.new(options).run($stderr, $stdout)
+        if exit_code != 0
+          raise 'There was problem to generate test examples for test suite'
+        end
+      end
+
+      def test_file_example_paths
+        raise "No report found at #{REPORT_PATH}" unless File.exists?(REPORT_PATH)
+
+        json_report = File.read(REPORT_PATH)
+        hash_report = JSON.parse(json_report)
+        hash_report
+          .fetch('examples')
+          .map { |e| e.fetch('id') }
+          .map { |path_with_example_id| test_file_hash_for(path_with_example_id) }
+      end
+
+      private
+
+      def adapter_class
+        KnapsackPro::Adapters::RSpecAdapter
+      end
+
+      def test_dir
+        KnapsackPro::Config::Env.test_dir || KnapsackPro::TestFilePattern.test_dir(adapter_class)
+      end
+
+      def test_file_pattern
+        KnapsackPro::TestFilePattern.call(adapter_class)
+      end
+
+      def ensure_report_dir_exists
+        FileUtils.mkdir_p(REPORT_DIR)
+      end
+
+      def remove_old_json_report
+        File.delete(REPORT_PATH) if File.exists?(REPORT_PATH)
+      end
+
+      def test_file_hash_for(test_file_path)
+        {
+          'path' => TestFileCleaner.clean(test_file_path)
+        }
+      end
+    end
+  end
+end

data/lib/knapsack_pro/test_file_pattern.rb

@@ -3,5 +3,10 @@ module KnapsackPro
     def self.call(adapter_class)
       KnapsackPro::Config::Env.test_file_pattern || adapter_class::TEST_DIR_PATTERN
     end
+
+    def self.test_dir(adapter_class)
+      test_file_pattern = call(adapter_class)
+      test_file_pattern.split('/').first.gsub(/({)/, '')
+    end
   end
 end

data/lib/knapsack_pro/tracker.rb

@@ -31,7 +31,7 @@ module KnapsackPro
 
     def current_test_path
       raise("current_test_path needs to be set by Knapsack Pro Adapter's bind method") unless @current_test_path
-      @current_test_path.sub(/^\.\//, '')
+      KnapsackPro::TestFileCleaner.clean(@current_test_path)
     end
 
     def set_prerun_tests(test_file_paths)

data/lib/knapsack_pro/version.rb

@@ -1,3 +1,3 @@
 module KnapsackPro
-  VERSION = '1.18.2'
+  VERSION = '1.19.0'
 end

data/lib/tasks/encrypted_test_file_names.rake

@@ -19,8 +19,15 @@ namespace :knapsack_pro do
                       raise('Provide adapter name like rspec, minitest, test_unit, cucumber, spinach')
                     end
 
-    test_file_pattern = KnapsackPro::TestFilePattern.call(adapter_class)
-    test_files = KnapsackPro::TestFileFinder.call(test_file_pattern)
+    test_files =
+      if adapter_class == KnapsackPro::Adapters::RSpecAdapter && KnapsackPro::Config::Env.rspec_split_by_test_examples?
+        detector = KnapsackPro::TestCaseDetectors::RSpecTestExampleDetector.new
+        detector.generate_json_report
+        detector.test_file_example_paths
+      else
+        test_file_pattern = KnapsackPro::TestFilePattern.call(adapter_class)
+        KnapsackPro::TestFileFinder.call(test_file_pattern)
+      end
 
     test_file_names = []
     test_files.each do |t|

data/lib/tasks/rspec.rake

@@ -4,4 +4,10 @@ namespace :knapsack_pro do
   task :rspec, [:rspec_args] do |_, args|
     KnapsackPro::Runners::RSpecRunner.run(args[:rspec_args])
   end
+
+  desc "Generate JSON report for test suite based on default test pattern or based on defined pattern with ENV vars"
+  task :rspec_test_example_detector do
+    detector = KnapsackPro::TestCaseDetectors::RSpecTestExampleDetector.new
+    detector.generate_json_report
+  end
 end

data/spec/knapsack_pro/adapters/rspec_adapter_spec.rb

@@ -70,37 +70,75 @@ describe KnapsackPro::Adapters::RSpecAdapter do
     describe '#bind_time_tracker' do
       let(:tracker) { instance_double(KnapsackPro::Tracker) }
       let(:logger) { instance_double(Logger) }
-      let(:test_path) { 'spec/a_spec.rb' }
       let(:global_time) { 'Global time: 01m 05s' }
-      let(:example_group) { double }
-      let(:current_example) do
-        OpenStruct.new(metadata: {
-          example_group: example_group
-        })
+
+      context 'when rspec split by test examples is disabled' do
+        let(:test_path) { 'spec/a_spec.rb' }
+        let(:example) { double }
+        let(:example_group) { double }
+        let(:current_example) do
+          OpenStruct.new(metadata: {
+            example_group: example_group
+          })
+        end
+
+        before do
+          expect(KnapsackPro::Config::Env).to receive(:rspec_split_by_test_examples?).and_return(false)
+        end
+
+        it do
+          expect(config).to receive(:around).with(:each).and_yield(example)
+          expect(config).to receive(:after).with(:suite).and_yield
+          expect(::RSpec).to receive(:configure).and_yield(config)
+
+          expect(::RSpec).to receive(:current_example).twice.and_return(current_example)
+          expect(described_class).to receive(:test_path).with(example_group).and_return(test_path)
+
+          allow(KnapsackPro).to receive(:tracker).and_return(tracker)
+          expect(tracker).to receive(:current_test_path=).with(test_path)
+          expect(tracker).to receive(:start_timer)
+
+          expect(example).to receive(:run)
+
+          expect(tracker).to receive(:stop_timer)
+
+          expect(KnapsackPro::Presenter).to receive(:global_time).and_return(global_time)
+          expect(KnapsackPro).to receive(:logger).and_return(logger)
+          expect(logger).to receive(:debug).with(global_time)
+
+          subject.bind_time_tracker
+        end
       end
-      let(:example) { double }
 
-      it do
-        expect(config).to receive(:around).with(:each).and_yield(example)
-        expect(config).to receive(:after).with(:suite).and_yield
-        expect(::RSpec).to receive(:configure).and_yield(config)
+      context 'when rspec split by test examples is enabled' do
+        let(:test_example_path) { 'spec/a_spec.rb[1:1]' }
+        let(:example) { double }
 
-        expect(::RSpec).to receive(:current_example).twice.and_return(current_example)
-        expect(described_class).to receive(:test_path).with(example_group).and_return(test_path)
+        before do
+          expect(KnapsackPro::Config::Env).to receive(:rspec_split_by_test_examples?).and_return(true)
+        end
+
+        it do
+          expect(config).to receive(:around).with(:each).and_yield(example)
+          expect(config).to receive(:after).with(:suite).and_yield
+          expect(::RSpec).to receive(:configure).and_yield(config)
 
-        allow(KnapsackPro).to receive(:tracker).and_return(tracker)
-        expect(tracker).to receive(:current_test_path=).with(test_path)
-        expect(tracker).to receive(:start_timer)
+          expect(example).to receive(:id).and_return(test_example_path)
 
-        expect(example).to receive(:run)
+          allow(KnapsackPro).to receive(:tracker).and_return(tracker)
+          expect(tracker).to receive(:current_test_path=).with(test_example_path)
+          expect(tracker).to receive(:start_timer)
 
-        expect(tracker).to receive(:stop_timer)
+          expect(example).to receive(:run)
 
-        expect(KnapsackPro::Presenter).to receive(:global_time).and_return(global_time)
-        expect(KnapsackPro).to receive(:logger).and_return(logger)
-        expect(logger).to receive(:debug).with(global_time)
+          expect(tracker).to receive(:stop_timer)
 
-        subject.bind_time_tracker
+          expect(KnapsackPro::Presenter).to receive(:global_time).and_return(global_time)
+          expect(KnapsackPro).to receive(:logger).and_return(logger)
+          expect(logger).to receive(:debug).with(global_time)
+
+          subject.bind_time_tracker
+        end
       end
     end
 

data/spec/knapsack_pro/allocator_builder_spec.rb

@@ -8,11 +8,8 @@ describe KnapsackPro::AllocatorBuilder do
     subject { allocator_builder.allocator }
 
     before do
-      test_file_pattern = double
-      expect(KnapsackPro::TestFilePattern).to receive(:call).with(adapter_class).and_return(test_file_pattern)
-
       test_files = double
-      expect(KnapsackPro::TestFileFinder).to receive(:call).with(test_file_pattern).and_return(test_files)
+      expect(allocator_builder).to receive(:test_files).and_return(test_files)
 
       repository_adapter = double
       expect(KnapsackPro::RepositoryAdapterInitiator).to receive(:call).and_return(repository_adapter)

data/spec/knapsack_pro/base_allocator_builder_spec.rb

@@ -99,4 +99,80 @@ describe KnapsackPro::BaseAllocatorBuilder do
       end
     end
   end
+
+  describe '#test_files' do
+    subject { allocator_builder.test_files }
+
+    context 'when looking for test files on disk by default' do
+      it do
+        test_file_pattern = double
+        expect(KnapsackPro::TestFilePattern).to receive(:call).with(adapter_class).and_return(test_file_pattern)
+
+        test_files = double
+        expect(KnapsackPro::TestFileFinder).to receive(:call).with(test_file_pattern).and_return(test_files)
+
+        expect(subject).to eq test_files
+      end
+    end
+
+    context 'when RSpec adapter and rspec split by test examples is enabled' do
+      let(:adapter_class) { KnapsackPro::Adapters::RSpecAdapter }
+      let(:test_files) { double(size: 1000) }
+      let(:cmd) { 'bundle exec rake knapsack_pro:rspec_test_example_detector' }
+
+      before do
+        expect(KnapsackPro::Config::Env).to receive(:rspec_split_by_test_examples?).and_return(true)
+
+        test_file_pattern = double
+        expect(KnapsackPro::TestFilePattern).to receive(:call).with(adapter_class).and_return(test_file_pattern)
+
+        expect(KnapsackPro::TestFileFinder).to receive(:call).with(test_file_pattern).and_return(test_files)
+
+        expect(Kernel).to receive(:system).with(cmd).and_return(cmd_result)
+      end
+
+      context 'when rake task to detect RSpec test examples works' do
+        let(:cmd_result) { true }
+        let(:test_file_example_paths) { double }
+        let(:logger) { instance_double(Logger) }
+
+        before do
+          rspec_test_example_detector = instance_double(KnapsackPro::TestCaseDetectors::RSpecTestExampleDetector)
+          expect(KnapsackPro::TestCaseDetectors::RSpecTestExampleDetector).to receive(:new).and_return(rspec_test_example_detector)
+          expect(rspec_test_example_detector).to receive(:test_file_example_paths).and_return(test_file_example_paths)
+
+          expect(KnapsackPro).to receive(:logger).at_least(1).and_return(logger)
+        end
+
+        context 'when up to 1000 test files detected on disk' do
+          let(:test_files) { double(size: 1000) }
+
+          it do
+            expect(logger).to receive(:warn).with("Generating RSpec test examples JSON report to prepare your test suite to be split by test examples (by individual 'it's. Thanks to that a single test file can be split across parallel CI nodes). Analyzing 1000 test files.")
+
+            expect(subject).to eq test_file_example_paths
+          end
+        end
+
+        context 'when more than 1000 test files detected on disk' do
+          let(:test_files) { double(size: 1001) }
+
+          it do
+            expect(logger).to receive(:warn).with("Generating RSpec test examples JSON report to prepare your test suite to be split by test examples (by individual 'it's. Thanks to that a single test file can be split across parallel CI nodes). Analyzing 1001 test files.")
+            expect(logger).to receive(:warn).with('You have more than 1000 test files, it may take longer to generate test examples. Please wait...')
+
+            expect(subject).to eq test_file_example_paths
+          end
+        end
+      end
+
+      context 'when rake task to detect RSpec test examples failed' do
+        let(:cmd_result) { false }
+
+        it do
+          expect { subject }.to raise_error(RuntimeError, 'Could not generate JSON report for RSpec. Rake task failed when running bundle exec rake knapsack_pro:rspec_test_example_detector')
+        end
+      end
+    end
+  end
 end

data/spec/knapsack_pro/config/env_spec.rb

@@ -609,6 +609,41 @@ describe KnapsackPro::Config::Env do
     end
   end
 
+  describe '.rspec_split_by_test_examples' do
+    subject { described_class.rspec_split_by_test_examples }
+
+    context 'when ENV exists' do
+      before { stub_const("ENV", { 'KNAPSACK_PRO_RSPEC_SPLIT_BY_TEST_EXAMPLES' => true }) }
+      it { should eq true }
+    end
+
+    context "when ENV doesn't exist" do
+      before { stub_const("ENV", {}) }
+      it { should be false }
+    end
+  end
+
+  describe '.rspec_split_by_test_examples?' do
+    subject { described_class.rspec_split_by_test_examples? }
+
+    context 'when ENV exists' do
+      context 'when KNAPSACK_PRO_RSPEC_SPLIT_BY_TEST_EXAMPLES=true' do
+        before { stub_const("ENV", { 'KNAPSACK_PRO_RSPEC_SPLIT_BY_TEST_EXAMPLES' => 'true' }) }
+        it { should be true }
+      end
+
+      context 'when KNAPSACK_PRO_RSPEC_SPLIT_BY_TEST_EXAMPLES=false' do
+        before { stub_const("ENV", { 'KNAPSACK_PRO_RSPEC_SPLIT_BY_TEST_EXAMPLES' => 'false' }) }
+        it { should be false }
+      end
+    end
+
+    context "when ENV doesn't exist" do
+      before { stub_const("ENV", {}) }
+      it { should be false }
+    end
+  end
+
   describe '.test_suite_token' do
     subject { described_class.test_suite_token }
 

data/spec/knapsack_pro/queue_allocator_builder_spec.rb

@@ -8,11 +8,8 @@ describe KnapsackPro::QueueAllocatorBuilder do
     subject { allocator_builder.allocator }
 
     before do
-      test_file_pattern = double
-      expect(KnapsackPro::TestFilePattern).to receive(:call).with(adapter_class).and_return(test_file_pattern)
-
       test_files = double
-      expect(KnapsackPro::TestFileFinder).to receive(:call).with(test_file_pattern).and_return(test_files)
+      expect(allocator_builder).to receive(:test_files).and_return(test_files)
 
       repository_adapter = double
       expect(KnapsackPro::RepositoryAdapterInitiator).to receive(:call).and_return(repository_adapter)

data/spec/knapsack_pro/runners/queue/rspec_runner_spec.rb

@@ -55,7 +55,7 @@ describe KnapsackPro::Runners::Queue::RSpecRunner do
         end
       end
 
-      context 'when format param is provided' do
+      context 'when format param is provided as --format' do
         let(:args) { '--format documentation' }
 
         it 'uses provided format param instead of default formatter progress' do
@@ -79,6 +79,31 @@ describe KnapsackPro::Runners::Queue::RSpecRunner do
           subject
         end
       end
+
+      context 'when format param is provided as -f' do
+        let(:args) { '-f d' }
+
+        it 'uses provided format param instead of default formatter progress' do
+          expected_exitstatus = 0
+          expected_accumulator = {
+            status: :completed,
+            exitstatus: expected_exitstatus
+          }
+          accumulator = {
+            status: :next,
+            runner: runner,
+            can_initialize_queue: true,
+            args: ['-f', 'd', '--format', 'KnapsackPro::Formatters::RSpecQueueSummaryFormatter', '--default-path', 'fake-test-dir'],
+            exitstatus: 0,
+            all_test_file_paths: [],
+          }
+          expect(described_class).to receive(:run_tests).with(accumulator).and_return(expected_accumulator)
+
+          expect(Kernel).to receive(:exit).with(expected_exitstatus)
+
+          subject
+        end
+      end
     end
 
     context 'when args not provided' do

data/spec/knapsack_pro/test_case_detectors/rspec_test_example_detector_spec.rb

@@ -0,0 +1,89 @@
+describe KnapsackPro::TestCaseDetectors::RSpecTestExampleDetector do
+  let(:report_dir) { 'tmp/knapsack_pro/test_case_detectors/rspec' }
+  let(:report_path) { 'tmp/knapsack_pro/test_case_detectors/rspec/rspec_dry_run_json_report.json' }
+
+  describe '#generate_json_report' do
+    subject { described_class.new.generate_json_report }
+
+    before do
+      expect(FileUtils).to receive(:mkdir_p).with(report_dir)
+
+      expect(File).to receive(:exists?).with(report_path).and_return(true)
+      expect(File).to receive(:delete).with(report_path)
+
+      test_file_pattern = double
+      adapter_class = KnapsackPro::Adapters::RSpecAdapter
+      expect(KnapsackPro::TestFilePattern).to receive(:call).with(adapter_class).and_return(test_file_pattern)
+
+      test_file_paths = [
+        { 'path' => 'spec/a_spec.rb' },
+        { 'path' => 'spec/b_spec.rb' },
+      ]
+      expect(KnapsackPro::TestFileFinder).to receive(:call).with(test_file_pattern).and_return(test_file_paths)
+
+      test_dir = 'spec'
+      expect(KnapsackPro::Config::Env).to receive(:test_dir).and_return(nil)
+      expect(KnapsackPro::TestFilePattern).to receive(:test_dir).with(adapter_class).and_return(test_dir)
+
+      options = double
+      expect(RSpec::Core::ConfigurationOptions).to receive(:new).with([
+        '--dry-run',
+        '--format', 'json',
+        '--out', report_path,
+        '--default-path', test_dir,
+        'spec/a_spec.rb', 'spec/b_spec.rb',
+      ]).and_return(options)
+
+      rspec_core_runner = double
+      expect(RSpec::Core::Runner).to receive(:new).with(options).and_return(rspec_core_runner)
+      expect(rspec_core_runner).to receive(:run).with($stderr, $stdout).and_return(exit_code)
+    end
+
+    context 'when exit code from RSpec::Core::Runner is 0' do
+      let(:exit_code) { 0 }
+
+      it do
+        expect(subject).to be_nil
+      end
+    end
+
+    context 'when exit code from RSpec::Core::Runner is 1' do
+      let(:exit_code) { 1 }
+
+      it do
+        expect { subject }.to raise_error(RuntimeError, 'There was problem to generate test examples for test suite')
+      end
+    end
+  end
+
+  describe '#test_file_example_paths' do
+    subject { described_class.new.test_file_example_paths }
+
+    context 'when json report exists' do
+      it do
+        expect(File).to receive(:exists?).with(report_path).and_return(true)
+
+        json_file = {
+          'examples' => [
+            { id: './spec/a_spec.rb[1:1]' },
+            { id: './spec/a_spec.rb[1:2]' },
+          ]
+        }.to_json
+        expect(File).to receive(:read).with(report_path).and_return(json_file)
+
+        expect(subject).to eq([
+          { 'path' => 'spec/a_spec.rb[1:1]' },
+          { 'path' => 'spec/a_spec.rb[1:2]' },
+        ])
+      end
+    end
+
+    context 'when json report does not exist' do
+      it do
+        expect(File).to receive(:exists?).with(report_path).and_return(false)
+
+        expect { subject }.to raise_error(RuntimeError, 'No report found at tmp/knapsack_pro/test_case_detectors/rspec/rspec_dry_run_json_report.json')
+      end
+    end
+  end
+end

data/spec/knapsack_pro/test_file_pattern_spec.rb

@@ -20,4 +20,30 @@ describe KnapsackPro::TestFilePattern do
       it { should eq 'test/**{,/*/**}/*_test.rb' }
     end
   end
+
+  describe '#test_dir' do
+    let(:adapter_class) { KnapsackPro::Adapters::BaseAdapter }
+
+    subject { described_class.test_dir(adapter_class) }
+
+    before do
+      expect(described_class).to receive(:call).with(adapter_class).and_return(test_file_pattern)
+    end
+
+    context 'when default test file pattern' do
+      let(:test_file_pattern) { 'spec/**{,/*/**}/*_spec.rb' }
+
+      it 'extracts test directory from the pattern' do
+        expect(subject).to eq 'spec'
+      end
+    end
+
+    context 'when test file pattern has multiple patterns' do
+      let(:test_file_pattern) { '{spec/*_spec.rb,spec2/controllers/**/*_spec.rb}' }
+
+      it 'extracts test directory from the first pattern' do
+        expect(subject).to eq 'spec'
+      end
+    end
+  end
 end

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: knapsack_pro
 version: !ruby/object:Gem::Version
-  version: 1.18.2
+  version: 1.19.0
 platform: ruby
 authors:
 - ArturT
 autorequire: 
 bindir: bin
 cert_chain: []
-date: 2020-03-18 00:00:00.000000000 Z
+date: 2020-04-05 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: rake
@@ -281,6 +281,7 @@ files:
 - lib/knapsack_pro/runners/spinach_runner.rb
 - lib/knapsack_pro/runners/test_unit_runner.rb
 - lib/knapsack_pro/task_loader.rb
+- lib/knapsack_pro/test_case_detectors/rspec_test_example_detector.rb
 - lib/knapsack_pro/test_file_cleaner.rb
 - lib/knapsack_pro/test_file_finder.rb
 - lib/knapsack_pro/test_file_pattern.rb
@@ -364,6 +365,7 @@ files:
 - spec/knapsack_pro/runners/spinach_runner_spec.rb
 - spec/knapsack_pro/runners/test_unit_runner_spec.rb
 - spec/knapsack_pro/task_loader_spec.rb
+- spec/knapsack_pro/test_case_detectors/rspec_test_example_detector_spec.rb
 - spec/knapsack_pro/test_file_cleaner_spec.rb
 - spec/knapsack_pro/test_file_finder_spec.rb
 - spec/knapsack_pro/test_file_pattern_spec.rb
@@ -477,6 +479,7 @@ test_files:
 - spec/knapsack_pro/runners/spinach_runner_spec.rb
 - spec/knapsack_pro/runners/test_unit_runner_spec.rb
 - spec/knapsack_pro/task_loader_spec.rb
+- spec/knapsack_pro/test_case_detectors/rspec_test_example_detector_spec.rb
 - spec/knapsack_pro/test_file_cleaner_spec.rb
 - spec/knapsack_pro/test_file_finder_spec.rb
 - spec/knapsack_pro/test_file_pattern_spec.rb