knapsack_pro 3.3.1 → 3.4.1

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 11036d33c04d07966ab4acadc55b1fb20a885b0e9b0d78fc8cf03fe798b7c3bf
-  data.tar.gz: 8ddfce179b4f7fce9bd28ae965bf15bec132b26b217326d5ce156e94e21cb145
+  metadata.gz: 6c1d109eab55a41f0ad16de94f9675299a8df07eca2c1bca2edf92c588fe2adb
+  data.tar.gz: 05d6fec7a5c6a8d594b78e80eecff4bad0282e4f10bb5e772eae25d659c39bb3
 SHA512:
-  metadata.gz: 4c27910d9850ad2d229864dac8b07cb17d9d23a9bd2d14aa05052b6e95a3e218667b32fb40f711324651d2bcd6ee8d180fc7d743ba5cd3b990908ab05d5c57de
-  data.tar.gz: a868ac3c2ef9c1fdefc655b2dbb59c8034cc21101717be46400fdeec087bd99c62c43c5357fbba1eccfe8a8cc23b032ee1e9e8eb14e75304357ccbe6a179a32b
+  metadata.gz: 01bd206ae95503b54abad29acc5bd29e658b5ee9b8fe0c7d31b1a78e8aee0b364a0d74f619306633e599f004a2e2fdb8c8b188b389e5e26255838c9b17b45c33
+  data.tar.gz: 7067285b1dc323859d8547df7c6d3edcb7c1cfa2f14251be9a857d957212d08666fdd82ae1426c7a614d93692b86131283606e6b57e73f9a1064b47b4af9355a

data/CHANGELOG.md

@@ -1,5 +1,23 @@
 # Change Log
 
+### 3.4.1
+
+* Improve the RSpec Queue Mode runner log output (add seed)
+
+  https://github.com/KnapsackPro/knapsack_pro-ruby/pull/178
+
+https://github.com/KnapsackPro/knapsack_pro-ruby/compare/v3.4.0...v3.4.1
+
+### 3.4.0
+
+* Update documentation and code because the encryption feature does not work with the RSpec split by examples feature
+
+    Update docs: https://github.com/KnapsackPro/knapsack_pro-ruby/pull/176
+
+    Update code: https://github.com/KnapsackPro/knapsack_pro-ruby/pull/177
+
+https://github.com/KnapsackPro/knapsack_pro-ruby/compare/v3.3.1...v3.4.0
+
 ### 3.3.1
 
 * Skip loading a test file path for Minitest in Queue Mode when it does not exist on the disk

data/README.md

@@ -97,7 +97,6 @@ The knapsack_pro has also [queue mode](#queue-mode) to get an optimal test suite
   - [Test file names encryption](#test-file-names-encryption)
     - [How to enable test file names encryption?](#how-to-enable-test-file-names-encryption)
     - [How to debug test file names?](#how-to-debug-test-file-names)
-      - [Preview encrypted RSpec test example paths?](#preview-encrypted-rspec-test-example-paths)
     - [How to enable branch names encryption?](#how-to-enable-branch-names-encryption)
     - [How to debug branch names?](#how-to-debug-branch-names)
   - [Supported CI providers](#supported-ci-providers)
@@ -719,72 +718,61 @@ This is optional way of using knapsack_pro when you don't want to add it to `Gem
 
 ### Test file names encryption
 
-knapsack_pro gem collects information about you test file names and time execution. Those data are stored on KnapsackPro.com server.
-If your test file names or branch names are sensitive data then you can encrypt the names before sending them to KnapsackPro.com API.
+The knapsack_pro gem collects information about your test file names, branch names, and execution times. The data are stored on the KnapsackPro.com server.
+If your test file names or branch names are considered sensitive data, then you can encrypt the data before sending it to the KnapsackPro.com API.
 
-By default, encryption is disabled because knapsack_pro can use your test files names to prepare better test suite split when the time execution data are not yet collected on KnapsackPro.com server.
-When you will enable test file names encryption then your first test suite split may be less optimal than it could be.
+Encryption is disabled by default because the knapsack_pro gem uses your test file names to prepare a better test suite split when the execution time data are not collected on the KnapsackPro.com server yet.
+When you enable the encryption, then your first test suite split may not be optimal.
 
-Each test file name is generated with `Digest::SHA2.hexdigest` method and 64 chars salt.
+Each test file name is generated with the `Digest::SHA2.hexdigest` method and 64 chars salt.
 
-Before you enable test file encryption please ensure you are using fresh API key. You should not use the same API key for encrypted and non encrypted test suite.
-You can generate API key for your test suite in [your dashboard](https://knapsackpro.com).
+Before you enable encryption, please ensure you are using a fresh API token. Do not use the same API token for the encrypted and non-encrypted test suite. You can generate an API token for your test suite in the [user dashboard](https://knapsackpro.com/dashboard).
 
-Next step is to generate salt which will be used to encrypt test files or branch names.
+The next step is to generate a salt to encrypt test files or branch names with it.
 
 ```bash
 bundle exec rake knapsack_pro:salt
 ```
 
-Add to your CI server generated environment variable `KNAPSACK_PRO_SALT`.
+Add the salt to your CI server as the `KNAPSACK_PRO_SALT` environment variable.
 
 #### How to enable test file names encryption?
 
-You need to add environment variable `KNAPSACK_PRO_TEST_FILES_ENCRYPTED=true` to your CI server.
+You need to add the `KNAPSACK_PRO_TEST_FILES_ENCRYPTED=true` environment variable to your CI server.
 
 #### How to debug test file names?
 
-If you need to check what is the encryption hash for particular test file you can check that with the rake task:
+If you need to check what is the hash value for a particular test file you can check that with the rake task:
 
 ```bash
-KNAPSACK_PRO_SALT=xxx bundle exec rake knapsack_pro:encrypted_test_file_names[rspec]
+KNAPSACK_PRO_SALT=xxx bundle exec rake "knapsack_pro:encrypted_test_file_names[rspec]"
 ```
 
-You can pass the name of test runner like `rspec`, `minitest`, `test_unit`, `cucumber`, `spinach` as argument to rake task.
-
-##### Preview encrypted RSpec test example paths?
-
-If you split RSpec tests by test examples (by individual `it`) you can preview encrypted test example paths this way:
-
-```bash
-KNAPSACK_PRO_RSPEC_SPLIT_BY_TEST_EXAMPLES=true \
-KNAPSACK_PRO_SALT=xxx \
-bundle exec rake knapsack_pro:encrypted_test_file_names[rspec]
-```
+You can pass the name of a test runner like `rspec`, `minitest`, `test_unit`, `cucumber`, `spinach` as an argument to the rake task.
 
 #### How to enable branch names encryption?
 
-You need to add environment variable `KNAPSACK_PRO_BRANCH_ENCRYPTED=true` to your CI server.
+You need to add the `KNAPSACK_PRO_BRANCH_ENCRYPTED=true` environment variable to your CI server.
 
-Note: there are a few branch names that won't be encrypted because we use them as fallback branches on Knapsack Pro API side to determine time execution for test files during split for newly created branches.
+Note a few branch names won't be encrypted because we use them as fallback branches on the Knapsack Pro API side to determine time execution for test files during split for newly created branches.
 
 * develop
 * development
 * dev
 * master
 * staging
-* [see full list of encryption excluded branch names](https://github.com/KnapsackPro/knapsack_pro-ruby/blob/master/lib/knapsack_pro/crypto/branch_encryptor.rb#L4)
+* [see the full list of excluded branch names from encryption](https://github.com/KnapsackPro/knapsack_pro-ruby/blob/master/lib/knapsack_pro/crypto/branch_encryptor.rb#L4)
 
 #### How to debug branch names?
 
-If you need to check what is the encryption hash for particular branch then use the rake task:
+If you need to check what is the hash for a particular branch, then use the rake task:
 
 ```bash
 # show all local branches and respective hashes
 $ KNAPSACK_PRO_SALT=xxx bundle exec rake knapsack_pro:encrypted_branch_names
 
-# show hash for branch provided as argument to rake task
-$ KNAPSACK_PRO_SALT=xxx bundle exec rake knapsack_pro:encrypted_branch_names[not-encrypted-branch-name]
+# show the hash for a branch name passed as an argument to the rake task
+$ KNAPSACK_PRO_SALT=xxx bundle exec rake "knapsack_pro:encrypted_branch_names[not-encrypted-branch-name]"
 ```
 
 ### Supported CI providers

data/lib/knapsack_pro/adapters/rspec_adapter.rb

@@ -12,16 +12,15 @@ module KnapsackPro
       end
 
       def self.has_tag_option?(cli_args)
-        # use start_with? because user can define tag option in a few ways:
-        # -t mytag
-        # -tmytag
-        # --tag mytag
-        # --tag=mytag
-        cli_args.any? { |arg| arg.start_with?('-t') || arg.start_with?('--tag') }
+        !!parsed_options(cli_args)&.[](:inclusion_filter)
       end
 
       def self.has_format_option?(cli_args)
-        cli_args.any? { |arg| arg.start_with?('-f') || arg.start_with?('--format') }
+        !!parsed_options(cli_args)&.[](:formatters)
+      end
+
+      def self.order_option(cli_args)
+        parsed_options(cli_args)&.[](:order)
       end
 
       def self.test_path(example)
@@ -107,6 +106,12 @@ module KnapsackPro
       def self.rspec_configuration
         ::RSpec.configuration
       end
+
+      def self.parsed_options(cli_args)
+        ::RSpec::Core::Parser.parse(cli_args)
+      rescue SystemExit
+        nil
+      end
     end
 
     # This is added to provide backwards compatibility

data/lib/knapsack_pro/runners/queue/rspec_runner.rb

@@ -2,6 +2,8 @@ module KnapsackPro
   module Runners
     module Queue
       class RSpecRunner < BaseRunner
+        @@used_seed = nil
+
         def self.run(args)
           require 'rspec/core'
           require_relative '../../formatters/rspec_queue_summary_formatter'
@@ -11,7 +13,6 @@ module KnapsackPro
           ENV['KNAPSACK_PRO_QUEUE_RECORDING_ENABLED'] = 'true'
           ENV['KNAPSACK_PRO_QUEUE_ID'] = KnapsackPro::Config::EnvGenerator.set_queue_id
 
-          adapter_class = KnapsackPro::Adapters::RSpecAdapter
           KnapsackPro::Config::Env.set_test_runner_adapter(adapter_class)
           runner = new(adapter_class)
 
@@ -61,6 +62,8 @@ module KnapsackPro
               KnapsackPro::Formatters::RSpecQueueSummaryFormatter.print_summary
               KnapsackPro::Formatters::RSpecQueueProfileFormatterExtension.print_summary
 
+              args += ['--seed', @@used_seed] if @@used_seed
+
               log_rspec_command(args, all_test_file_paths, :end_of_queue)
             end
 
@@ -82,12 +85,15 @@ module KnapsackPro
             all_test_file_paths += test_file_paths
             cli_args = args + test_file_paths
 
-            log_rspec_command(args, test_file_paths, :subset_queue)
-
             options = ::RSpec::Core::ConfigurationOptions.new(cli_args)
-            exit_code = ::RSpec::Core::Runner.new(options).run($stderr, $stdout)
+            rspec_runner = ::RSpec::Core::Runner.new(options)
+
+            exit_code = rspec_runner.run($stderr, $stdout)
             exitstatus = exit_code if exit_code != 0
 
+            printable_args = args_with_seed_option_added_when_viable(args, rspec_runner)
+            log_rspec_command(printable_args, test_file_paths, :subset_queue)
+
             rspec_clear_examples
 
             KnapsackPro::Hooks::Queue.call_after_subset_queue
@@ -107,19 +113,22 @@ module KnapsackPro
 
         private
 
+        def self.adapter_class
+          KnapsackPro::Adapters::RSpecAdapter
+        end
+
         def self.log_rspec_command(cli_args, test_file_paths, type)
           case type
           when :subset_queue
-            KnapsackPro.logger.info("To retry the last batch of tests fetched from the API Queue, please run the following command on your machine. (If you use the `-- order random` option, remember to add correct `--seed 123` that you can find at the end of the RSpec output.)")
+            KnapsackPro.logger.info("To retry the last batch of tests fetched from the API Queue, please run the following command on your machine:")
           when :end_of_queue
             KnapsackPro.logger.info("To retry all the tests assigned to this CI node, please run the following command on your machine:")
           end
 
-          stringify_cli_args = cli_args.join(' ')
-          stringify_cli_args.slice!("--format #{KnapsackPro::Formatters::RSpecQueueSummaryFormatter}")
+          stringified_cli_args = cli_args.join(' ').sub(" --format #{KnapsackPro::Formatters::RSpecQueueSummaryFormatter}", '')
 
           KnapsackPro.logger.info(
-            "bundle exec rspec #{stringify_cli_args} " +
+            "bundle exec rspec #{stringified_cli_args} " +
             KnapsackPro::TestFilePresenter.stringify_paths(test_file_paths)
           )
         end
@@ -151,6 +160,24 @@ module KnapsackPro
             ::RSpec.configuration.reset_filters
           end
         end
+
+        def self.args_with_seed_option_added_when_viable(args, rspec_runner)
+          order_option = adapter_class.order_option(args)
+
+          if order_option
+            # Don't add the seed option for order other than random, e.g. `defined`
+            return args unless order_option.include?('rand')
+            # Don't add the seed option if the seed is already set in args, e.g. `rand:12345`
+            return args if order_option.to_s.split(':')[1]
+          end
+
+          # Don't add the seed option if the seed was not used (i.e. a different order is being used, e.g. `defined`)
+          return args unless rspec_runner.configuration.seed_used?
+
+          @@used_seed = rspec_runner.configuration.seed.to_s
+
+          args + ['--seed', @@used_seed]
+        end
       end
     end
   end

data/lib/knapsack_pro/version.rb

@@ -1,3 +1,3 @@
 module KnapsackPro
-  VERSION = '3.3.1'
+  VERSION = '3.4.1'
 end

data/lib/tasks/encrypted_test_file_names.rake

@@ -19,15 +19,8 @@ namespace :knapsack_pro do
                       raise('Provide adapter name like rspec, minitest, test_unit, cucumber, spinach')
                     end
 
-    test_files =
-      if adapter_class == KnapsackPro::Adapters::RSpecAdapter && KnapsackPro::Config::Env.rspec_split_by_test_examples?
-        detector = KnapsackPro::TestCaseDetectors::RSpecTestExampleDetector.new
-        detector.generate_json_report
-        detector.test_file_example_paths
-      else
-        test_file_pattern = KnapsackPro::TestFilePattern.call(adapter_class)
-        KnapsackPro::TestFileFinder.call(test_file_pattern)
-      end
+    test_file_pattern = KnapsackPro::TestFilePattern.call(adapter_class)
+    test_files = KnapsackPro::TestFileFinder.call(test_file_pattern)
 
     test_file_names = []
     test_files.each do |t|

data/spec/knapsack_pro/adapters/rspec_adapter_spec.rb

@@ -110,6 +110,58 @@ describe KnapsackPro::Adapters::RSpecAdapter do
     end
   end
 
+  describe '.order_option' do
+    subject { described_class.order_option(cli_args) }
+
+    context "when order is 'defined'" do
+      let(:cli_args) { ['--order', 'defined'] }
+
+      it { expect(subject).to eq 'defined' }
+    end
+
+    context "when order is 'recently-modified'" do
+      let(:cli_args) { ['--order', 'recently-modified'] }
+
+      it { expect(subject).to eq 'recently-modified' }
+    end
+
+    context "when order is 'rand'" do
+      let(:cli_args) { ['--order', 'rand'] }
+
+      it { expect(subject).to eq 'rand' }
+
+      context 'with the seed' do
+        let(:cli_args) { ['--order', 'rand:123456'] }
+
+        it { expect(subject).to eq 'rand:123456' }
+      end
+    end
+
+    context "when order is 'random'" do
+      let(:cli_args) { ['--order', 'random'] }
+
+      it { expect(subject).to eq 'random' }
+
+      context 'with the seed' do
+        let(:cli_args) { ['--order', 'random:123456'] }
+
+        it { expect(subject).to eq 'random:123456' }
+      end
+    end
+
+    context 'when some custom order is specified' do
+      let(:cli_args) { ['--order', 'some-custom-order'] }
+
+      it { expect(subject).to eq 'some-custom-order' }
+    end
+
+    context "when the seed is given with the --seed command" do
+      let(:cli_args) { ['--seed', '123456'] }
+
+      it { expect(subject).to eq 'rand:123456' }
+    end
+  end
+
   describe '.test_path' do
     let(:example_group) do
       {

data/spec/knapsack_pro/runners/queue/rspec_runner_spec.rb

@@ -177,7 +177,7 @@ describe KnapsackPro::Runners::Queue::RSpecRunner do
   describe '.run_tests' do
     let(:runner) { instance_double(described_class) }
     let(:can_initialize_queue) { double(:can_initialize_queue) }
-    let(:args) { ['--example-arg', 'example-value', '--default-path', 'fake-test-dir'] }
+    let(:args) { ['--no-color', '--default-path', 'fake-test-dir'] }
     let(:exitstatus) { double }
     let(:all_test_file_paths) { [] }
     let(:accumulator) do
@@ -198,6 +198,8 @@ describe KnapsackPro::Runners::Queue::RSpecRunner do
 
     context 'when test files exist' do
       let(:test_file_paths) { ['a_spec.rb', 'b_spec.rb'] }
+      let(:logger) { double }
+      let(:rspec_seed) { 7771 }
 
       before do
         subset_queue_id = 'fake-subset-queue-id'
@@ -212,7 +214,7 @@ describe KnapsackPro::Runners::Queue::RSpecRunner do
 
         options = double
         expect(RSpec::Core::ConfigurationOptions).to receive(:new).with([
-          '--example-arg', 'example-value',
+          '--no-color',
           '--default-path', 'fake-test-dir',
           'a_spec.rb', 'b_spec.rb',
         ]).and_return(options)
@@ -226,6 +228,16 @@ describe KnapsackPro::Runners::Queue::RSpecRunner do
         expect(KnapsackPro::Hooks::Queue).to receive(:call_after_subset_queue)
 
         expect(KnapsackPro::Report).to receive(:save_subset_queue_to_file)
+
+        configuration = double
+        expect(rspec_core_runner).to receive(:configuration).twice.and_return(configuration)
+        expect(configuration).to receive(:seed_used?).and_return(true)
+        expect(configuration).to receive(:seed).and_return(rspec_seed)
+
+        expect(KnapsackPro).to receive(:logger).twice.and_return(logger)
+        expect(logger).to receive(:info)
+          .with("To retry the last batch of tests fetched from the API Queue, please run the following command on your machine:")
+        expect(logger).to receive(:info).with(/#{args.join(' ')} --seed #{rspec_seed}/)
       end
 
       context 'when exit code is zero' do
@@ -264,8 +276,13 @@ describe KnapsackPro::Runners::Queue::RSpecRunner do
 
       context 'when all_test_file_paths exist' do
         let(:all_test_file_paths) { ['a_spec.rb'] }
+        let(:logger) { double }
+
+        before do
+          described_class.class_variable_set(:@@used_seed, used_seed)
+
+          expect(KnapsackPro).to receive(:logger).twice.and_return(logger)
 
-        it do
           expect(KnapsackPro::Adapters::RSpecAdapter).to receive(:verify_bind_method_called)
 
           expect(KnapsackPro::Formatters::RSpecQueueSummaryFormatter).to receive(:print_summary)
@@ -274,10 +291,33 @@ describe KnapsackPro::Runners::Queue::RSpecRunner do
           expect(KnapsackPro::Hooks::Queue).to receive(:call_after_queue)
           expect(KnapsackPro::Report).to receive(:save_node_queue_to_api)
 
-          expect(subject).to eq({
-            status: :completed,
-            exitstatus: exitstatus,
-          })
+          expect(logger).to receive(:info)
+            .with('To retry all the tests assigned to this CI node, please run the following command on your machine:')
+          expect(logger).to receive(:info).with(logged_rspec_command_matcher)
+        end
+
+        context 'when @@used_seed has been set' do
+          let(:used_seed) { '8333' }
+          let(:logged_rspec_command_matcher) { /#{args.join(' ')} --seed #{used_seed} \"a_spec.rb"/ }
+
+          it do
+            expect(subject).to eq({
+              status: :completed,
+              exitstatus: exitstatus,
+            })
+          end
+        end
+
+        context 'when @@used_seed has not been set' do
+          let(:used_seed) { nil }
+          let(:logged_rspec_command_matcher) { /#{args.join(' ')} \"a_spec.rb"/ }
+
+          it do
+            expect(subject).to eq({
+              status: :completed,
+              exitstatus: exitstatus,
+            })
+          end
         end
       end
 
@@ -287,6 +327,7 @@ describe KnapsackPro::Runners::Queue::RSpecRunner do
         it do
           expect(KnapsackPro::Hooks::Queue).to receive(:call_after_queue)
           expect(KnapsackPro::Report).to receive(:save_node_queue_to_api)
+          expect(KnapsackPro).to_not receive(:logger)
 
           expect(subject).to eq({
             status: :completed,

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: knapsack_pro
 version: !ruby/object:Gem::Version
-  version: 3.3.1
+  version: 3.4.1
 platform: ruby
 authors:
 - ArturT
-autorequire:
+autorequire: 
 bindir: bin
 cert_chain: []
-date: 2022-07-19 00:00:00.000000000 Z
+date: 2022-10-13 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: rake
@@ -390,7 +390,7 @@ metadata:
   documentation_uri: https://docs.knapsackpro.com/integration/
   homepage_uri: https://knapsackpro.com
   source_code_uri: https://github.com/KnapsackPro/knapsack_pro-ruby
-post_install_message:
+post_install_message: 
 rdoc_options: []
 require_paths:
 - lib
@@ -405,8 +405,8 @@ required_rubygems_version: !ruby/object:Gem::Requirement
     - !ruby/object:Gem::Version
       version: '0'
 requirements: []
-rubygems_version: 3.3.7
-signing_key:
+rubygems_version: 3.1.6
+signing_key: 
 specification_version: 4
 summary: Knapsack Pro splits tests across parallel CI nodes and ensures each parallel
   job finish work at a similar time.