knapsack_pro 1.8.0 → 1.9.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 4e439d9522688ef5f7ed0774c471fb6bffed329affd7e01c43cabce63709a348
-  data.tar.gz: b6838efdb36311c9f5d1c96f1fe18479bc33625703f3e6663a0fa754a25da97c
+  metadata.gz: 5d7bf901385b7f5c6010f0ec20241cf9d74e4c954cf69ae75ca7f5ac1d4c549b
+  data.tar.gz: 0a24d55e70d9055e70ae052ef152e0cf1a85716d4d93025e800a5b1769853f0d
 SHA512:
-  metadata.gz: 45cd07d1faa6ba56d558d853c9068701e9bef059d6f36f0aefe40b7267aa287f6f976f5d276d0791da26fdf38347bfda4209aa9c9548de66a426fa4654cdcc54
-  data.tar.gz: 9fa2019aa18ef91a7a10b658501b89ae752395973f1d01d5e9ecbaedca08d095153d03f659132e8e4a8775c21b1327e57b2191b4f45b3a042c3c2702bcd3a26c
+  metadata.gz: '0482cd76a5a08df97ce43abf6d89763f1f8fb83cd3aca349c09de993513a87c623bfe0abb31a05d9dd5701101ebf19789e0a41145b3f2e435567a244a462a992'
+  data.tar.gz: 0b958c6d1bf0d8f9cdfcb4a97b5f48a0ad3b046acdc6c6b7198e0a7865f37928fe8f2d4d62acb0306489d5779bd07b613cb5774aca426dd774b81fd6cf431435

data/CHANGELOG.md

@@ -1,5 +1,13 @@
 # Change Log
 
+### 1.9.0
+
+* Reduce data transfer and speed up usage of Knapsack Pro API for Queue Mode
+
+    https://github.com/KnapsackPro/knapsack_pro-ruby/pull/81
+
+https://github.com/KnapsackPro/knapsack_pro-ruby/compare/v1.8.0...v1.9.0
+
 ### 1.8.0
 
 * Run Fallback Mode when `OpenSSL::SSL::SSLError` certificate verify failed for API

data/README.md

@@ -165,6 +165,8 @@ You can see list of questions for common problems and tips in below [Table of Co
     - [How to set `before(:suite)` and `after(:suite)` RSpec hooks in Queue Mode (Percy.io example)?](#how-to-set-beforesuite-and-aftersuite-rspec-hooks-in-queue-mode-percyio-example)
     - [How to call `before(:suite)` and `after(:suite)` RSpec hooks only once in Queue Mode?](#how-to-call-beforesuite-and-aftersuite-rspec-hooks-only-once-in-queue-mode)
     - [How to run knapsack_pro with parallel_tests gem?](#how-to-run-knapsack_pro-with-parallel_tests-gem)
+      - [parallel_tests with knapsack_pro on parallel CI nodes](#parallel_tests-with-knapsack_pro-on-parallel-ci-nodes)
+      - [parallel_tests with knapsack_pro on single CI machine](#parallel_tests-with-knapsack_pro-on-single-ci-machine)
     - [How to retry failed tests (flaky tests)?](#how-to-retry-failed-tests-flaky-tests)
     - [How can I run tests from multiple directories?](#how-can-i-run-tests-from-multiple-directories)
     - [Why I don't see all test files being recorded in user dashboard](#why-i-dont-see-all-test-files-being-recorded-in-user-dashboard)
@@ -2116,6 +2118,8 @@ end
 
 #### How to run knapsack_pro with parallel_tests gem?
 
+##### parallel_tests with knapsack_pro on parallel CI nodes
+
 You can run knapsack_pro with [parallel_tests](https://github.com/grosser/parallel_tests) gem to run multiple concurrent knapsack_pro commands per CI node.
 
 Let's consider this example. We have 2 CI node. On each CI node we want to run 2 concurrent knapsack_pro commands by parallel_tests gem (`PARALLEL_TESTS_CONCURRENCY=2`).
@@ -2199,6 +2203,49 @@ To ensure everything works you can check output for each CI node.
     (tests output here)
     ```
 
+##### parallel_tests with knapsack_pro on single CI machine
+
+This tip is only relevant to you if you cannot use multiple parallel CI nodes on your CI provider. In such case, you can run your tests on a single CI machine with knapsack_pro Queue Mode in order to auto balance execution of tests and thanks to this better utilize CI machine resources.
+
+You can run knapsack_pro with [parallel_tests](https://github.com/grosser/parallel_tests) gem to run multiple concurrent knapsack_pro commands on single CI node.
+
+Create in your project directory an executable file `bin/parallel_tests_knapsack_pro_single_machine`:
+
+```bash
+#!/bin/bash
+# bin/parallel_tests_knapsack_pro_single_machine
+
+export KNAPSACK_PRO_CI_NODE_TOTAL=$PARALLEL_TESTS_CONCURRENCY
+
+if [ "$TEST_ENV_NUMBER" == "" ]; then
+  export KNAPSACK_PRO_CI_NODE_INDEX=0
+else
+  export KNAPSACK_PRO_CI_NODE_INDEX=$(( $TEST_ENV_NUMBER - 1 ))
+fi
+
+echo KNAPSACK_PRO_CI_NODE_TOTAL=$KNAPSACK_PRO_CI_NODE_TOTAL KNAPSACK_PRO_CI_NODE_INDEX=$KNAPSACK_PRO_CI_NODE_INDEX PARALLEL_TESTS_CONCURRENCY=$PARALLEL_TESTS_CONCURRENCY
+
+bundle exec rake knapsack_pro:queue:rspec
+```
+
+Then you need another script `bin/parallel_tests_knapsack_pro_single_machine_run` to run above script with `parallel_tests`:
+
+```bash
+#!/bin/bash
+# bin/parallel_tests_knapsack_pro_single_machine_run
+
+export PARALLEL_TESTS_CONCURRENCY=2;
+
+RAILS_ENV=test \
+  KNAPSACK_PRO_TEST_SUITE_TOKEN_RSPEC=xxx \
+  KNAPSACK_PRO_REPOSITORY_ADAPTER=git \
+  KNAPSACK_PRO_PROJECT_DIR=/home/user/rails-app-repo \
+  PARALLEL_TESTS_CONCURRENCY=$PARALLEL_TESTS_CONCURRENCY \
+  bundle exec parallel_test -n $PARALLEL_TESTS_CONCURRENCY -e './bin/parallel_tests_knapsack_pro_single_machine'
+```
+
+Now you can run `bin/parallel_tests_knapsack_pro_single_machine_run` and it will execute 2 parallel processes with `parallel_tests`. Each process will run knapsack_pro Queue Mode to autobalance test files distribution across the processes.
+
 #### How to retry failed tests (flaky tests)?
 
 Flaky (nondeterministic) tests, are tests that exhibit both a passing and a failing result with the same code.

data/lib/knapsack_pro/client/api/v1/queues.rb

@@ -5,19 +5,26 @@ module KnapsackPro
         class Queues < Base
           class << self
             def queue(args)
+              request_hash = {
+                :fixed_queue_split => KnapsackPro::Config::Env.fixed_queue_split,
+                :can_initialize_queue => args.fetch(:can_initialize_queue),
+                :commit_hash => args.fetch(:commit_hash),
+                :branch => args.fetch(:branch),
+                :node_total => args.fetch(:node_total),
+                :node_index => args.fetch(:node_index),
+                :node_build_id => KnapsackPro::Config::Env.ci_node_build_id,
+              }
+
+              if request_hash[:can_initialize_queue]
+                request_hash.merge!({
+                  :test_files => args.fetch(:test_files)
+                })
+              end
+
               action_class.new(
                 endpoint_path: '/v1/queues/queue',
                 http_method: :post,
-                request_hash: {
-                  :fixed_queue_split => KnapsackPro::Config::Env.fixed_queue_split,
-                  :can_initialize_queue => args.fetch(:can_initialize_queue),
-                  :commit_hash => args.fetch(:commit_hash),
-                  :branch => args.fetch(:branch),
-                  :node_total => args.fetch(:node_total),
-                  :node_index => args.fetch(:node_index),
-                  :node_build_id => KnapsackPro::Config::Env.ci_node_build_id,
-                  :test_files => args.fetch(:test_files)
-                }
+                request_hash: request_hash
               )
             end
           end

data/lib/knapsack_pro/version.rb

@@ -1,3 +1,3 @@
 module KnapsackPro
-  VERSION = '1.8.0'
+  VERSION = '1.9.0'
 end

data/spec/knapsack_pro/client/api/v1/queues_spec.rb

@@ -1,12 +1,12 @@
 describe KnapsackPro::Client::API::V1::Queues do
   describe '.queue' do
     let(:fixed_queue_split) { double }
-    let(:can_initialize_queue) { double }
     let(:commit_hash) { double }
     let(:branch) { double }
     let(:node_total) { double }
     let(:node_index) { double }
     let(:test_files) { double }
+    let(:node_build_id) { double }
 
     subject do
       described_class.queue(
@@ -21,28 +21,52 @@ describe KnapsackPro::Client::API::V1::Queues do
 
     before do
       expect(KnapsackPro::Config::Env).to receive(:fixed_queue_split).and_return(fixed_queue_split)
+      expect(KnapsackPro::Config::Env).to receive(:ci_node_build_id).and_return(node_build_id)
     end
 
-    it do
-      node_build_id = double
-      expect(KnapsackPro::Config::Env).to receive(:ci_node_build_id).and_return(node_build_id)
+    context 'when can_initialize_queue=true' do
+      let(:can_initialize_queue) { true }
+
+      it 'sends test_files among other params' do
+        action = double
+        expect(KnapsackPro::Client::API::Action).to receive(:new).with({
+          endpoint_path: '/v1/queues/queue',
+          http_method: :post,
+          request_hash: {
+            fixed_queue_split: fixed_queue_split,
+            can_initialize_queue: can_initialize_queue,
+            commit_hash: commit_hash,
+            branch: branch,
+            node_total: node_total,
+            node_index: node_index,
+            node_build_id: node_build_id,
+            test_files: test_files
+          }
+        }).and_return(action)
+        expect(subject).to eq action
+      end
+    end
+
+    context 'when can_initialize_queue=false' do
+      let(:can_initialize_queue) { false }
 
-      action = double
-      expect(KnapsackPro::Client::API::Action).to receive(:new).with({
-        endpoint_path: '/v1/queues/queue',
-        http_method: :post,
-        request_hash: {
-          fixed_queue_split: fixed_queue_split,
-          can_initialize_queue: can_initialize_queue,
-          commit_hash: commit_hash,
-          branch: branch,
-          node_total: node_total,
-          node_index: node_index,
-          node_build_id: node_build_id,
-          test_files: test_files
-        }
-      }).and_return(action)
-      expect(subject).to eq action
+      it 'does not send test_files among other params' do
+        action = double
+        expect(KnapsackPro::Client::API::Action).to receive(:new).with({
+          endpoint_path: '/v1/queues/queue',
+          http_method: :post,
+          request_hash: {
+            fixed_queue_split: fixed_queue_split,
+            can_initialize_queue: can_initialize_queue,
+            commit_hash: commit_hash,
+            branch: branch,
+            node_total: node_total,
+            node_index: node_index,
+            node_build_id: node_build_id,
+          }
+        }).and_return(action)
+        expect(subject).to eq action
+      end
     end
   end
 end

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: knapsack_pro
 version: !ruby/object:Gem::Version
-  version: 1.8.0
+  version: 1.9.0
 platform: ruby
 authors:
 - ArturT
 autorequire: 
 bindir: bin
 cert_chain: []
-date: 2019-03-29 00:00:00.000000000 Z
+date: 2019-04-13 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: rake
@@ -398,7 +398,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
     - !ruby/object:Gem::Version
       version: '0'
 requirements: []
-rubygems_version: 3.0.1
+rubygems_version: 3.0.3
 signing_key: 
 specification_version: 4
 summary: Knapsack Pro splits tests across CI nodes and makes sure that tests will