knapsack_pro 0.28.0 → 0.28.1
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/CHANGELOG.md +8 -0
- data/README.md +82 -2
- data/lib/knapsack_pro/test_file_presenter.rb +4 -1
- data/lib/knapsack_pro/version.rb +1 -1
- data/spec/knapsack_pro/test_file_presenter_spec.rb +1 -1
- metadata +2 -2
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA1:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: ed7d84e6a7751a94b9b7d9dc71f8a67b081dcc85
|
4
|
+
data.tar.gz: 48eb86e58ac7e812b94fad49fe16ed9a231d3d7b
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: aa166fb2006447742c0759483884e766c826f4cf9ecc4ab95f1acecb007ec976ad6dd42e2bf011b0bc3950893be0c0cef90ed2b2b18a706cd3b4045a6786f67f
|
7
|
+
data.tar.gz: 5bc4cf8a32f606f7874bb974823d82cd8118eefcae389d4d38f3186200e994de06a4852b94aec079ad42a341946e73cab195e8df960d0a1c1a13cc484f133c3e
|
data/CHANGELOG.md
CHANGED
@@ -2,6 +2,14 @@
|
|
2
2
|
|
3
3
|
* TODO
|
4
4
|
|
5
|
+
### 0.28.1
|
6
|
+
|
7
|
+
* Add support for test files in directory with spaces.
|
8
|
+
|
9
|
+
https://github.com/KnapsackPro/knapsack_pro-ruby/issues/27
|
10
|
+
|
11
|
+
https://github.com/KnapsackPro/knapsack_pro-ruby/compare/v0.28.0...v0.28.1
|
12
|
+
|
5
13
|
### 0.28.0
|
6
14
|
|
7
15
|
* Show at the end of `knapsack_pro:queue:rspec` command the example how to run all tests executed for the CI node in the development environment.
|
data/README.md
CHANGED
@@ -99,6 +99,8 @@ The knapsack_pro has also [queue mode](#queue-mode) to get most optimal test sui
|
|
99
99
|
- [for knapack_pro regular mode](#for-knapack_pro-regular-mode)
|
100
100
|
- [for knapsack_pro queue mode](#for-knapsack_pro-queue-mode)
|
101
101
|
- [What happens when Knapsack Pro API is not available/not reachable temporarily?](#what-happens-when-knapsack-pro-api-is-not-availablenot-reachable-temporarily)
|
102
|
+
- [for knapack_pro regular mode](#for-knapack_pro-regular-mode-1)
|
103
|
+
- [for knapsack_pro queue mode](#for-knapsack_pro-queue-mode-1)
|
102
104
|
- [How can I change log level?](#how-can-i-change-log-level)
|
103
105
|
- [How to split tests based on test level instead of test file level?](#how-to-split-tests-based-on-test-level-instead-of-test-file-level)
|
104
106
|
- [A. Create multiple small test files](#a-create-multiple-small-test-files)
|
@@ -111,6 +113,11 @@ The knapsack_pro has also [queue mode](#queue-mode) to get most optimal test sui
|
|
111
113
|
- [Why when I use Queue Mode for RSpec and test fails then I see multiple times info about failed test in RSpec result?](#why-when-i-use-queue-mode-for-rspec-and-test-fails-then-i-see-multiple-times-info-about-failed-test-in-rspec-result)
|
112
114
|
- [Why when I use Queue Mode for RSpec then I see multiple times the same pending tests?](#why-when-i-use-queue-mode-for-rspec-then-i-see-multiple-times-the-same-pending-tests)
|
113
115
|
- [Does in Queue Mode the RSpec is initialized many times that causes Rails load over and over again?](#does-in-queue-mode-the-rspec-is-initialized-many-times-that-causes-rails-load-over-and-over-again)
|
116
|
+
- [How to use junit formatter?](#how-to-use-junit-formatter)
|
117
|
+
- [Why I see API error commit_hash parameter is required?](#why-i-see-api-error-commit_hash-parameter-is-required)
|
118
|
+
- [How many API keys I need?](#how-many-api-keys-i-need)
|
119
|
+
- [What is optimal order of test commands?](#what-is-optimal-order-of-test-commands)
|
120
|
+
- [Why my tests are executed twice in queue mode? Why CI node runs whole test suite again?](#why-my-tests-are-executed-twice-in-queue-mode-why-ci-node-runs-whole-test-suite-again)
|
114
121
|
- [Gem tests](#gem-tests)
|
115
122
|
- [Spec](#spec)
|
116
123
|
- [Contributing](#contributing)
|
@@ -341,6 +348,13 @@ If above command fails then you may need to explicitly pass an argument to requi
|
|
341
348
|
Note if you will run queue mode command for the first time it might be slower.
|
342
349
|
The second build should have better optimal test suite split.
|
343
350
|
|
351
|
+
If you will encounter problem with stack level too deep then you may want to ensure you load your dependencies only once in `spec/rails_helper.rb` or `spec/spec_helper.rb`. The Queue Mode may load multiple times the `rails_helper.rb` hence the problem. For instance the problem occurs for capybara-screenshot gem. Here is the example how you should load the gem.
|
352
|
+
|
353
|
+
unless ENV['KNAPSACK_PRO_RSPEC_DEPENDENCIES_LOADED']
|
354
|
+
ENV['KNAPSACK_PRO_RSPEC_DEPENDENCIES_LOADED'] = 'true'
|
355
|
+
require 'capybara-screenshot/rspec'
|
356
|
+
end
|
357
|
+
|
344
358
|
### Additional info about queue mode
|
345
359
|
|
346
360
|
* You should use different API token for queue mode than for regular mode to avoid problem with test suite split in case you would like to go back to regular mode.
|
@@ -701,7 +715,7 @@ If you were running your tests with `--order random` on your CI then you can add
|
|
701
715
|
|
702
716
|
There are a few ways to reproduce tests executed on CI node in your development environment.
|
703
717
|
|
704
|
-
* At the end of `knapsack_pro:queue:rspec` results you will find example of command that you can copy and paste to your development machine. It will run all tests executed on the CI node in a single run.
|
718
|
+
* At the end of `knapsack_pro:queue:rspec` results you will find example of command that you can copy and paste to your development machine. It will run all tests executed on the CI node in a single run. I recommend this approach.
|
705
719
|
|
706
720
|
* For each intermediate request to Knapsack Pro API queue you will also find example of command to run a subset of tests fetched from API. This might be helpful when you use `--order random` for rspec and you would like to reproduce the tests with the same seed.
|
707
721
|
|
@@ -709,6 +723,8 @@ There are a few ways to reproduce tests executed on CI node in your development
|
|
709
723
|
|
710
724
|
To retry the particular CI node do this on your machine:
|
711
725
|
|
726
|
+
RACK_ENV=test \
|
727
|
+
RAILS_ENV=test \
|
712
728
|
KNAPSACK_PRO_TEST_SUITE_TOKEN_RSPEC=token \
|
713
729
|
KNAPSACK_PRO_REPOSITORY_ADAPTER=git \
|
714
730
|
KNAPSACK_PRO_PROJECT_DIR=~/projects/rails-app \
|
@@ -728,6 +744,8 @@ There are a few ways to reproduce tests executed on CI node in your development
|
|
728
744
|
|
729
745
|
You can pass the seed in your local environment to reproduce the tests in the same order as they were executed on CI node:
|
730
746
|
|
747
|
+
RACK_ENV=test \
|
748
|
+
RAILS_ENV=test \
|
731
749
|
KNAPSACK_PRO_TEST_SUITE_TOKEN_RSPEC=token \
|
732
750
|
KNAPSACK_PRO_REPOSITORY_ADAPTER=git \
|
733
751
|
KNAPSACK_PRO_PROJECT_DIR=~/projects/rails-app \
|
@@ -738,7 +756,13 @@ There are a few ways to reproduce tests executed on CI node in your development
|
|
738
756
|
|
739
757
|
### What happens when Knapsack Pro API is not available/not reachable temporarily?
|
740
758
|
|
741
|
-
|
759
|
+
#### for knapack_pro regular mode
|
760
|
+
|
761
|
+
knapack_pro gem will retry requests to Knapsack Pro API multiple times every few seconds til it switch to fallback behaviour and it will split test files across CI nodes based on popular test directory names.
|
762
|
+
|
763
|
+
#### for knapsack_pro queue mode
|
764
|
+
|
765
|
+
knapack_pro gem will retry requests to Knapsack Pro API multiple times every few seconds til it fails.
|
742
766
|
|
743
767
|
### How can I change log level?
|
744
768
|
|
@@ -864,6 +888,62 @@ You can see the list of all pending test files at the end of knapack_pro queue m
|
|
864
888
|
|
865
889
|
No. In Queue Mode the RSpec configuration is updated every time when knapsack_pro gem gets a new set of test files from the Knapsack Pro API and it looks in knapsack_pro output like RSpec was loaded many times but in fact, it loads your project environment only once.
|
866
890
|
|
891
|
+
### How to use junit formatter?
|
892
|
+
|
893
|
+
You can use junit formatter for rspec thanks to gem [rspec_junit_formatter](https://github.com/sj26/rspec_junit_formatter).
|
894
|
+
Here you can find example how to generate `rspec.xml` file with junit format and at the same time show normal documentation format output for RSpec.
|
895
|
+
|
896
|
+
# Regular Mode
|
897
|
+
bundle exec rake "knapsack_pro:rspec[--format documentation --format RspecJunitFormatter --out tmp/rspec.xml]"
|
898
|
+
|
899
|
+
# Queue Mode
|
900
|
+
# The xml report will contain all tests executed across intermediate test subset runs based on queue
|
901
|
+
bundle exec rake "knapsack_pro:queue:rspec[--format documentation --format RspecJunitFormatter --out tmp/rspec.xml]"
|
902
|
+
|
903
|
+
### Why I see API error commit_hash parameter is required?
|
904
|
+
|
905
|
+
ERROR -- : [knapsack_pro] {"errors"=>[{"commit_hash"=>["parameter is required"]}]}
|
906
|
+
|
907
|
+
When Knapsack Pro API returns error like above the problem is because you use CI provider not supported by knapack_pro which means
|
908
|
+
knapack_pro gem cannot determine the git commit hash and branch name. To fix this problem you can do:
|
909
|
+
|
910
|
+
* if you have git installed on CI node then you can use it to determine git commit hash and branch name. [See this](#when-you-set-global-variable-knapsack_pro_repository_adaptergit-required-when-ci-provider-is-not-supported)
|
911
|
+
* if you have no git installed on CI node then you should manually set `KNAPSACK_PRO_BRANCH` and `KNAPSACK_PRO_COMMIT_HASH`. For instance this might be useful when you use Jenkins. [See this](#when-you-not-set-global-variable-knapsack_pro_repository_adapter-default)
|
912
|
+
|
913
|
+
### How many API keys I need?
|
914
|
+
|
915
|
+
Basically you need as many API keys as you have steps in your build.
|
916
|
+
|
917
|
+
Here is example:
|
918
|
+
|
919
|
+
* Step 1. API_KEY_A for `bundle exec rake knapsack_pro:cucumber`
|
920
|
+
* Step 2. API_KEY_B for `bundle exec rake knapsack_pro:rspec`
|
921
|
+
* Step 3. API_KEY_C for `KNAPSACK_PRO_TEST_FILE_PATTERN="specs/features/*_spec.rb" bundle exec rake knapsack_pro:rspec`
|
922
|
+
* Step 4. API_KEY_D for `bundle exec rake knapsack_pro:rspec[--tag tagA]`
|
923
|
+
* Step 5. API_KEY_E for `bundle exec rake knapsack_pro:rspec[--tag ~tagA]`
|
924
|
+
* Step 6. API_KEY_F for `bundle exec rake knapsack_pro:queue:rspec`
|
925
|
+
|
926
|
+
Note:
|
927
|
+
|
928
|
+
* If you specified `KNAPSACK_PRO_TEST_FILE_PATTERN` then you run subset of your whole test suite hence you need separate API key because we want to track only tests for this subset.
|
929
|
+
* If you pass `--tag tagA` or `--tag ~tagA` then you run subset of your whole test suite hence you need separate API key.
|
930
|
+
* If you use regular or queue mode then you need separate API key for each mode.
|
931
|
+
|
932
|
+
### What is optimal order of test commands?
|
933
|
+
|
934
|
+
I recommend to run first the test commands in the regular mode and later the commands in the queue mode.
|
935
|
+
|
936
|
+
* Step 1. `bundle exec rake knapsack_pro:cucumber` (regular mode)
|
937
|
+
* Step 2. `bundle exec rake knapsack_pro:queue:rspec` (queue mode)
|
938
|
+
|
939
|
+
Thanks to that when for some reason the tests executed for cucumber in regular mode will not be well balanced across CI nodes (for instance when one of CI node has bad performance) then the rspec tests executed later in the queue mode will autobalance your build.
|
940
|
+
|
941
|
+
### Why my tests are executed twice in queue mode? Why CI node runs whole test suite again?
|
942
|
+
|
943
|
+
This may happen when one of your CI node started work when all other CI nodes already executed whole test suite.
|
944
|
+
This slow CI node will initialize a new queue hence the tests executed twice. To solve this problem you should set `KNAPSACK_PRO_FIXED_QUEUE_SPLIT=true`.
|
945
|
+
Please [read this](#knapsack_pro_fixed_queue_split-remember-queue-split-on-retry-ci-node).
|
946
|
+
|
867
947
|
## Gem tests
|
868
948
|
|
869
949
|
### Spec
|
data/lib/knapsack_pro/version.rb
CHANGED
metadata
CHANGED
@@ -1,14 +1,14 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: knapsack_pro
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 0.28.
|
4
|
+
version: 0.28.1
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- ArturT
|
8
8
|
autorequire:
|
9
9
|
bindir: bin
|
10
10
|
cert_chain: []
|
11
|
-
date: 2017-
|
11
|
+
date: 2017-03-22 00:00:00.000000000 Z
|
12
12
|
dependencies:
|
13
13
|
- !ruby/object:Gem::Dependency
|
14
14
|
name: rake
|