knapsack_pro 1.6.0 → 1.7.0
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/CHANGELOG.md +8 -0
- data/README.md +67 -38
- data/lib/knapsack_pro.rb +6 -0
- data/lib/knapsack_pro/config/env.rb +4 -0
- data/lib/knapsack_pro/version.rb +1 -1
- data/spec/knapsack_pro/config/env_spec.rb +14 -0
- data/spec/knapsack_pro_spec.rb +33 -0
- metadata +1 -1
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA256:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: 71ea9aaa9e707803ac96ff56bf1ba2a520b4aaec005f70d831b0864a14178108
|
4
|
+
data.tar.gz: e520da2e94ce7df0edce0ec518733f155cc1f9502910f61c8b45504307a5952c
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: 8beba449b5e473b987b61d29fa0082f24dc05c667d57529de40d98c3aad33dacaef2d9050b51473a4e64b353ef61016fb261afec0c07b277fa106c177d775fb3
|
7
|
+
data.tar.gz: 4cf162bd3bb0d002cf6be9ccfed0d7eeabb0eb3fa86696b91acb6781758e1d9217c207adf01f7b77270a57e0a4ec93da72dd35cb6437b87e734f3237e8307ea9
|
data/CHANGELOG.md
CHANGED
@@ -1,5 +1,13 @@
|
|
1
1
|
# Change Log
|
2
2
|
|
3
|
+
### 1.7.0
|
4
|
+
|
5
|
+
* Add `KNAPSACK_PRO_LOG_DIR` to set directory where to write logs
|
6
|
+
|
7
|
+
https://github.com/KnapsackPro/knapsack_pro-ruby/pull/79
|
8
|
+
|
9
|
+
https://github.com/KnapsackPro/knapsack_pro-ruby/compare/v1.6.0...v1.7.0
|
10
|
+
|
3
11
|
### 1.6.0
|
4
12
|
|
5
13
|
* Retry request 3 times when API returns 5xx HTTP status
|
data/README.md
CHANGED
@@ -138,13 +138,16 @@ You can see list of questions for common problems and tips in below [Table of Co
|
|
138
138
|
- [How to find seed in RSpec output when I use Queue Mode for RSpec?](#how-to-find-seed-in-rspec-output-when-i-use-queue-mode-for-rspec)
|
139
139
|
- [General questions](#general-questions)
|
140
140
|
- [How to run tests for particular CI node in your development environment](#how-to-run-tests-for-particular-ci-node-in-your-development-environment)
|
141
|
-
- [for
|
141
|
+
- [for knapsack_pro regular mode](#for-knapsack_pro-regular-mode)
|
142
142
|
- [for knapsack_pro queue mode](#for-knapsack_pro-queue-mode)
|
143
143
|
- [What happens when Knapsack Pro API is not available/not reachable temporarily?](#what-happens-when-knapsack-pro-api-is-not-availablenot-reachable-temporarily)
|
144
|
-
- [for
|
144
|
+
- [for knapsack_pro regular mode](#for-knapsack_pro-regular-mode-1)
|
145
145
|
- [for knapsack_pro queue mode](#for-knapsack_pro-queue-mode-1)
|
146
146
|
- [How can I change log level?](#how-can-i-change-log-level)
|
147
|
-
- [How to write
|
147
|
+
- [How to write knapsack_pro logs to a file?](#how-to-write-knapsack_pro-logs-to-a-file)
|
148
|
+
- [set directory where to write log file (option 1 - recommended)](#set-directory-where-to-write-log-file-option-1---recommended)
|
149
|
+
- [set custom logger config (option 2)](#set-custom-logger-config-option-2)
|
150
|
+
- [How to preserve logs on my CI after CI build completed?](#how-to-preserve-logs-on-my-ci-after-ci-build-completed)
|
148
151
|
- [How to split tests based on test level instead of test file level?](#how-to-split-tests-based-on-test-level-instead-of-test-file-level)
|
149
152
|
- [A. Create multiple small test files](#a-create-multiple-small-test-files)
|
150
153
|
- [B. Use tags to mark set of tests in particular test file](#b-use-tags-to-mark-set-of-tests-in-particular-test-file)
|
@@ -481,7 +484,7 @@ There might be some cached test suite splits for git commits you have run in pas
|
|
481
484
|
* If you retry CI node in first hour since the CI build started for the first time then the retried CI node won't execute tests because the queue was consumed. There is important reason why it works like that. For instance some CI providers like Buildkite allows to start CI node later than the others so sometimes the particular CI node may start work while all other CI nodes finished work. In that case we don't want to run tests on the CI node because queue was already consumed. We don't know whether the CI node is part of the build or it is retried CI node hence the 1 hour lock on initializing a new queue.
|
482
485
|
* If you retry CI node after 1 hour since the CI build started for the first time then the retried CI node will initialize a new queue and it will run whole test suite from the queue because there will be no other CI nodes running connected to the queue. The order of tests on retried CI node will be different than on the first run. You probably would expect the retried CI node to run the tests that were executed there on the first place. To achieve that you need to [enable it](#knapsack_pro_fixed_queue_split-remember-queue-split-on-retry-ci-node).
|
483
486
|
|
484
|
-
* When you use unsupported CI provider by
|
487
|
+
* When you use unsupported CI provider by knapsack_pro gem or you forget to set unique `KNAPSACK_PRO_CI_NODE_BUILD_ID` per CI build then:
|
485
488
|
* when you retry single CI node then it will initialize a new queue and it will run whole test suite from the queue because there will be no other CI nodes running connected to the queue. The order of tests on retried CI node will be different than on the first run.
|
486
489
|
* when you retry all CI nodes then a new queue will be initialized and all CI nodes will connect to it.
|
487
490
|
|
@@ -522,9 +525,9 @@ There might be some cached test suite splits for git commits you have run in pas
|
|
522
525
|
|
523
526
|
* `KNAPSACK_PRO_MODIFY_DEFAULT_RSPEC_FORMATTERS=true` (default)
|
524
527
|
|
525
|
-
By default, the
|
528
|
+
By default, the knapsack_pro will monkey patch [RSpec Formatters](https://www.relishapp.com/rspec/rspec-core/v/2-6/docs/command-line/format-option) in order to
|
526
529
|
hide the summary of pending and failed tests after each intermediate run of tests fetched from the work queue on Knapsack Pro API.
|
527
|
-
|
530
|
+
knapsack_pro shows summary of all pending and failed tests at the very end when work queue ended. If you use your custom formatter and you have problem with it then you can disable `KNAPSACK_PRO_MODIFY_DEFAULT_RSPEC_FORMATTERS=false` monkey patching.
|
528
531
|
|
529
532
|
* `KNAPSACK_PRO_MODIFY_DEFAULT_RSPEC_FORMATTERS=false`
|
530
533
|
|
@@ -550,7 +553,7 @@ In case when you use other CI provider for instance [Jenkins](https://jenkins-ci
|
|
550
553
|
|
551
554
|
#### KNAPSACK_PRO_FIXED_TEST_SUITE_SPLITE (test suite split based on seed)
|
552
555
|
|
553
|
-
Note this is for
|
556
|
+
Note this is for knapsack_pro regular mode only.
|
554
557
|
|
555
558
|
* `KNAPSACK_PRO_FIXED_TEST_SUITE_SPLIT=true` (default)
|
556
559
|
|
@@ -799,7 +802,7 @@ Please remember to add additional containers for your project in CircleCI settin
|
|
799
802
|
|
800
803
|
##### CircleCI and knapsack_pro Queue Mode
|
801
804
|
|
802
|
-
If you use
|
805
|
+
If you use knapsack_pro Queue Mode with CircleCI you may want to [collect metadata](https://circleci.com/docs/1.0/test-metadata/#metadata-collection-in-custom-test-steps) like junit xml report about your RSpec test suite.
|
803
806
|
|
804
807
|
Here you can read how to configure [junit formatter](#how-to-use-junit-formatter-with-knapsack_pro-queue-mode). Step for CircleCI is to copy the xml report to `$CIRCLE_TEST_REPORTS` directory. Below is full config for your `spec_helper.rb`:
|
805
808
|
|
@@ -875,7 +878,7 @@ More info about global and matrix ENV configuration in [travis docs](https://doc
|
|
875
878
|
|
876
879
|
##### Semaphore 2.0
|
877
880
|
|
878
|
-
|
881
|
+
knapsack_pro gem supports environment variables provided by Semaphore CI 2.0 to run your tests. You will have to define a few things in `.semaphore/semaphore.yml` config file.
|
879
882
|
|
880
883
|
* You need to set `KNAPSACK_PRO_TEST_SUITE_TOKEN_RSPEC`. If you don't want to commit secrets in yml file then you can [follow this guide](https://docs.semaphoreci.com/article/66-environment-variables-and-secrets).
|
881
884
|
* You need to create as many jobs with unique names (Node 0 - Knapsack Pro, Node 1 - Knapsack Pro etc) as many parallel jobs you want to run. If your test suite is long you should use more parallel jobs.
|
@@ -1349,8 +1352,8 @@ If you are going to relay on rspec to autobalance build when cucumber tests were
|
|
1349
1352
|
|
1350
1353
|
ERROR -- : [knapsack_pro] {"errors"=>[{"commit_hash"=>["parameter is required"]}]}
|
1351
1354
|
|
1352
|
-
When Knapsack Pro API returns error like above the problem is because you use CI provider not supported by
|
1353
|
-
|
1355
|
+
When Knapsack Pro API returns error like above the problem is because you use CI provider not supported by knapsack_pro which means
|
1356
|
+
knapsack_pro gem cannot determine the git commit hash and branch name. To fix this problem you can do:
|
1354
1357
|
|
1355
1358
|
* if you have git installed on CI node then you can use it to determine git commit hash and branch name. [See this](#when-should-you-set-global-variable-knapsack_pro_repository_adaptergit-required-when-ci-provider-is-not-supported)
|
1356
1359
|
* if you have no git installed on CI node then you should manually set `KNAPSACK_PRO_BRANCH` and `KNAPSACK_PRO_COMMIT_HASH`. For instance this might be useful when you use Jenkins. [See this](#when-you-not-set-global-variable-knapsack_pro_repository_adapter-default)
|
@@ -1369,11 +1372,11 @@ Please ensure you are actually using only Test::Unit runner. You may use some hy
|
|
1369
1372
|
|
1370
1373
|
#### Why I see HEAD as branch name in user dashboard for Build metrics for my API token?
|
1371
1374
|
|
1372
|
-
|
1375
|
+
knapsack_pro detects your branch name from environment variables of [supported CI providers](#supported-ci-providers). Sometimes the CI provider may expose the `HEAD` instead of branch name (for instance for pull request merge commits).
|
1373
1376
|
|
1374
1377
|
The same can happen for CI provider not supported by default by knapsack_pro when you use [KNAPSACK_PRO_REPOSITORY_ADAPTER=git](#when-should-you-set-global-variable-knapsack_pro_repository_adaptergit-required-when-ci-provider-is-not-supported) to use local git installed on CI node to detect the branch name and git commit.
|
1375
1378
|
|
1376
|
-
|
1379
|
+
knapsack_pro uses git command `git -C /home/user/project_dir rev-parse --abbrev-ref HEAD` to detect branch name. See [source of knapsack_pro](https://github.com/KnapsackPro/knapsack_pro-ruby/blob/master/lib/knapsack_pro/repository_adapters/git_adapter.rb). In most of cases it's good way to detect branch name. But if your CI provider during CI build checkouts to specific git commit then git cannot provide the name of the branch. In such scenario you would see `HEAD` as your branch name. It is good enough situation and knapsack_pro will work correctly. The benefit of knowing exactly the branch name allows KnapsackPro API to better track history of test files timing changes across branches in order to better do split of test suite. The difference should be rather very small so it's not a problem that you have `HEAD` as branch name.
|
1377
1380
|
|
1378
1381
|
If you would like to see exact branch name instead of `HEAD` in your `build metrics` history in [user dashboard](https://knapsackpro.com/dashboard) then you can explicitly provide the branch name with `KNAPSACK_PRO_BRANCH` for each CI build.
|
1379
1382
|
|
@@ -1441,7 +1444,7 @@ end
|
|
1441
1444
|
Workaround is to replace `task` with explicit association:
|
1442
1445
|
|
1443
1446
|
```ruby
|
1444
|
-
# this will work in
|
1447
|
+
# this will work in knapsack_pro Queue Mode
|
1445
1448
|
FactoryBot.define do
|
1446
1449
|
factory :assignment do
|
1447
1450
|
association :task
|
@@ -1542,17 +1545,17 @@ The 0.1s is a default time execution used when test file is an empty file or its
|
|
1542
1545
|
|
1543
1546
|
The problem may happen when you use old knapsack_pro `< 0.33.0` or if you use custom rspec formatter, or when you set flag [KNAPSACK_PRO_MODIFY_DEFAULT_RSPEC_FORMATTERS=false](#knapsack_pro_modify_default_rspec_formatters-hide-duplicated-summary-of-pending-and-failed-tests).
|
1544
1547
|
|
1545
|
-
When you use Queue Mode then
|
1548
|
+
When you use Queue Mode then knapsack_pro does multiple requests to Knapsack Pro API and fetches a few test files to execute.
|
1546
1549
|
This means RSpec will remember failed tests so far and it will present them at the end of each executed test subset if flag `KNAPSACK_PRO_MODIFY_DEFAULT_RSPEC_FORMATTERS=false`.
|
1547
|
-
You can see the list of all failed test files at the end of
|
1550
|
+
You can see the list of all failed test files at the end of knapsack_pro queue mode command.
|
1548
1551
|
|
1549
1552
|
##### Why when I use Queue Mode for RSpec then I see multiple times the same pending tests?
|
1550
1553
|
|
1551
1554
|
The problem may happen when you use old knapsack_pro `< 0.33.0` or if you use custom rspec formatter, or when you set flag [KNAPSACK_PRO_MODIFY_DEFAULT_RSPEC_FORMATTERS=false](#knapsack_pro_modify_default_rspec_formatters-hide-duplicated-summary-of-pending-and-failed-tests).
|
1552
1555
|
|
1553
|
-
When you use Queue Mode then
|
1556
|
+
When you use Queue Mode then knapsack_pro does multiple requests to Knapsack Pro API and fetches a few test files to execute.
|
1554
1557
|
This means RSpec will remember pending tests so far and it will present them at the end of each executed test subset if flag `KNAPSACK_PRO_MODIFY_DEFAULT_RSPEC_FORMATTERS=false`.
|
1555
|
-
You can see the list of all pending test files at the end of
|
1558
|
+
You can see the list of all pending test files at the end of knapsack_pro queue mode command.
|
1556
1559
|
|
1557
1560
|
##### Does in Queue Mode the RSpec is initialized many times that causes Rails load over and over again?
|
1558
1561
|
|
@@ -1560,7 +1563,7 @@ No. In Queue Mode the RSpec configuration is updated every time when knapsack_pr
|
|
1560
1563
|
|
1561
1564
|
##### Why my tests are executed twice in queue mode? Why CI node runs whole test suite again?
|
1562
1565
|
|
1563
|
-
This may happen when you use not supported CI provider by
|
1566
|
+
This may happen when you use not supported CI provider by knapsack_pro. It's because of missing value of CI build ID. You can set unique `KNAPSACK_PRO_CI_NODE_BUILD_ID` for each CI build. The problem with test suite run again happens when one of your CI node started work later when all other CI nodes already executed whole test suite.
|
1564
1567
|
The slow CI node that started work late will initialize a new queue hence the tests executed twice.
|
1565
1568
|
|
1566
1569
|
To solve this problem you can set `KNAPSACK_PRO_CI_NODE_BUILD_ID` as mentioned above or you can set `KNAPSACK_PRO_FIXED_QUEUE_SPLIT=true`.
|
@@ -1617,7 +1620,7 @@ Learn more about [log levels](#how-can-i-change-log-level).
|
|
1617
1620
|
|
1618
1621
|
##### How to find seed in RSpec output when I use Queue Mode for RSpec?
|
1619
1622
|
|
1620
|
-
In output for RSpec in
|
1623
|
+
In output for RSpec in knapsack_pro Queue Mode you may see message:
|
1621
1624
|
|
1622
1625
|
> INFO -- : [knapsack_pro] To retry in development the subset of tests fetched from API queue please run below command on your machine. If you use --order random then remember to add proper --seed 123 that you will find at the end of rspec command.
|
1623
1626
|
>
|
@@ -1647,7 +1650,7 @@ If you don't use RSpec argument `--order random` then you don't need to provide
|
|
1647
1650
|
|
1648
1651
|
#### How to run tests for particular CI node in your development environment
|
1649
1652
|
|
1650
|
-
##### for
|
1653
|
+
##### for knapsack_pro regular mode
|
1651
1654
|
|
1652
1655
|
In your development environment you can debug tests that were run on the particular CI node.
|
1653
1656
|
For instance to run subset of tests for the first CI node with specified seed you can do.
|
@@ -1732,13 +1735,13 @@ There are a few ways to reproduce tests executed on CI node in your development
|
|
1732
1735
|
|
1733
1736
|
#### What happens when Knapsack Pro API is not available/not reachable temporarily?
|
1734
1737
|
|
1735
|
-
##### for
|
1738
|
+
##### for knapsack_pro regular mode
|
1736
1739
|
|
1737
|
-
|
1740
|
+
knapsack_pro gem will retry requests to Knapsack Pro API multiple times every few seconds til it switch to fallback behavior and it will split test files across CI nodes based on popular test directory names. When knapsack_pro starts fallback mode then you will see a warning in the output.
|
1738
1741
|
|
1739
1742
|
##### for knapsack_pro queue mode
|
1740
1743
|
|
1741
|
-
|
1744
|
+
knapsack_pro gem will retry requests to Knapsack Pro API multiple times every few seconds till it switches to fallback behavior and it will split test files across CI nodes based on popular test directory names. Note that if one of CI nodes will lose connection to Knapsack Pro API but other not then you may see that some of the test files will be executed on multiple CI nodes. Fallback mode guarantees each of test files is run at least once across CI nodes. Thanks to that we know if the whole test suite is green or not. When knapsack_pro starts fallback mode then you will see a warning in the output.
|
1742
1745
|
|
1743
1746
|
#### How can I change log level?
|
1744
1747
|
|
@@ -1753,17 +1756,43 @@ Recommended log levels you can use:
|
|
1753
1756
|
* `debug` is default log level and it is recommended to log details about requests to Knapsack Pro API. Thanks to that you can debug things or ensure everything works. For instance in [user dashboard](https://knapsackpro.com/dashboard) you can find tips referring to debug logs.
|
1754
1757
|
* `info` level shows message like how to retry tests in development or info why something works this way or the other (for instance why tests were not executed on the CI node). You can use `info` level when you really don't want to see all debug messages from default log level.
|
1755
1758
|
|
1756
|
-
#### How to write
|
1759
|
+
#### How to write knapsack_pro logs to a file?
|
1757
1760
|
|
1758
|
-
|
1761
|
+
##### set directory where to write log file (option 1 - recommended)
|
1762
|
+
|
1763
|
+
Set `KNAPSACK_PRO_LOG_DIR=log` environment variable in order to notify knapsack_pro gem to write logs to `log` directory instead of stdout.
|
1764
|
+
If you have Rails project then this should work for you.
|
1765
|
+
|
1766
|
+
knapsack_pro will create a file with CI node index in name. For instance if you run tests on 2 CI nodes:
|
1767
|
+
|
1768
|
+
* `log/knapsack_pro_node_0.log`
|
1769
|
+
* `log/knapsack_pro_node_1.log`
|
1770
|
+
|
1771
|
+
`KNAPSACK_PRO_LOG_DIR` has higher priority than custom log set in `rails_helper.rb` as shown below (option 2).
|
1772
|
+
|
1773
|
+
You can change log level with [KNAPSACK_PRO_LOG_LEVEL environment variable](#how-can-i-change-log-level).
|
1774
|
+
|
1775
|
+
##### set custom logger config (option 2)
|
1776
|
+
|
1777
|
+
In your `rails_helper.rb` you can set custom Knapsack Pro logger and write to custom log file.
|
1759
1778
|
|
1760
1779
|
```ruby
|
1780
|
+
# Ensure you load Rails before using Rails const below.
|
1781
|
+
# This line should be already in your rails_helper.rb
|
1782
|
+
require File.expand_path('../../config/environment', __FILE__)
|
1783
|
+
|
1761
1784
|
require 'logger'
|
1762
1785
|
KnapsackPro.logger = Logger.new(Rails.root.join('log', "knapsack_pro_node_#{KnapsackPro::Config::Env.ci_node_index}.log"))
|
1763
1786
|
KnapsackPro.logger.level = Logger::DEBUG
|
1764
1787
|
```
|
1765
1788
|
|
1766
|
-
Note if you run knapsack_pro
|
1789
|
+
Note if you run knapsack_pro then the very first request to Knapsack Pro API still will be shown to stdout because we need to have set of test files needed to run RSpec before we load `rails_helper.rb` where the configuration of logger actually is loaded for the first time.
|
1790
|
+
|
1791
|
+
That is why you may prefer to use option 1 instead of this.
|
1792
|
+
|
1793
|
+
##### How to preserve logs on my CI after CI build completed?
|
1794
|
+
|
1795
|
+
Follow this tip if you use one of above options to write knapsack_pro log to the file.
|
1767
1796
|
|
1768
1797
|
If you would like to keep knapsack_pro logs after your CI build finished then you could use artifacts or some cache mechanize for your CI provider.
|
1769
1798
|
|
@@ -1796,7 +1825,7 @@ A lot of small test files will give you better test suite split results.
|
|
1796
1825
|
|
1797
1826
|
Another way is to use tags to mark subset of tests in particular test file and then split tests based on tags.
|
1798
1827
|
|
1799
|
-
This example is for
|
1828
|
+
This example is for knapsack_pro Regular Mode. You can also use knapsack_pro Queue Mode with tags.
|
1800
1829
|
|
1801
1830
|
Here is example of test file with specified tags for describe groups:
|
1802
1831
|
|
@@ -2043,7 +2072,7 @@ When you have short test suite, for instance in javascript then you could distri
|
|
2043
2072
|
* CI 1
|
2044
2073
|
* Step 1: `bundle exec rake knapsack_pro:queue:rspec`
|
2045
2074
|
|
2046
|
-
You will run your javascript tests on single CI node and the
|
2075
|
+
You will run your javascript tests on single CI node and the knapsack_pro will auto-balance CI build with Queue Mode. Thanks to that CI build time execution will be flat and optimal (as fast as possible).
|
2047
2076
|
|
2048
2077
|
#### How to set `before(:suite)` and `after(:suite)` RSpec hooks in Queue Mode (Percy.io example)?
|
2049
2078
|
|
@@ -2090,7 +2119,7 @@ end
|
|
2090
2119
|
You can run knapsack_pro with [parallel_tests](https://github.com/grosser/parallel_tests) gem to run multiple concurrent knapsack_pro commands per CI node.
|
2091
2120
|
|
2092
2121
|
Let's consider this example. We have 2 CI node. On each CI node we want to run 2 concurrent knapsack_pro commands by parallel_tests gem (`PARALLEL_TESTS_CONCURRENCY=2`).
|
2093
|
-
This means we would have 4 parallel
|
2122
|
+
This means we would have 4 parallel knapsack_pro commands in total across all CI nodes. So from knapsack_pro perspective you will have 4 nodes in total.
|
2094
2123
|
|
2095
2124
|
Create in your project directory an executable file `bin/parallel_tests`:
|
2096
2125
|
|
@@ -2189,7 +2218,7 @@ end
|
|
2189
2218
|
Then you can execute rspec with only failed tests after main knapsack_pro command finish.
|
2190
2219
|
|
2191
2220
|
```bash
|
2192
|
-
# Run
|
2221
|
+
# Run knapsack_pro in Queue Mode and it will save failed tests in tmp/rspec_examples.txt
|
2193
2222
|
bundle exec rake knapsack_pro:queue:rspec
|
2194
2223
|
|
2195
2224
|
# run only failed tests from tmp/rspec_examples.txt
|
@@ -2251,7 +2280,7 @@ If you would like to run only non feature tests then use negation `~type:feature
|
|
2251
2280
|
KNAPSACK_PRO_TEST_SUITE_TOKEN_RSPEC=$API_TOKEN_FOR_NON_FEATURE_TESTS bundle exec rake "knapsack_pro:queue:rspec[--tag ~type:feature]"
|
2252
2281
|
```
|
2253
2282
|
|
2254
|
-
Note above examples are for
|
2283
|
+
Note above examples are for knapsack_pro Queue Mode and when you will run tests you may notice that all test files are run by RSpec but only tests specified by tag like `tag type:feature` will be executed. Basically RSpec will just load all files but run just specified tags.
|
2255
2284
|
|
2256
2285
|
**Option 2: specify directory pattern**
|
2257
2286
|
|
@@ -2304,9 +2333,9 @@ You can check CodeClimate docs about [parallel tests](https://docs.codeclimate.c
|
|
2304
2333
|
|
2305
2334
|
#### How to run knapsack_pro only on a few parallel CI nodes instead of all?
|
2306
2335
|
|
2307
|
-
You may want to run
|
2336
|
+
You may want to run knapsack_pro only on a few CI nodes when you would like to run a different job on other CI nodes.
|
2308
2337
|
|
2309
|
-
For instance, you have 3 parallel CI nodes. You would like to run
|
2338
|
+
For instance, you have 3 parallel CI nodes. You would like to run knapsack_pro only on two CI nodes. The last CI node you want to use for the different job like running linters etc.
|
2310
2339
|
|
2311
2340
|
In such case, you can override the number of total CI nodes available by your CI provider. For instance, Heroku CI provider exposes in ENV variables `CI_NODE_TOTAL=3`.
|
2312
2341
|
|
@@ -2316,15 +2345,15 @@ You can then run knapsack_pro command this way on the first and the second CI no
|
|
2316
2345
|
KNAPSACK_PRO_CI_NODE_TOTAL=$((CI_NODE_TOTAL-1)) bundle exec rake knapsack_pro:rspec
|
2317
2346
|
```
|
2318
2347
|
|
2319
|
-
We decrease the number of CI node total by 1 that
|
2348
|
+
We decrease the number of CI node total by 1 that knapsack_pro can see. This way you can run tests with knapsack_pro only on two CI nodes.
|
2320
2349
|
On the 3rd CI node, you can run other things like linters etc.
|
2321
2350
|
|
2322
2351
|
If you would like to check what is the CI node total ENV variable name exposed by your CI provider you can check that in your CI provider environment variables docs
|
2323
|
-
or preview the [ENV variables that
|
2352
|
+
or preview the [ENV variables that knapsack_pro can read](https://github.com/KnapsackPro/knapsack_pro-ruby/tree/master/lib/knapsack_pro/config/ci) for supported CI providers.
|
2324
2353
|
|
2325
2354
|
#### How to use simplecov in Queue Mode?
|
2326
2355
|
|
2327
|
-
If you would like to make [simplecov](https://github.com/colszowka/simplecov) gem work with
|
2356
|
+
If you would like to make [simplecov](https://github.com/colszowka/simplecov) gem work with knapsack_pro Queue Mode to correctly track code coverage for parallel CI nodes please do:
|
2328
2357
|
|
2329
2358
|
```ruby
|
2330
2359
|
# spec_helper.rb or rails_helper.rb
|
@@ -2342,7 +2371,7 @@ This way there should be no conflict between code coverage reports generated per
|
|
2342
2371
|
|
2343
2372
|
#### Do I need to use separate API token for Queue Mode and Regular Mode?
|
2344
2373
|
|
2345
|
-
I recommend to record timing of a new test suite with `API token A` and knapsack_pro Regular Mode. After you recorded test suite timing then you should use the `API token A` to run your tests in
|
2374
|
+
I recommend to record timing of a new test suite with `API token A` and knapsack_pro Regular Mode. After you recorded test suite timing then you should use the `API token A` to run your tests in knapsack_pro Queue Mode. This way Queue Mode will leverage test suite timing recorded in a fast way with Regular Mode so the first run in Queue Mode won't be slow due to recording test files timing for the first time.
|
2346
2375
|
|
2347
2376
|
When you want to go back from Queue Mode to Regular Mode then the fact of using the same API token could cause edge cases that some builds might not be well balanced in Regular Mode. That is why I recommend using separate API token for Regular Mode and Queue Mode. If you plan to use only Queue Mode then no worry.
|
2348
2377
|
|
data/lib/knapsack_pro.rb
CHANGED
@@ -79,6 +79,12 @@ module KnapsackPro
|
|
79
79
|
end
|
80
80
|
|
81
81
|
def logger
|
82
|
+
if KnapsackPro::Config::Env.log_dir
|
83
|
+
default_logger = Logger.new("#{KnapsackPro::Config::Env.log_dir}/knapsack_pro_node_#{KnapsackPro::Config::Env.ci_node_index}.log")
|
84
|
+
default_logger.level = KnapsackPro::Config::Env.log_level
|
85
|
+
self.logger = default_logger
|
86
|
+
end
|
87
|
+
|
82
88
|
unless @logger
|
83
89
|
default_logger = ::Logger.new(STDOUT)
|
84
90
|
default_logger.level = KnapsackPro::Config::Env.log_level
|
data/lib/knapsack_pro/version.rb
CHANGED
@@ -701,4 +701,18 @@ describe KnapsackPro::Config::Env do
|
|
701
701
|
it { should eql ::Logger::DEBUG }
|
702
702
|
end
|
703
703
|
end
|
704
|
+
|
705
|
+
describe '.log_dir' do
|
706
|
+
subject { described_class.log_dir }
|
707
|
+
|
708
|
+
context 'when ENV set to directory path' do
|
709
|
+
let(:log_dir) { 'log' }
|
710
|
+
before { stub_const('ENV', { 'KNAPSACK_PRO_LOG_DIR' => log_dir }) }
|
711
|
+
it { should eql 'log' }
|
712
|
+
end
|
713
|
+
|
714
|
+
context "when ENV doesn't exist" do
|
715
|
+
it { should be_nil }
|
716
|
+
end
|
717
|
+
end
|
704
718
|
end
|
data/spec/knapsack_pro_spec.rb
CHANGED
@@ -13,6 +13,39 @@ describe KnapsackPro do
|
|
13
13
|
before { described_class.reset_logger! }
|
14
14
|
after { described_class.reset_logger! }
|
15
15
|
|
16
|
+
context 'when KNAPSACK_PRO_LOG_DIR is set' do
|
17
|
+
let(:logger) { instance_double(Logger) }
|
18
|
+
|
19
|
+
context 'when KNAPSACK_PRO_CI_NODE_INDEX is set' do
|
20
|
+
before do
|
21
|
+
stub_const('ENV', {
|
22
|
+
'KNAPSACK_PRO_LOG_DIR' => 'log',
|
23
|
+
'KNAPSACK_PRO_CI_NODE_INDEX' => 1,
|
24
|
+
})
|
25
|
+
|
26
|
+
expect(Logger).to receive(:new).with('log/knapsack_pro_node_1.log').and_return(logger)
|
27
|
+
expect(logger).to receive(:level=).with(Logger::DEBUG)
|
28
|
+
expect(KnapsackPro::LoggerWrapper).to receive(:new).with(logger).and_return(logger_wrapper)
|
29
|
+
end
|
30
|
+
|
31
|
+
it { should eql logger_wrapper }
|
32
|
+
end
|
33
|
+
|
34
|
+
context 'when KNAPSACK_PRO_CI_NODE_INDEX is not set' do
|
35
|
+
before do
|
36
|
+
stub_const('ENV', {
|
37
|
+
'KNAPSACK_PRO_LOG_DIR' => 'log',
|
38
|
+
})
|
39
|
+
|
40
|
+
expect(Logger).to receive(:new).with('log/knapsack_pro_node_0.log').and_return(logger)
|
41
|
+
expect(logger).to receive(:level=).with(Logger::DEBUG)
|
42
|
+
expect(KnapsackPro::LoggerWrapper).to receive(:new).with(logger).and_return(logger_wrapper)
|
43
|
+
end
|
44
|
+
|
45
|
+
it { should eql logger_wrapper }
|
46
|
+
end
|
47
|
+
end
|
48
|
+
|
16
49
|
context 'when default logger' do
|
17
50
|
let(:logger) { instance_double(Logger) }
|
18
51
|
|