knapsack_pro 0.43.0 → 0.44.0
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/CHANGELOG.md +8 -0
- data/README.md +107 -51
- data/lib/knapsack_pro/base_allocator_builder.rb +1 -1
- data/lib/knapsack_pro/config/env.rb +4 -0
- data/lib/knapsack_pro/version.rb +1 -1
- data/spec/knapsack_pro/base_allocator_builder_spec.rb +21 -7
- data/spec/knapsack_pro/config/env_spec.rb +14 -0
- metadata +2 -2
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA1:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: c8d9401540ef36e33e333a330b03807d83a902de
|
4
|
+
data.tar.gz: 86fe1eeb6c82a2a37ca8c81b0cc7eb465d63ba79
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: 04e2e06185f9d02bb16c69cdd89e352b5a4e2bdf8c2509ae0f4c22be2381c0e1f4bebcfc29c950df6c6c9ac34fb4bdbbd917da85b90e34ae4cce562b7ea1144f
|
7
|
+
data.tar.gz: bfd28948e2c3cd157b2ee9f51ca40d822fa5e97af13131c42ee67b29ded3f5d8008a42a91dba0b9bc0b7c80287c221d6973ba55d3b506c9b1e721e0aee71f01f
|
data/CHANGELOG.md
CHANGED
@@ -2,6 +2,14 @@
|
|
2
2
|
|
3
3
|
* TODO
|
4
4
|
|
5
|
+
### 0.44.0
|
6
|
+
|
7
|
+
* Add ability to set test_dir using an environment variable.
|
8
|
+
|
9
|
+
https://github.com/KnapsackPro/knapsack_pro-ruby/pull/45
|
10
|
+
|
11
|
+
https://github.com/KnapsackPro/knapsack_pro-ruby/compare/v0.43.0...v0.44.0
|
12
|
+
|
5
13
|
### 0.43.0
|
6
14
|
|
7
15
|
* Extract correct test directory from test file pattern that has multiple patterns.
|
data/README.md
CHANGED
@@ -5,7 +5,7 @@
|
|
5
5
|
[![Code Climate](https://codeclimate.com/github/KnapsackPro/knapsack_pro-ruby/badges/gpa.svg)](https://codeclimate.com/github/KnapsackPro/knapsack_pro-ruby)
|
6
6
|
[![Test Coverage](https://codeclimate.com/github/KnapsackPro/knapsack-pro-ruby/badges/coverage.svg)](https://codeclimate.com/github/KnapsackPro/knapsack-pro-ruby/coverage)
|
7
7
|
|
8
|
-
Knapsack Pro gem splits tests across CI nodes and makes sure that tests will run comparable time on each node. It uses [KnapsackPro.com API](http://docs.knapsackpro.com).
|
8
|
+
Knapsack Pro gem splits tests across CI nodes and makes sure that tests will run comparable time on each node. It uses [KnapsackPro.com API](http://docs.knapsackpro.com). You can learn more at [https://knapsackpro.com](https://knapsackpro.com?utm_source=github&utm_medium=readme&utm_campaign=knapsack_pro-ruby_gem&utm_content=learn_more)
|
9
9
|
|
10
10
|
The knapsack_pro gem supports:
|
11
11
|
|
@@ -15,33 +15,33 @@ The knapsack_pro gem supports:
|
|
15
15
|
* [Spinach](https://github.com/codegram/spinach)
|
16
16
|
* [Turnip](https://github.com/jnicklas/turnip)
|
17
17
|
|
18
|
-
__Would you like to try knapsack_pro gem?__ You can [get API token here](http://knapsackpro.com?utm_source=github&utm_medium=readme&utm_campaign=knapsack_pro-ruby_gem&utm_content=get_api_token).
|
18
|
+
__Would you like to try knapsack_pro gem?__ You can [get an API token here](http://knapsackpro.com?utm_source=github&utm_medium=readme&utm_campaign=knapsack_pro-ruby_gem&utm_content=get_api_token).
|
19
19
|
|
20
20
|
## Is knapsack_pro gem free?
|
21
21
|
|
22
22
|
* If your __project is open source__ then you can use Knapsack Pro for free. Please let me know via email (arturtrzop@gmail.com) and I will mark your account on KnapsackPro.com as open source.
|
23
23
|
|
24
|
-
* If your __project is commercial__ then I'd like to get feedback from you and work closely together to validate if the solution I'm building provide a value for the users. Switching to paid plan is a good way to validate that and a way to get support from happy users. Maybe you will be the next one who will join and support the project. Thanks!
|
24
|
+
* If your __project is commercial__ then I'd like to get feedback from you and work closely together to validate if the solution I'm building provide a value for the users. Switching to a paid plan is a good way to validate that and a way to get support from happy users. Maybe you will be the next one who will join and support the project. Thanks!
|
25
25
|
|
26
|
-
# How knapsack_pro
|
26
|
+
# How does knapsack_pro work?
|
27
27
|
|
28
28
|
## Basics
|
29
29
|
|
30
30
|
Basically it will track your branches, commits and for how many CI nodes you are running tests.
|
31
|
-
Collected data about test time execution will be
|
32
|
-
|
31
|
+
Collected data about test time execution will be sent to the API where the test suite split is done.
|
32
|
+
The next time you run your tests, each CI node will get an appropriate set of test files in order to achieve comparable time execution on each CI node.
|
33
33
|
|
34
34
|
## Details
|
35
35
|
|
36
|
-
For instance when you
|
36
|
+
For instance when you run tests with `rake knapsack_pro:rspec`:
|
37
37
|
|
38
38
|
* information about all your existing test files are sent to API http://docs.knapsackpro.com/api/v1/#build_distributions_subset_post
|
39
|
-
* API returns which files should be executed on particular CI node (example KNAPSACK_PRO_CI_NODE_INDEX=0)
|
40
|
-
*
|
41
|
-
* knapsack_pro will run test files which got from API
|
42
|
-
*
|
39
|
+
* the API returns which files should be executed on a particular CI node (example KNAPSACK_PRO_CI_NODE_INDEX=0)
|
40
|
+
* if the API server has data about previous test runs then it will use this to return more accurate test split results, otherwise the API returns a simple split based on directory names
|
41
|
+
* knapsack_pro will run the set of test files which it got from API
|
42
|
+
* once tests are finished, knapsack_pro will send information about time execution of each file to API http://docs.knapsackpro.com/api/v1/#build_subsets_post so data can be used for future test runs
|
43
43
|
|
44
|
-
The knapsack_pro has also [queue mode](#queue-mode) to get optimal test suite split.
|
44
|
+
The knapsack_pro has also [queue mode](#queue-mode) to get an optimal test suite split.
|
45
45
|
|
46
46
|
# Requirements
|
47
47
|
|
@@ -67,9 +67,9 @@ The knapsack_pro has also [queue mode](#queue-mode) to get optimal test suite sp
|
|
67
67
|
- [Set knapsack_pro command to execute tests](#set-knapsack_pro-command-to-execute-tests)
|
68
68
|
- [Repository adapter (How to set up 3 of 3)](#repository-adapter-how-to-set-up-3-of-3)
|
69
69
|
- [When you NOT set global variable `KNAPSACK_PRO_REPOSITORY_ADAPTER` (default)](#when-you-not-set-global-variable-knapsack_pro_repository_adapter-default)
|
70
|
-
- [When you set global variable `KNAPSACK_PRO_REPOSITORY_ADAPTER=git` (required when CI provider is not supported)](#when-you-set-global-variable-knapsack_pro_repository_adaptergit-required-when-ci-provider-is-not-supported)
|
70
|
+
- [When should you set global variable `KNAPSACK_PRO_REPOSITORY_ADAPTER=git` (required when CI provider is not supported)](#when-should-you-set-global-variable-knapsack_pro_repository_adaptergit-required-when-ci-provider-is-not-supported)
|
71
71
|
- [Queue Mode](#queue-mode)
|
72
|
-
- [How queue mode
|
72
|
+
- [How does queue mode work?](#how-does-queue-mode-work)
|
73
73
|
- [How to use queue mode?](#how-to-use-queue-mode)
|
74
74
|
- [Additional info about queue mode](#additional-info-about-queue-mode)
|
75
75
|
- [Extra configuration for Queue Mode](#extra-configuration-for-queue-mode)
|
@@ -102,6 +102,7 @@ The knapsack_pro has also [queue mode](#queue-mode) to get optimal test suite sp
|
|
102
102
|
- [FAQ](#faq)
|
103
103
|
- [Common problems](#common-problems)
|
104
104
|
- [Why I see API error commit_hash parameter is required?](#why-i-see-api-error-commit_hash-parameter-is-required)
|
105
|
+
- [Why I see `LoadError: cannot load such file -- spec_helper`?](#why-i-see-loaderror-cannot-load-such-file----spec_helper)
|
105
106
|
- [Queue Mode problems](#queue-mode-problems)
|
106
107
|
- [Why I don't see collected time execution data for my build in user dashboard?](#why-i-dont-see-collected-time-execution-data-for-my-build-in-user-dashboard)
|
107
108
|
- [Why when I use Queue Mode for RSpec and test fails then I see multiple times info about failed test in RSpec result?](#why-when-i-use-queue-mode-for-rspec-and-test-fails-then-i-see-multiple-times-info-about-failed-test-in-rspec-result)
|
@@ -130,6 +131,7 @@ The knapsack_pro has also [queue mode](#queue-mode) to get optimal test suite sp
|
|
130
131
|
- [How to call `before(:suite)` and `after(:suite)` RSpec hooks only once in Queue Mode?](#how-to-call-beforesuite-and-aftersuite-rspec-hooks-only-once-in-queue-mode)
|
131
132
|
- [How to run knapsack_pro with parallel_tests gem?](#how-to-run-knapsack_pro-with-parallel_tests-gem)
|
132
133
|
- [How to retry failed tests (flaky tests)?](#how-to-retry-failed-tests-flaky-tests)
|
134
|
+
- [How can I run tests from multiple directories?](#how-can-i-run-tests-from-multiple-directories)
|
133
135
|
- [Questions around data usage and security](#questions-around-data-usage-and-security)
|
134
136
|
- [What data is sent to your servers?](#what-data-is-sent-to-your-servers)
|
135
137
|
- [How is that data secured?](#how-is-that-data-secured)
|
@@ -137,16 +139,17 @@ The knapsack_pro has also [queue mode](#queue-mode) to get optimal test suite sp
|
|
137
139
|
- [Gem tests](#gem-tests)
|
138
140
|
- [Spec](#spec)
|
139
141
|
- [Contributing](#contributing)
|
142
|
+
- [Mentions](#mentions)
|
140
143
|
|
141
144
|
<!-- END doctoc generated TOC please keep comment here to allow auto update -->
|
142
145
|
|
143
146
|
## Update gem
|
144
147
|
|
145
|
-
Please check [changelog](CHANGELOG.md) before
|
148
|
+
Please check [changelog](CHANGELOG.md) before updating gem. Knapsack Pro follows [semantic versioning](http://semver.org).
|
146
149
|
|
147
150
|
## Installation
|
148
151
|
|
149
|
-
Add
|
152
|
+
Add these lines to your application's Gemfile:
|
150
153
|
|
151
154
|
```ruby
|
152
155
|
group :test, :development do
|
@@ -159,7 +162,7 @@ And then execute:
|
|
159
162
|
$ bundle install
|
160
163
|
|
161
164
|
|
162
|
-
Add this
|
165
|
+
Add this line at the bottom of `Rakefile` if your project has one:
|
163
166
|
|
164
167
|
```ruby
|
165
168
|
KnapsackPro.load_tasks if defined?(KnapsackPro)
|
@@ -167,16 +170,16 @@ KnapsackPro.load_tasks if defined?(KnapsackPro)
|
|
167
170
|
|
168
171
|
__Please check [online installation guide](http://docs.knapsackpro.com/knapsack_pro-ruby/guide/#questions) to get started.__ It will ask you a few questions and generate instruction steps for your project.
|
169
172
|
|
170
|
-
|
173
|
+
_You only need to read the next section if you want to understand optional gem configuration and features._
|
171
174
|
|
172
175
|
## How to set up
|
173
176
|
|
174
|
-
If you use [VCR](https://github.com/vcr/vcr), [WebMock](https://github.com/bblimke/webmock) or [FakeWeb](https://github.com/chrisk/fakeweb) gems then you need to allow them to make requests to Knapsack Pro API.
|
177
|
+
If you use [VCR](https://github.com/vcr/vcr), [WebMock](https://github.com/bblimke/webmock) or [FakeWeb](https://github.com/chrisk/fakeweb) gems then you need to allow them to make requests to the Knapsack Pro API.
|
175
178
|
|
176
179
|
For VCR add Knapsack Pro API subdomain to [ignore hosts](https://www.relishapp.com/vcr/vcr/v/2-9-3/docs/configuration/ignore-request):
|
177
180
|
|
178
181
|
```ruby
|
179
|
-
# spec/spec_helper.rb or wherever
|
182
|
+
# spec/spec_helper.rb or wherever your VCR configuration is
|
180
183
|
|
181
184
|
require 'vcr'
|
182
185
|
VCR.configure do |config|
|
@@ -193,7 +196,7 @@ require 'fakeweb'
|
|
193
196
|
FakeWeb.allow_net_connect = %r[^https?://api\.knapsackpro\.com]
|
194
197
|
```
|
195
198
|
|
196
|
-
Ensure you have require false for webmock gem when VCR is
|
199
|
+
Ensure you have `require false` in your Gemfile for webmock gem (see below) when VCR is hooked into it. That ensures that the webmock configuration in `spec_helper.rb` (above) is loaded properly.
|
197
200
|
|
198
201
|
```ruby
|
199
202
|
# Gemfile
|
@@ -207,7 +210,7 @@ end
|
|
207
210
|
|
208
211
|
### Usage (How to set up 1 of 3)
|
209
212
|
|
210
|
-
__Tip:__ You can find here example of rails app with already configured
|
213
|
+
__Tip:__ You can find here an example of a rails app with knapsack_pro already configured.
|
211
214
|
|
212
215
|
https://github.com/KnapsackPro/rails-app-with-knapsack_pro
|
213
216
|
|
@@ -262,7 +265,7 @@ KnapsackPro::Adapters::SpinachAdapter.bind
|
|
262
265
|
|
263
266
|
#### Custom configuration
|
264
267
|
|
265
|
-
You can change default Knapsack Pro configuration for RSpec, Cucumber, Minitest or Spinach tests. Here are examples what you can do. Put
|
268
|
+
You can change the default Knapsack Pro configuration for RSpec, Cucumber, Minitest or Spinach tests. Here are examples what you can do. Put the configuration below in place of `CUSTOM_CONFIG_GOES_HERE` (in the configuration samples above).
|
266
269
|
|
267
270
|
```ruby
|
268
271
|
# you can use your own logger
|
@@ -271,13 +274,13 @@ KnapsackPro.logger = Logger.new(STDOUT)
|
|
271
274
|
KnapsackPro.logger.level = Logger::DEBUG
|
272
275
|
```
|
273
276
|
|
274
|
-
Debug is default log level and
|
277
|
+
Debug is default log level and this is recommended. [Read more](#how-can-i-change-log-level).
|
275
278
|
|
276
279
|
### Setup your CI server (How to set up 2 of 3)
|
277
280
|
|
278
281
|
#### Set API key token
|
279
282
|
|
280
|
-
Set one or
|
283
|
+
Set one or more tokens depending on how many test suites you run on CI server.
|
281
284
|
|
282
285
|
* `KNAPSACK_PRO_TEST_SUITE_TOKEN_RSPEC` - as value set token for rspec test suite. Token can be generated when you sign in to [knapsackpro.com](http://www.knapsackpro.com).
|
283
286
|
* `KNAPSACK_PRO_TEST_SUITE_TOKEN_CUCUMBER` - token for cucumber test suite.
|
@@ -316,9 +319,9 @@ You can add `KNAPSACK_PRO_TEST_FILE_PATTERN` if your tests are not in default di
|
|
316
319
|
# Step for Spinach
|
317
320
|
$ KNAPSACK_PRO_TEST_FILE_PATTERN="directory_with_features/**{,/*/**}/*.feature" KNAPSACK_PRO_CI_NODE_TOTAL=2 KNAPSACK_PRO_CI_NODE_INDEX=0 bundle exec rake knapsack_pro:spinach
|
318
321
|
|
319
|
-
__Tip:__ If you use one of supported CI providers then instead of above steps you should [take a look
|
322
|
+
__Tip:__ If you use one of the supported CI providers then instead of the above steps you should [take a look at this](#supported-ci-providers).
|
320
323
|
|
321
|
-
__Tip 2:__ If you use one of unsupported CI providers ([here is list of supported CI providers](#supported-ci-providers)) then you should [set KNAPSACK_PRO_REPOSITORY_ADAPTER=git](#when-you-set-global-variable-knapsack_pro_repository_adaptergit-required-when-ci-provider-is-not-supported).
|
324
|
+
__Tip 2:__ If you use one of unsupported CI providers ([here is list of supported CI providers](#supported-ci-providers)) then you should [set KNAPSACK_PRO_REPOSITORY_ADAPTER=git](#when-should-you-set-global-variable-knapsack_pro_repository_adaptergit-required-when-ci-provider-is-not-supported).
|
322
325
|
|
323
326
|
### Repository adapter (How to set up 3 of 3)
|
324
327
|
|
@@ -332,58 +335,60 @@ By default `KNAPSACK_PRO_REPOSITORY_ADAPTER` variable has no value so knapsack_p
|
|
332
335
|
|
333
336
|
You can also use git as repository adapter to determine branch and commit hash, please see below section.
|
334
337
|
|
335
|
-
#### When you set global variable `KNAPSACK_PRO_REPOSITORY_ADAPTER=git` (required when CI provider is not supported)
|
338
|
+
#### When should you set global variable `KNAPSACK_PRO_REPOSITORY_ADAPTER=git` (required when CI provider is not supported)
|
336
339
|
|
337
|
-
`KNAPSACK_PRO_REPOSITORY_ADAPTER` - When it has value `git
|
340
|
+
`KNAPSACK_PRO_REPOSITORY_ADAPTER` - When it has the value `git`, your local version of git on CI server will be used to get the branch name and commit hash. You also need to set `KNAPSACK_PRO_PROJECT_DIR` with the project directory path.
|
338
341
|
|
339
|
-
`KNAPSACK_PRO_PROJECT_DIR` - Path to the project on CI node for instance `/home/ubuntu/my-app-repository`. It should be
|
342
|
+
`KNAPSACK_PRO_PROJECT_DIR` - Path to the project on the CI node, for instance `/home/ubuntu/my-app-repository`. It should be the top-level directory of your repository.
|
340
343
|
|
341
344
|
## Queue Mode
|
342
345
|
|
343
|
-
knapsack_pro has built
|
344
|
-
CI node overload and
|
345
|
-
|
346
|
+
knapsack_pro has a built-in queue mode designed to determine the optimal test suite split even when there is an unpredictably longer time execution of test files on one node (e.g. by
|
347
|
+
CI node overload and decrease of performance that may affect how long the tests take on that node, or
|
348
|
+
things like external requests done in individual tests).
|
346
349
|
|
347
|
-
### How queue mode
|
350
|
+
### How does queue mode work?
|
348
351
|
|
349
|
-
On the Knapsack Pro API side, there is test
|
350
|
-
that should be executed. Thanks to that each CI node will finish tests at the same time.
|
352
|
+
On the Knapsack Pro API side, there is test file queue generated for your CI build. Each CI node periodically requests the Knapsack Pro API for test files
|
353
|
+
that should be executed next. Thanks to that each CI node will finish tests at the same time.
|
351
354
|
|
352
355
|
### How to use queue mode?
|
353
356
|
|
354
|
-
Please use
|
357
|
+
Please use a separate API token for queue mode from one used already for regular mode.
|
355
358
|
|
356
359
|
Use this command to run queue mode:
|
357
360
|
|
358
361
|
bundle exec rake knapsack_pro:queue:rspec
|
359
362
|
|
360
|
-
If above command fails then you may need to explicitly pass an argument to require `rails_helper` file or `spec_helper` in case you are not doing this in some of your test files:
|
363
|
+
If the above command fails then you may need to explicitly pass an argument to require the `rails_helper` file or `spec_helper` in case you are not doing this in some of your test files:
|
361
364
|
|
362
365
|
bundle exec rake "knapsack_pro:queue:rspec[--require rails_helper]"
|
363
366
|
|
364
|
-
Note
|
365
|
-
The second build should have
|
367
|
+
Note: when you run queue mode command for the first time it might be slower.
|
368
|
+
The second build should have a more optimal test suite split.
|
366
369
|
|
367
370
|
Please ensure you have explicitly set `RAILS_ENV=test` on your CI nodes.
|
368
371
|
|
369
|
-
If you use capybara-screenshot gem then please [follow this step](#how-to-fix-capybara-screenshot-fail-with-systemstackerror-stack-level-too-deep-when-using-queue-mode-for-rspec).
|
372
|
+
If you use the capybara-screenshot gem then please [follow this step](#how-to-fix-capybara-screenshot-fail-with-systemstackerror-stack-level-too-deep-when-using-queue-mode-for-rspec).
|
370
373
|
|
371
|
-
If you use rspec_junit_formatter gem then please [follow this step](#how-to-use-junit-formatter-with-knapsack_pro-queue-mode).
|
374
|
+
If you use the rspec_junit_formatter gem then please [follow this step](#how-to-use-junit-formatter-with-knapsack_pro-queue-mode).
|
375
|
+
|
376
|
+
If your test suite is very long and the RSpec output is too long for your CI node then you can set log level `KNAPSACK_PRO_LOG_LEVEL=info` to don't show debug messages in RSpec output. [Read more about log level](#how-can-i-change-log-level).
|
372
377
|
|
373
378
|
### Additional info about queue mode
|
374
379
|
|
375
|
-
* You should use
|
376
|
-
There might be some cached test suite splits for git commits you run in past for API token you used in queue mode because of the [flag `KNAPSACK_PRO_FIXED_TEST_SUITE_SPLIT=true` for regular mode which is default](#knapsack_pro_fixed_test_suite_splite-test-suite-split-based-on-seed).
|
380
|
+
* You should use a separate API token for queue mode than for regular mode to avoid problems with test suite split (especially in case you would like to go back to regular mode).
|
381
|
+
There might be some cached test suite splits for git commits you have run in past for API token you used in queue mode because of the [flag `KNAPSACK_PRO_FIXED_TEST_SUITE_SPLIT=true` for regular mode which is default](#knapsack_pro_fixed_test_suite_splite-test-suite-split-based-on-seed).
|
377
382
|
|
378
|
-
* If you are not using one of [supported CI providers](#supported-ci-providers) then please note that knapsack_pro gem doesn't
|
383
|
+
* If you are not using one of the [supported CI providers](#supported-ci-providers) then please note that the knapsack_pro gem doesn't have a CI build ID in order to generate a queue for each particular CI build. This may result in two different CI builds taking tests from the same queue when CI builds are running at the same time against the same git commit.
|
379
384
|
|
380
|
-
To avoid this you should specify unique `KNAPSACK_PRO_CI_NODE_BUILD_ID` environment variable for each CI build. This mean that each CI node that is part of particular CI build should have the same value for `KNAPSACK_PRO_CI_NODE_BUILD_ID`.
|
385
|
+
To avoid this you should specify a unique `KNAPSACK_PRO_CI_NODE_BUILD_ID` environment variable for each CI build. This mean that each CI node that is part of particular CI build should have the same value for `KNAPSACK_PRO_CI_NODE_BUILD_ID`.
|
381
386
|
|
382
387
|
* Note that in the Queue Mode by default you cannot retry the failed CI node with exactly the same subset of tests that were run on the CI node in the first place. It's possible in regular mode ([read more](#knapsack_pro_fixed_test_suite_splite-test-suite-split-based-on-seed)). If you want to have similar behavior in Queue Mode you need to explicitly [enable it](#knapsack_pro_fixed_queue_split-remember-queue-split-on-retry-ci-node).
|
383
388
|
|
384
389
|
By default the Queue Mode works this way:
|
385
390
|
|
386
|
-
* If you retry the failed build and your all CI nodes
|
391
|
+
* If you retry the failed build and your all CI nodes start this new build then there will be a new dynamic test suite split across CI nodes. The reason is that the most of the CI providers schedule a new CI build with a different ID when you retry CI build. They retry all CI nodes again. In that case you don't have to worry with below edge cases because the CI build ID will be different so a new queue will be initialized on Knapsack Pro API side and all retried CI node will connect to that queue.
|
387
392
|
|
388
393
|
Edge cases:
|
389
394
|
|
@@ -636,7 +641,12 @@ Here is another example for CircleCI 2.0 platform.
|
|
636
641
|
# auto-balancing CI build time execution to be flat and optimal (as fast as possible).
|
637
642
|
# Queue Mode does dynamic tests allocation so the previous not balanced run command won't
|
638
643
|
# create a bottleneck on the CI node
|
639
|
-
- run:
|
644
|
+
- run:
|
645
|
+
name: RSpec via knapsack_pro Queue Mode
|
646
|
+
command: |
|
647
|
+
# export word is important here!
|
648
|
+
export RAILS_ENV=test
|
649
|
+
bundle exec rake "knapsack_pro:queue:rspec[--format documentation]"
|
640
650
|
```
|
641
651
|
|
642
652
|
Please remember to add additional containers for your project in CircleCI settings.
|
@@ -731,6 +741,8 @@ Here you can find article [how to set up a new pipeline for your project in Buil
|
|
731
741
|
* [Buildkite Rails Parallel Example with Knapsack Pro](https://github.com/KnapsackPro/buildkite-rails-parallel-example-with-knapsack_pro)
|
732
742
|
* [Buildkite Rails Docker Parallel Example with Knapsack Pro](https://github.com/KnapsackPro/buildkite-rails-docker-parallel-example-with-knapsack_pro)
|
733
743
|
|
744
|
+
If you want to use Buildkite retry single agent feature to retry just failed tests on particular agent (CI node) then you should set [`KNAPSACK_PRO_FIXED_QUEUE_SPLIT=true`](#knapsack_pro_fixed_queue_split-remember-queue-split-on-retry-ci-node).
|
745
|
+
|
734
746
|
#### Info for Gitlab CI users
|
735
747
|
|
736
748
|
Gitlab CI does not provide parallel jobs environment variables so you will have to define `KNAPSACK_PRO_CI_NODE_TOTAL` and `KNAPSACK_PRO_CI_NODE_INDEX` for each parallel job running as part of the same `test` stage. Below is relevant part of `.gitlab-ci.yml` configuration for 2 parallel jobs.
|
@@ -857,9 +869,17 @@ If you are going to relay on rspec to autobalance build when cucumber tests were
|
|
857
869
|
When Knapsack Pro API returns error like above the problem is because you use CI provider not supported by knapack_pro which means
|
858
870
|
knapack_pro gem cannot determine the git commit hash and branch name. To fix this problem you can do:
|
859
871
|
|
860
|
-
* if you have git installed on CI node then you can use it to determine git commit hash and branch name. [See this](#when-you-set-global-variable-knapsack_pro_repository_adaptergit-required-when-ci-provider-is-not-supported)
|
872
|
+
* if you have git installed on CI node then you can use it to determine git commit hash and branch name. [See this](#when-should-you-set-global-variable-knapsack_pro_repository_adaptergit-required-when-ci-provider-is-not-supported)
|
861
873
|
* if you have no git installed on CI node then you should manually set `KNAPSACK_PRO_BRANCH` and `KNAPSACK_PRO_COMMIT_HASH`. For instance this might be useful when you use Jenkins. [See this](#when-you-not-set-global-variable-knapsack_pro_repository_adapter-default)
|
862
874
|
|
875
|
+
#### Why I see `LoadError: cannot load such file -- spec_helper`?
|
876
|
+
|
877
|
+
When your tests fails with:
|
878
|
+
|
879
|
+
LoadError: cannot load such file -- spec_helper
|
880
|
+
|
881
|
+
then problem might be related to the fact you specified complex `KNAPSACK_PRO_TEST_FILE_PATTERN` and knapsack_pro gem cannot detect correct main test directory with spec_helper. You should set `KNAPSACK_PRO_TEST_DIR=spec`. Please [read also example](#how-can-i-run-tests-from-multiple-directories).
|
882
|
+
|
863
883
|
#### Queue Mode problems
|
864
884
|
|
865
885
|
##### Why I don't see collected time execution data for my build in user dashboard?
|
@@ -1114,7 +1134,7 @@ The xml report will contain all tests executed across intermediate test subset r
|
|
1114
1134
|
old_xml_file = 'tmp/rspec.xml'
|
1115
1135
|
# move results to new_xml_file so the results won't accumulate with duplicated xml tags in old_xml_file
|
1116
1136
|
new_xml_file = 'tmp/rspec_final_results.xml'
|
1117
|
-
FileUtils.mv(old_xml_file, new_xml_file) if File.
|
1137
|
+
FileUtils.mv(old_xml_file, new_xml_file) if File.exist?(old_xml_file)
|
1118
1138
|
end
|
1119
1139
|
|
1120
1140
|
#### How many API keys I need?
|
@@ -1306,6 +1326,35 @@ bundle exec rake knapsack_pro:queue:rspec
|
|
1306
1326
|
bundle exec rspec --only-failures
|
1307
1327
|
```
|
1308
1328
|
|
1329
|
+
#### How can I run tests from multiple directories?
|
1330
|
+
|
1331
|
+
The test file pattern config option supports any glob pattern handled by [`Dir.glob`](http://ruby-doc.org/core-2.4.1/Dir.html#method-c-glob) and can be configured to pull test files from multiple directories. An example of this when using RSpec would be `""{spec,engines/*/spec}/**/*_spec.rb" "`. For complex cases like this, the test directory can't be extracted and must be specified manually using the `KNAPSACK_PRO_TEST_DIR` environment variable:
|
1332
|
+
|
1333
|
+
```
|
1334
|
+
# This is example where you have in engines directory multiple projects
|
1335
|
+
# and each project directory has a spec folder and you would like to run tests for it.
|
1336
|
+
# You want to use the spec_helper from the main spec directory.
|
1337
|
+
#
|
1338
|
+
# Tree:
|
1339
|
+
# * spec
|
1340
|
+
# * engines
|
1341
|
+
# * project_a
|
1342
|
+
# * spec
|
1343
|
+
# * project_b
|
1344
|
+
# * spec
|
1345
|
+
$ KNAPSACK_PRO_TEST_DIR=spec KNAPSACK_PRO_TEST_FILE_PATTERN="{spec,engines/*/spec}/**/*_spec.rb" bundle exec rake knapsack_pro:queue:rspec
|
1346
|
+
```
|
1347
|
+
|
1348
|
+
`KNAPSACK_PRO_TEST_DIR` will be your default path for rspec so you should put there your `spec_helper.rb`. Please ensure you will require it in your test files this way if something doesn't work:
|
1349
|
+
|
1350
|
+
```ruby
|
1351
|
+
# good
|
1352
|
+
require_relative 'spec_helper'
|
1353
|
+
|
1354
|
+
# bad - won't work
|
1355
|
+
require 'spec_helper'
|
1356
|
+
```
|
1357
|
+
|
1309
1358
|
### Questions around data usage and security
|
1310
1359
|
|
1311
1360
|
#### What data is sent to your servers?
|
@@ -1317,7 +1366,7 @@ Here is the [full specification of the API](http://docs.knapsackpro.com/api/v1/)
|
|
1317
1366
|
#### How is that data secured?
|
1318
1367
|
|
1319
1368
|
The test file paths and/or branch names can be [encrypted](#test-file-names-encryption) on your CI node with a salt and later send to knapsackpro.com API.
|
1320
|
-
You generate the salt locally and
|
1369
|
+
You generate the salt locally and __only you__ can decrypt the test file paths or branch names with the salt. Here you can [see how the data are encrypted](lib/knapsack_pro/crypto/digestor.rb).
|
1321
1370
|
|
1322
1371
|
Connection with knapsackpro.com server is via https.
|
1323
1372
|
|
@@ -1346,3 +1395,10 @@ To run specs for Knapsack Pro gem type:
|
|
1346
1395
|
4. Push to the branch (`git push origin my-new-feature`)
|
1347
1396
|
5. You can create example tests in related repository with example of [rails application and knapsack_pro gem usage](https://github.com/KnapsackPro/rails-app-with-knapsack_pro).
|
1348
1397
|
6. Create a new Pull Request
|
1398
|
+
|
1399
|
+
## Mentions
|
1400
|
+
|
1401
|
+
List of articles where people mentioned Knapsack Pro:
|
1402
|
+
|
1403
|
+
* [Treat your Build Pipeline as a Product](https://medium.com/mydr-engineering/treat-your-build-pipeline-as-a-product-61a1b24ae538).
|
1404
|
+
* Video - [Your Build Pipeline is a Product](https://www.youtube.com/watch?v=7e8Qk3H6xhg&t=21m14s)
|
data/lib/knapsack_pro/version.rb
CHANGED
@@ -14,19 +14,33 @@ describe KnapsackPro::BaseAllocatorBuilder do
|
|
14
14
|
subject { allocator_builder.test_dir }
|
15
15
|
|
16
16
|
before do
|
17
|
-
expect(KnapsackPro::
|
17
|
+
expect(KnapsackPro::Config::Env).to receive(:test_dir).and_return(test_dir)
|
18
18
|
end
|
19
19
|
|
20
|
-
context 'when
|
21
|
-
let(:
|
20
|
+
context 'when test_dir is defined in ENV' do
|
21
|
+
let(:test_dir) { double }
|
22
22
|
|
23
|
-
it { should eq
|
23
|
+
it { should eq test_dir }
|
24
24
|
end
|
25
25
|
|
26
|
-
context 'when
|
27
|
-
let(:
|
26
|
+
context 'when test_dir is not defined in ENV' do
|
27
|
+
let(:test_dir) { nil }
|
28
28
|
|
29
|
-
|
29
|
+
before do
|
30
|
+
expect(KnapsackPro::TestFilePattern).to receive(:call).with(adapter_class).and_return(test_file_pattern)
|
31
|
+
end
|
32
|
+
|
33
|
+
context 'when single pattern' do
|
34
|
+
let(:test_file_pattern) { 'spec/**{,/*/**}/*_spec.rb' }
|
35
|
+
|
36
|
+
it { should eq 'spec' }
|
37
|
+
end
|
38
|
+
|
39
|
+
context 'when multiple patterns' do
|
40
|
+
let(:test_file_pattern) { '{spec/controllers/**/*.rb,spec/decorators/**/*.rb}' }
|
41
|
+
|
42
|
+
it { should eq 'spec' }
|
43
|
+
end
|
30
44
|
end
|
31
45
|
end
|
32
46
|
end
|
@@ -153,6 +153,20 @@ describe KnapsackPro::Config::Env do
|
|
153
153
|
end
|
154
154
|
end
|
155
155
|
|
156
|
+
describe '.test_dir' do
|
157
|
+
subject { described_class.test_dir }
|
158
|
+
|
159
|
+
context 'when ENV exists' do
|
160
|
+
let(:test_dir) { 'spec' }
|
161
|
+
before { stub_const("ENV", { 'KNAPSACK_PRO_TEST_DIR' => test_dir }) }
|
162
|
+
it { should eql test_dir }
|
163
|
+
end
|
164
|
+
|
165
|
+
context "when ENV doesn't exist" do
|
166
|
+
it { should be_nil }
|
167
|
+
end
|
168
|
+
end
|
169
|
+
|
156
170
|
describe '.repository_adapter' do
|
157
171
|
subject { described_class.repository_adapter }
|
158
172
|
|
metadata
CHANGED
@@ -1,14 +1,14 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: knapsack_pro
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 0.
|
4
|
+
version: 0.44.0
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- ArturT
|
8
8
|
autorequire:
|
9
9
|
bindir: bin
|
10
10
|
cert_chain: []
|
11
|
-
date: 2017-
|
11
|
+
date: 2017-07-31 00:00:00.000000000 Z
|
12
12
|
dependencies:
|
13
13
|
- !ruby/object:Gem::Dependency
|
14
14
|
name: rake
|