rspec-tracer 0.6.1 → 0.9.0

Sign up to get free protection for your applications and to get access to all the features.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 4dff7e48382045aef3821a6f863521c9f74823b5bbd12c16b09c41b47277124e
4
- data.tar.gz: cff630db632b7fa96be3d3de675b994325f815c69613b84d891bc8313688ab67
3
+ metadata.gz: 1df1d0c90323ae123f24629f74f654bce0fff93bba8e8b052388d582dfb64d8f
4
+ data.tar.gz: 5caabede1fb71ffcbbd70f0e9bfe8ca355303e843dbb99cacf14f146149afe5a
5
5
  SHA512:
6
- metadata.gz: 83348356f2a12e930d73cec63e867a13ffab85e428696fc3d91dfeab5a691bac4641d848330d0225e08bc7156d0902faffc8ecd8f4a8871b24c093513c6a8b0d
7
- data.tar.gz: 2cc9084dbeb9ca0fa26997b33b486360e3c1fe8b7efbb91f28b3a1786bf0e966b32f75b5d7b98a3488c5ced8d1704005f1c1a5972e876e25d3877a40d8068379
6
+ metadata.gz: 5ad9ec360d867ed745039fc2cf2a4aef314912a5b6a596c8c79e9d6615b00f2b578f5d3a164c2725ed710d9b0aa055855c607eebe52f7b032d1b959d483cb079
7
+ data.tar.gz: 347236ad3e59372ff1cf33aa3c812ae5a22911ff6b46bfdfced44b96ef80161c4da0798a9afecf96846bc44963713bac4aa625c714266431b975c3152ebd92b1
data/CHANGELOG.md CHANGED
@@ -1,3 +1,29 @@
1
+ ## [0.9.0] - 2021-09-15
2
+
3
+ ### Added
4
+
5
+ - Handling all examples filtered by RSpec (#34)
6
+ - Warn on incorrect analysis to stop using RSpec Tracer (#35)
7
+ - Run `SimpleCov.at_exit` hook (#36)
8
+
9
+ ## [0.8.0] - 2021-09-13
10
+
11
+ ### Fixed
12
+
13
+ Unable to find cache in case of history rewrites (#33)
14
+
15
+ ## [0.7.0] - 2021-09-10
16
+
17
+ ### Fixed
18
+
19
+ Missing spec files for the gem
20
+
21
+ ## [0.6.2] - 2021-09-07
22
+
23
+ ### Added
24
+
25
+ Improvements towards reducing dependency and coverage processing time (#26)
26
+
1
27
  ## [0.6.1] - 2021-09-06
2
28
 
3
29
  ### Fixed
data/README.md CHANGED
@@ -1,8 +1,12 @@
1
1
  ![](./readme_files/rspec_tracer.png)
2
2
 
3
- RSpec Tracer is a **specs dependency analysis tool** and a **test skipper for RSpec**.
4
- It maintains a list of files for each test, enabling itself to skip tests in the
5
- subsequent runs if none of the dependent files are changed.
3
+ [![Maintainability](https://api.codeclimate.com/v1/badges/eabce2757839c08d8f8d/maintainability)](https://codeclimate.com/github/avmnu-sng/rspec-tracer/maintainability)
4
+ [![Test Coverage](https://api.codeclimate.com/v1/badges/eabce2757839c08d8f8d/test_coverage)](https://codeclimate.com/github/avmnu-sng/rspec-tracer/test_coverage)
5
+ [![Gem Version](https://badge.fury.io/rb/rspec-tracer.svg)](https://badge.fury.io/rb/rspec-tracer)
6
+
7
+ RSpec Tracer is a **specs dependency analyzer**, **flaky tests detector**, **tests accelerator**,
8
+ and **coverage reporter** tool. It maintains a list of files for each test, enabling
9
+ itself to skip tests in the subsequent runs if none of the dependent files are changed.
6
10
 
7
11
  It uses [Ruby's built-in coverage library](https://ruby-doc.org/stdlib/libdoc/coverage/rdoc/Coverage.html)
8
12
  to keep track of the coverage for each test. For each test executed, the coverage
@@ -18,45 +22,25 @@ Knowing the examples and files dependency gives us a better insight into the cod
18
22
  and we have **a clear idea of what to test for when making any changes**. With this data,
19
23
  we can also analyze the coupling between different components and much more.
20
24
 
21
- ## Note
25
+ RSpec Tracer requires **Ruby 2.5+** and **rspec-core >= 3.6.0**. To use with **Rails 5+**,
26
+ make sure to use **rspec-rails >= 4.0.0**. If you are using SimpleCov, it is
27
+ recommended to use **simplecov >= 0.12.0**. To use RSpec Tracer **cache on CI**, you
28
+ need to have an **S3 bucket** and **[AWS CLI](https://aws.amazon.com/cli/)**
29
+ installed.
22
30
 
23
- You should take some time and go through the **[document](./RSPEC_TRACER.md)** describing
24
- the **intention** and implementation details of **managing dependency**, **managing flaky tests**,
25
- **skipping tests**, and **caching on CI**. You must go through the README file before
26
- integrating the gem into your project to better understand what is happening.
31
+ You should take some time and go through the **[document](./RSPEC_TRACER.md)**
32
+ describing the **intention** and implementation details of **managing dependency**,
33
+ **managing flaky tests**, **skipping tests**, and **caching on CI**.
27
34
 
28
35
  ## Table of Contents
29
36
 
30
37
  * [Demo](#demo)
31
- * [Installation](#installation)
32
- * [Compatibility](#compatibility)
33
- * [Additional Tools](#additional-tools)
34
38
  * [Getting Started](#getting-started)
35
- * [Environment Variables](#environment-variables)
36
- * [BUNDLE_PATH](#bundle_path)
37
- * [CI](#ci)
38
- * [LOCAL_AWS](#local_aws)
39
- * [RSPEC_TRACER_NO_SKIP](#rspec_tracer_no_skip)
40
- * [RSPEC_TRACER_S3_URI](#rspec_tracer_s3_uri)
41
- * [RSPEC_TRACER_UPLOAD_LOCAL_CACHE](#rspec_tracer_upload_local_cache)
42
- * [RSPEC_TRACER_VERBOSE](#rspec_tracer_verbose)
43
- * [TEST_SUITES](#test_suites)
44
- * [TEST_SUITE_ID](#test_suite_id)
45
- * [Sample Reports](#sample-reports)
46
- * [Examples](#examples)
47
- * [Flaky Examples](#flaky-examples)
48
- * [Examples Dependency](#examples-dependency)
49
- * [Files Dependency](#files-dependency)
50
- * [Configuring RSpec Tracer](#configuring-rspec-tracer)
39
+ * [Configuring CI Caching](#configuring-ci-caching)
40
+ * [Advanced Configuration](#advanced-configuration)
51
41
  * [Filters](#filters)
52
- * [Defining Custom Filteres](#defining-custom-filteres)
53
- * [String Filter](#string-filter)
54
- * [Regex Filter](#regex-filter)
55
- * [Block Filter](#block-filter)
56
- * [Array Filter](#array-filter)
57
- * [Contributing](#contributing)
58
- * [License](#license)
59
- * [Code of Conduct](#code-of-conduct)
42
+ * [Environment Variables](#environment-variables)
43
+ * [When Should You Not Use RSpec Tracer](#when-should-you-not-use-rspec-tracer)
60
44
 
61
45
  ## Demo
62
46
 
@@ -66,317 +50,406 @@ integrating the gem into your project to better understand what is happening.
66
50
  **Next Run**
67
51
  ![](./readme_files/next_run.gif)
68
52
 
53
+ You get the following three reports:
69
54
 
70
- ## Installation
55
+ ### All Examples Report
71
56
 
72
- Add this line to your `Gemfile` and `bundle install`:
73
- ```ruby
74
- gem 'rspec-tracer', group: :test, require: false
75
- ```
57
+ These reports provide basic test information:
76
58
 
77
- And, add the followings to your `.gitignore`:
78
- ```
79
- /rspec_tracer_cache/
80
- /rspec_tracer_coverage/
81
- /rspec_tracer_report/
82
- ```
59
+ **First Run**
83
60
 
84
- ### Compatibility
61
+ ![](./readme_files/examples_report_first_run.png)
85
62
 
86
- RSpec Tracer requires **Ruby 2.5+** and **rspec-core >= 3.6.0**. To use with **Rails 5+**,
87
- make sure to use **rspec-rails >= 4.0.0**. If you are using SimpleCov, it is
88
- recommended to use **simplecov >= 0.12.0**.
63
+ **Next Run**
89
64
 
90
- ### Additional Tools
65
+ ![](./readme_files/examples_report_next_run.png)
91
66
 
92
- To use RSpec Tracer on CI, you need to have an **S3 bucket** and
93
- **[AWS CLI](https://aws.amazon.com/cli/)** installed.
67
+ ### Flaky Examples Report
94
68
 
95
- ## Getting Started
69
+ These reports provide flaky tests information. Assuming **the following two tests
70
+ failed in the first run.**
96
71
 
97
- 1. **Load and Start RSpec Tracer**
72
+ **Next Run**
98
73
 
99
- - **With SimpleCov**
74
+ ![](./readme_files/flaky_examples_report_first_run.png)
100
75
 
101
- If you are using `SimpleCov`, load RSpec Tracer right after the SimpleCov load
102
- and launch:
76
+ **Another Run**
103
77
 
104
- ```ruby
105
- require 'simplecov'
106
- SimpleCov.start
78
+ ![](./readme_files/flaky_examples_report_next_run.png)
107
79
 
108
- # Load RSpec Tracer
109
- require 'rspec_tracer'
110
- RSpecTracer.start
111
- ```
80
+ ### Examples Dependency Report
112
81
 
113
- Currently using RSpec Tracer with SimpleCov has the following two limitations:
114
- - SimpleCov **won't be able to provide branch coverage report** even when enabled.
115
- - RSpec Tracer **nullifies the `SimpleCov.at_exit`** callback.
82
+ These reports show a list of dependent files for each test.
116
83
 
117
- - **Without SimpleCov**
84
+ ![](./readme_files/examples_dependency_report.png)
118
85
 
119
- Load and launch RSpec Tracer at the very top of `spec_helper.rb` (or `rails_helper.rb`,
120
- `test/test_helper.rb`). Note that `RSpecTracer.start` must be issued **before loading
121
- any of the application code.**
86
+ ### Files Dependency Report
122
87
 
123
- ```ruby
124
- # Load RSpec Tracer
125
- require 'rspec_tracer'
126
- RSpecTracer.start
127
- ```
88
+ These reports provide information on the total number of tests that will run after changing this particular file.
128
89
 
129
- 2. To enable RSpec Tracer to share cache between different builds on CI, update the
130
- Rakefile in your project to have the following:
90
+ ![](./readme_files/files_dependency_report.png)
131
91
 
132
- ```ruby
133
- spec = Gem::Specification.find_by_name('rspec-tracer')
134
92
 
135
- load "#{spec.gem_dir}/lib/rspec_tracer/remote_cache/Rakefile"
136
- ```
137
- 3. Before running tests, download the remote cache using the following rake task:
93
+ ## Getting Started
138
94
 
139
- ```sh
140
- bundle exec rake rspec_tracer:remote_cache:download
95
+ 1. Add this line to your `Gemfile` and `bundle install`:
96
+ ```ruby
97
+ gem 'rspec-tracer', '~> 0.9', group: :test, require: false
141
98
  ```
142
- 4. Run the tests with RSpec using `bundle exec rspec`.
143
- 5. After running tests, upload the local cache using the following rake task:
144
99
 
145
- ```sh
146
- bundle exec rake rspec_tracer:remote_cache:upload
100
+ And, add the followings to your `.gitignore`:
147
101
  ```
148
- 6. After running your tests, open `rspec_tracer_report/index.html` in the
149
- browser of your choice.
150
-
151
- ## Environment Variables
152
-
153
- To get better control on execution, you can use the following environment variables
154
- whenever required.
155
-
156
- ### BUNDLE_PATH
157
-
158
- Since the bundler uses a vendor directory inside the project, it might cause slowness
159
- depending on the vendor size. You can configure the bundle path outside of the project
160
- using `BUNDLE_PATH` environment variable, for example, `BUNDLE_PATH=$HOME/vendor/bundle`.
161
- Make sure to cache this directory in the CI configuration.
162
-
163
- ### CI
164
-
165
- Mostly all the CI have `CI=true`. If not, you should explicitly set it to `true`.
166
-
167
- ### LOCAL_AWS
168
-
169
- In case you want to test out the caching feature in the local development environment.
170
- You can install [localstack](https://github.com/localstack/localstack) and
171
- [awscli-local](https://github.com/localstack/awscli-local) and then invoke the
172
- rake tasks with `LOCAL_AWS=true`.
173
-
174
- ### RSPEC_TRACER_NO_SKIP
175
-
176
- The default value is `false.` If set to `true`, the RSpec Tracer will not skip
177
- any tests. Note that it will continue to maintain cache files and generate reports.
178
-
179
- ```ruby
180
- RSPEC_TRACER_NO_SKIP=true bundle exec rspec
181
- ```
182
-
183
- ### RSPEC_TRACER_S3_URI
184
-
185
- You should provide the S3 bucket path to store the cache files.
186
-
187
- ```ruby
188
- export RSPEC_TRACER_S3_URI=s3://ci-artifacts-bucket/rspec-tracer-cache
189
- ```
102
+ /rspec_tracer_cache/
103
+ /rspec_tracer_coverage/
104
+ /rspec_tracer_report/
105
+ ```
106
+ 2. Load and launch RSpec Tracer at the very top of `spec_helper.rb` (or `rails_helper.rb`,
107
+ `test/test_helper.rb`). Note that `RSpecTracer.start` must be issued **before loading
108
+ any of the application code.**
190
109
 
191
- ### RSPEC_TRACER_UPLOAD_LOCAL_CACHE
110
+ ```ruby
111
+ # Load RSpec Tracer
112
+ require 'rspec_tracer'
113
+ RSpecTracer.start
114
+ ```
192
115
 
193
- By default, RSpec Tracer does not upload local cache files. You can set this
194
- environment variable to `true` to upload the local cache to S3.
116
+ **If you are using SimpleCov**, load RSpec Tracer right after the SimpleCov load
117
+ and launch:
195
118
 
196
- ### RSPEC_TRACER_VERBOSE
119
+ ```ruby
120
+ require 'simplecov'
121
+ SimpleCov.start
197
122
 
198
- To print the intermediate steps and time taken, use this environment variable:
123
+ # Load RSpec Tracer
124
+ require 'rspec_tracer'
125
+ RSpecTracer.start
126
+ ```
199
127
 
200
- ```sh
201
- export RSPEC_TRACER_VERBOSE=true
202
- ```
128
+ If you use RSpec Tracer with SimpleCov, then **SimpleCov would not report branch
129
+ coverage results even when enabled**.
203
130
 
204
- ### TEST_SUITES
131
+ 3. After running your tests, open `rspec_tracer_report/index.html` in the browser
132
+ of your choice.
205
133
 
206
- Set this environment variable when using test suite id. It determines the total
207
- number of different test suites you are running.
134
+ ## Configuring CI Caching
208
135
 
136
+ To enable RSpec Tracer to share cache between different builds on CI, update the
137
+ Rakefile in your project to have the following:
209
138
  ```ruby
210
- export TEST_SUITES=8
211
- ```
212
-
213
- ### TEST_SUITE_ID
139
+ spec = Gem::Specification.find_by_name('rspec-tracer')
214
140
 
215
- If you have a large set of tests to run, it is recommended to run them in
216
- separate groups. This way, RSpec Tracer is not overwhelmed with loading massive
217
- cached data in the memory. Also, it generates and uses cache for specific test suites
218
- and not merge them.
219
-
220
- ```ruby
221
- TEST_SUITE_ID=1 bundle exec rspec spec/models
222
- TEST_SUITE_ID=2 bundle exec rspec spec/helpers
141
+ load "#{spec.gem_dir}/lib/rspec_tracer/remote_cache/Rakefile"
223
142
  ```
224
143
 
225
- If you run parallel builds on the CI, you should specify the test suite ID and
226
- the total number of test suites when downloading the cache files.
227
-
144
+ Before running tests, download the remote cache using the following rake task:
228
145
  ```sh
229
- $ TEST_SUITES=5 TEST_SUITE_ID=1 bundle exec rake rspec_tracer:remote_cache:download
146
+ bundle exec rake rspec_tracer:remote_cache:download
230
147
  ```
231
148
 
232
- In this case, the appropriate cache should have all the cache files available on
233
- the S3 for each test suite, not just for the current one. Also, while uploading,
234
- make sure to provide the test suite id.
235
-
149
+ After running tests, upload the local cache using the following rake task:
236
150
  ```sh
237
- $ TEST_SUITE_ID=1 bundle exec rake rspec_tracer:remote_cache:upload
151
+ bundle exec rake rspec_tracer:remote_cache:upload
238
152
  ```
239
153
 
240
- ## Sample Reports
241
-
242
- You get the following three reports:
243
-
244
- ### Examples
245
-
246
- These reports provide basic test information:
247
-
248
- **First Run**
249
-
250
- ![](./readme_files/examples_report_first_run.png)
251
-
252
- **Next Run**
253
-
254
- ![](./readme_files/examples_report_next_run.png)
154
+ You must set the following two environment variables:
255
155
 
256
- ### Flaky Examples
156
+ - **`GIT_BRANCH`** is the git branch name you are running the CI build on.
157
+ - **`RSPEC_TRACER_S3_URI`** is the S3 bucket path to store the cache files.
158
+ ```sh
159
+ export RSPEC_TRACER_S3_URI=s3://ci-artifacts-bucket/rspec-tracer-cache
160
+ ```
257
161
 
258
- These reports provide flaky tests information. Assuming **the following two tests
259
- failed in the first run.**
162
+ ## Advanced Configuration
260
163
 
261
- **Next Run**
164
+ Configuration settings can be applied in three formats, which are completely equivalent:
262
165
 
263
- ![](./readme_files/flaky_examples_report_first_run.png)
166
+ - The most common way is to configure it directly in your start block:
167
+ ```ruby
168
+ RSpecTracer.start do
169
+ config_option 'foo'
170
+ end
171
+ ```
172
+ - You can also set all configuration options directly:
173
+ ```ruby
174
+ RSpecTracer.config_option 'foo'
175
+ ```
264
176
 
265
- **Another Run**
177
+ - If you do not want to start tracer immediately after launch or want to add
178
+ additional configuration later on in a concise way, use:
179
+ ```ruby
180
+ RSpecTracer.configure do
181
+ config_option 'foo'
182
+ end
183
+ ```
266
184
 
267
- ![](./readme_files/flaky_examples_report_next_run.png)
185
+ The available configuration options are:
268
186
 
269
- ### Examples Dependency
187
+ - **`root dir`** to set the project root. The default value is the current working
188
+ directory.
189
+ - **`add_filter filter`** to apply [filters](#filters) on the source files to
190
+ exclude them from the dependent files list.
191
+ - **`filters.clear`** to remove the default configured dependent files filters.
192
+ - **`add_coverage_filter filter`** to apply [filters](#filters) on the source files
193
+ to exclude them from the coverage report.
194
+ - **`coverage_filters.clear`** to remove the default configured coverage files filters.
195
+ - **`coverage_track_files glob`** to include files in the given glob pattern in
196
+ the coverage report if these files are not already present.
270
197
 
271
- These reports show a list of dependent files for each test.
198
+ ```ruby
199
+ RSpecTracer.start do
200
+ # Configure project root
201
+ root '/tmp/my_project'
272
202
 
273
- ![](./readme_files/examples_dependency_report.png)
203
+ # Clear existing filters
204
+ filters.clear
205
+ # Add dependent files filter
206
+ add_filter %r{^/tasks/}
274
207
 
275
- ### Files Dependency
208
+ # Clear existing coverage filters
209
+ coverage_filters.clear
210
+ # Add coverage files filter
211
+ add_coverage_filter %w[/features/ /spec/ /tests/]
276
212
 
277
- These reports provide information on the total number of tests that will run after changing this particular file.
213
+ # Define glob to track files in the coverage report
214
+ coverage_track_files '{app,lib}/**/*.rb'
215
+ end
216
+ ```
278
217
 
279
- ![](./readme_files/files_dependency_report.png)
218
+ You can configure the RSpec Tracer reports directories using the following environment
219
+ variables:
220
+
221
+ - **`RSPEC_TRACER_CACHE_DIR`** to update the default cache directory (`rspec_tracer_cache`).
222
+ ```sh
223
+ export RSPEC_TRACER_CACHE_DIR=/tmp/rspec_tracer_cache
224
+ ```
225
+ - **`RSPEC_TRACER_COVERAGE_DIR`** to update the default coverage directory (`rspec_tracer_coverage`).
226
+ ```sh
227
+ export RSPEC_TRACER_CACHE_DIR=/tmp/rspec_tracer_coverage
228
+ ```
229
+ - **`RSPEC_TRACER_REPORT_DIR`** to update the default html reports directory (`rspec_tracer_report`).
230
+ ```sh
231
+ export RSPEC_TRACER_CACHE_DIR=/tmp/rspec_tracer_report
232
+ ```
233
+
234
+ These settings are available through environment variables because the rake tasks
235
+ to download and upload the cache files need to use the same directories.
280
236
 
281
- ## Configuring RSpec Tracer
237
+ ## Filters
282
238
 
283
- Configuration settings can be applied in three formats, which are completely equivalent:
239
+ By default, RSpec Tracer ignores all the files outside of the project root directory -
240
+ otherwise you would end up with the source files in the gems you are using in the
241
+ project. It also applies the following filters:
242
+ ```ruby
243
+ RSpecTracer.configure do
244
+ add_filter '/vendor/bundle/'
245
+
246
+ add_coverage_filter %w[
247
+ /autotest/
248
+ /features/
249
+ /spec/
250
+ /test/
251
+ /vendor/bundle/
252
+ ].freeze
253
+ end
254
+ ```
284
255
 
285
- - The most common way is to configure it directly in your start block:
256
+ ### Defining Custom Filteres
286
257
 
287
- ```ruby
288
- RSpecTracer.start do
289
- config_option 'foo'
258
+ You can currently define a filter using either a String or Regexp (that will then
259
+ be Regexp-matched against each source file's name relative to the project root),
260
+ a block or by passing in your own Filter class.
261
+
262
+ - **String Filter**: The string filter matches files that have the given string
263
+ in their name. For example, the following string filter will remove all files that
264
+ have `"/helpers/"` in their name.
265
+ ```ruby
266
+ RSpecTracer.start do
267
+ add_filter '/helpers/'
268
+ end
269
+ ```
270
+
271
+ - **Regex Filter**: The regex filter removes all files that have a successful name
272
+ match against the given regex expression. This simple regex filter will remove
273
+ all files that start with `%r{^/helper/}` in their name:
274
+ ```ruby
275
+ RSpecTracer.start do
276
+ add_filter %r{^/helpers/}
277
+ end
278
+ ```
279
+
280
+ - **Block Filter**: Block filters receive a `Hash` object and expect your block
281
+ to return either **true** (if the file is to be removed from the result) or **false**
282
+ (if the result should be kept). In the below example, the filter will remove all
283
+ files that match `"/helpers/"` in their path.
284
+ ```ruby
285
+ RSpecTracer.start do
286
+ add_filter do |source_file|
287
+ source_file[:file_path].include?('/helpers/')
290
288
  end
291
- ```
292
- - You can also set all configuration options directly:
293
-
294
- ```ruby
295
- RSpecTracer.config_option 'foo'
296
- ```
297
-
298
- - If you do not want to start tracer immediately after launch or want to add
299
- additional configuration later on in a concise way, use:
289
+ end
290
+ ```
300
291
 
301
- ```ruby
302
- RSpecTracer.configure do
303
- config_option 'foo'
304
- end
305
- ```
292
+ You can also use `source_file[:name]` to define the return value of the block
293
+ filter for the given source file.
306
294
 
307
- ## Filters
295
+ - **Array Filter**: You can pass in an array containing any of the other filter types:
296
+ ```ruby
297
+ RSpecTracer.start do
298
+ add_filter ['/helpers/', %r{^/utils/}]
299
+ end
300
+ ```
308
301
 
309
- RSpec Tracer supports two types of filters:
302
+ ## Environment Variables
310
303
 
311
- - To exclude selected files from the dependency list of tests:
304
+ To get better control on execution, you can use the following environment variables
305
+ whenever required.
312
306
 
313
- ```ruby
314
- RSpecTracer.start do
315
- add_filter %r{^/helpers/}
316
- end
317
- ```
318
- - To exclude selected files from the coverage data. You should only use this
319
- when not using SimpleCov.
307
+ - **`LOCAL_AWS (default: false)`:** In case you want to test out the caching feature in the local
308
+ development environment. You can install [localstack](https://github.com/localstack/localstack)
309
+ and [awscli-local](https://github.com/localstack/awscli-local) and then invoke the
310
+ rake tasks with `LOCAL_AWS=true`.
320
311
 
321
- ```ruby
322
- RSpecTracer.start do
323
- add_coverage_filter %r{^/tasks/}
324
- end
325
- ```
312
+ - **`RSPEC_TRACER_NO_SKIP (default: false)`:** Use this environment variables to
313
+ not skip any tests. Note that it will continue to maintain cache files and generate
314
+ reports.
326
315
 
327
- By default, a filter is applied that removes all files OUTSIDE of your project's
328
- root directory - otherwise you'd end up with the source files in the gems you are
329
- using as tests dependency.
316
+ - **`RSPEC_TRACER_UPLOAD_LOCAL_CACHE (default: false)`:** By default, RSpec Tracer
317
+ does not upload local cache files. You can set this environment variable to `true`
318
+ to upload the local cache to S3.
330
319
 
331
- ### Defining Custom Filteres
320
+ - **`RSPEC_TRACER_VERBOSE (default: false)`:** To print the intermediate steps
321
+ and time taken, use this environment variable.
332
322
 
333
- You can currently define a filter using either a String or Regexp (that will then
334
- be Regexp-matched against each source file's path), a block or by passing in your
335
- own Filter class.
323
+ - **`TEST_SUITES`:** Set this environment variable when running parallel builds
324
+ in the CI. It determines the total number of different test suites you are running.
325
+ ```sh
326
+ export TEST_SUITES=8
327
+ ```
336
328
 
337
- #### String Filter
329
+ - **`TEST_SUITE_ID`:** If you have a large set of tests to run, it is recommended
330
+ to run them in separate groups. This way, RSpec Tracer is not overwhelmed with
331
+ loading massive cached data in the memory. Also, it generates and uses cache for
332
+ specific test suites and not merge them.
333
+ ```sh
334
+ TEST_SUITE_ID=1 bundle exec rspec spec/models
335
+ TEST_SUITE_ID=2 bundle exec rspec spec/helpers
336
+ ```
338
337
 
339
- ```ruby
340
- RSpecTracer.start do
341
- add_filter '/helpers/'
342
- end
343
- ```
338
+ ## When Should You Not Use RSpec Tracer
344
339
 
345
- This simple string filter will remove all files that match "/helpers/" in their path.
340
+ To uniquely identify the examples is one of the requirements for the correctness
341
+ of the RSpec Tracer. Sometimes, it would not be possible to do so depending upon
342
+ how we have written the specs. The following attributes determine the uniqueness:
346
343
 
347
- #### Regex Filter
344
+ - The example group
345
+ - The example full description
346
+ - The spec file location, i.e., file name and line number
347
+ - All the shared examples and contexts
348
348
 
349
+ Consider the following `Calculator` module:
349
350
  ```ruby
350
- RSpecTracer.start do
351
- add_filter %r{^/helpers/}
351
+ module Calculator
352
+ module_function
353
+
354
+ def add(a, b) a + b; end
355
+ def sub(a, b) a - b; end
356
+ def mul(a, b) a * b; end
352
357
  end
353
358
  ```
354
359
 
355
- This simple regex filter will remove all files that start with /helper/ in their path.
360
+ And the corresponding spec file `spec/calculator_spec.rb`:
361
+ ```ruby
362
+ RSpec.describe Calculator do
363
+ describe '#add' do
364
+ [
365
+ [1, 2, 3],
366
+ [0, 0, 0],
367
+ [5, 32, 37],
368
+ [-1, -8, -9],
369
+ [10, -10, 0]
370
+ ].each { |a, b, r| it { expect(described_class.add(a, b)).to eq(r) } }
371
+ end
356
372
 
357
- #### Block Filter
373
+ describe '#sub' do
374
+ [
375
+ [1, 2, -1],
376
+ [10, 0, 10],
377
+ [37, 5, 32],
378
+ [-1, -8, 7],
379
+ [10, 10, 0]
380
+ ].each do |a, b, r|
381
+ it 'performs subtraction' do
382
+ expect(described_class.sub(a, b)).to eq(r)
383
+ end
384
+ end
385
+ end
358
386
 
359
- ```ruby
360
- RSpecTracer.start do
361
- add_filter do |source_file|
362
- source_file[:file_path].include?('/helpers/')
387
+ describe '#mul' do
388
+ [
389
+ [1, 2, -2],
390
+ [10, 0, 0],
391
+ [5, 7, 35],
392
+ [-1, -8, 8],
393
+ [10, 10, 100]
394
+ ].each do |a, b, r|
395
+ it "multiplies #{a} and #{b} to #{r}" do
396
+ expect(described_class.mul(a, b)).to eq(r)
397
+ end
398
+ end
363
399
  end
364
400
  end
365
401
  ```
366
402
 
367
- Block filters receive a `Hash` object and expect your block to return either true
368
- (if the file is to be removed from the result) or false (if the result should be kept).
369
- In the above example, the filter will remove all files that match "/helpers/" in their path.
403
+ Running the spec with `bundle exec rspec spec/calculator_spec.rb` generates the
404
+ following output:
405
+ ```
406
+ Calculator
407
+ #mul
408
+ multiplies 5 and 7 to 35
409
+ multiplies 10 and 10 to 100
410
+ multiplies 10 and 0 to 0
411
+ multiplies 1 and 2 to -2 (FAILED - 1)
412
+ multiplies -1 and -8 to 8
413
+ #add
414
+ example at ./spec/calculator_spec.rb:13
415
+ example at ./spec/calculator_spec.rb:13
416
+ example at ./spec/calculator_spec.rb:13
417
+ example at ./spec/calculator_spec.rb:13
418
+ example at ./spec/calculator_spec.rb:13
419
+ #sub
420
+ performs subtraction
421
+ performs subtraction
422
+ performs subtraction
423
+ performs subtraction
424
+ performs subtraction
425
+ ```
370
426
 
371
- #### Array Filter
427
+ In this scenario, RSpec Tracer cannot determine the `Calculator#add` and
428
+ `Calculator#sub` group examples. Also, it will ask you not to use the gem unless
429
+ you have made some changes to your spec files.
372
430
 
373
- ```ruby
374
- RSpecTracer.start do
375
- add_filter ['/helpers/', %r{^/utils/}]
376
- end
377
431
  ```
378
-
379
- You can pass in an array containing any of the other filter types.
432
+ ================================================================================
433
+ IMPORTANT NOTICE -- DO NOT USE RSPEC TRACER
434
+ ================================================================================
435
+ It would be best to make changes so that the RSpec tracer can uniquely
436
+ identify all the examples, and then you can enable the RSpec tracer back.
437
+ ================================================================================
438
+
439
+ RSpec tracer could not uniquely identify the following 10 examples:
440
+ - Example ID: eabd51a899db4f64d5839afe96004f03 (5 examples)
441
+ * Calculator#add (spec/calculator_spec.rb:13)
442
+ * Calculator#add (spec/calculator_spec.rb:13)
443
+ * Calculator#add (spec/calculator_spec.rb:13)
444
+ * Calculator#add (spec/calculator_spec.rb:13)
445
+ * Calculator#add (spec/calculator_spec.rb:13)
446
+ - Example ID: 72171b502c5a42b9aa133f165cf09ec2 (5 examples)
447
+ * Calculator#sub performs subtraction (spec/calculator_spec.rb:24)
448
+ * Calculator#sub performs subtraction (spec/calculator_spec.rb:24)
449
+ * Calculator#sub performs subtraction (spec/calculator_spec.rb:24)
450
+ * Calculator#sub performs subtraction (spec/calculator_spec.rb:24)
451
+ * Calculator#sub performs subtraction (spec/calculator_spec.rb:24)
452
+ ```
380
453
 
381
454
  ## Contributing
382
455