rspec-tracer 0.7.0 → 0.8.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 4825c225fdac05b09d4c56c5b6b191d0b32ad4198fc39b72a215bef3d066f7bd
4
- data.tar.gz: 4bdcb243ed2cf200f6dae4b00b4dfd50fbc8bb70b9f59552056059ec3be643b1
3
+ metadata.gz: 675be2bd6603501deb282508ea234808f0bbdf60c2b626611919f84c32bb4ba9
4
+ data.tar.gz: 591be21b4e97c50f33385111ac758665a97e83ca82e154cd3b86d32133f7ef52
5
5
  SHA512:
6
- metadata.gz: 80dff35e9e15e2b28bd5c0658fdc027b659a9a1cf4265e17d7207ce0327bc975b05890d9e23fd9b22faa81b6a471d9a3d68d5bec0e501314907534796ff6088c
7
- data.tar.gz: 77d1e9a4a9c4fe26713da06d594e7c6088655a2bc84c7e38ccd145c27f60247a9554043631c526c4a262637304c730c59ae767b3d4e8462ebc4e7e397e330055
6
+ metadata.gz: 1ab8ca453da65dd7185fa4869633d9b9ab85fb5c73cfb45929928a492d58d0e16d24e3bcef2466ceae4408c7e1ad2e5ab3f10137061ab684b1decf885809607d
7
+ data.tar.gz: 8dc1d3078f2c16c8ee5142c03d05222dd0bd458c979bca0a35ee1b1e44fead38ed2c1a6d84795a93f2a4d7a15c58da40562f4dae360f22e65f1518fc0ddecf75
data/CHANGELOG.md CHANGED
@@ -1,3 +1,9 @@
1
+ ## [0.8.0] - 2021-09-13
2
+
3
+ ### Fixed
4
+
5
+ Unable to find cache in case of history rewrites (#33)
6
+
1
7
  ## [0.7.0] - 2021-09-10
2
8
 
3
9
  ### Fixed
data/README.md CHANGED
@@ -1,8 +1,12 @@
1
1
  ![](./readme_files/rspec_tracer.png)
2
2
 
3
- RSpec Tracer is a **specs dependency analysis tool** and a **test skipper for RSpec**.
4
- It maintains a list of files for each test, enabling itself to skip tests in the
5
- subsequent runs if none of the dependent files are changed.
3
+ [![Maintainability](https://api.codeclimate.com/v1/badges/eabce2757839c08d8f8d/maintainability)](https://codeclimate.com/github/avmnu-sng/rspec-tracer/maintainability)
4
+ [![Test Coverage](https://api.codeclimate.com/v1/badges/eabce2757839c08d8f8d/test_coverage)](https://codeclimate.com/github/avmnu-sng/rspec-tracer/test_coverage)
5
+ [![Gem Version](https://badge.fury.io/rb/rspec-tracer.svg)](https://badge.fury.io/rb/rspec-tracer)
6
+
7
+ RSpec Tracer is a **specs dependency analyzer**, **flaky tests detector**, **tests accelerator**,
8
+ and **coverage reporter** tool. It maintains a list of files for each test, enabling
9
+ itself to skip tests in the subsequent runs if none of the dependent files are changed.
6
10
 
7
11
  It uses [Ruby's built-in coverage library](https://ruby-doc.org/stdlib/libdoc/coverage/rdoc/Coverage.html)
8
12
  to keep track of the coverage for each test. For each test executed, the coverage
@@ -18,45 +22,24 @@ Knowing the examples and files dependency gives us a better insight into the cod
18
22
  and we have **a clear idea of what to test for when making any changes**. With this data,
19
23
  we can also analyze the coupling between different components and much more.
20
24
 
21
- ## Note
25
+ RSpec Tracer requires **Ruby 2.5+** and **rspec-core >= 3.6.0**. To use with **Rails 5+**,
26
+ make sure to use **rspec-rails >= 4.0.0**. If you are using SimpleCov, it is
27
+ recommended to use **simplecov >= 0.12.0**. To use RSpec Tracer **cache on CI**, you
28
+ need to have an **S3 bucket** and **[AWS CLI](https://aws.amazon.com/cli/)**
29
+ installed.
22
30
 
23
- You should take some time and go through the **[document](./RSPEC_TRACER.md)** describing
24
- the **intention** and implementation details of **managing dependency**, **managing flaky tests**,
25
- **skipping tests**, and **caching on CI**. You must go through the README file before
26
- integrating the gem into your project to better understand what is happening.
31
+ You should take some time and go through the **[document](./RSPEC_TRACER.md)**
32
+ describing the **intention** and implementation details of **managing dependency**,
33
+ **managing flaky tests**, **skipping tests**, and **caching on CI**.
27
34
 
28
35
  ## Table of Contents
29
36
 
30
37
  * [Demo](#demo)
31
- * [Installation](#installation)
32
- * [Compatibility](#compatibility)
33
- * [Additional Tools](#additional-tools)
34
38
  * [Getting Started](#getting-started)
35
- * [Environment Variables](#environment-variables)
36
- * [BUNDLE_PATH](#bundle_path)
37
- * [CI](#ci)
38
- * [LOCAL_AWS](#local_aws)
39
- * [RSPEC_TRACER_NO_SKIP](#rspec_tracer_no_skip)
40
- * [RSPEC_TRACER_S3_URI](#rspec_tracer_s3_uri)
41
- * [RSPEC_TRACER_UPLOAD_LOCAL_CACHE](#rspec_tracer_upload_local_cache)
42
- * [RSPEC_TRACER_VERBOSE](#rspec_tracer_verbose)
43
- * [TEST_SUITES](#test_suites)
44
- * [TEST_SUITE_ID](#test_suite_id)
45
- * [Sample Reports](#sample-reports)
46
- * [Examples](#examples)
47
- * [Flaky Examples](#flaky-examples)
48
- * [Examples Dependency](#examples-dependency)
49
- * [Files Dependency](#files-dependency)
50
- * [Configuring RSpec Tracer](#configuring-rspec-tracer)
39
+ * [Configuring CI Caching](#configuring-ci-caching)
40
+ * [Advanced Configuration](#advanced-configuration)
51
41
  * [Filters](#filters)
52
- * [Defining Custom Filteres](#defining-custom-filteres)
53
- * [String Filter](#string-filter)
54
- * [Regex Filter](#regex-filter)
55
- * [Block Filter](#block-filter)
56
- * [Array Filter](#array-filter)
57
- * [Contributing](#contributing)
58
- * [License](#license)
59
- * [Code of Conduct](#code-of-conduct)
42
+ * [Environment Variables](#environment-variables)
60
43
 
61
44
  ## Demo
62
45
 
@@ -66,267 +49,210 @@ integrating the gem into your project to better understand what is happening.
66
49
  **Next Run**
67
50
  ![](./readme_files/next_run.gif)
68
51
 
52
+ You get the following three reports:
69
53
 
70
- ## Installation
54
+ ### All Examples Report
71
55
 
72
- Add this line to your `Gemfile` and `bundle install`:
73
- ```ruby
74
- gem 'rspec-tracer', '~> 0.6', group: :test, require: false
75
- ```
56
+ These reports provide basic test information:
76
57
 
77
- And, add the followings to your `.gitignore`:
78
- ```
79
- /rspec_tracer_cache/
80
- /rspec_tracer_coverage/
81
- /rspec_tracer_report/
82
- ```
58
+ **First Run**
83
59
 
84
- ### Compatibility
60
+ ![](./readme_files/examples_report_first_run.png)
85
61
 
86
- RSpec Tracer requires **Ruby 2.5+** and **rspec-core >= 3.6.0**. To use with **Rails 5+**,
87
- make sure to use **rspec-rails >= 4.0.0**. If you are using SimpleCov, it is
88
- recommended to use **simplecov >= 0.12.0**.
62
+ **Next Run**
89
63
 
90
- ### Additional Tools
64
+ ![](./readme_files/examples_report_next_run.png)
91
65
 
92
- To use RSpec Tracer on CI, you need to have an **S3 bucket** and
93
- **[AWS CLI](https://aws.amazon.com/cli/)** installed.
66
+ ### Flaky Examples Report
94
67
 
95
- ## Getting Started
68
+ These reports provide flaky tests information. Assuming **the following two tests
69
+ failed in the first run.**
96
70
 
97
- 1. **Load and Start RSpec Tracer**
71
+ **Next Run**
98
72
 
99
- - **With SimpleCov**
73
+ ![](./readme_files/flaky_examples_report_first_run.png)
100
74
 
101
- If you are using `SimpleCov`, load RSpec Tracer right after the SimpleCov load
102
- and launch:
75
+ **Another Run**
103
76
 
104
- ```ruby
105
- require 'simplecov'
106
- SimpleCov.start
77
+ ![](./readme_files/flaky_examples_report_next_run.png)
107
78
 
108
- # Load RSpec Tracer
109
- require 'rspec_tracer'
110
- RSpecTracer.start
111
- ```
79
+ ### Examples Dependency Report
112
80
 
113
- Currently using RSpec Tracer with SimpleCov has the following two limitations:
114
- - SimpleCov **won't be able to provide branch coverage report** even when enabled.
115
- - RSpec Tracer **nullifies the `SimpleCov.at_exit`** callback.
81
+ These reports show a list of dependent files for each test.
116
82
 
117
- - **Without SimpleCov**
83
+ ![](./readme_files/examples_dependency_report.png)
118
84
 
119
- Load and launch RSpec Tracer at the very top of `spec_helper.rb` (or `rails_helper.rb`,
120
- `test/test_helper.rb`). Note that `RSpecTracer.start` must be issued **before loading
121
- any of the application code.**
85
+ ### Files Dependency Report
122
86
 
123
- ```ruby
124
- # Load RSpec Tracer
125
- require 'rspec_tracer'
126
- RSpecTracer.start
127
- ```
87
+ These reports provide information on the total number of tests that will run after changing this particular file.
128
88
 
129
- 2. To enable RSpec Tracer to share cache between different builds on CI, update the
130
- Rakefile in your project to have the following:
89
+ ![](./readme_files/files_dependency_report.png)
131
90
 
132
- ```ruby
133
- spec = Gem::Specification.find_by_name('rspec-tracer')
134
91
 
135
- load "#{spec.gem_dir}/lib/rspec_tracer/remote_cache/Rakefile"
136
- ```
137
- 3. Before running tests, download the remote cache using the following rake task:
92
+ ## Getting Started
138
93
 
139
- ```sh
140
- bundle exec rake rspec_tracer:remote_cache:download
94
+ 1. Add this line to your `Gemfile` and `bundle install`:
95
+ ```ruby
96
+ gem 'rspec-tracer', '~> 0.7', group: :test, require: false
141
97
  ```
142
- 4. Run the tests with RSpec using `bundle exec rspec`.
143
- 5. After running tests, upload the local cache using the following rake task:
144
98
 
145
- ```sh
146
- bundle exec rake rspec_tracer:remote_cache:upload
99
+ And, add the followings to your `.gitignore`:
147
100
  ```
148
- 6. After running your tests, open `rspec_tracer_report/index.html` in the
149
- browser of your choice.
150
-
151
- ## Environment Variables
101
+ /rspec_tracer_cache/
102
+ /rspec_tracer_coverage/
103
+ /rspec_tracer_report/
104
+ ```
105
+ 2. Load and launch RSpec Tracer at the very top of `spec_helper.rb` (or `rails_helper.rb`,
106
+ `test/test_helper.rb`). Note that `RSpecTracer.start` must be issued **before loading
107
+ any of the application code.**
152
108
 
153
- To get better control on execution, you can use the following environment variables
154
- whenever required.
109
+ ```ruby
110
+ # Load RSpec Tracer
111
+ require 'rspec_tracer'
112
+ RSpecTracer.start
113
+ ```
155
114
 
156
- ### BUNDLE_PATH
115
+ **If you are using SimpleCov**, load RSpec Tracer right after the SimpleCov load
116
+ and launch:
157
117
 
158
- Since the bundler uses a vendor directory inside the project, it might cause slowness
159
- depending on the vendor size. You can configure the bundle path outside of the project
160
- using `BUNDLE_PATH` environment variable, for example, `BUNDLE_PATH=$HOME/vendor/bundle`.
161
- Make sure to cache this directory in the CI configuration.
118
+ ```ruby
119
+ require 'simplecov'
120
+ SimpleCov.start
162
121
 
163
- ### CI
122
+ # Load RSpec Tracer
123
+ require 'rspec_tracer'
124
+ RSpecTracer.start
125
+ ```
164
126
 
165
- Mostly all the CI have `CI=true`. If not, you should explicitly set it to `true`.
127
+ Currently using RSpec Tracer with SimpleCov has the following two limitations:
166
128
 
167
- ### LOCAL_AWS
129
+ - SimpleCov **won't be able to provide branch coverage report** even when enabled.
130
+ - RSpec Tracer **nullifies the `SimpleCov.at_exit`** callback.
168
131
 
169
- In case you want to test out the caching feature in the local development environment.
170
- You can install [localstack](https://github.com/localstack/localstack) and
171
- [awscli-local](https://github.com/localstack/awscli-local) and then invoke the
172
- rake tasks with `LOCAL_AWS=true`.
132
+ 3. After running your tests, open `rspec_tracer_report/index.html` in the browser
133
+ of your choice.
173
134
 
174
- ### RSPEC_TRACER_NO_SKIP
175
-
176
- The default value is `false.` If set to `true`, the RSpec Tracer will not skip
177
- any tests. Note that it will continue to maintain cache files and generate reports.
135
+ ## Configuring CI Caching
178
136
 
137
+ To enable RSpec Tracer to share cache between different builds on CI, update the
138
+ Rakefile in your project to have the following:
179
139
  ```ruby
180
- RSPEC_TRACER_NO_SKIP=true bundle exec rspec
181
- ```
182
-
183
- ### RSPEC_TRACER_S3_URI
140
+ spec = Gem::Specification.find_by_name('rspec-tracer')
184
141
 
185
- You should provide the S3 bucket path to store the cache files.
186
-
187
- ```ruby
188
- export RSPEC_TRACER_S3_URI=s3://ci-artifacts-bucket/rspec-tracer-cache
142
+ load "#{spec.gem_dir}/lib/rspec_tracer/remote_cache/Rakefile"
189
143
  ```
190
144
 
191
- ### RSPEC_TRACER_UPLOAD_LOCAL_CACHE
192
-
193
- By default, RSpec Tracer does not upload local cache files. You can set this
194
- environment variable to `true` to upload the local cache to S3.
195
-
196
- ### RSPEC_TRACER_VERBOSE
197
-
198
- To print the intermediate steps and time taken, use this environment variable:
199
-
145
+ Before running tests, download the remote cache using the following rake task:
200
146
  ```sh
201
- export RSPEC_TRACER_VERBOSE=true
147
+ bundle exec rake rspec_tracer:remote_cache:download
202
148
  ```
203
149
 
204
- ### TEST_SUITES
205
-
206
- Set this environment variable when using test suite id. It determines the total
207
- number of different test suites you are running.
208
-
209
- ```ruby
210
- export TEST_SUITES=8
211
- ```
212
-
213
- ### TEST_SUITE_ID
214
-
215
- If you have a large set of tests to run, it is recommended to run them in
216
- separate groups. This way, RSpec Tracer is not overwhelmed with loading massive
217
- cached data in the memory. Also, it generates and uses cache for specific test suites
218
- and not merge them.
219
-
220
- ```ruby
221
- TEST_SUITE_ID=1 bundle exec rspec spec/models
222
- TEST_SUITE_ID=2 bundle exec rspec spec/helpers
223
- ```
224
-
225
- If you run parallel builds on the CI, you should specify the test suite ID and
226
- the total number of test suites when downloading the cache files.
227
-
150
+ After running tests, upload the local cache using the following rake task:
228
151
  ```sh
229
- $ TEST_SUITES=5 TEST_SUITE_ID=1 bundle exec rake rspec_tracer:remote_cache:download
152
+ bundle exec rake rspec_tracer:remote_cache:upload
230
153
  ```
231
154
 
232
- In this case, the appropriate cache should have all the cache files available on
233
- the S3 for each test suite, not just for the current one. Also, while uploading,
234
- make sure to provide the test suite id.
155
+ You must set the following two environment variables:
235
156
 
236
- ```sh
237
- $ TEST_SUITE_ID=1 bundle exec rake rspec_tracer:remote_cache:upload
238
- ```
157
+ - **`GIT_BRANCH`** is the git branch name you are running the CI build on.
158
+ - **`RSPEC_TRACER_S3_URI`** is the S3 bucket path to store the cache files.
159
+ ```sh
160
+ export RSPEC_TRACER_S3_URI=s3://ci-artifacts-bucket/rspec-tracer-cache
161
+ ```
239
162
 
240
- ## Sample Reports
241
-
242
- You get the following three reports:
243
-
244
- ### Examples
245
-
246
- These reports provide basic test information:
247
-
248
- **First Run**
249
-
250
- ![](./readme_files/examples_report_first_run.png)
251
-
252
- **Next Run**
253
-
254
- ![](./readme_files/examples_report_next_run.png)
255
-
256
- ### Flaky Examples
257
-
258
- These reports provide flaky tests information. Assuming **the following two tests
259
- failed in the first run.**
260
-
261
- **Next Run**
262
-
263
- ![](./readme_files/flaky_examples_report_first_run.png)
264
-
265
- **Another Run**
266
-
267
- ![](./readme_files/flaky_examples_report_next_run.png)
268
-
269
- ### Examples Dependency
270
-
271
- These reports show a list of dependent files for each test.
272
-
273
- ![](./readme_files/examples_dependency_report.png)
274
-
275
- ### Files Dependency
276
-
277
- These reports provide information on the total number of tests that will run after changing this particular file.
278
-
279
- ![](./readme_files/files_dependency_report.png)
280
-
281
- ## Configuring RSpec Tracer
163
+ ## Advanced Configuration
282
164
 
283
165
  Configuration settings can be applied in three formats, which are completely equivalent:
284
166
 
285
167
  - The most common way is to configure it directly in your start block:
286
-
287
- ```ruby
288
- RSpecTracer.start do
289
- config_option 'foo'
290
- end
291
- ```
168
+ ```ruby
169
+ RSpecTracer.start do
170
+ config_option 'foo'
171
+ end
172
+ ```
292
173
  - You can also set all configuration options directly:
293
-
294
- ```ruby
295
- RSpecTracer.config_option 'foo'
296
- ```
174
+ ```ruby
175
+ RSpecTracer.config_option 'foo'
176
+ ```
297
177
 
298
178
  - If you do not want to start tracer immediately after launch or want to add
299
179
  additional configuration later on in a concise way, use:
180
+ ```ruby
181
+ RSpecTracer.configure do
182
+ config_option 'foo'
183
+ end
184
+ ```
300
185
 
301
- ```ruby
302
- RSpecTracer.configure do
303
- config_option 'foo'
304
- end
305
- ```
186
+ The available configuration options are:
306
187
 
307
- ## Filters
188
+ - **`root dir`** to set the project root. The default value is the current working
189
+ directory.
190
+ - **`add_filter filter`** to apply [filters](#filters) on the source files to
191
+ exclude them from the dependent files list.
192
+ - **`filters.clear`** to remove the default configured dependent files filters.
193
+ - **`add_coverage_filter filter`** to apply [filters](#filters) on the source files
194
+ to exclude them from the coverage report.
195
+ - **`coverage_filters.clear`** to remove the default configured coverage files filters.
196
+ - **`coverage_track_files glob`** to include files in the given glob pattern in
197
+ the coverage report if these files are not already present.
308
198
 
309
- RSpec Tracer supports two types of filters:
199
+ ```ruby
200
+ RSpecTracer.start do
201
+ # Configure project root
202
+ root '/tmp/my_project'
310
203
 
311
- - To exclude selected files from the dependency list of tests:
204
+ # Clear existing filters
205
+ filters.clear
206
+ # Add dependent files filter
207
+ add_filter %r{^/tasks/}
312
208
 
313
- ```ruby
314
- RSpecTracer.start do
315
- add_filter %r{^/helpers/}
316
- end
317
- ```
318
- - To exclude selected files from the coverage data. You should only use this
319
- when not using SimpleCov.
209
+ # Clear existing coverage filters
210
+ coverage_filters.clear
211
+ # Add coverage files filter
212
+ add_coverage_filter %w[/features/ /spec/ /tests/]
320
213
 
321
- ```ruby
322
- RSpecTracer.start do
323
- add_coverage_filter %r{^/tasks/}
324
- end
325
- ```
214
+ # Define glob to track files in the coverage report
215
+ coverage_track_files '{app,lib}/**/*.rb'
216
+ end
217
+ ```
218
+
219
+ You can configure the RSpec Tracer reports directories using the following environment
220
+ variables:
221
+
222
+ - **`RSPEC_TRACER_CACHE_DIR`** to update the default cache directory (`rspec_tracer_cache`).
223
+ ```sh
224
+ export RSPEC_TRACER_CACHE_DIR=/tmp/rspec_tracer_cache
225
+ ```
226
+ - **`RSPEC_TRACER_COVERAGE_DIR`** to update the default coverage directory (`rspec_tracer_coverage`).
227
+ ```sh
228
+ export RSPEC_TRACER_CACHE_DIR=/tmp/rspec_tracer_coverage
229
+ ```
230
+ - **`RSPEC_TRACER_REPORT_DIR`** to update the default html reports directory (`rspec_tracer_report`).
231
+ ```sh
232
+ export RSPEC_TRACER_CACHE_DIR=/tmp/rspec_tracer_report
233
+ ```
234
+
235
+ These settings are available through environment variables because the rake tasks
236
+ to download and upload the cache files need to use the same directories.
237
+
238
+ ## Filters
326
239
 
327
- By default, a filter is applied that removes all files OUTSIDE of your project's
328
- root directory - otherwise you'd end up with the source files in the gems you are
329
- using as tests dependency.
240
+ By default, RSpec Tracer ignores all the files outside of the project root directory -
241
+ otherwise you would end up with the source files in the gems you are using in the
242
+ project. It also applies the following filters:
243
+ ```ruby
244
+ RSpecTracer.configure do
245
+ add_filter '/vendor/bundle/'
246
+
247
+ add_coverage_filter %w[
248
+ /autotest/
249
+ /features/
250
+ /spec/
251
+ /test/
252
+ /vendor/bundle/
253
+ ].freeze
254
+ end
255
+ ```
330
256
 
331
257
  ### Defining Custom Filteres
332
258
 
@@ -378,6 +304,42 @@ end
378
304
 
379
305
  You can pass in an array containing any of the other filter types.
380
306
 
307
+ ## Environment Variables
308
+
309
+ To get better control on execution, you can use the following environment variables
310
+ whenever required.
311
+
312
+ - **`LOCAL_AWS (default: false)`:** In case you want to test out the caching feature in the local
313
+ development environment. You can install [localstack](https://github.com/localstack/localstack)
314
+ and [awscli-local](https://github.com/localstack/awscli-local) and then invoke the
315
+ rake tasks with `LOCAL_AWS=true`.
316
+
317
+ - **`RSPEC_TRACER_NO_SKIP (default: false)`:** Use this environment variables to
318
+ not skip any tests. Note that it will continue to maintain cache files and generate
319
+ reports.
320
+
321
+ - **`RSPEC_TRACER_UPLOAD_LOCAL_CACHE (default: false)`:** By default, RSpec Tracer
322
+ does not upload local cache files. You can set this environment variable to `true`
323
+ to upload the local cache to S3.
324
+
325
+ - **`RSPEC_TRACER_VERBOSE (default: false)`:** To print the intermediate steps
326
+ and time taken, use this environment variable.
327
+
328
+ - **`TEST_SUITES`:** Set this environment variable when running parallel builds
329
+ in the CI. It determines the total number of different test suites you are running.
330
+ ```sh
331
+ export TEST_SUITES=8
332
+ ```
333
+
334
+ - **`TEST_SUITE_ID`:** If you have a large set of tests to run, it is recommended
335
+ to run them in separate groups. This way, RSpec Tracer is not overwhelmed with
336
+ loading massive cached data in the memory. Also, it generates and uses cache for
337
+ specific test suites and not merge them.
338
+ ```sh
339
+ TEST_SUITE_ID=1 bundle exec rspec spec/models
340
+ TEST_SUITE_ID=2 bundle exec rspec spec/helpers
341
+ ```
342
+
381
343
  ## Contributing
382
344
 
383
345
  Read the [contribution guide](https://github.com/avmnu-sng/rspec-tracer/blob/main/.github/CONTRIBUTING.md).