simplecov 0.16.1 → 0.22.0

Sign up to get free protection for your applications and to get access to all the features.
Files changed (50) hide show
  1. checksums.yaml +5 -5
  2. data/CHANGELOG.md +112 -402
  3. data/README.md +391 -127
  4. data/doc/alternate-formatters.md +21 -1
  5. data/doc/commercial-services.md +5 -0
  6. data/lib/minitest/simplecov_plugin.rb +15 -0
  7. data/lib/simplecov/combine/branches_combiner.rb +32 -0
  8. data/lib/simplecov/combine/files_combiner.rb +24 -0
  9. data/lib/simplecov/combine/lines_combiner.rb +43 -0
  10. data/lib/simplecov/combine/results_combiner.rb +60 -0
  11. data/lib/simplecov/combine.rb +30 -0
  12. data/lib/simplecov/command_guesser.rb +6 -3
  13. data/lib/simplecov/configuration.rb +210 -15
  14. data/lib/simplecov/coverage_statistics.rb +56 -0
  15. data/lib/simplecov/default_formatter.rb +20 -0
  16. data/lib/simplecov/defaults.rb +15 -12
  17. data/lib/simplecov/exit_codes/exit_code_handling.rb +29 -0
  18. data/lib/simplecov/exit_codes/maximum_coverage_drop_check.rb +83 -0
  19. data/lib/simplecov/exit_codes/minimum_coverage_by_file_check.rb +54 -0
  20. data/lib/simplecov/exit_codes/minimum_overall_coverage_check.rb +53 -0
  21. data/lib/simplecov/exit_codes.rb +5 -0
  22. data/lib/simplecov/file_list.rb +72 -13
  23. data/lib/simplecov/filter.rb +9 -6
  24. data/lib/simplecov/formatter/multi_formatter.rb +5 -7
  25. data/lib/simplecov/formatter/simple_formatter.rb +4 -4
  26. data/lib/simplecov/formatter.rb +2 -2
  27. data/lib/simplecov/last_run.rb +3 -1
  28. data/lib/simplecov/lines_classifier.rb +5 -5
  29. data/lib/simplecov/no_defaults.rb +1 -1
  30. data/lib/simplecov/process.rb +19 -0
  31. data/lib/simplecov/profiles/hidden_filter.rb +5 -0
  32. data/lib/simplecov/profiles/rails.rb +1 -1
  33. data/lib/simplecov/profiles.rb +9 -7
  34. data/lib/simplecov/result.rb +18 -12
  35. data/lib/simplecov/result_adapter.rb +30 -0
  36. data/lib/simplecov/result_merger.rb +130 -59
  37. data/lib/simplecov/simulate_coverage.rb +29 -0
  38. data/lib/simplecov/source_file/branch.rb +84 -0
  39. data/lib/simplecov/source_file/line.rb +72 -0
  40. data/lib/simplecov/source_file.rb +279 -127
  41. data/lib/simplecov/useless_results_remover.rb +18 -0
  42. data/lib/simplecov/version.rb +1 -1
  43. data/lib/simplecov.rb +310 -121
  44. metadata +48 -50
  45. data/CONTRIBUTING.md +0 -51
  46. data/ISSUE_TEMPLATE.md +0 -23
  47. data/lib/simplecov/jruby_fix.rb +0 -44
  48. data/lib/simplecov/railtie.rb +0 -9
  49. data/lib/simplecov/railties/tasks.rake +0 -13
  50. data/lib/simplecov/raw_coverage.rb +0 -41
data/README.md CHANGED
@@ -1,5 +1,6 @@
1
- SimpleCov [![Build Status](https://travis-ci.org/colszowka/simplecov.svg)][Continuous Integration] [![Dependency Status](https://gemnasium.com/colszowka/simplecov.svg)][Dependencies] [![Code Climate](https://codeclimate.com/github/colszowka/simplecov.svg)](https://codeclimate.com/github/colszowka/simplecov) [![Inline docs](http://inch-ci.org/github/colszowka/simplecov.svg)](http://inch-ci.org/github/colszowka/simplecov)
1
+ SimpleCov [![Gem Version](https://badge.fury.io/rb/simplecov.svg)](https://badge.fury.io/rb/simplecov) [![Build Status](https://github.com/simplecov-ruby/simplecov/workflows/stable/badge.svg?branch=main)][Continuous Integration] [![Maintainability](https://api.codeclimate.com/v1/badges/c071d197d61953a7e482/maintainability)](https://codeclimate.com/github/simplecov-ruby/simplecov/maintainability) [![Inline docs](http://inch-ci.org/github/simplecov-ruby/simplecov.svg?branch=main)](http://inch-ci.org/github/simplecov-ruby/simplecov)
2
2
  =========
3
+
3
4
  **Code coverage for Ruby**
4
5
 
5
6
  * [Source Code]
@@ -8,39 +9,42 @@ SimpleCov [![Build Status](https://travis-ci.org/colszowka/simplecov.svg)][Conti
8
9
  * [Rubygem]
9
10
  * [Continuous Integration]
10
11
 
11
- [Coverage]: http://www.ruby-doc.org/stdlib-2.1.0/libdoc/coverage/rdoc/Coverage.html "API doc for Ruby's Coverage library"
12
- [Source Code]: https://github.com/colszowka/simplecov "Source Code @ GitHub"
12
+ [Coverage]: https://ruby-doc.org/stdlib/libdoc/coverage/rdoc/Coverage.html "API doc for Ruby's Coverage library"
13
+ [Source Code]: https://github.com/simplecov-ruby/simplecov "Source Code @ GitHub"
13
14
  [API documentation]: http://rubydoc.info/gems/simplecov/frames "RDoc API Documentation at Rubydoc.info"
14
15
  [Configuration]: http://rubydoc.info/gems/simplecov/SimpleCov/Configuration "Configuration options API documentation"
15
- [Changelog]: https://github.com/colszowka/simplecov/blob/master/CHANGELOG.md "Project Changelog"
16
+ [Changelog]: https://github.com/simplecov-ruby/simplecov/blob/main/CHANGELOG.md "Project Changelog"
16
17
  [Rubygem]: http://rubygems.org/gems/simplecov "SimpleCov @ rubygems.org"
17
- [Continuous Integration]: http://travis-ci.org/colszowka/simplecov "SimpleCov is built around the clock by travis-ci.org"
18
- [Dependencies]: https://gemnasium.com/colszowka/simplecov "SimpleCov dependencies on Gemnasium"
19
- [simplecov-html]: https://github.com/colszowka/simplecov-html "SimpleCov HTML Formatter Source Code @ GitHub"
18
+ [Continuous Integration]: https://github.com/simplecov-ruby/simplecov/actions?query=workflow%3Astable "SimpleCov is built around the clock by github.com"
19
+ [Dependencies]: https://gemnasium.com/simplecov-ruby/simplecov "SimpleCov dependencies on Gemnasium"
20
+ [simplecov-html]: https://github.com/simplecov-ruby/simplecov-html "SimpleCov HTML Formatter Source Code @ GitHub"
20
21
 
21
22
  SimpleCov is a code coverage analysis tool for Ruby. It uses [Ruby's built-in Coverage][Coverage] library to gather code
22
23
  coverage data, but makes processing its results much easier by providing a clean API to filter, group, merge, format,
23
24
  and display those results, giving you a complete code coverage suite that can be set up with just a couple lines of
24
25
  code.
26
+ SimpleCov/Coverage track covered ruby code, gathering coverage for common templating solutions like erb, slim and haml is not supported.
25
27
 
26
28
  In most cases, you'll want overall coverage results for your projects, including all types of tests, Cucumber features,
27
29
  etc. SimpleCov automatically takes care of this by caching and merging results when generating reports, so your
28
30
  report actually includes coverage across your test suites and thereby gives you a better picture of blank spots.
29
31
 
30
- The official formatter of SimpleCov is packaged as a separate gem called [simplecov-html], but will be installed and configured
31
- automatically when you launch SimpleCov. If you're curious, you can find it [on GitHub, too][simplecov-html].
32
+ The official formatter of SimpleCov is packaged as a separate gem called [simplecov-html], but will be installed and
33
+ configured automatically when you launch SimpleCov. If you're curious, you can find it [on GitHub, too][simplecov-html].
32
34
 
33
35
 
34
36
  ## Contact
35
37
 
36
38
  *Code and Bug Reports*
37
39
 
38
- * [Issue Tracker](https://github.com/colszowka/simplecov/issues)
39
- * See [CONTRIBUTING](https://github.com/colszowka/simplecov/blob/master/CONTRIBUTING.md) for how to contribute along with some common problems to check out before creating an issue.
40
+ * [Issue Tracker](https://github.com/simplecov-ruby/simplecov/issues)
41
+ * See [CONTRIBUTING](https://github.com/simplecov-ruby/simplecov/blob/main/CONTRIBUTING.md) for how to contribute along
42
+ with some common problems to check out before creating an issue.
40
43
 
41
44
  *Questions, Problems, Suggestions, etc.*
42
45
 
43
- * [Mailing List](https://groups.google.com/forum/#!forum/simplecov) "Open mailing list for discussion and announcements on Google Groups"
46
+ * [Mailing List](https://groups.google.com/forum/#!forum/simplecov) "Open mailing list for discussion and announcements
47
+ on Google Groups"
44
48
 
45
49
  Getting started
46
50
  ---------------
@@ -65,13 +69,16 @@ Getting started
65
69
  The `SimpleCov.start` **must** be issued **before any of your application
66
70
  code is required!**
67
71
 
72
+ This is especially true if you use anything that keeps your tests application loaded like spring, check out the **[spring section](#want-to-use-spring-with-simplecov)**.
73
+
68
74
  SimpleCov must be running in the process that you want the code coverage
69
75
  analysis to happen on. When testing a server process (e.g. a JSON API
70
76
  endpoint) via a separate test process (e.g. when using Selenium) where you
71
77
  want to see all code executed by the `rails server`, and not just code
72
- executed in your actual test files, you'll want to add something like this
73
- to the top of `script/rails` (or `bin/rails` for Rails 4), but below the
74
- "shebang" line (`#! /usr/bin/env ruby`):
78
+ executed in your actual test files, you need to require SimpleCov in the
79
+ server process. For rails for instance, you'll want to add something like this
80
+ to the top of `bin/rails`, but below the "shebang" line (`#! /usr/bin/env
81
+ ruby`) and after config/boot is required:
75
82
 
76
83
  ```ruby
77
84
  if ENV['RAILS_ENV'] == 'test'
@@ -81,23 +88,37 @@ Getting started
81
88
  end
82
89
  ```
83
90
 
84
- 3. Run your tests, open up `coverage/index.html` in your browser and check out
85
- what you've missed so far.
86
- 4. Add the following to your `.gitignore` file to ensure that coverage results
91
+ 3. Run your full test suite to see the percent coverage that your application has.
92
+ 4. After running your tests, open `coverage/index.html` in the browser of your choice. For example, in a Mac Terminal,
93
+ run the following command from your application's root directory:
94
+
95
+ ```
96
+ open coverage/index.html
97
+ ```
98
+ in a debian/ubuntu Terminal,
99
+
100
+ ```
101
+ xdg-open coverage/index.html
102
+ ```
103
+
104
+ **Note:** [This guide](https://dwheeler.com/essays/open-files-urls.html) can help if you're unsure which command your particular
105
+ operating system requires.
106
+
107
+ 5. Add the following to your `.gitignore` file to ensure that coverage results
87
108
  are not tracked by Git (optional):
88
109
 
89
- ```
90
- coverage
91
- ```
110
+ ```
111
+ echo coverage >> .gitignore
112
+ ```
92
113
 
93
- If you're making a Rails application, SimpleCov comes with built-in configurations (see below for information on profiles)
94
- that will get you started with groups for your Controllers, Views, Models and Helpers. To use it, the first two lines of
95
- your test_helper should be like this:
114
+ If you're making a Rails application, SimpleCov comes with built-in configurations (see below for information on
115
+ profiles) that will get you started with groups for your Controllers, Models and Helpers. To use it, the
116
+ first two lines of your test_helper should be like this:
96
117
 
97
- ```ruby
98
- require 'simplecov'
99
- SimpleCov.start 'rails'
100
- ```
118
+ ```ruby
119
+ require 'simplecov'
120
+ SimpleCov.start 'rails'
121
+ ```
101
122
 
102
123
  ## Example output
103
124
 
@@ -123,8 +144,9 @@ require 'simplecov'
123
144
  SimpleCov.start 'rails'
124
145
  ```
125
146
 
126
- You could even track what kind of code your UI testers are touching if you want to go overboard with things. SimpleCov does not
127
- care what kind of framework it is running in; it just looks at what code is being executed and generates a report about it.
147
+ You could even track what kind of code your UI testers are touching if you want to go overboard with things. SimpleCov
148
+ does not care what kind of framework it is running in; it just looks at what code is being executed and generates a
149
+ report about it.
128
150
 
129
151
  ### Notes on specific frameworks and test utilities
130
152
 
@@ -135,28 +157,28 @@ to use SimpleCov with them. Here's an overview of the known ones:
135
157
  <tr><th>Framework</th><th>Notes</th><th>Issue</th></tr>
136
158
  <tr>
137
159
  <th>
138
- bootsnap
160
+ parallel_tests
139
161
  </th>
140
162
  <td>
141
- <a href="#want-to-use-bootsnap-with-simplecov">See section below.</a>
163
+ As of 0.8.0, SimpleCov should correctly recognize parallel_tests and
164
+ supplement your test suite names with their corresponding test env
165
+ numbers. SimpleCov locks the resultset cache while merging, ensuring no
166
+ race conditions occur when results are merged.
142
167
  </td>
143
168
  <td>
144
- <a href="https://github.com/Shopify/bootsnap/issues/35">Shopify/bootsnap#35</a>
169
+ <a href="https://github.com/simplecov-ruby/simplecov/issues/64">#64</a> &amp;
170
+ <a href="https://github.com/simplecov-ruby/simplecov/pull/185">#185</a>
145
171
  </td>
146
172
  </tr>
147
173
  <tr>
148
174
  <th>
149
- parallel_tests
175
+ knapsack_pro
150
176
  </th>
151
177
  <td>
152
- As of 0.8.0, SimpleCov should correctly recognize parallel_tests and
153
- supplement your test suite names with their corresponding test env
154
- numbers. SimpleCov locks the resultset cache while merging, ensuring no
155
- race conditions occur when results are merged.
178
+ To make SimpleCov work with Knapsack Pro Queue Mode to split tests in parallel on CI jobs you need to provide CI node index number to the <code>SimpleCov.command_name</code> in <code>KnapsackPro::Hooks::Queue.before_queue</code> hook.
156
179
  </td>
157
180
  <td>
158
- <a href="https://github.com/colszowka/simplecov/issues/64">#64</a> &amp;
159
- <a href="https://github.com/colszowka/simplecov/pull/185">#185</a>
181
+ <a href="https://knapsackpro.com/faq/question/how-to-use-simplecov-in-queue-mode">Tip</a>
160
182
  </td>
161
183
  </tr>
162
184
  <tr>
@@ -169,7 +191,7 @@ to use SimpleCov with them. Here's an overview of the known ones:
169
191
  to explicitly set the output root using `SimpleCov.root('foo/bar/baz')`
170
192
  </td>
171
193
  <td>
172
- <a href="https://github.com/colszowka/simplecov/issues/95">#95</a>
194
+ <a href="https://github.com/simplecov-ruby/simplecov/issues/95">#95</a>
173
195
  </td>
174
196
  </tr>
175
197
  <tr>
@@ -180,11 +202,11 @@ to use SimpleCov with them. Here's an overview of the known ones:
180
202
  Because of how Spork works internally (using preforking), there used to
181
203
  be trouble when using SimpleCov with it, but that has apparently been
182
204
  resolved with a specific configuration strategy. See <a
183
- href="https://github.com/colszowka/simplecov/issues/42#issuecomment-4440284">this</a>
205
+ href="https://github.com/simplecov-ruby/simplecov/issues/42#issuecomment-4440284">this</a>
184
206
  comment.
185
207
  </td>
186
208
  <td>
187
- <a href="https://github.com/colszowka/simplecov/issues/42#issuecomment-4440284">#42</a>
209
+ <a href="https://github.com/simplecov-ruby/simplecov/issues/42#issuecomment-4440284">#42</a>
188
210
  </td>
189
211
  </tr>
190
212
  <tr>
@@ -195,7 +217,7 @@ to use SimpleCov with them. Here's an overview of the known ones:
195
217
  <a href="#want-to-use-spring-with-simplecov">See section below.</a>
196
218
  </td>
197
219
  <td>
198
- <a href="https://github.com/colszowka/simplecov/issues/381">#381</a>
220
+ <a href="https://github.com/simplecov-ruby/simplecov/issues/381">#381</a>
199
221
  </td>
200
222
  </tr>
201
223
  <tr>
@@ -208,7 +230,7 @@ to use SimpleCov with them. Here's an overview of the known ones:
208
230
  (Dec 11th, 2011) should have this problem resolved.
209
231
  </td>
210
232
  <td>
211
- <a href="https://github.com/colszowka/simplecov/issues/45">#45</a> &amp;
233
+ <a href="https://github.com/simplecov-ruby/simplecov/issues/45">#45</a> &amp;
212
234
  <a href="https://github.com/test-unit/test-unit/pull/12">test-unit/test-unit#12</a>
213
235
  </td>
214
236
  </tr>
@@ -230,7 +252,8 @@ to use SimpleCov with them. Here's an overview of the known ones:
230
252
  ```ruby
231
253
  SimpleCov.some_config_option 'foo'
232
254
  ```
233
- * If you do not want to start coverage immediately after launch or want to add additional configuration later on in a concise way, use:
255
+ * If you do not want to start coverage immediately after launch or want to add additional configuration later on in a
256
+ concise way, use:
234
257
 
235
258
  ```ruby
236
259
  SimpleCov.configure do
@@ -242,11 +265,12 @@ Please check out the [Configuration] API documentation to find out what you can
242
265
 
243
266
  ## Using .simplecov for centralized config
244
267
 
245
- If you use SimpleCov to merge multiple test suite results (e.g. Test/Unit and Cucumber) into a single report, you'd normally have to
246
- set up all your config options twice, once in `test_helper.rb` and once in `env.rb`.
268
+ If you use SimpleCov to merge multiple test suite results (e.g. Test/Unit and Cucumber) into a single report, you'd
269
+ normally have to set up all your config options twice, once in `test_helper.rb` and once in `env.rb`.
247
270
 
248
- To avoid this, you can place a file called `.simplecov` in your project root. You can then just leave the `require 'simplecov'` in each
249
- test setup helper (**at the top**) and move the `SimpleCov.start` code with all your custom config options into `.simplecov`:
271
+ To avoid this, you can place a file called `.simplecov` in your project root. You can then just leave the
272
+ `require 'simplecov'` in each test setup helper (**at the top**) and move the `SimpleCov.start` code with all your
273
+ custom config options into `.simplecov`:
250
274
 
251
275
  ```ruby
252
276
  # test/test_helper.rb
@@ -261,21 +285,100 @@ SimpleCov.start 'rails' do
261
285
  end
262
286
  ```
263
287
 
264
- Using `.simplecov` rather than separately requiring SimpleCov multiple times is recommended if you are merging multiple test frameworks like Cucumber and RSpec that rely on each other, as invoking SimpleCov multiple times can cause coverage information to be lost.
288
+ Using `.simplecov` rather than separately requiring SimpleCov multiple times is recommended if you are merging multiple
289
+ test frameworks like Cucumber and RSpec that rely on each other, as invoking SimpleCov multiple times can cause coverage
290
+ information to be lost.
291
+
292
+ ## Branch coverage (ruby "~> 2.5")
293
+ Add branch coverage measurement statistics to your results. Supported in CRuby versions 2.5+.
294
+
295
+ ```ruby
296
+ SimpleCov.start do
297
+ enable_coverage :branch
298
+ end
299
+ ```
300
+
301
+ Branch coverage is a feature introduced in Ruby 2.5 concerning itself with whether a
302
+ particular branch of a condition had been executed. Line coverage on the other hand
303
+ is only interested in whether a line of code has been executed.
304
+
305
+ This comes in handy for instance for one line conditionals:
306
+
307
+ ```ruby
308
+ number.odd? ? "odd" : "even"
309
+ ```
310
+
311
+ In line coverage this line would always be marked as executed but you'd never know if both
312
+ conditions were met. Guard clauses have a similar story:
313
+
314
+ ```ruby
315
+ return if number.odd?
316
+
317
+ # more code
318
+ ```
319
+
320
+ If all the code in that method was covered you'd never know if the guard clause was ever
321
+ triggered! With line coverage as just evaluating the condition marks it as covered.
322
+
323
+ In the HTML report the lines of code will be annotated like `branch_type: hit_count`:
324
+
325
+ * `then: 2` - the then branch (of an `if`) was executed twice
326
+ * `else: 0` - the else branch (of an `if` or `case`) was never executed
327
+
328
+ Not that even if you don't declare an `else` branch it will still show up in the coverage
329
+ reports meaning that the condition of the `if` was not hit or that no `when` of `case`
330
+ was hit during the test runs.
331
+
332
+ **Is branch coverage strictly better?** No. Branch coverage really only concerns itself with
333
+ conditionals - meaning coverage of sequential code is of no interest to it. A file without
334
+ conditional logic will have no branch coverage data and SimpleCov will report 0 of 0
335
+ branches covered as 100% (as everything that can be covered was covered).
336
+
337
+ Hence, we recommend looking at both metrics together. Branch coverage might also be a good
338
+ overall metric to look at - while you might be missing only 10% of your lines that might
339
+ account for 50% of your branches for instance.
340
+
341
+ ## Primary Coverage
342
+
343
+ By default, the primary coverage type is `line`. To set the primary coverage to something else, use the following:
344
+
345
+ ```ruby
346
+ # or in configure SimpleCov.primary_coverage :branch
347
+ SimpleCov.start do
348
+ enable_coverage :branch
349
+ primary_coverage :branch
350
+ end
351
+ ```
352
+
353
+ Primary coverage determines what will come in first all output, and the type of coverage to check if you don't specify the type of coverage when customizing exit behavior (`SimpleCov.minimum_coverage 90`).
354
+
355
+ Note that coverage must first be enabled for non-default coverage types.
356
+
357
+ ## Coverage for eval
358
+
359
+ You can measure coverage for code that is evaluated by `Kernel#eval`. Supported in CRuby versions 3.2+.
360
+
361
+ ```ruby
362
+ SimpleCov.start do
363
+ enable_coverage_for_eval
364
+ end
365
+ ```
366
+
367
+ This is typically useful for ERB. Set `ERB#filename=` to make it possible for SimpleCov to trace the original .erb source file.
265
368
 
266
369
  ## Filters
267
370
 
268
- Filters can be used to remove selected files from your coverage data. By default, a filter is applied that removes all files
269
- OUTSIDE of your project's root directory - otherwise you'd end up with billions of coverage reports for source files in the
270
- gems you are using.
371
+ Filters can be used to remove selected files from your coverage data. By default, a filter is applied that removes all
372
+ files OUTSIDE of your project's root directory - otherwise you'd end up with billions of coverage reports for source
373
+ files in the gems you are using.
271
374
 
272
375
  You can define your own to remove things like configuration files, tests or whatever you don't need in your coverage
273
376
  report.
274
377
 
275
378
  ### Defining custom filters
276
379
 
277
- You can currently define a filter using either a String or Regexp (that will then be Regexp-matched against each source file's path),
278
- a block or by passing in your own Filter class.
380
+ You can currently define a filter using either a String or Regexp (that will then be Regexp-matched against each source
381
+ file's path), a block or by passing in your own Filter class.
279
382
 
280
383
  #### String filter
281
384
 
@@ -307,9 +410,10 @@ SimpleCov.start do
307
410
  end
308
411
  ```
309
412
 
310
- Block filters receive a SimpleCov::SourceFile instance and expect your block to return either true (if the file is to be removed
311
- from the result) or false (if the result should be kept). Please check out the RDoc for SimpleCov::SourceFile to learn about the
312
- methods available to you. In the above example, the filter will remove all files that have less than 5 lines of code.
413
+ Block filters receive a SimpleCov::SourceFile instance and expect your block to return either true (if the file is to be
414
+ removed from the result) or false (if the result should be kept). Please check out the RDoc for SimpleCov::SourceFile to
415
+ learn about the methods available to you. In the above example, the filter will remove all files that have less than 5
416
+ lines of code.
313
417
 
314
418
  #### Custom filter class
315
419
 
@@ -323,9 +427,10 @@ end
323
427
  SimpleCov.add_filter LineFilter.new(5)
324
428
  ```
325
429
 
326
- Defining your own filters is pretty easy: Just inherit from SimpleCov::Filter and define a method 'matches?(source_file)'. When running
327
- the filter, a true return value from this method will result in the removal of the given source_file. The filter_argument method
328
- is being set in the SimpleCov::Filter initialize method and thus is set to 5 in this example.
430
+ Defining your own filters is pretty easy: Just inherit from SimpleCov::Filter and define a method
431
+ 'matches?(source_file)'. When running the filter, a true return value from this method will result in the removal of the
432
+ given source_file. The filter_argument method is being set in the SimpleCov::Filter initialize method and thus is set to
433
+ 5 in this example.
329
434
 
330
435
  #### Array filter
331
436
 
@@ -350,16 +455,18 @@ end
350
455
  # :nocov:
351
456
  ```
352
457
 
353
- The name of the token can be changed to your liking. [Learn more about the nocov feature.]( https://github.com/colszowka/simplecov/blob/master/features/config_nocov_token.feature)
458
+ The name of the token can be changed to your liking. [Learn more about the nocov feature.]( https://github.com/simplecov-ruby/simplecov/blob/main/features/config_nocov_token.feature)
354
459
 
355
- **Note:** You shouldn't have to use the nocov token to skip private methods that are being included in your coverage. If you appropriately test the public interface of your classes and objects you should automatically get full coverage of your private methods.
460
+ **Note:** You shouldn't have to use the nocov token to skip private methods that are being included in your coverage. If
461
+ you appropriately test the public interface of your classes and objects you should automatically get full coverage of
462
+ your private methods.
356
463
 
357
464
  ## Default root filter and coverage for things outside of it
358
465
 
359
466
  By default, SimpleCov filters everything outside of the `SimpleCov.root` directory. However, sometimes you may want
360
467
  to include coverage reports for things you include as a gem, for example a Rails Engine.
361
468
 
362
- Here's an example by [@lsaffie](https://github.com/lsaffie) from [#221](https://github.com/colszowka/simplecov/issues/221)
469
+ Here's an example by [@lsaffie](https://github.com/lsaffie) from [#221](https://github.com/simplecov-ruby/simplecov/issues/221)
363
470
  that shows how you can achieve just that:
364
471
 
365
472
  ```ruby
@@ -375,8 +482,8 @@ end
375
482
 
376
483
  You can separate your source files into groups. For example, in a Rails app, you'll want to have separate listings for
377
484
  Models, Controllers, Helpers, and Libs. Group definition works similarly to Filters (and also accepts custom
378
- filter classes), but source files end up in a group when the filter passes (returns true), as opposed to filtering results,
379
- which exclude files from results when the filter results in a true value.
485
+ filter classes), but source files end up in a group when the filter passes (returns true), as opposed to filtering
486
+ results, which exclude files from results when the filter results in a true value.
380
487
 
381
488
  Add your groups with:
382
489
 
@@ -396,12 +503,11 @@ end
396
503
 
397
504
  You normally want to have your coverage analyzed across ALL of your test suites, right?
398
505
 
399
- Simplecov automatically caches coverage results in your (coverage_path)/.resultset.json. Those results will then
400
- be automatically merged when generating the result, so when coverage is set up properly for Cucumber and your
401
- unit / functional / integration tests, all of those test suites will be taken into account when building the
402
- coverage report.
403
-
404
- There are two things to note here though:
506
+ Simplecov automatically caches coverage results in your
507
+ (coverage_path)/.resultset.json, and will merge or override those with
508
+ subsequent runs, depending on whether simplecov considers those subsequent runs
509
+ as different test suites or as the same test suite as the cached results. To
510
+ make this distinction, simplecov has the concept of "test suite names".
405
511
 
406
512
  ### Test suite names
407
513
 
@@ -455,24 +561,143 @@ SimpleCov.command_name "features" + (ENV['TEST_ENV_NUMBER'] || '')
455
561
 
456
562
  [simplecov-html] prints the used test suites in the footer of the generated coverage report.
457
563
 
458
- ### Timeout for merge
459
564
 
460
- Of course, your cached coverage data is likely to become invalid at some point. Thus, result sets that are older than
461
- `SimpleCov.merge_timeout` will not be used any more. By default, the timeout is 600 seconds (10 minutes), and you can
462
- raise (or lower) it by specifying `SimpleCov.merge_timeout 3600` (1 hour), or, inside a configure/start block, with
463
- just `merge_timeout 3600`.
565
+ ### Merging test runs under the same execution environment
566
+
567
+ Test results are automatically merged with previous runs in the same execution
568
+ environment when generating the result, so when coverage is set up properly for
569
+ Cucumber and your unit / functional / integration tests, all of those test
570
+ suites will be taken into account when building the coverage report.
571
+
572
+ #### Timeout for merge
573
+
574
+ Of course, your cached coverage data is likely to become invalid at some point. Thus, when automatically merging
575
+ subsequent test runs, result sets that are older than `SimpleCov.merge_timeout` will not be used any more. By default,
576
+ the timeout is 600 seconds (10 minutes), and you can raise (or lower) it by specifying `SimpleCov.merge_timeout 3600`
577
+ (1 hour), or, inside a configure/start block, with just `merge_timeout 3600`.
578
+
579
+ You can deactivate this automatic merging altogether with `SimpleCov.use_merging false`.
580
+
581
+ ### Merging test runs under different execution environments
582
+
583
+ If your tests are done in parallel across multiple build machines, you can fetch them all and merge them into a single
584
+ result set using the `SimpleCov.collate` method. This can be added to a Rakefile or script file, having downloaded a set of
585
+ `.resultset.json` files from each parallel test run.
586
+
587
+ ```ruby
588
+ # lib/tasks/coverage_report.rake
589
+ namespace :coverage do
590
+ desc "Collates all result sets generated by the different test runners"
591
+ task :report do
592
+ require 'simplecov'
593
+
594
+ SimpleCov.collate Dir["simplecov-resultset-*/.resultset.json"]
595
+ end
596
+ end
597
+ ```
598
+
599
+ `SimpleCov.collate` also takes an optional simplecov profile and an optional
600
+ block for configuration, just the same as `SimpleCov.start` or
601
+ `SimpleCov.configure`. This means you can configure a separate formatter for
602
+ the collated output. For instance, you can make the formatter in
603
+ `SimpleCov.start` the `SimpleCov::Formatter::SimpleFormatter`, and only use more
604
+ complex formatters in the final `SimpleCov.collate` run.
605
+
606
+ ```ruby
607
+ # spec/spec_helper.rb
608
+ require 'simplecov'
609
+
610
+ SimpleCov.start 'rails' do
611
+ # Disambiguates individual test runs
612
+ command_name "Job #{ENV["TEST_ENV_NUMBER"]}" if ENV["TEST_ENV_NUMBER"]
613
+
614
+ if ENV['CI']
615
+ formatter SimpleCov::Formatter::SimpleFormatter
616
+ else
617
+ formatter SimpleCov::Formatter::MultiFormatter.new([
618
+ SimpleCov::Formatter::SimpleFormatter,
619
+ SimpleCov::Formatter::HTMLFormatter
620
+ ])
621
+ end
622
+
623
+ track_files "**/*.rb"
624
+ end
625
+ ```
626
+
627
+ ```ruby
628
+ # lib/tasks/coverage_report.rake
629
+ namespace :coverage do
630
+ task :report do
631
+ require 'simplecov'
632
+
633
+ SimpleCov.collate Dir["simplecov-resultset-*/.resultset.json"], 'rails' do
634
+ formatter SimpleCov::Formatter::MultiFormatter.new([
635
+ SimpleCov::Formatter::SimpleFormatter,
636
+ SimpleCov::Formatter::HTMLFormatter
637
+ ])
638
+ end
639
+ end
640
+ end
641
+ ```
642
+
643
+ ## Running simplecov against subprocesses
644
+
645
+ `SimpleCov.enable_for_subprocesses` will allow SimpleCov to observe subprocesses starting using `Process.fork`.
646
+ This modifies ruby's core Process.fork method so that SimpleCov can see into it, appending `" (subprocess #{pid})"`
647
+ to the `SimpleCov.command_name`, with results that can be merged together using SimpleCov's merging feature.
648
+
649
+ To configure this, use `.at_fork`.
650
+
651
+ ```ruby
652
+ SimpleCov.enable_for_subprocesses true
653
+ SimpleCov.at_fork do |pid|
654
+ # This needs a unique name so it won't be ovewritten
655
+ SimpleCov.command_name "#{SimpleCov.command_name} (subprocess: #{pid})"
656
+ # be quiet, the parent process will be in charge of output and checking coverage totals
657
+ SimpleCov.print_error_status = false
658
+ SimpleCov.formatter SimpleCov::Formatter::SimpleFormatter
659
+ SimpleCov.minimum_coverage 0
660
+ # start
661
+ SimpleCov.start
662
+ end
663
+ ```
664
+
665
+ NOTE: SimpleCov must have already been started before `Process.fork` was called.
666
+
667
+ ### Running simplecov against spawned subprocesses
464
668
 
465
- You can deactivate merging altogether with `SimpleCov.use_merging false`.
669
+ Perhaps you're testing a ruby script with `PTY.spawn` or `Open3.popen`, or `Process.spawn` or etc.
670
+ SimpleCov can cover this too.
671
+
672
+ Add a .simplecov_spawn.rb file to your project root
673
+ ```ruby
674
+ # .simplecov_spawn.rb
675
+ require 'simplecov' # this will also pick up whatever config is in .simplecov
676
+ # so ensure it just contains configuration, and doesn't call SimpleCov.start.
677
+ SimpleCov.command_name 'spawn' # As this is not for a test runner directly, script doesn't have a pre-defined base command_name
678
+ SimpleCov.at_fork.call(Process.pid) # Use the per-process setup described previously
679
+ SimpleCov.start # only now can we start.
680
+ ```
681
+ Then, instead of calling your script directly, like:
682
+ ```ruby
683
+ PTY.spawn('my_script.rb') do # ...
684
+ ```
685
+ Use bin/ruby to require the new .simplecov_spawn file, then your script
686
+ ```ruby
687
+ PTY.spawn('ruby -r./.simplecov_spawn my_script.rb') do # ...
688
+ ```
466
689
 
467
690
  ## Running coverage only on demand
468
691
 
469
- The Ruby STDLIB Coverage library that SimpleCov builds upon is *very* fast (on a ~10 min Rails test suite, the speed drop was
470
- only a couple seconds for me), and therefore it's SimpleCov's policy to just generate coverage every time you run your tests because
471
- it doesn't do your test speed any harm and you're always equipped with the latest and greatest coverage results.
692
+ The Ruby STDLIB Coverage library that SimpleCov builds upon is *very* fast (on a ~10 min Rails test suite, the speed
693
+ drop was only a couple seconds for me), and therefore it's SimpleCov's policy to just generate coverage every time you
694
+ run your tests because it doesn't do your test speed any harm and you're always equipped with the latest and greatest
695
+ coverage results.
472
696
 
473
697
  Because of this, SimpleCov has no explicit built-in mechanism to run coverage only on demand.
474
698
 
475
- However, you can still accomplish this very easily by introducing an ENV variable conditional into your SimpleCov setup block, like this:
699
+ However, you can still accomplish this very easily by introducing an ENV variable conditional into your SimpleCov setup
700
+ block, like this:
476
701
 
477
702
  ```ruby
478
703
  SimpleCov.start if ENV["COVERAGE"]
@@ -484,6 +709,21 @@ Then, SimpleCov will only run if you execute your tests like this:
484
709
  COVERAGE=true rake test
485
710
  ```
486
711
 
712
+ ## Errors and exit statuses
713
+
714
+ To aid in debugging issues, if an error is raised, SimpleCov will print a message to `STDERR`
715
+ with the exit status of the error, like:
716
+
717
+ ```
718
+ SimpleCov failed with exit 1
719
+ ```
720
+
721
+ This `STDERR` message can be disabled with:
722
+
723
+ ```
724
+ SimpleCov.print_error_status = false
725
+ ```
726
+
487
727
  ## Profiles
488
728
 
489
729
  By default, SimpleCov's only config assumption is that you only want coverage reports for files inside your project
@@ -521,8 +761,8 @@ end
521
761
 
522
762
  ### Custom profiles
523
763
 
524
- You can load additional profiles with the SimpleCov.load_profile('xyz') method. This allows you to build upon an existing
525
- profile and customize it so you can reuse it in unit tests and Cucumber features. For example:
764
+ You can load additional profiles with the SimpleCov.load_profile('xyz') method. This allows you to build upon an
765
+ existing profile and customize it so you can reuse it in unit tests and Cucumber features. For example:
526
766
 
527
767
  ```ruby
528
768
  # lib/simplecov_custom_profile.rb
@@ -559,14 +799,23 @@ You can define the minimum coverage percentage expected. SimpleCov will return n
559
799
 
560
800
  ```ruby
561
801
  SimpleCov.minimum_coverage 90
802
+ # same as above (the default is to check line coverage)
803
+ SimpleCov.minimum_coverage line: 90
804
+ # check for a minimum line coverage of 90% and minimum 80% branch coverage
805
+ SimpleCov.minimum_coverage line: 90, branch: 80
562
806
  ```
563
807
 
564
808
  ### Minimum coverage by file
565
809
 
566
- You can define the minimum coverage by file percentage expected. SimpleCov will return non-zero if unmet. This is useful to help ensure coverage is relatively consistent, rather than being skewed by particularly good or bad areas of the code.
810
+ You can define the minimum coverage by file percentage expected. SimpleCov will return non-zero if unmet. This is useful
811
+ to help ensure coverage is relatively consistent, rather than being skewed by particularly good or bad areas of the code.
567
812
 
568
813
  ```ruby
569
814
  SimpleCov.minimum_coverage_by_file 80
815
+ # same as above (the default is to check line coverage by file)
816
+ SimpleCov.minimum_coverage_by_file line: 80
817
+ # check for a minimum line coverage by file of 90% and minimum 80% branch coverage
818
+ SimpleCov.minimum_coverage_by_file line: 90, branch: 80
570
819
  ```
571
820
 
572
821
  ### Maximum coverage drop
@@ -575,6 +824,10 @@ You can define the maximum coverage drop percentage at once. SimpleCov will retu
575
824
 
576
825
  ```ruby
577
826
  SimpleCov.maximum_coverage_drop 5
827
+ # same as above (the default is to check line drop)
828
+ SimpleCov.maximum_coverage_drop line: 5
829
+ # check for a maximum line drop of 5% and maximum 10% branch drop
830
+ SimpleCov.maximum_coverage_drop line: 5, branch: 10
578
831
  ```
579
832
 
580
833
  ### Refuse dropping coverage
@@ -583,6 +836,10 @@ You can also entirely refuse dropping coverage between test runs:
583
836
 
584
837
  ```ruby
585
838
  SimpleCov.refuse_coverage_drop
839
+ # same as above (the default is to only refuse line drop)
840
+ SimpleCov.refuse_coverage_drop :line
841
+ # refuse drop for line and branch
842
+ SimpleCov.refuse_coverage_drop :line, :branch
586
843
  ```
587
844
 
588
845
  ## Using your own formatter
@@ -593,21 +850,37 @@ You can use your own formatter with:
593
850
  SimpleCov.formatter = SimpleCov::Formatter::HTMLFormatter
594
851
  ```
595
852
 
596
- When calling SimpleCov.result.format!, it will be invoked with SimpleCov::Formatter::YourFormatter.new.format(result), "result"
597
- being an instance of SimpleCov::Result. Do whatever your wish with that!
853
+ Calling `SimpleCov.result.format!` will be invoked with `SimpleCov::Formatter::YourFormatter.new.format(result)`,
854
+ and `result` is an instance of `SimpleCov::Result`. Do whatever your wish with that!
598
855
 
599
856
 
600
857
  ## Using multiple formatters
601
858
 
602
- As of SimpleCov 0.9, you can specify multiple result formats:
859
+ As of SimpleCov 0.9, you can specify multiple result formats. Formatters besides the default HTML formatter require separate gems, however.
603
860
 
604
861
  ```ruby
862
+ require "simplecov-html"
863
+
605
864
  SimpleCov.formatters = SimpleCov::Formatter::MultiFormatter.new([
606
865
  SimpleCov::Formatter::HTMLFormatter,
607
866
  SimpleCov::Formatter::CSVFormatter,
608
867
  ])
609
868
  ```
610
869
 
870
+ ## JSON formatter
871
+
872
+ SimpleCov is packaged with a separate gem called [simplecov_json_formatter](https://github.com/codeclimate-community/simplecov_json_formatter) that provides you with a JSON formatter, this formatter could be useful for different use cases, such as for CI consumption or for reporting to external services.
873
+
874
+ In order to use it you will need to manually load the installed gem like so:
875
+
876
+ ```ruby
877
+ require "simplecov_json_formatter"
878
+ SimpleCov.formatter = SimpleCov::Formatter::JSONFormatter
879
+ ```
880
+
881
+ > _Note:_ In case you plan to report your coverage results to CodeClimate services, know that SimpleCov will automatically use the
882
+ > JSON formatter along with the HTML formatter when the `CC_TEST_REPORTER_ID` variable is present in the environment.
883
+
611
884
  ## Available formatters, editor integrations and hosted services
612
885
 
613
886
  * [Open Source formatter and integration plugins for SimpleCov](doc/alternate-formatters.md)
@@ -616,11 +889,9 @@ SimpleCov.formatters = SimpleCov::Formatter::MultiFormatter.new([
616
889
 
617
890
  ## Ruby version compatibility
618
891
 
619
- Only Ruby 1.9+ ships with the coverage library that SimpleCov depends upon and that's what SimpleCov supports. Additionally JRuby 9.1+ is supported as well, while JRuby 1.7 and 9.0 should work they're not "officially" supported.
620
- SimpleCov is also built against Ruby 1.8 in [Continuous Integration], but this happens only to ensure that SimpleCov
621
- does not make your test suite crash right now.
892
+ SimpleCov is built in [Continuous Integration] on Ruby 2.7+ as well as JRuby 9.3+.
622
893
 
623
- SimpleCov is built in [Continuous Integration] on Ruby 1.9.3, 2.0.0, 2.1, 2.2, 2.3, 2.4, 2.5 as well as JRuby 9.1.
894
+ Note for JRuby => You need to pass JRUBY_OPTS="--debug" or create .jrubyrc and add debug.fullTrace=true
624
895
 
625
896
  ## Want to find dead code in production?
626
897
 
@@ -628,48 +899,36 @@ Try [Coverband](https://github.com/danmayer/coverband).
628
899
 
629
900
  ## Want to use Spring with SimpleCov?
630
901
 
631
- If you're using [Spring](https://github.com/rails/spring) to speed up test suite runs and want to run SimpleCov along with them, you'll find that it often misreports coverage with the default config due to some sort of eager loading issue. Don't despair!
902
+ If you're using [Spring](https://github.com/rails/spring) to speed up test suite runs and want to run SimpleCov along
903
+ with them, you'll find that it often misreports coverage with the default config due to some sort of eager loading
904
+ issue. Don't despair!
632
905
 
633
- 1. Change the following settings in `test.rb`.
906
+ One solution is to [explicitly call eager
907
+ load](https://github.com/simplecov-ruby/simplecov/issues/381#issuecomment-347651728)
908
+ in your `test_helper.rb` / `spec_helper.rb` after calling `SimpleCov.start`.
634
909
 
635
- ```ruby
636
- # For Rails
637
- # Do not eager load code on boot
638
- config.eager_load = false
639
- ```
640
- 2. Add your SimpleCov config, as you normally would, to your `spec_helper.rb`
641
- (or `rails_helper.rb` for RSpec 3). If you have a `config/spring.rb` file
642
- (or anything similar), add it to the start of such file. Here's a simple
643
- version of what the config should look like:
644
-
645
- ```ruby
646
- if ENV['RAILS_ENV'] == 'test'
647
- require 'simplecov'
648
- SimpleCov.start
649
- end
650
- ```
651
- 3. Run `spring rspec <path>` as normal. Remember to run `spring stop` after
652
- making important changes to your app or its specs!
653
-
654
- ## Want to use bootsnap with SimpleCov?
910
+ ```ruby
911
+ require 'simplecov'
912
+ SimpleCov.start 'rails'
913
+ Rails.application.eager_load!
914
+ ```
655
915
 
656
- As mentioned in [this issue](https://github.com/Shopify/bootsnap/issues/35) iseq
657
- loading/dumping doesn't work with coverage. Hence you need to deactivate it when
658
- you run coverage so for instance when you use the environment `COVERAGE=true` to
659
- decide that you want to gather coverage you can do:
916
+ Alternatively, you could disable Spring while running SimpleCov:
660
917
 
661
- ```ruby
662
- Bootsnap.setup(
663
- compile_cache_iseq: !ENV["COVERAGE"], # Compile Ruby code into ISeq cache, breaks coverage reporting.
664
- # all those other options
665
- )
666
918
  ```
919
+ DISABLE_SPRING=1 rake test
920
+ ```
921
+
922
+ Or you could remove `gem 'spring'` from your `Gemfile`.
667
923
 
668
924
  ## Troubleshooting
669
925
 
670
- The **most common problem is that simplecov isn't required and started before everything else**. In order to track coverage for your whole application **simplecov needs to be the first one** so that it (and the underlying coverage library) can subsequently track loaded files and their usage.
926
+ The **most common problem is that simplecov isn't required and started before everything else**. In order to track
927
+ coverage for your whole application **simplecov needs to be the first one** so that it (and the underlying coverage
928
+ library) can subsequently track loaded files and their usage.
671
929
 
672
- If you are missing coverage for some code a simple trick is to put a puts statement in there and right after `SimpleCov.start` so you can see if the file really was loaded after simplecov was started.
930
+ If you are missing coverage for some code a simple trick is to put a puts statement in there and right after
931
+ `SimpleCov.start` so you can see if the file really was loaded after simplecov was started.
673
932
 
674
933
  ```ruby
675
934
  # my_code.rb
@@ -697,9 +956,14 @@ MyCode is being loaded!
697
956
 
698
957
  then it's good otherwise you likely have a problem :)
699
958
 
959
+ ## Code of Conduct
960
+
961
+ Everyone participating in this project's development, issue trackers and other channels is expected to follow our
962
+ [Code of Conduct](./CODE_OF_CONDUCT.md)
963
+
700
964
  ## Contributing
701
965
 
702
- See the [contributing guide](https://github.com/colszowka/simplecov/blob/master/CONTRIBUTING.md).
966
+ See the [contributing guide](https://github.com/simplecov-ruby/simplecov/blob/main/CONTRIBUTING.md).
703
967
 
704
968
  ## Kudos
705
969