simplecov 0.17.1 → 0.21.0

Sign up to get free protection for your applications and to get access to all the features.
Files changed (48) hide show
  1. checksums.yaml +4 -4
  2. data/CHANGELOG.md +97 -437
  3. data/README.md +375 -93
  4. data/doc/alternate-formatters.md +16 -1
  5. data/doc/commercial-services.md +5 -0
  6. data/lib/minitest/simplecov_plugin.rb +15 -0
  7. data/lib/simplecov.rb +294 -128
  8. data/lib/simplecov/combine.rb +30 -0
  9. data/lib/simplecov/combine/branches_combiner.rb +32 -0
  10. data/lib/simplecov/combine/files_combiner.rb +24 -0
  11. data/lib/simplecov/combine/lines_combiner.rb +43 -0
  12. data/lib/simplecov/combine/results_combiner.rb +60 -0
  13. data/lib/simplecov/command_guesser.rb +6 -3
  14. data/lib/simplecov/configuration.rb +191 -15
  15. data/lib/simplecov/coverage_statistics.rb +56 -0
  16. data/lib/simplecov/default_formatter.rb +20 -0
  17. data/lib/simplecov/defaults.rb +14 -13
  18. data/lib/simplecov/exit_codes.rb +5 -0
  19. data/lib/simplecov/exit_codes/exit_code_handling.rb +29 -0
  20. data/lib/simplecov/exit_codes/maximum_coverage_drop_check.rb +73 -0
  21. data/lib/simplecov/exit_codes/minimum_coverage_by_file_check.rb +54 -0
  22. data/lib/simplecov/exit_codes/minimum_overall_coverage_check.rb +53 -0
  23. data/lib/simplecov/file_list.rb +72 -13
  24. data/lib/simplecov/filter.rb +9 -6
  25. data/lib/simplecov/formatter.rb +2 -2
  26. data/lib/simplecov/formatter/multi_formatter.rb +5 -7
  27. data/lib/simplecov/formatter/simple_formatter.rb +4 -4
  28. data/lib/simplecov/last_run.rb +3 -1
  29. data/lib/simplecov/lines_classifier.rb +5 -5
  30. data/lib/simplecov/no_defaults.rb +1 -1
  31. data/lib/simplecov/process.rb +19 -0
  32. data/lib/simplecov/profiles.rb +9 -7
  33. data/lib/simplecov/result.rb +18 -12
  34. data/lib/simplecov/result_adapter.rb +30 -0
  35. data/lib/simplecov/result_merger.rb +130 -59
  36. data/lib/simplecov/simulate_coverage.rb +29 -0
  37. data/lib/simplecov/source_file.rb +272 -126
  38. data/lib/simplecov/source_file/branch.rb +84 -0
  39. data/lib/simplecov/source_file/line.rb +72 -0
  40. data/lib/simplecov/useless_results_remover.rb +18 -0
  41. data/lib/simplecov/version.rb +1 -1
  42. metadata +44 -158
  43. data/CONTRIBUTING.md +0 -51
  44. data/ISSUE_TEMPLATE.md +0 -23
  45. data/lib/simplecov/jruby_fix.rb +0 -44
  46. data/lib/simplecov/railtie.rb +0 -9
  47. data/lib/simplecov/railties/tasks.rake +0 -13
  48. data/lib/simplecov/raw_coverage.rb +0 -41
data/README.md CHANGED
@@ -1,5 +1,6 @@
1
- SimpleCov [![Build Status](https://travis-ci.org/colszowka/simplecov.svg)][Continuous Integration] [![Code Climate](https://codeclimate.com/github/colszowka/simplecov.svg)](https://codeclimate.com/github/colszowka/simplecov) [![Inline docs](http://inch-ci.org/github/colszowka/simplecov.svg)](http://inch-ci.org/github/colszowka/simplecov)
1
+ SimpleCov [![Gem Version](https://badge.fury.io/rb/simplecov.svg)](https://badge.fury.io/rb/simplecov) [![Build Status](https://github.com/simplecov-ruby/simplecov/workflows/stable/badge.svg?branch=main)][Continuous Integration] [![Maintainability](https://api.codeclimate.com/v1/badges/c071d197d61953a7e482/maintainability)](https://codeclimate.com/github/simplecov-ruby/simplecov/maintainability) [![Inline docs](http://inch-ci.org/github/simplecov-ruby/simplecov.svg?branch=main)](http://inch-ci.org/github/simplecov-ruby/simplecov)
2
2
  =========
3
+
3
4
  **Code coverage for Ruby**
4
5
 
5
6
  * [Source Code]
@@ -8,39 +9,42 @@ SimpleCov [![Build Status](https://travis-ci.org/colszowka/simplecov.svg)][Conti
8
9
  * [Rubygem]
9
10
  * [Continuous Integration]
10
11
 
11
- [Coverage]: http://www.ruby-doc.org/stdlib-2.1.0/libdoc/coverage/rdoc/Coverage.html "API doc for Ruby's Coverage library"
12
- [Source Code]: https://github.com/colszowka/simplecov "Source Code @ GitHub"
12
+ [Coverage]: https://ruby-doc.org/stdlib/libdoc/coverage/rdoc/Coverage.html "API doc for Ruby's Coverage library"
13
+ [Source Code]: https://github.com/simplecov-ruby/simplecov "Source Code @ GitHub"
13
14
  [API documentation]: http://rubydoc.info/gems/simplecov/frames "RDoc API Documentation at Rubydoc.info"
14
15
  [Configuration]: http://rubydoc.info/gems/simplecov/SimpleCov/Configuration "Configuration options API documentation"
15
- [Changelog]: https://github.com/colszowka/simplecov/blob/master/CHANGELOG.md "Project Changelog"
16
+ [Changelog]: https://github.com/simplecov-ruby/simplecov/blob/main/CHANGELOG.md "Project Changelog"
16
17
  [Rubygem]: http://rubygems.org/gems/simplecov "SimpleCov @ rubygems.org"
17
- [Continuous Integration]: http://travis-ci.org/colszowka/simplecov "SimpleCov is built around the clock by travis-ci.org"
18
- [Dependencies]: https://gemnasium.com/colszowka/simplecov "SimpleCov dependencies on Gemnasium"
19
- [simplecov-html]: https://github.com/colszowka/simplecov-html "SimpleCov HTML Formatter Source Code @ GitHub"
18
+ [Continuous Integration]: https://github.com/simplecov-ruby/simplecov/actions?query=workflow%3Astable "SimpleCov is built around the clock by github.com"
19
+ [Dependencies]: https://gemnasium.com/simplecov-ruby/simplecov "SimpleCov dependencies on Gemnasium"
20
+ [simplecov-html]: https://github.com/simplecov-ruby/simplecov-html "SimpleCov HTML Formatter Source Code @ GitHub"
20
21
 
21
22
  SimpleCov is a code coverage analysis tool for Ruby. It uses [Ruby's built-in Coverage][Coverage] library to gather code
22
23
  coverage data, but makes processing its results much easier by providing a clean API to filter, group, merge, format,
23
24
  and display those results, giving you a complete code coverage suite that can be set up with just a couple lines of
24
25
  code.
26
+ SimpleCov/Coverage track covered ruby code, gathering coverage for common templating solutions like erb, slim and haml is not supported.
25
27
 
26
28
  In most cases, you'll want overall coverage results for your projects, including all types of tests, Cucumber features,
27
29
  etc. SimpleCov automatically takes care of this by caching and merging results when generating reports, so your
28
30
  report actually includes coverage across your test suites and thereby gives you a better picture of blank spots.
29
31
 
30
- The official formatter of SimpleCov is packaged as a separate gem called [simplecov-html], but will be installed and configured
31
- automatically when you launch SimpleCov. If you're curious, you can find it [on GitHub, too][simplecov-html].
32
+ The official formatter of SimpleCov is packaged as a separate gem called [simplecov-html], but will be installed and
33
+ configured automatically when you launch SimpleCov. If you're curious, you can find it [on GitHub, too][simplecov-html].
32
34
 
33
35
 
34
36
  ## Contact
35
37
 
36
38
  *Code and Bug Reports*
37
39
 
38
- * [Issue Tracker](https://github.com/colszowka/simplecov/issues)
39
- * See [CONTRIBUTING](https://github.com/colszowka/simplecov/blob/master/CONTRIBUTING.md) for how to contribute along with some common problems to check out before creating an issue.
40
+ * [Issue Tracker](https://github.com/simplecov-ruby/simplecov/issues)
41
+ * See [CONTRIBUTING](https://github.com/simplecov-ruby/simplecov/blob/main/CONTRIBUTING.md) for how to contribute along
42
+ with some common problems to check out before creating an issue.
40
43
 
41
44
  *Questions, Problems, Suggestions, etc.*
42
45
 
43
- * [Mailing List](https://groups.google.com/forum/#!forum/simplecov) "Open mailing list for discussion and announcements on Google Groups"
46
+ * [Mailing List](https://groups.google.com/forum/#!forum/simplecov) "Open mailing list for discussion and announcements
47
+ on Google Groups"
44
48
 
45
49
  Getting started
46
50
  ---------------
@@ -69,9 +73,10 @@ Getting started
69
73
  analysis to happen on. When testing a server process (e.g. a JSON API
70
74
  endpoint) via a separate test process (e.g. when using Selenium) where you
71
75
  want to see all code executed by the `rails server`, and not just code
72
- executed in your actual test files, you'll want to add something like this
73
- to the top of `script/rails` (or `bin/rails` for Rails 4), but below the
74
- "shebang" line (`#! /usr/bin/env ruby`):
76
+ executed in your actual test files, you need to require SimpleCov in the
77
+ server process. For rails for instance, you'll want to add something like this
78
+ to the top of `bin/rails`, but below the "shebang" line (`#! /usr/bin/env
79
+ ruby`) and after config/boot is required:
75
80
 
76
81
  ```ruby
77
82
  if ENV['RAILS_ENV'] == 'test'
@@ -81,27 +86,41 @@ Getting started
81
86
  end
82
87
  ```
83
88
 
84
- 3. Run your tests, open up `coverage/index.html` in your browser and check out
85
- what you've missed so far.
86
- 4. Add the following to your `.gitignore` file to ensure that coverage results
89
+ 3. Run your full test suite to see the percent coverage that your application has.
90
+ 4. After running your tests, open `coverage/index.html` in the browser of your choice. For example, in a Mac Terminal,
91
+ run the following command from your application's root directory:
92
+
93
+ ```
94
+ open coverage/index.html
95
+ ```
96
+ in a debian/ubuntu Terminal,
97
+
98
+ ```
99
+ xdg-open coverage/index.html
100
+ ```
101
+
102
+ **Note:** [This guide](https://dwheeler.com/essays/open-files-urls.html) can help if you're unsure which command your particular
103
+ operating system requires.
104
+
105
+ 5. Add the following to your `.gitignore` file to ensure that coverage results
87
106
  are not tracked by Git (optional):
88
107
 
89
- ```
90
- echo "coverage" >> .gitignore
91
- ```
92
- Or if you use Windows:
93
- ```
94
- echo coverage >> .gitignore
95
- ```
108
+ ```
109
+ echo "coverage" >> .gitignore
110
+ ```
111
+ Or if you use Windows:
112
+ ```
113
+ echo coverage >> .gitignore
114
+ ```
96
115
 
97
- If you're making a Rails application, SimpleCov comes with built-in configurations (see below for information on profiles)
98
- that will get you started with groups for your Controllers, Views, Models and Helpers. To use it, the first two lines of
99
- your test_helper should be like this:
116
+ If you're making a Rails application, SimpleCov comes with built-in configurations (see below for information on
117
+ profiles) that will get you started with groups for your Controllers, Models and Helpers. To use it, the
118
+ first two lines of your test_helper should be like this:
100
119
 
101
- ```ruby
102
- require 'simplecov'
103
- SimpleCov.start 'rails'
104
- ```
120
+ ```ruby
121
+ require 'simplecov'
122
+ SimpleCov.start 'rails'
123
+ ```
105
124
 
106
125
  ## Example output
107
126
 
@@ -127,8 +146,9 @@ require 'simplecov'
127
146
  SimpleCov.start 'rails'
128
147
  ```
129
148
 
130
- You could even track what kind of code your UI testers are touching if you want to go overboard with things. SimpleCov does not
131
- care what kind of framework it is running in; it just looks at what code is being executed and generates a report about it.
149
+ You could even track what kind of code your UI testers are touching if you want to go overboard with things. SimpleCov
150
+ does not care what kind of framework it is running in; it just looks at what code is being executed and generates a
151
+ report about it.
132
152
 
133
153
  ### Notes on specific frameworks and test utilities
134
154
 
@@ -148,8 +168,19 @@ to use SimpleCov with them. Here's an overview of the known ones:
148
168
  race conditions occur when results are merged.
149
169
  </td>
150
170
  <td>
151
- <a href="https://github.com/colszowka/simplecov/issues/64">#64</a> &amp;
152
- <a href="https://github.com/colszowka/simplecov/pull/185">#185</a>
171
+ <a href="https://github.com/simplecov-ruby/simplecov/issues/64">#64</a> &amp;
172
+ <a href="https://github.com/simplecov-ruby/simplecov/pull/185">#185</a>
173
+ </td>
174
+ </tr>
175
+ <tr>
176
+ <th>
177
+ knapsack_pro
178
+ </th>
179
+ <td>
180
+ To make SimpleCov work with Knapsack Pro Queue Mode to split tests in parallel on CI jobs you need to provide CI node index number to the <code>SimpleCov.command_name</code> in <code>KnapsackPro::Hooks::Queue.before_queue</code> hook.
181
+ </td>
182
+ <td>
183
+ <a href="https://knapsackpro.com/faq/question/how-to-use-simplecov-in-queue-mode">Tip</a>
153
184
  </td>
154
185
  </tr>
155
186
  <tr>
@@ -162,7 +193,7 @@ to use SimpleCov with them. Here's an overview of the known ones:
162
193
  to explicitly set the output root using `SimpleCov.root('foo/bar/baz')`
163
194
  </td>
164
195
  <td>
165
- <a href="https://github.com/colszowka/simplecov/issues/95">#95</a>
196
+ <a href="https://github.com/simplecov-ruby/simplecov/issues/95">#95</a>
166
197
  </td>
167
198
  </tr>
168
199
  <tr>
@@ -173,11 +204,11 @@ to use SimpleCov with them. Here's an overview of the known ones:
173
204
  Because of how Spork works internally (using preforking), there used to
174
205
  be trouble when using SimpleCov with it, but that has apparently been
175
206
  resolved with a specific configuration strategy. See <a
176
- href="https://github.com/colszowka/simplecov/issues/42#issuecomment-4440284">this</a>
207
+ href="https://github.com/simplecov-ruby/simplecov/issues/42#issuecomment-4440284">this</a>
177
208
  comment.
178
209
  </td>
179
210
  <td>
180
- <a href="https://github.com/colszowka/simplecov/issues/42#issuecomment-4440284">#42</a>
211
+ <a href="https://github.com/simplecov-ruby/simplecov/issues/42#issuecomment-4440284">#42</a>
181
212
  </td>
182
213
  </tr>
183
214
  <tr>
@@ -188,7 +219,7 @@ to use SimpleCov with them. Here's an overview of the known ones:
188
219
  <a href="#want-to-use-spring-with-simplecov">See section below.</a>
189
220
  </td>
190
221
  <td>
191
- <a href="https://github.com/colszowka/simplecov/issues/381">#381</a>
222
+ <a href="https://github.com/simplecov-ruby/simplecov/issues/381">#381</a>
192
223
  </td>
193
224
  </tr>
194
225
  <tr>
@@ -201,7 +232,7 @@ to use SimpleCov with them. Here's an overview of the known ones:
201
232
  (Dec 11th, 2011) should have this problem resolved.
202
233
  </td>
203
234
  <td>
204
- <a href="https://github.com/colszowka/simplecov/issues/45">#45</a> &amp;
235
+ <a href="https://github.com/simplecov-ruby/simplecov/issues/45">#45</a> &amp;
205
236
  <a href="https://github.com/test-unit/test-unit/pull/12">test-unit/test-unit#12</a>
206
237
  </td>
207
238
  </tr>
@@ -223,7 +254,8 @@ to use SimpleCov with them. Here's an overview of the known ones:
223
254
  ```ruby
224
255
  SimpleCov.some_config_option 'foo'
225
256
  ```
226
- * If you do not want to start coverage immediately after launch or want to add additional configuration later on in a concise way, use:
257
+ * If you do not want to start coverage immediately after launch or want to add additional configuration later on in a
258
+ concise way, use:
227
259
 
228
260
  ```ruby
229
261
  SimpleCov.configure do
@@ -235,11 +267,12 @@ Please check out the [Configuration] API documentation to find out what you can
235
267
 
236
268
  ## Using .simplecov for centralized config
237
269
 
238
- If you use SimpleCov to merge multiple test suite results (e.g. Test/Unit and Cucumber) into a single report, you'd normally have to
239
- set up all your config options twice, once in `test_helper.rb` and once in `env.rb`.
270
+ If you use SimpleCov to merge multiple test suite results (e.g. Test/Unit and Cucumber) into a single report, you'd
271
+ normally have to set up all your config options twice, once in `test_helper.rb` and once in `env.rb`.
240
272
 
241
- To avoid this, you can place a file called `.simplecov` in your project root. You can then just leave the `require 'simplecov'` in each
242
- test setup helper (**at the top**) and move the `SimpleCov.start` code with all your custom config options into `.simplecov`:
273
+ To avoid this, you can place a file called `.simplecov` in your project root. You can then just leave the
274
+ `require 'simplecov'` in each test setup helper (**at the top**) and move the `SimpleCov.start` code with all your
275
+ custom config options into `.simplecov`:
243
276
 
244
277
  ```ruby
245
278
  # test/test_helper.rb
@@ -254,21 +287,88 @@ SimpleCov.start 'rails' do
254
287
  end
255
288
  ```
256
289
 
257
- Using `.simplecov` rather than separately requiring SimpleCov multiple times is recommended if you are merging multiple test frameworks like Cucumber and RSpec that rely on each other, as invoking SimpleCov multiple times can cause coverage information to be lost.
290
+ Using `.simplecov` rather than separately requiring SimpleCov multiple times is recommended if you are merging multiple
291
+ test frameworks like Cucumber and RSpec that rely on each other, as invoking SimpleCov multiple times can cause coverage
292
+ information to be lost.
293
+
294
+ ## Branch coverage (ruby "~> 2.5")
295
+ Add branch coverage measurement statistics to your results. Supported in CRuby versions 2.5+.
296
+
297
+ ```ruby
298
+ SimpleCov.start do
299
+ enable_coverage :branch
300
+ end
301
+ ```
302
+
303
+ Branch coverage is a feature introduced in Ruby 2.5 concerning itself with whether a
304
+ particular branch of a condition had been executed. Line coverage on the other hand
305
+ is only interested in whether a line of code has been executed.
306
+
307
+ This comes in handy for instance for one line conditionals:
308
+
309
+ ```ruby
310
+ number.odd? ? "odd" : "even"
311
+ ```
312
+
313
+ In line coverage this line would always be marked as executed but you'd never know if both
314
+ conditions were met. Guard clauses have a similar story:
315
+
316
+ ```ruby
317
+ return if number.odd?
318
+
319
+ # more code
320
+ ```
321
+
322
+ If all the code in that method was covered you'd never know if the guard clause was ever
323
+ triggered! With line coverage as just evaluating the condition marks it as covered.
324
+
325
+ In the HTML report the lines of code will be annotated like `branch_type: hit_count`:
326
+
327
+ * `then: 2` - the then branch (of an `if`) was executed twice
328
+ * `else: 0` - the else branch (of an `if` or `case`) was never executed
329
+
330
+ Not that even if you don't declare an `else` branch it will still show up in the coverage
331
+ reports meaning that the condition of the `if` was not hit or that no `when` of `case`
332
+ was hit during the test runs.
333
+
334
+ **Is branch coverage strictly better?** No. Branch coverage really only concerns itself with
335
+ conditionals - meaning coverage of sequential code is of no interest to it. A file without
336
+ conditional logic will have no branch coverage data and SimpleCov will report 0 of 0
337
+ branches covered as 100% (as everything that can be covered was covered).
338
+
339
+ Hence, we recommend looking at both metrics together. Branch coverage might also be a good
340
+ overall metric to look at - while you might be missing only 10% of your lines that might
341
+ account for 50% of your branches for instance.
342
+
343
+ ## Primary Coverage
344
+
345
+ By default, the primary coverage type is `line`. To set the primary coverage to something else, use the following:
346
+
347
+ ```ruby
348
+ # or in configure SimpleCov.primary_coverage :branch
349
+ SimpleCov.start do
350
+ enable_coverage :branch
351
+ primary_coverage :branch
352
+ end
353
+ ```
354
+
355
+ Primary coverage determines what will come in first all output, and the type of coverage to check if you don't specify the type of coverage when customizing exit behavior (`SimpleCov.minimum_coverage 90`).
356
+
357
+ Note that coverage must first be enabled for non-default coverage types.
258
358
 
259
359
  ## Filters
260
360
 
261
- Filters can be used to remove selected files from your coverage data. By default, a filter is applied that removes all files
262
- OUTSIDE of your project's root directory - otherwise you'd end up with billions of coverage reports for source files in the
263
- gems you are using.
361
+ Filters can be used to remove selected files from your coverage data. By default, a filter is applied that removes all
362
+ files OUTSIDE of your project's root directory - otherwise you'd end up with billions of coverage reports for source
363
+ files in the gems you are using.
264
364
 
265
365
  You can define your own to remove things like configuration files, tests or whatever you don't need in your coverage
266
366
  report.
267
367
 
268
368
  ### Defining custom filters
269
369
 
270
- You can currently define a filter using either a String or Regexp (that will then be Regexp-matched against each source file's path),
271
- a block or by passing in your own Filter class.
370
+ You can currently define a filter using either a String or Regexp (that will then be Regexp-matched against each source
371
+ file's path), a block or by passing in your own Filter class.
272
372
 
273
373
  #### String filter
274
374
 
@@ -300,9 +400,10 @@ SimpleCov.start do
300
400
  end
301
401
  ```
302
402
 
303
- Block filters receive a SimpleCov::SourceFile instance and expect your block to return either true (if the file is to be removed
304
- from the result) or false (if the result should be kept). Please check out the RDoc for SimpleCov::SourceFile to learn about the
305
- methods available to you. In the above example, the filter will remove all files that have less than 5 lines of code.
403
+ Block filters receive a SimpleCov::SourceFile instance and expect your block to return either true (if the file is to be
404
+ removed from the result) or false (if the result should be kept). Please check out the RDoc for SimpleCov::SourceFile to
405
+ learn about the methods available to you. In the above example, the filter will remove all files that have less than 5
406
+ lines of code.
306
407
 
307
408
  #### Custom filter class
308
409
 
@@ -316,9 +417,10 @@ end
316
417
  SimpleCov.add_filter LineFilter.new(5)
317
418
  ```
318
419
 
319
- Defining your own filters is pretty easy: Just inherit from SimpleCov::Filter and define a method 'matches?(source_file)'. When running
320
- the filter, a true return value from this method will result in the removal of the given source_file. The filter_argument method
321
- is being set in the SimpleCov::Filter initialize method and thus is set to 5 in this example.
420
+ Defining your own filters is pretty easy: Just inherit from SimpleCov::Filter and define a method
421
+ 'matches?(source_file)'. When running the filter, a true return value from this method will result in the removal of the
422
+ given source_file. The filter_argument method is being set in the SimpleCov::Filter initialize method and thus is set to
423
+ 5 in this example.
322
424
 
323
425
  #### Array filter
324
426
 
@@ -343,16 +445,18 @@ end
343
445
  # :nocov:
344
446
  ```
345
447
 
346
- The name of the token can be changed to your liking. [Learn more about the nocov feature.]( https://github.com/colszowka/simplecov/blob/master/features/config_nocov_token.feature)
448
+ The name of the token can be changed to your liking. [Learn more about the nocov feature.]( https://github.com/simplecov-ruby/simplecov/blob/main/features/config_nocov_token.feature)
347
449
 
348
- **Note:** You shouldn't have to use the nocov token to skip private methods that are being included in your coverage. If you appropriately test the public interface of your classes and objects you should automatically get full coverage of your private methods.
450
+ **Note:** You shouldn't have to use the nocov token to skip private methods that are being included in your coverage. If
451
+ you appropriately test the public interface of your classes and objects you should automatically get full coverage of
452
+ your private methods.
349
453
 
350
454
  ## Default root filter and coverage for things outside of it
351
455
 
352
456
  By default, SimpleCov filters everything outside of the `SimpleCov.root` directory. However, sometimes you may want
353
457
  to include coverage reports for things you include as a gem, for example a Rails Engine.
354
458
 
355
- Here's an example by [@lsaffie](https://github.com/lsaffie) from [#221](https://github.com/colszowka/simplecov/issues/221)
459
+ Here's an example by [@lsaffie](https://github.com/lsaffie) from [#221](https://github.com/simplecov-ruby/simplecov/issues/221)
356
460
  that shows how you can achieve just that:
357
461
 
358
462
  ```ruby
@@ -368,8 +472,8 @@ end
368
472
 
369
473
  You can separate your source files into groups. For example, in a Rails app, you'll want to have separate listings for
370
474
  Models, Controllers, Helpers, and Libs. Group definition works similarly to Filters (and also accepts custom
371
- filter classes), but source files end up in a group when the filter passes (returns true), as opposed to filtering results,
372
- which exclude files from results when the filter results in a true value.
475
+ filter classes), but source files end up in a group when the filter passes (returns true), as opposed to filtering
476
+ results, which exclude files from results when the filter results in a true value.
373
477
 
374
478
  Add your groups with:
375
479
 
@@ -389,12 +493,11 @@ end
389
493
 
390
494
  You normally want to have your coverage analyzed across ALL of your test suites, right?
391
495
 
392
- Simplecov automatically caches coverage results in your (coverage_path)/.resultset.json. Those results will then
393
- be automatically merged when generating the result, so when coverage is set up properly for Cucumber and your
394
- unit / functional / integration tests, all of those test suites will be taken into account when building the
395
- coverage report.
396
-
397
- There are two things to note here though:
496
+ Simplecov automatically caches coverage results in your
497
+ (coverage_path)/.resultset.json, and will merge or override those with
498
+ subsequent runs, depending on whether simplecov considers those subsequent runs
499
+ as different test suites or as the same test suite as the cached results. To
500
+ make this distinction, simplecov has the concept of "test suite names".
398
501
 
399
502
  ### Test suite names
400
503
 
@@ -448,24 +551,143 @@ SimpleCov.command_name "features" + (ENV['TEST_ENV_NUMBER'] || '')
448
551
 
449
552
  [simplecov-html] prints the used test suites in the footer of the generated coverage report.
450
553
 
451
- ### Timeout for merge
452
554
 
453
- Of course, your cached coverage data is likely to become invalid at some point. Thus, result sets that are older than
454
- `SimpleCov.merge_timeout` will not be used any more. By default, the timeout is 600 seconds (10 minutes), and you can
455
- raise (or lower) it by specifying `SimpleCov.merge_timeout 3600` (1 hour), or, inside a configure/start block, with
456
- just `merge_timeout 3600`.
555
+ ### Merging test runs under the same execution environment
556
+
557
+ Test results are automatically merged with previous runs in the same execution
558
+ environment when generating the result, so when coverage is set up properly for
559
+ Cucumber and your unit / functional / integration tests, all of those test
560
+ suites will be taken into account when building the coverage report.
561
+
562
+ #### Timeout for merge
563
+
564
+ Of course, your cached coverage data is likely to become invalid at some point. Thus, when automatically merging
565
+ subsequent test runs, result sets that are older than `SimpleCov.merge_timeout` will not be used any more. By default,
566
+ the timeout is 600 seconds (10 minutes), and you can raise (or lower) it by specifying `SimpleCov.merge_timeout 3600`
567
+ (1 hour), or, inside a configure/start block, with just `merge_timeout 3600`.
568
+
569
+ You can deactivate this automatic merging altogether with `SimpleCov.use_merging false`.
570
+
571
+ ### Merging test runs under different execution environments
572
+
573
+ If your tests are done in parallel across multiple build machines, you can fetch them all and merge them into a single
574
+ result set using the `SimpleCov.collate` method. This can be added to a Rakefile or script file, having downloaded a set of
575
+ `.resultset.json` files from each parallel test run.
576
+
577
+ ```ruby
578
+ # lib/tasks/coverage_report.rake
579
+ namespace :coverage do
580
+ desc "Collates all result sets generated by the different test runners"
581
+ task :report do
582
+ require 'simplecov'
583
+
584
+ SimpleCov.collate Dir["simplecov-resultset-*/.resultset.json"]
585
+ end
586
+ end
587
+ ```
588
+
589
+ `SimpleCov.collate` also takes an optional simplecov profile and an optional
590
+ block for configuration, just the same as `SimpleCov.start` or
591
+ `SimpleCov.configure`. This means you can configure a separate formatter for
592
+ the collated output. For instance, you can make the formatter in
593
+ `SimpleCov.start` the `SimpleCov::Formatter::SimpleFormatter`, and only use more
594
+ complex formatters in the final `SimpleCov.collate` run.
595
+
596
+ ```ruby
597
+ # spec/spec_helper.rb
598
+ require 'simplecov'
599
+
600
+ SimpleCov.start 'rails' do
601
+ # Disambiguates individual test runs
602
+ command_name "Job #{ENV["TEST_ENV_NUMBER"]}" if ENV["TEST_ENV_NUMBER"]
603
+
604
+ if ENV['CI']
605
+ formatter SimpleCov::Formatter::SimpleFormatter
606
+ else
607
+ formatter SimpleCov::Formatter::MultiFormatter.new([
608
+ SimpleCov::Formatter::SimpleFormatter,
609
+ SimpleCov::Formatter::HTMLFormatter
610
+ ])
611
+ end
612
+
613
+ track_files "**/*.rb"
614
+ end
615
+ ```
616
+
617
+ ```ruby
618
+ # lib/tasks/coverage_report.rake
619
+ namespace :coverage do
620
+ task :report do
621
+ require 'simplecov'
622
+
623
+ SimpleCov.collate Dir["simplecov-resultset-*/.resultset.json"], 'rails' do
624
+ formatter SimpleCov::Formatter::MultiFormatter.new([
625
+ SimpleCov::Formatter::SimpleFormatter,
626
+ SimpleCov::Formatter::HTMLFormatter
627
+ ])
628
+ end
629
+ end
630
+ end
631
+ ```
632
+
633
+ ## Running simplecov against subprocesses
634
+
635
+ `SimpleCov.enable_for_subprocesses` will allow SimpleCov to observe subprocesses starting using `Process.fork`.
636
+ This modifies ruby's core Process.fork method so that SimpleCov can see into it, appending `" (subprocess #{pid})"`
637
+ to the `SimpleCov.command_name`, with results that can be merged together using SimpleCov's merging feature.
638
+
639
+ To configure this, use `.at_fork`.
640
+
641
+ ```ruby
642
+ SimpleCov.enable_for_subprocesses true
643
+ SimpleCov.at_fork do |pid|
644
+ # This needs a unique name so it won't be ovewritten
645
+ SimpleCov.command_name "#{SimpleCov.command_name} (subprocess: #{pid})"
646
+ # be quiet, the parent process will be in charge of output and checking coverage totals
647
+ SimpleCov.print_error_status = false
648
+ SimpleCov.formatter SimpleCov::Formatter::SimpleFormatter
649
+ SimpleCov.minimum_coverage 0
650
+ # start
651
+ SimpleCov.start
652
+ end
653
+ ```
654
+
655
+ NOTE: SimpleCov must have already been started before `Process.fork` was called.
457
656
 
458
- You can deactivate merging altogether with `SimpleCov.use_merging false`.
657
+ ### Running simplecov against spawned subprocesses
658
+
659
+ Perhaps you're testing a ruby script with `PTY.spawn` or `Open3.popen`, or `Process.spawn` or etc.
660
+ SimpleCov can cover this too.
661
+
662
+ Add a .simplecov_spawn.rb file to your project root
663
+ ```ruby
664
+ # .simplecov_spawn.rb
665
+ require 'simplecov' # this will also pick up whatever config is in .simplecov
666
+ # so ensure it just contains configuration, and doesn't call SimpleCov.start.
667
+ SimpleCov.command_name 'spawn' # As this is not for a test runner directly, script doesn't have a pre-defined base command_name
668
+ SimpleCov.at_fork.call(Process.pid) # Use the per-process setup described previously
669
+ SimpleCov.start # only now can we start.
670
+ ```
671
+ Then, instead of calling your script directly, like:
672
+ ```ruby
673
+ PTY.spawn('my_script.rb') do # ...
674
+ ```
675
+ Use bin/ruby to require the new .simplecov_spawn file, then your script
676
+ ```ruby
677
+ PTY.spawn('ruby -r./.simplecov_spawn my_script.rb') do # ...
678
+ ```
459
679
 
460
680
  ## Running coverage only on demand
461
681
 
462
- The Ruby STDLIB Coverage library that SimpleCov builds upon is *very* fast (on a ~10 min Rails test suite, the speed drop was
463
- only a couple seconds for me), and therefore it's SimpleCov's policy to just generate coverage every time you run your tests because
464
- it doesn't do your test speed any harm and you're always equipped with the latest and greatest coverage results.
682
+ The Ruby STDLIB Coverage library that SimpleCov builds upon is *very* fast (on a ~10 min Rails test suite, the speed
683
+ drop was only a couple seconds for me), and therefore it's SimpleCov's policy to just generate coverage every time you
684
+ run your tests because it doesn't do your test speed any harm and you're always equipped with the latest and greatest
685
+ coverage results.
465
686
 
466
687
  Because of this, SimpleCov has no explicit built-in mechanism to run coverage only on demand.
467
688
 
468
- However, you can still accomplish this very easily by introducing an ENV variable conditional into your SimpleCov setup block, like this:
689
+ However, you can still accomplish this very easily by introducing an ENV variable conditional into your SimpleCov setup
690
+ block, like this:
469
691
 
470
692
  ```ruby
471
693
  SimpleCov.start if ENV["COVERAGE"]
@@ -477,6 +699,21 @@ Then, SimpleCov will only run if you execute your tests like this:
477
699
  COVERAGE=true rake test
478
700
  ```
479
701
 
702
+ ## Errors and exit statuses
703
+
704
+ To aid in debugging issues, if an error is raised, SimpleCov will print a message to `STDERR`
705
+ with the exit status of the error, like:
706
+
707
+ ```
708
+ SimpleCov failed with exit 1
709
+ ```
710
+
711
+ This `STDERR` message can be disabled with:
712
+
713
+ ```
714
+ SimpleCov.print_error_status = false
715
+ ```
716
+
480
717
  ## Profiles
481
718
 
482
719
  By default, SimpleCov's only config assumption is that you only want coverage reports for files inside your project
@@ -514,8 +751,8 @@ end
514
751
 
515
752
  ### Custom profiles
516
753
 
517
- You can load additional profiles with the SimpleCov.load_profile('xyz') method. This allows you to build upon an existing
518
- profile and customize it so you can reuse it in unit tests and Cucumber features. For example:
754
+ You can load additional profiles with the SimpleCov.load_profile('xyz') method. This allows you to build upon an
755
+ existing profile and customize it so you can reuse it in unit tests and Cucumber features. For example:
519
756
 
520
757
  ```ruby
521
758
  # lib/simplecov_custom_profile.rb
@@ -552,14 +789,23 @@ You can define the minimum coverage percentage expected. SimpleCov will return n
552
789
 
553
790
  ```ruby
554
791
  SimpleCov.minimum_coverage 90
792
+ # same as above (the default is to check line coverage)
793
+ SimpleCov.minimum_coverage line: 90
794
+ # check for a minimum line coverage of 90% and minimum 80% branch coverage
795
+ SimpleCov.minimum_coverage line: 90, branch: 80
555
796
  ```
556
797
 
557
798
  ### Minimum coverage by file
558
799
 
559
- You can define the minimum coverage by file percentage expected. SimpleCov will return non-zero if unmet. This is useful to help ensure coverage is relatively consistent, rather than being skewed by particularly good or bad areas of the code.
800
+ You can define the minimum coverage by file percentage expected. SimpleCov will return non-zero if unmet. This is useful
801
+ to help ensure coverage is relatively consistent, rather than being skewed by particularly good or bad areas of the code.
560
802
 
561
803
  ```ruby
562
804
  SimpleCov.minimum_coverage_by_file 80
805
+ # same as above (the default is to check line coverage by file)
806
+ SimpleCov.minimum_coverage_by_file line: 80
807
+ # check for a minimum line coverage by file of 90% and minimum 80% branch coverage
808
+ SimpleCov.minimum_coverage_by_file line: 90, branch: 80
563
809
  ```
564
810
 
565
811
  ### Maximum coverage drop
@@ -568,6 +814,10 @@ You can define the maximum coverage drop percentage at once. SimpleCov will retu
568
814
 
569
815
  ```ruby
570
816
  SimpleCov.maximum_coverage_drop 5
817
+ # same as above (the default is to check line drop)
818
+ SimpleCov.maximum_coverage_drop line: 5
819
+ # check for a maximum line drop of 5% and maximum 10% branch drop
820
+ SimpleCov.maximum_coverage_drop line: 5, branch: 10
571
821
  ```
572
822
 
573
823
  ### Refuse dropping coverage
@@ -576,6 +826,10 @@ You can also entirely refuse dropping coverage between test runs:
576
826
 
577
827
  ```ruby
578
828
  SimpleCov.refuse_coverage_drop
829
+ # same as above (the default is to only refuse line drop)
830
+ SimpleCov.refuse_coverage_drop :line
831
+ # refuse drop for line and branch
832
+ SimpleCov.refuse_coverage_drop :line, :branch
579
833
  ```
580
834
 
581
835
  ## Using your own formatter
@@ -586,8 +840,8 @@ You can use your own formatter with:
586
840
  SimpleCov.formatter = SimpleCov::Formatter::HTMLFormatter
587
841
  ```
588
842
 
589
- When calling SimpleCov.result.format!, it will be invoked with SimpleCov::Formatter::YourFormatter.new.format(result), "result"
590
- being an instance of SimpleCov::Result. Do whatever your wish with that!
843
+ When calling SimpleCov.result.format!, it will be invoked with SimpleCov::Formatter::YourFormatter.new.format(result),
844
+ "result" being an instance of SimpleCov::Result. Do whatever your wish with that!
591
845
 
592
846
 
593
847
  ## Using multiple formatters
@@ -601,6 +855,20 @@ SimpleCov.formatters = SimpleCov::Formatter::MultiFormatter.new([
601
855
  ])
602
856
  ```
603
857
 
858
+ ## JSON formatter
859
+
860
+ SimpleCov is packaged with a separate gem called [simplecov_json_formatter](https://github.com/codeclimate-community/simplecov_json_formatter) that provides you with a JSON formatter, this formatter could be useful for different use cases, such as for CI consumption or for reporting to external services.
861
+
862
+ In order to use it you will need to manually load the installed gem like so:
863
+
864
+ ```ruby
865
+ require "simplecov_json_formatter"
866
+ SimpleCov.formatter = SimpleCov::Formatter::JSONFormatter
867
+ ```
868
+
869
+ > _Note:_ In case you plan to report your coverage results to CodeClimate services, know that SimpleCov will automatically use the
870
+ > JSON formatter along with the HTML formatter when the `CC_TEST_REPORTER_ID` variable is present in the environment.
871
+
604
872
  ## Available formatters, editor integrations and hosted services
605
873
 
606
874
  * [Open Source formatter and integration plugins for SimpleCov](doc/alternate-formatters.md)
@@ -609,11 +877,9 @@ SimpleCov.formatters = SimpleCov::Formatter::MultiFormatter.new([
609
877
 
610
878
  ## Ruby version compatibility
611
879
 
612
- Only Ruby 1.9+ ships with the coverage library that SimpleCov depends upon and that's what SimpleCov supports. Additionally JRuby 9.1+ is supported as well, while JRuby 1.7 and 9.0 should work they're not "officially" supported.
613
- SimpleCov is also built against Ruby 1.8 in [Continuous Integration], but this happens only to ensure that SimpleCov
614
- does not make your test suite crash right now.
880
+ SimpleCov is built in [Continuous Integration] on Ruby 2.5+ as well as JRuby 9.2+.
615
881
 
616
- SimpleCov is built in [Continuous Integration] on Ruby 1.9.3, 2.0.0, 2.1, 2.2, 2.3, 2.4, 2.5 as well as JRuby 9.1.
882
+ Note for JRuby => You need to pass JRUBY_OPTS="--debug" or create .jrubyrc and add debug.fullTrace=true
617
883
 
618
884
  ## Want to find dead code in production?
619
885
 
@@ -621,10 +887,12 @@ Try [Coverband](https://github.com/danmayer/coverband).
621
887
 
622
888
  ## Want to use Spring with SimpleCov?
623
889
 
624
- If you're using [Spring](https://github.com/rails/spring) to speed up test suite runs and want to run SimpleCov along with them, you'll find that it often misreports coverage with the default config due to some sort of eager loading issue. Don't despair!
890
+ If you're using [Spring](https://github.com/rails/spring) to speed up test suite runs and want to run SimpleCov along
891
+ with them, you'll find that it often misreports coverage with the default config due to some sort of eager loading
892
+ issue. Don't despair!
625
893
 
626
894
  One solution is to [explicitly call eager
627
- load](https://github.com/colszowka/simplecov/issues/381#issuecomment-347651728)
895
+ load](https://github.com/simplecov-ruby/simplecov/issues/381#issuecomment-347651728)
628
896
  in your `test_helper.rb` / `spec_helper.rb` after calling `SimpleCov.start`.
629
897
 
630
898
  ```ruby
@@ -633,13 +901,22 @@ SimpleCov.start 'rails'
633
901
  Rails.application.eager_load!
634
902
  ```
635
903
 
904
+ Alternatively, you could disable Spring while running SimpleCov:
905
+
906
+ ```
907
+ DISABLE_SPRING=1 rake test
908
+ ```
909
+
636
910
  Or you could remove `gem 'spring'` from your `Gemfile`.
637
911
 
638
912
  ## Troubleshooting
639
913
 
640
- The **most common problem is that simplecov isn't required and started before everything else**. In order to track coverage for your whole application **simplecov needs to be the first one** so that it (and the underlying coverage library) can subsequently track loaded files and their usage.
914
+ The **most common problem is that simplecov isn't required and started before everything else**. In order to track
915
+ coverage for your whole application **simplecov needs to be the first one** so that it (and the underlying coverage
916
+ library) can subsequently track loaded files and their usage.
641
917
 
642
- If you are missing coverage for some code a simple trick is to put a puts statement in there and right after `SimpleCov.start` so you can see if the file really was loaded after simplecov was started.
918
+ If you are missing coverage for some code a simple trick is to put a puts statement in there and right after
919
+ `SimpleCov.start` so you can see if the file really was loaded after simplecov was started.
643
920
 
644
921
  ```ruby
645
922
  # my_code.rb
@@ -667,9 +944,14 @@ MyCode is being loaded!
667
944
 
668
945
  then it's good otherwise you likely have a problem :)
669
946
 
947
+ ## Code of Conduct
948
+
949
+ Everyone participating in this project's development, issue trackers and other channels is expected to follow our
950
+ [Code of Conduct](./CODE_OF_CONDUCT.md)
951
+
670
952
  ## Contributing
671
953
 
672
- See the [contributing guide](https://github.com/colszowka/simplecov/blob/master/CONTRIBUTING.md).
954
+ See the [contributing guide](https://github.com/simplecov-ruby/simplecov/blob/main/CONTRIBUTING.md).
673
955
 
674
956
  ## Kudos
675
957