simplecov 0.17.1 → 0.18.1

Sign up to get free protection for your applications and to get access to all the features.
Files changed (36) hide show
  1. checksums.yaml +4 -4
  2. data/CHANGELOG.md +77 -1
  3. data/CODE_OF_CONDUCT.md +76 -0
  4. data/README.md +275 -76
  5. data/doc/alternate-formatters.md +5 -0
  6. data/lib/simplecov.rb +225 -61
  7. data/lib/simplecov/combine.rb +30 -0
  8. data/lib/simplecov/combine/branches_combiner.rb +32 -0
  9. data/lib/simplecov/combine/files_combiner.rb +24 -0
  10. data/lib/simplecov/combine/lines_combiner.rb +43 -0
  11. data/lib/simplecov/combine/results_combiner.rb +60 -0
  12. data/lib/simplecov/command_guesser.rb +6 -3
  13. data/lib/simplecov/configuration.rb +110 -9
  14. data/lib/simplecov/coverage_statistics.rb +56 -0
  15. data/lib/simplecov/defaults.rb +4 -2
  16. data/lib/simplecov/file_list.rb +66 -13
  17. data/lib/simplecov/filter.rb +2 -1
  18. data/lib/simplecov/formatter/multi_formatter.rb +2 -2
  19. data/lib/simplecov/formatter/simple_formatter.rb +4 -4
  20. data/lib/simplecov/last_run.rb +3 -1
  21. data/lib/simplecov/lines_classifier.rb +2 -2
  22. data/lib/simplecov/profiles.rb +9 -7
  23. data/lib/simplecov/result.rb +39 -6
  24. data/lib/simplecov/result_adapter.rb +30 -0
  25. data/lib/simplecov/result_merger.rb +18 -11
  26. data/lib/simplecov/simulate_coverage.rb +29 -0
  27. data/lib/simplecov/source_file.rb +226 -125
  28. data/lib/simplecov/source_file/branch.rb +84 -0
  29. data/lib/simplecov/source_file/line.rb +72 -0
  30. data/lib/simplecov/useless_results_remover.rb +16 -0
  31. data/lib/simplecov/version.rb +1 -1
  32. metadata +32 -166
  33. data/lib/simplecov/jruby_fix.rb +0 -44
  34. data/lib/simplecov/railtie.rb +0 -9
  35. data/lib/simplecov/railties/tasks.rake +0 -13
  36. data/lib/simplecov/raw_coverage.rb +0 -41
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 6f3e7730302a3a31266a22a830aa6a098e17cc4042f664e4f9a079eef408897f
4
- data.tar.gz: 423ed5a50493415c4c8bb17ff9c82ffd84e75dc98fc14cc0b4e82f17491d483e
3
+ metadata.gz: 6139c9ec905e1f20dec5bdfd8fef48fe8b9091ce24ae1b55e89b850093cfcf69
4
+ data.tar.gz: 3e15d4dbbc0f68b9453241603c0a1beade1b29ca2fbf64190f742f69db307b13
5
5
  SHA512:
6
- metadata.gz: 5b46e9d714b0a675de3b2e42284069942479af4fb0df627b3806fb48e8d373ea7f555fc8cf482682e15e8a6e335ea82358e4b4db28b1257183ae22d018c17f21
7
- data.tar.gz: 82eedda10bb5786656190ec0ca7dd1a3e18c38f6526db25f90c9b614849e05a14b85a34257343c5f48f62cffc7ddbdb74505b896a7c72236992aadd83c945a24
6
+ metadata.gz: 3af5aa2cad84980d28f95c606d0e7cc8282d5c81689bdaa637cb29a971ffe2842b7cd91ba2d602a6214f7c917851886ead8a62b50a8e071a12edbddbb7355a45
7
+ data.tar.gz: a3afd317b84ae5b11b907384c3ee17436e3a033c956c7ebdc000db0fca852442e4e658b72dc09c07f3c510f659297ab386f07af4fbee1c43b8a81794da595f6a
@@ -1,3 +1,79 @@
1
+ 0.18.1 (2020-01-31)
2
+ ===================
3
+
4
+ Small Bugfix release.
5
+
6
+ ## Bugfixes
7
+ * Just putting `# :nocov:` on top of a file or having an uneven number of them in general works again and acts as if ignoring until the end of the file. See [#846](https://github.com/colszowka/simplecov/issues/846) and thanks [@DannyBen](https://github.com/DannyBen) for the report.
8
+
9
+ 0.18.0 (2020-01-28)
10
+ ===================
11
+
12
+ Huge release! Highlights are support for branch coverage (Ruby 2.5+) and dropping support for EOL'ed Ruby versions (< 2.4).
13
+ Please also read the other beta patch notes.
14
+
15
+ You can run with branch coverage by putting `enable_coverage :branch` into your SimpleCov configuration (like the `SimpleCov.start do .. end` block)
16
+
17
+ ## Enhancements
18
+ * You can now define the minimum expected coverage by criterion like `minimum_coverage line: 90, branch: 80`
19
+ * Memoized some internal data structures that didn't change to reduce SimpleCov overhead
20
+ * Both `FileList` and `SourceFile` now have a `coverage` method that returns a hash that points from a coverage criterion to a `CoverageStatistics` object for uniform access to overall coverage statistics for both line and branch coverage
21
+
22
+ ## Bugfixes
23
+ * we were losing precision by rounding the covered strength early, that has been removed. **For Formatters** this also means that you may need to round it yourself now.
24
+ * Removed an inconsistency in how we treat skipped vs. irrelevant lines (see [#565](https://github.com/colszowka/simplecov/issues/565)) - SimpleCov's definition of 100% is now "You covered everything that you could" so if coverage is 0/0 that's counted as a 100% no matter if the lines were irrelevant or ignored/skipped
25
+
26
+ ## Noteworthy
27
+ * `FileList` stopped inheriting from Array, it includes Enumerable so if you didn't use Array specific methods on it in formatters you should be fine
28
+
29
+ 0.18.0.beta3 (2020-01-20)
30
+ ========================
31
+
32
+ ## Enhancements
33
+ * Instead of ignoring old `.resultset.json`s that are inside the merge timeout, adapt and respect them
34
+
35
+ ## Bugfixes
36
+ * Remove the constant warning printing if you still have a `.resultset.json` in pre 0.18 layout that is within your merge timeout
37
+
38
+ 0.18.0.beta2 (2020-01-19)
39
+ ===================
40
+
41
+ ## Enhancements
42
+ * only turn on the requested coverage criteria (when activating branch coverage before SimpleCov would also instruct Ruby to take Method coverage)
43
+ * Change how branch coverage is displayed, now it's `branch_type: hit_count` which should be more self explanatory. See [#830](https://github.com/colszowka/simplecov/pull/830) for an example and feel free to give feedback!
44
+ * Allow early running exit tasks and avoid the `at_exit` hook through the `SimpleCov.run_exit_tasks!` method. (thanks [@macumber](https://github.com/macumber))
45
+ * Allow manual collation of result sets through the `SimpleCov.collate` entrypoint. See the README for more details (thanks [@ticky](https://github.com/ticky))
46
+ * Within `case`, even if there is no `else` branch declared show missing coverage for it (aka no branch of it). See [#825](https://github.com/colszowka/simplecov/pull/825)
47
+ * Stop symbolizing all keys when loading cache (should lead to be faster and consume less memory)
48
+ * Cache whether we can use/are using branch coverage (should be slightly faster)
49
+
50
+ ## Bugfixes
51
+ * Fix a crash that happened when an old version of our internal cache file `.resultset.json` was still present
52
+
53
+ 0.18.0.beta1 (2020-01-05)
54
+ ===================
55
+
56
+ This is a huge release highlighted by changing our support for ruby versions to 2.4+ (so things that aren't EOL'ed) and finally adding branch coverage support!
57
+
58
+ This release is still beta because we'd love for you to test out branch coverage and get your feedback before doing a full release.
59
+
60
+ On a personal note from [@PragTob](https://github.com/PragTob/) thanks to [ruby together](https://rubytogether.org/) for sponsoring this work on SimpleCov making it possible to deliver this and subsequent releases.
61
+
62
+ ## Breaking
63
+ * Dropped support for all EOL'ed rubies meaning we only support 2.4+. Simplecov can no longer be installed on older rubies, but older simplecov releases should still work. (thanks [@deivid-rodriguez](https://github.com/deivid-rodriguez))
64
+ * Dropped the `rake simplecov` task that "magically" integreated with rails. It was always undocumented, caused some issues and [had some issues](https://github.com/colszowka/simplecov/issues/689#issuecomment-561572327). Use the integration as described in the README please :)
65
+
66
+ ## Enhancements
67
+
68
+ * Branch coverage is here! Please try it out and test it! You can activate it with `enable_coverage :branch`. See the README for more details. This is thanks to a bunch of people most notably [@som4ik](https://github.com/som4ik), [@tycooon](https://github.com/tycooon), [@stepozer](https://github.com/stepozer), [@klyonrad](https://github.com/klyonrad) and your humble maintainers also contributed ;)
69
+ * If the minimum coverage is set to be greater than 100, a warning will be shown. See [#737](https://github.com/colszowka/simplecov/pull/737) (thanks [@belfazt](https://github.com/belfazt))
70
+ * Add a configuration option to disable the printing of non-successful exit statuses. See [#747](https://github.com/colszowka/simplecov/pull/746) (thanks [@JacobEvelyn](https://github.com/JacobEvelyn))
71
+ * Calculating 100% coverage is now stricter, so 100% means 100%. See [#680](https://github.com/colszowka/simplecov/pull/680) thanks [@gleseur](https://github.com/gleseur)
72
+
73
+ ## Bugfixes
74
+
75
+ * Add new instance of `Minitest` constant. The `MiniTest` constant (with the capital T) will be removed in the next major release of Minitest. See [#757](https://github.com/colszowka/simplecov/pull/757) (thanks [@adam12](https://github.com/adam12))
76
+
1
77
  0.17.1 (2019-09-16)
2
78
  ===================
3
79
 
@@ -11,7 +87,7 @@ Bugfix release for problems with ParallelTests.
11
87
  ===================
12
88
 
13
89
  Maintenance release with nice convenience features and important bugfixes.
14
- Notably this **will be the last release to support ruby versions that have reached their end of life**. Moving forward official CRuby support will be 2.4+ and JRuby support will be 9.1+. Older versions might still work but no guarantees.
90
+ Notably this **will be the last release to support ruby versions that have reached their end of life**. Moving forward official CRuby support will be 2.4+ and JRuby support will be 9.2+. Older versions might still work but no guarantees.
15
91
 
16
92
  ## Enhancements
17
93
 
@@ -0,0 +1,76 @@
1
+ # SimpleCov Code of Conduct
2
+
3
+ ## Our Pledge
4
+
5
+ In the interest of fostering an open and welcoming environment, we as
6
+ contributors and maintainers pledge to making participation in our project and
7
+ our community a harassment-free experience for everyone, regardless of age, body
8
+ size, disability, ethnicity, sex characteristics, gender identity and expression,
9
+ level of experience, education, socio-economic status, nationality, personal
10
+ appearance, race, religion, or sexual identity and orientation.
11
+
12
+ ## Our Standards
13
+
14
+ Examples of behavior that contributes to creating a positive environment
15
+ include:
16
+
17
+ * Using welcoming and inclusive language
18
+ * Being respectful of differing viewpoints and experiences
19
+ * Gracefully accepting constructive criticism
20
+ * Focusing on what is best for the community
21
+ * Showing empathy towards other community members
22
+
23
+ Examples of unacceptable behavior by participants include:
24
+
25
+ * The use of sexualized language or imagery and unwelcome sexual attention or
26
+ advances
27
+ * Trolling, insulting/derogatory comments, and personal or political attacks
28
+ * Public or private harassment
29
+ * Publishing others' private information, such as a physical or electronic
30
+ address, without explicit permission
31
+ * Other conduct which could reasonably be considered inappropriate in a
32
+ professional setting
33
+
34
+ ## Our Responsibilities
35
+
36
+ Project maintainers are responsible for clarifying the standards of acceptable
37
+ behavior and are expected to take appropriate and fair corrective action in
38
+ response to any instances of unacceptable behavior.
39
+
40
+ Project maintainers have the right and responsibility to remove, edit, or
41
+ reject comments, commits, code, wiki edits, issues, and other contributions
42
+ that are not aligned to this Code of Conduct, or to ban temporarily or
43
+ permanently any contributor for other behaviors that they deem inappropriate,
44
+ threatening, offensive, or harmful.
45
+
46
+ ## Scope
47
+
48
+ This Code of Conduct applies both within project spaces and in public spaces
49
+ when an individual is representing the project or its community. Examples of
50
+ representing a project or community include using an official project e-mail
51
+ address, posting via an official social media account, or acting as an appointed
52
+ representative at an online or offline event. Representation of a project may be
53
+ further defined and clarified by project maintainers.
54
+
55
+ ## Enforcement
56
+
57
+ Instances of abusive, harassing, or otherwise unacceptable behavior may be
58
+ reported by contacting the project team at simplecov.team@gmail.com. All
59
+ complaints will be reviewed and investigated and will result in a response that
60
+ is deemed necessary and appropriate to the circumstances. The project team is
61
+ obligated to maintain confidentiality with regard to the reporter of an incident.
62
+ Further details of specific enforcement policies may be posted separately.
63
+
64
+ Project maintainers who do not follow or enforce the Code of Conduct in good
65
+ faith may face temporary or permanent repercussions as determined by other
66
+ members of the project's leadership.
67
+
68
+ ## Attribution
69
+
70
+ This Code of Conduct is adapted from the [Contributor Covenant][homepage], version 1.4,
71
+ available at https://www.contributor-covenant.org/version/1/4/code-of-conduct.html
72
+
73
+ [homepage]: https://www.contributor-covenant.org
74
+
75
+ For answers to common questions about this code of conduct, see
76
+ https://www.contributor-covenant.org/faq
data/README.md CHANGED
@@ -1,5 +1,6 @@
1
- SimpleCov [![Build Status](https://travis-ci.org/colszowka/simplecov.svg)][Continuous Integration] [![Code Climate](https://codeclimate.com/github/colszowka/simplecov.svg)](https://codeclimate.com/github/colszowka/simplecov) [![Inline docs](http://inch-ci.org/github/colszowka/simplecov.svg)](http://inch-ci.org/github/colszowka/simplecov)
1
+ SimpleCov [![Gem Version](https://badge.fury.io/rb/simplecov.svg)](https://badge.fury.io/rb/simplecov) [![Build Status](https://github.com/colszowka/simplecov/workflows/stable/badge.svg?branch=master)][Continuous Integration] [![Code Climate](https://codeclimate.com/github/colszowka/simplecov.svg)](https://codeclimate.com/github/colszowka/simplecov) [![Inline docs](http://inch-ci.org/github/colszowka/simplecov.svg)](http://inch-ci.org/github/colszowka/simplecov)
2
2
  =========
3
+
3
4
  **Code coverage for Ruby**
4
5
 
5
6
  * [Source Code]
@@ -8,13 +9,13 @@ SimpleCov [![Build Status](https://travis-ci.org/colszowka/simplecov.svg)][Conti
8
9
  * [Rubygem]
9
10
  * [Continuous Integration]
10
11
 
11
- [Coverage]: http://www.ruby-doc.org/stdlib-2.1.0/libdoc/coverage/rdoc/Coverage.html "API doc for Ruby's Coverage library"
12
+ [Coverage]: https://ruby-doc.org/stdlib/libdoc/coverage/rdoc/Coverage.html "API doc for Ruby's Coverage library"
12
13
  [Source Code]: https://github.com/colszowka/simplecov "Source Code @ GitHub"
13
14
  [API documentation]: http://rubydoc.info/gems/simplecov/frames "RDoc API Documentation at Rubydoc.info"
14
15
  [Configuration]: http://rubydoc.info/gems/simplecov/SimpleCov/Configuration "Configuration options API documentation"
15
16
  [Changelog]: https://github.com/colszowka/simplecov/blob/master/CHANGELOG.md "Project Changelog"
16
17
  [Rubygem]: http://rubygems.org/gems/simplecov "SimpleCov @ rubygems.org"
17
- [Continuous Integration]: http://travis-ci.org/colszowka/simplecov "SimpleCov is built around the clock by travis-ci.org"
18
+ [Continuous Integration]: https://github.com/colszowka/simplecov/actions?query=workflow%3Astable "SimpleCov is built around the clock by github.com"
18
19
  [Dependencies]: https://gemnasium.com/colszowka/simplecov "SimpleCov dependencies on Gemnasium"
19
20
  [simplecov-html]: https://github.com/colszowka/simplecov-html "SimpleCov HTML Formatter Source Code @ GitHub"
20
21
 
@@ -22,13 +23,14 @@ SimpleCov is a code coverage analysis tool for Ruby. It uses [Ruby's built-in Co
22
23
  coverage data, but makes processing its results much easier by providing a clean API to filter, group, merge, format,
23
24
  and display those results, giving you a complete code coverage suite that can be set up with just a couple lines of
24
25
  code.
26
+ SimpleCov/Coverage track covered ruby code, gathering coverage for common templating solutions like erb, slim and haml is not supported.
25
27
 
26
28
  In most cases, you'll want overall coverage results for your projects, including all types of tests, Cucumber features,
27
29
  etc. SimpleCov automatically takes care of this by caching and merging results when generating reports, so your
28
30
  report actually includes coverage across your test suites and thereby gives you a better picture of blank spots.
29
31
 
30
- The official formatter of SimpleCov is packaged as a separate gem called [simplecov-html], but will be installed and configured
31
- automatically when you launch SimpleCov. If you're curious, you can find it [on GitHub, too][simplecov-html].
32
+ The official formatter of SimpleCov is packaged as a separate gem called [simplecov-html], but will be installed and
33
+ configured automatically when you launch SimpleCov. If you're curious, you can find it [on GitHub, too][simplecov-html].
32
34
 
33
35
 
34
36
  ## Contact
@@ -36,11 +38,13 @@ automatically when you launch SimpleCov. If you're curious, you can find it [on
36
38
  *Code and Bug Reports*
37
39
 
38
40
  * [Issue Tracker](https://github.com/colszowka/simplecov/issues)
39
- * See [CONTRIBUTING](https://github.com/colszowka/simplecov/blob/master/CONTRIBUTING.md) for how to contribute along with some common problems to check out before creating an issue.
41
+ * See [CONTRIBUTING](https://github.com/colszowka/simplecov/blob/master/CONTRIBUTING.md) for how to contribute along
42
+ with some common problems to check out before creating an issue.
40
43
 
41
44
  *Questions, Problems, Suggestions, etc.*
42
45
 
43
- * [Mailing List](https://groups.google.com/forum/#!forum/simplecov) "Open mailing list for discussion and announcements on Google Groups"
46
+ * [Mailing List](https://groups.google.com/forum/#!forum/simplecov) "Open mailing list for discussion and announcements
47
+ on Google Groups"
44
48
 
45
49
  Getting started
46
50
  ---------------
@@ -70,8 +74,8 @@ Getting started
70
74
  endpoint) via a separate test process (e.g. when using Selenium) where you
71
75
  want to see all code executed by the `rails server`, and not just code
72
76
  executed in your actual test files, you'll want to add something like this
73
- to the top of `script/rails` (or `bin/rails` for Rails 4), but below the
74
- "shebang" line (`#! /usr/bin/env ruby`):
77
+ to the top of `bin/rails`, but below the "shebang" line (`#! /usr/bin/env
78
+ ruby`):
75
79
 
76
80
  ```ruby
77
81
  if ENV['RAILS_ENV'] == 'test'
@@ -81,27 +85,41 @@ Getting started
81
85
  end
82
86
  ```
83
87
 
84
- 3. Run your tests, open up `coverage/index.html` in your browser and check out
85
- what you've missed so far.
86
- 4. Add the following to your `.gitignore` file to ensure that coverage results
88
+ 3. Run your full test suite to see the percent coverage that your application has.
89
+ 4. After running your tests, open `coverage/index.html` in the browser of your choice. For example, in a Mac Terminal,
90
+ run the following command from your application's root directory:
91
+
92
+ ```
93
+ open coverage/index.html
94
+ ```
95
+ in a debian/ubuntu Terminal,
96
+
97
+ ```
98
+ xdg-open coverage/index.html
99
+ ```
100
+
101
+ **Note:** [This guide](https://dwheeler.com/essays/open-files-urls.html) can help if you're unsure which command your particular
102
+ operating system requires.
103
+
104
+ 5. Add the following to your `.gitignore` file to ensure that coverage results
87
105
  are not tracked by Git (optional):
88
106
 
89
- ```
90
- echo "coverage" >> .gitignore
91
- ```
92
- Or if you use Windows:
93
- ```
94
- echo coverage >> .gitignore
95
- ```
107
+ ```
108
+ echo "coverage" >> .gitignore
109
+ ```
110
+ Or if you use Windows:
111
+ ```
112
+ echo coverage >> .gitignore
113
+ ```
96
114
 
97
- If you're making a Rails application, SimpleCov comes with built-in configurations (see below for information on profiles)
98
- that will get you started with groups for your Controllers, Views, Models and Helpers. To use it, the first two lines of
99
- your test_helper should be like this:
115
+ If you're making a Rails application, SimpleCov comes with built-in configurations (see below for information on
116
+ profiles) that will get you started with groups for your Controllers, Models and Helpers. To use it, the
117
+ first two lines of your test_helper should be like this:
100
118
 
101
- ```ruby
102
- require 'simplecov'
103
- SimpleCov.start 'rails'
104
- ```
119
+ ```ruby
120
+ require 'simplecov'
121
+ SimpleCov.start 'rails'
122
+ ```
105
123
 
106
124
  ## Example output
107
125
 
@@ -127,8 +145,9 @@ require 'simplecov'
127
145
  SimpleCov.start 'rails'
128
146
  ```
129
147
 
130
- You could even track what kind of code your UI testers are touching if you want to go overboard with things. SimpleCov does not
131
- care what kind of framework it is running in; it just looks at what code is being executed and generates a report about it.
148
+ You could even track what kind of code your UI testers are touching if you want to go overboard with things. SimpleCov
149
+ does not care what kind of framework it is running in; it just looks at what code is being executed and generates a
150
+ report about it.
132
151
 
133
152
  ### Notes on specific frameworks and test utilities
134
153
 
@@ -152,6 +171,17 @@ to use SimpleCov with them. Here's an overview of the known ones:
152
171
  <a href="https://github.com/colszowka/simplecov/pull/185">#185</a>
153
172
  </td>
154
173
  </tr>
174
+ <tr>
175
+ <th>
176
+ knapsack_pro
177
+ </th>
178
+ <td>
179
+ To make SimpleCov work with Knapsack Pro Queue Mode to split tests in parallel on CI jobs you need to provide CI node index number to the <code>SimpleCov.command_name</code> in <code>KnapsackPro::Hooks::Queue.before_queue</code> hook.
180
+ </td>
181
+ <td>
182
+ <a href="https://knapsackpro.com/faq/question/how-to-use-simplecov-in-queue-mode">Tip</a>
183
+ </td>
184
+ </tr>
155
185
  <tr>
156
186
  <th>
157
187
  RubyMine
@@ -223,7 +253,8 @@ to use SimpleCov with them. Here's an overview of the known ones:
223
253
  ```ruby
224
254
  SimpleCov.some_config_option 'foo'
225
255
  ```
226
- * If you do not want to start coverage immediately after launch or want to add additional configuration later on in a concise way, use:
256
+ * If you do not want to start coverage immediately after launch or want to add additional configuration later on in a
257
+ concise way, use:
227
258
 
228
259
  ```ruby
229
260
  SimpleCov.configure do
@@ -235,11 +266,12 @@ Please check out the [Configuration] API documentation to find out what you can
235
266
 
236
267
  ## Using .simplecov for centralized config
237
268
 
238
- If you use SimpleCov to merge multiple test suite results (e.g. Test/Unit and Cucumber) into a single report, you'd normally have to
239
- set up all your config options twice, once in `test_helper.rb` and once in `env.rb`.
269
+ If you use SimpleCov to merge multiple test suite results (e.g. Test/Unit and Cucumber) into a single report, you'd
270
+ normally have to set up all your config options twice, once in `test_helper.rb` and once in `env.rb`.
240
271
 
241
- To avoid this, you can place a file called `.simplecov` in your project root. You can then just leave the `require 'simplecov'` in each
242
- test setup helper (**at the top**) and move the `SimpleCov.start` code with all your custom config options into `.simplecov`:
272
+ To avoid this, you can place a file called `.simplecov` in your project root. You can then just leave the
273
+ `require 'simplecov'` in each test setup helper (**at the top**) and move the `SimpleCov.start` code with all your
274
+ custom config options into `.simplecov`:
243
275
 
244
276
  ```ruby
245
277
  # test/test_helper.rb
@@ -254,21 +286,73 @@ SimpleCov.start 'rails' do
254
286
  end
255
287
  ```
256
288
 
257
- Using `.simplecov` rather than separately requiring SimpleCov multiple times is recommended if you are merging multiple test frameworks like Cucumber and RSpec that rely on each other, as invoking SimpleCov multiple times can cause coverage information to be lost.
289
+ Using `.simplecov` rather than separately requiring SimpleCov multiple times is recommended if you are merging multiple
290
+ test frameworks like Cucumber and RSpec that rely on each other, as invoking SimpleCov multiple times can cause coverage
291
+ information to be lost.
292
+
293
+ ## Branch coverage (ruby "~> 2.5")
294
+ Add branch coverage measurement statistics to your results. Supported in CRuby versions 2.5+.
295
+
296
+ ```ruby
297
+ # or in configure or just SimpleCov.enable_coverage :branch
298
+ SimpleCov.start do
299
+ enable_coverage :branch
300
+ end
301
+ ```
302
+
303
+ Branch coverage is a feature introduced in Ruby 2.5 concerning itself with whether a
304
+ particular branch of a condition had been executed. Line coverage on the other hand
305
+ is only interested in whether a line of code has been executed.
306
+
307
+ This comes in handy for instance for one line conditionals:
308
+
309
+ ```ruby
310
+ number.odd? ? "odd" : "even"
311
+ ```
312
+
313
+ In line coverage this line would always be marked as executed but you'd never know if both
314
+ conditions were met. Guard clauses have a similar story:
315
+
316
+ ```ruby
317
+ return if number.odd?
318
+
319
+ # more code
320
+ ```
321
+
322
+ If all the code in that method was covered you'd never know if the guard clause was ever
323
+ triggered! With line coverage as just evaluating the condition marks it as covered.
324
+
325
+ In the HTML report the lines of code will be annotated like `branch_type: hit_count`:
326
+
327
+ * `then: 2` - the then branch (of an `if`) was executed twice
328
+ * `else: 0` - the else branch (of an `if` or `case`) was never executed
329
+
330
+ Not that even if you don't declare an `else` branch it will still show up in the coverage
331
+ reports meaning that the condition of the `if` was not hit or that no `when` of `case`
332
+ was hit during the test runs.
333
+
334
+ **Is branch coverage strictly better?** No. Branch coverage really only concerns itself with
335
+ conditionals - meaning coverage of sequential code is of no interest to it. A file without
336
+ conditional logic will have no branch coverage data and SimpleCov will report 0 of 0
337
+ branches covered as 100% (as everything that can be covered was covered).
338
+
339
+ Hence, we recommend looking at both metrics together. Branch coverage might also be a good
340
+ overall metric to look at - while you might be missing only 10% of your lines that might
341
+ account for 50% of your branches for instance.
258
342
 
259
343
  ## Filters
260
344
 
261
- Filters can be used to remove selected files from your coverage data. By default, a filter is applied that removes all files
262
- OUTSIDE of your project's root directory - otherwise you'd end up with billions of coverage reports for source files in the
263
- gems you are using.
345
+ Filters can be used to remove selected files from your coverage data. By default, a filter is applied that removes all
346
+ files OUTSIDE of your project's root directory - otherwise you'd end up with billions of coverage reports for source
347
+ files in the gems you are using.
264
348
 
265
349
  You can define your own to remove things like configuration files, tests or whatever you don't need in your coverage
266
350
  report.
267
351
 
268
352
  ### Defining custom filters
269
353
 
270
- You can currently define a filter using either a String or Regexp (that will then be Regexp-matched against each source file's path),
271
- a block or by passing in your own Filter class.
354
+ You can currently define a filter using either a String or Regexp (that will then be Regexp-matched against each source
355
+ file's path), a block or by passing in your own Filter class.
272
356
 
273
357
  #### String filter
274
358
 
@@ -300,9 +384,10 @@ SimpleCov.start do
300
384
  end
301
385
  ```
302
386
 
303
- Block filters receive a SimpleCov::SourceFile instance and expect your block to return either true (if the file is to be removed
304
- from the result) or false (if the result should be kept). Please check out the RDoc for SimpleCov::SourceFile to learn about the
305
- methods available to you. In the above example, the filter will remove all files that have less than 5 lines of code.
387
+ Block filters receive a SimpleCov::SourceFile instance and expect your block to return either true (if the file is to be
388
+ removed from the result) or false (if the result should be kept). Please check out the RDoc for SimpleCov::SourceFile to
389
+ learn about the methods available to you. In the above example, the filter will remove all files that have less than 5
390
+ lines of code.
306
391
 
307
392
  #### Custom filter class
308
393
 
@@ -316,9 +401,10 @@ end
316
401
  SimpleCov.add_filter LineFilter.new(5)
317
402
  ```
318
403
 
319
- Defining your own filters is pretty easy: Just inherit from SimpleCov::Filter and define a method 'matches?(source_file)'. When running
320
- the filter, a true return value from this method will result in the removal of the given source_file. The filter_argument method
321
- is being set in the SimpleCov::Filter initialize method and thus is set to 5 in this example.
404
+ Defining your own filters is pretty easy: Just inherit from SimpleCov::Filter and define a method
405
+ 'matches?(source_file)'. When running the filter, a true return value from this method will result in the removal of the
406
+ given source_file. The filter_argument method is being set in the SimpleCov::Filter initialize method and thus is set to
407
+ 5 in this example.
322
408
 
323
409
  #### Array filter
324
410
 
@@ -345,7 +431,9 @@ end
345
431
 
346
432
  The name of the token can be changed to your liking. [Learn more about the nocov feature.]( https://github.com/colszowka/simplecov/blob/master/features/config_nocov_token.feature)
347
433
 
348
- **Note:** You shouldn't have to use the nocov token to skip private methods that are being included in your coverage. If you appropriately test the public interface of your classes and objects you should automatically get full coverage of your private methods.
434
+ **Note:** You shouldn't have to use the nocov token to skip private methods that are being included in your coverage. If
435
+ you appropriately test the public interface of your classes and objects you should automatically get full coverage of
436
+ your private methods.
349
437
 
350
438
  ## Default root filter and coverage for things outside of it
351
439
 
@@ -368,8 +456,8 @@ end
368
456
 
369
457
  You can separate your source files into groups. For example, in a Rails app, you'll want to have separate listings for
370
458
  Models, Controllers, Helpers, and Libs. Group definition works similarly to Filters (and also accepts custom
371
- filter classes), but source files end up in a group when the filter passes (returns true), as opposed to filtering results,
372
- which exclude files from results when the filter results in a true value.
459
+ filter classes), but source files end up in a group when the filter passes (returns true), as opposed to filtering
460
+ results, which exclude files from results when the filter results in a true value.
373
461
 
374
462
  Add your groups with:
375
463
 
@@ -389,12 +477,11 @@ end
389
477
 
390
478
  You normally want to have your coverage analyzed across ALL of your test suites, right?
391
479
 
392
- Simplecov automatically caches coverage results in your (coverage_path)/.resultset.json. Those results will then
393
- be automatically merged when generating the result, so when coverage is set up properly for Cucumber and your
394
- unit / functional / integration tests, all of those test suites will be taken into account when building the
395
- coverage report.
396
-
397
- There are two things to note here though:
480
+ Simplecov automatically caches coverage results in your
481
+ (coverage_path)/.resultset.json, and will merge or override those with
482
+ subsequent runs, depending on whether simplecov considers those subsequent runs
483
+ as different test suites or as the same test suite as the cached results. To
484
+ make this distinction, simplecov has the concept of "test suite names".
398
485
 
399
486
  ### Test suite names
400
487
 
@@ -448,24 +535,96 @@ SimpleCov.command_name "features" + (ENV['TEST_ENV_NUMBER'] || '')
448
535
 
449
536
  [simplecov-html] prints the used test suites in the footer of the generated coverage report.
450
537
 
451
- ### Timeout for merge
452
538
 
453
- Of course, your cached coverage data is likely to become invalid at some point. Thus, result sets that are older than
454
- `SimpleCov.merge_timeout` will not be used any more. By default, the timeout is 600 seconds (10 minutes), and you can
455
- raise (or lower) it by specifying `SimpleCov.merge_timeout 3600` (1 hour), or, inside a configure/start block, with
456
- just `merge_timeout 3600`.
539
+ ### Merging test runs under the same execution environment
540
+
541
+ Test results are automatically merged with previous runs in the same execution
542
+ environment when generating the result, so when coverage is set up properly for
543
+ Cucumber and your unit / functional / integration tests, all of those test
544
+ suites will be taken into account when building the coverage report.
545
+
546
+ #### Timeout for merge
547
+
548
+ Of course, your cached coverage data is likely to become invalid at some point. Thus, when automatically merging
549
+ subsequent test runs, result sets that are older than `SimpleCov.merge_timeout` will not be used any more. By default,
550
+ the timeout is 600 seconds (10 minutes), and you can raise (or lower) it by specifying `SimpleCov.merge_timeout 3600`
551
+ (1 hour), or, inside a configure/start block, with just `merge_timeout 3600`.
457
552
 
458
- You can deactivate merging altogether with `SimpleCov.use_merging false`.
553
+ You can deactivate this automatic merging altogether with `SimpleCov.use_merging false`.
554
+
555
+ ### Merging test runs under different execution environments
556
+
557
+ If your tests are done in parallel across multiple build machines, you can fetch them all and merge them into a single
558
+ result set using the `SimpleCov.collate` method. This can be added to a Rakefile or script file, having downloaded a set of
559
+ `.resultset.json` files from each parallel test run.
560
+
561
+ ```ruby
562
+ # lib/tasks/coverage_report.rake
563
+ namespace :coverage do
564
+ desc "Collates all result sets generated by the different test runners"
565
+ task :report do
566
+ require 'simplecov'
567
+
568
+ SimpleCov.collate Dir["simplecov-resultset-*/.resultset.json"]
569
+ end
570
+ end
571
+ ```
572
+
573
+ `SimpleCov.collate` also takes an optional simplecov profile and an optional
574
+ block for configuration, just the same as `SimpleCov.start` or
575
+ `SimpleCov.configure`. This means you can configure a separate formatter for
576
+ the collated output. For instance, you can make the formatter in
577
+ `SimpleCov.start` the `SimpleCov::Formatter::SimpleFormatter`, and only use more
578
+ complex formatters in the final `SimpleCov.collate` run.
579
+
580
+ ```ruby
581
+ # spec/spec_helper.rb
582
+ require 'simplecov'
583
+
584
+ SimpleCov.start 'rails' do
585
+ # Disambiguates individual test runs
586
+ command_name "Job #{ENV["TEST_ENV_NUMBER"]}" if ENV["TEST_ENV_NUMBER"]
587
+
588
+ if ENV['CI']
589
+ formatter SimpleCov::Formatter::SimpleFormatter
590
+ else
591
+ formatter SimpleCov::Formatter::MultiFormatter.new([
592
+ SimpleCov::Formatter::SimpleFormatter,
593
+ SimpleCov::Formatter::HTMLFormatter
594
+ ])
595
+ end
596
+
597
+ track_files "**/*.rb"
598
+ end
599
+ ```
600
+
601
+ ```ruby
602
+ # lib/tasks/coverage_report.rake
603
+ namespace :coverage do
604
+ task :report do
605
+ require 'simplecov'
606
+
607
+ SimpleCov.collate Dir["simplecov-resultset-*/.resultset.json"], 'rails' do
608
+ formatter SimpleCov::Formatter::MultiFormatter.new([
609
+ SimpleCov::Formatter::SimpleFormatter,
610
+ SimpleCov::Formatter::HTMLFormatter
611
+ ])
612
+ end
613
+ end
614
+ end
615
+ ```
459
616
 
460
617
  ## Running coverage only on demand
461
618
 
462
- The Ruby STDLIB Coverage library that SimpleCov builds upon is *very* fast (on a ~10 min Rails test suite, the speed drop was
463
- only a couple seconds for me), and therefore it's SimpleCov's policy to just generate coverage every time you run your tests because
464
- it doesn't do your test speed any harm and you're always equipped with the latest and greatest coverage results.
619
+ The Ruby STDLIB Coverage library that SimpleCov builds upon is *very* fast (on a ~10 min Rails test suite, the speed
620
+ drop was only a couple seconds for me), and therefore it's SimpleCov's policy to just generate coverage every time you
621
+ run your tests because it doesn't do your test speed any harm and you're always equipped with the latest and greatest
622
+ coverage results.
465
623
 
466
624
  Because of this, SimpleCov has no explicit built-in mechanism to run coverage only on demand.
467
625
 
468
- However, you can still accomplish this very easily by introducing an ENV variable conditional into your SimpleCov setup block, like this:
626
+ However, you can still accomplish this very easily by introducing an ENV variable conditional into your SimpleCov setup
627
+ block, like this:
469
628
 
470
629
  ```ruby
471
630
  SimpleCov.start if ENV["COVERAGE"]
@@ -477,6 +636,21 @@ Then, SimpleCov will only run if you execute your tests like this:
477
636
  COVERAGE=true rake test
478
637
  ```
479
638
 
639
+ ## Errors and exit statuses
640
+
641
+ To aid in debugging issues, if an error is raised, SimpleCov will print a message to `STDERR`
642
+ with the exit status of the error, like:
643
+
644
+ ```
645
+ SimpleCov failed with exit 1
646
+ ```
647
+
648
+ This `STDERR` message can be disabled with:
649
+
650
+ ```
651
+ SimpleCov.print_error_status = false
652
+ ```
653
+
480
654
  ## Profiles
481
655
 
482
656
  By default, SimpleCov's only config assumption is that you only want coverage reports for files inside your project
@@ -514,8 +688,8 @@ end
514
688
 
515
689
  ### Custom profiles
516
690
 
517
- You can load additional profiles with the SimpleCov.load_profile('xyz') method. This allows you to build upon an existing
518
- profile and customize it so you can reuse it in unit tests and Cucumber features. For example:
691
+ You can load additional profiles with the SimpleCov.load_profile('xyz') method. This allows you to build upon an
692
+ existing profile and customize it so you can reuse it in unit tests and Cucumber features. For example:
519
693
 
520
694
  ```ruby
521
695
  # lib/simplecov_custom_profile.rb
@@ -552,16 +726,23 @@ You can define the minimum coverage percentage expected. SimpleCov will return n
552
726
 
553
727
  ```ruby
554
728
  SimpleCov.minimum_coverage 90
729
+ # same as above (the default is to check line coverage)
730
+ SimpleCov.minimum_coverage line: 90
731
+ # check for a minimum line coverage of 90% and minimum 80% branch coverage
732
+ SimpleCov.minimum_coverage line: 90, branch: 80
555
733
  ```
556
734
 
557
735
  ### Minimum coverage by file
558
736
 
559
- You can define the minimum coverage by file percentage expected. SimpleCov will return non-zero if unmet. This is useful to help ensure coverage is relatively consistent, rather than being skewed by particularly good or bad areas of the code.
737
+ You can define the minimum coverage by file percentage expected. SimpleCov will return non-zero if unmet. This is useful
738
+ to help ensure coverage is relatively consistent, rather than being skewed by particularly good or bad areas of the code.
560
739
 
561
740
  ```ruby
562
741
  SimpleCov.minimum_coverage_by_file 80
563
742
  ```
564
743
 
744
+ (not yet supported for branch coverage)
745
+
565
746
  ### Maximum coverage drop
566
747
 
567
748
  You can define the maximum coverage drop percentage at once. SimpleCov will return non-zero if exceeded.
@@ -570,6 +751,8 @@ You can define the maximum coverage drop percentage at once. SimpleCov will retu
570
751
  SimpleCov.maximum_coverage_drop 5
571
752
  ```
572
753
 
754
+ (not yet supported for branch coverage)
755
+
573
756
  ### Refuse dropping coverage
574
757
 
575
758
  You can also entirely refuse dropping coverage between test runs:
@@ -578,6 +761,8 @@ You can also entirely refuse dropping coverage between test runs:
578
761
  SimpleCov.refuse_coverage_drop
579
762
  ```
580
763
 
764
+ (not yet supported for branch coverage)
765
+
581
766
  ## Using your own formatter
582
767
 
583
768
  You can use your own formatter with:
@@ -586,8 +771,8 @@ You can use your own formatter with:
586
771
  SimpleCov.formatter = SimpleCov::Formatter::HTMLFormatter
587
772
  ```
588
773
 
589
- When calling SimpleCov.result.format!, it will be invoked with SimpleCov::Formatter::YourFormatter.new.format(result), "result"
590
- being an instance of SimpleCov::Result. Do whatever your wish with that!
774
+ When calling SimpleCov.result.format!, it will be invoked with SimpleCov::Formatter::YourFormatter.new.format(result),
775
+ "result" being an instance of SimpleCov::Result. Do whatever your wish with that!
591
776
 
592
777
 
593
778
  ## Using multiple formatters
@@ -609,11 +794,9 @@ SimpleCov.formatters = SimpleCov::Formatter::MultiFormatter.new([
609
794
 
610
795
  ## Ruby version compatibility
611
796
 
612
- Only Ruby 1.9+ ships with the coverage library that SimpleCov depends upon and that's what SimpleCov supports. Additionally JRuby 9.1+ is supported as well, while JRuby 1.7 and 9.0 should work they're not "officially" supported.
613
- SimpleCov is also built against Ruby 1.8 in [Continuous Integration], but this happens only to ensure that SimpleCov
614
- does not make your test suite crash right now.
797
+ SimpleCov is built in [Continuous Integration] on Ruby 2.4+ as well as JRuby 9.2+.
615
798
 
616
- SimpleCov is built in [Continuous Integration] on Ruby 1.9.3, 2.0.0, 2.1, 2.2, 2.3, 2.4, 2.5 as well as JRuby 9.1.
799
+ Note for JRuby => You need to pass JRUBY_OPTS="--debug" or create .jrubyrc and add debug.fullTrace=true
617
800
 
618
801
  ## Want to find dead code in production?
619
802
 
@@ -621,7 +804,9 @@ Try [Coverband](https://github.com/danmayer/coverband).
621
804
 
622
805
  ## Want to use Spring with SimpleCov?
623
806
 
624
- If you're using [Spring](https://github.com/rails/spring) to speed up test suite runs and want to run SimpleCov along with them, you'll find that it often misreports coverage with the default config due to some sort of eager loading issue. Don't despair!
807
+ If you're using [Spring](https://github.com/rails/spring) to speed up test suite runs and want to run SimpleCov along
808
+ with them, you'll find that it often misreports coverage with the default config due to some sort of eager loading
809
+ issue. Don't despair!
625
810
 
626
811
  One solution is to [explicitly call eager
627
812
  load](https://github.com/colszowka/simplecov/issues/381#issuecomment-347651728)
@@ -633,13 +818,22 @@ SimpleCov.start 'rails'
633
818
  Rails.application.eager_load!
634
819
  ```
635
820
 
821
+ Alternatively, you could disable Spring while running SimpleCov:
822
+
823
+ ```
824
+ DISABLE_SPRING=1 rake test
825
+ ```
826
+
636
827
  Or you could remove `gem 'spring'` from your `Gemfile`.
637
828
 
638
829
  ## Troubleshooting
639
830
 
640
- The **most common problem is that simplecov isn't required and started before everything else**. In order to track coverage for your whole application **simplecov needs to be the first one** so that it (and the underlying coverage library) can subsequently track loaded files and their usage.
831
+ The **most common problem is that simplecov isn't required and started before everything else**. In order to track
832
+ coverage for your whole application **simplecov needs to be the first one** so that it (and the underlying coverage
833
+ library) can subsequently track loaded files and their usage.
641
834
 
642
- If you are missing coverage for some code a simple trick is to put a puts statement in there and right after `SimpleCov.start` so you can see if the file really was loaded after simplecov was started.
835
+ If you are missing coverage for some code a simple trick is to put a puts statement in there and right after
836
+ `SimpleCov.start` so you can see if the file really was loaded after simplecov was started.
643
837
 
644
838
  ```ruby
645
839
  # my_code.rb
@@ -667,6 +861,11 @@ MyCode is being loaded!
667
861
 
668
862
  then it's good otherwise you likely have a problem :)
669
863
 
864
+ ## Code of Conduct
865
+
866
+ Everyone participating in this project's development, issue trackers and other channels is expected to follow our
867
+ [Code of Conduct](./CODE_OF_CONDUCT.md)
868
+
670
869
  ## Contributing
671
870
 
672
871
  See the [contributing guide](https://github.com/colszowka/simplecov/blob/master/CONTRIBUTING.md).