simplecov 0.6.0 → 0.22.0

Sign up to get free protection for your applications and to get access to all the features.
Files changed (131) hide show
  1. checksums.yaml +7 -0
  2. data/CHANGELOG.md +163 -80
  3. data/LICENSE +1 -1
  4. data/README.md +776 -277
  5. data/doc/alternate-formatters.md +71 -0
  6. data/doc/commercial-services.md +25 -0
  7. data/doc/editor-integration.md +18 -0
  8. data/lib/minitest/simplecov_plugin.rb +15 -0
  9. data/lib/simplecov/combine/branches_combiner.rb +32 -0
  10. data/lib/simplecov/combine/files_combiner.rb +24 -0
  11. data/lib/simplecov/combine/lines_combiner.rb +43 -0
  12. data/lib/simplecov/combine/results_combiner.rb +60 -0
  13. data/lib/simplecov/combine.rb +30 -0
  14. data/lib/simplecov/command_guesser.rb +53 -38
  15. data/lib/simplecov/configuration.rb +478 -193
  16. data/lib/simplecov/coverage_statistics.rb +56 -0
  17. data/lib/simplecov/default_formatter.rb +20 -0
  18. data/lib/simplecov/defaults.rb +40 -44
  19. data/lib/simplecov/exit_codes/exit_code_handling.rb +29 -0
  20. data/lib/simplecov/exit_codes/maximum_coverage_drop_check.rb +83 -0
  21. data/lib/simplecov/exit_codes/minimum_coverage_by_file_check.rb +54 -0
  22. data/lib/simplecov/exit_codes/minimum_overall_coverage_check.rb +53 -0
  23. data/lib/simplecov/exit_codes.rb +15 -0
  24. data/lib/simplecov/file_list.rb +112 -36
  25. data/lib/simplecov/filter.rb +54 -4
  26. data/lib/simplecov/formatter/multi_formatter.rb +32 -0
  27. data/lib/simplecov/formatter/simple_formatter.rb +21 -15
  28. data/lib/simplecov/formatter.rb +4 -1
  29. data/lib/simplecov/last_run.rb +28 -0
  30. data/lib/simplecov/lines_classifier.rb +48 -0
  31. data/lib/simplecov/load_global_config.rb +8 -0
  32. data/lib/simplecov/no_defaults.rb +4 -0
  33. data/lib/simplecov/process.rb +19 -0
  34. data/lib/simplecov/profiles/bundler_filter.rb +5 -0
  35. data/lib/simplecov/profiles/hidden_filter.rb +5 -0
  36. data/lib/simplecov/profiles/rails.rb +18 -0
  37. data/lib/simplecov/profiles/root_filter.rb +10 -0
  38. data/lib/simplecov/profiles/test_frameworks.rb +8 -0
  39. data/lib/simplecov/profiles.rb +35 -0
  40. data/lib/simplecov/result.rb +33 -64
  41. data/lib/simplecov/result_adapter.rb +30 -0
  42. data/lib/simplecov/result_merger.rb +178 -64
  43. data/lib/simplecov/simulate_coverage.rb +29 -0
  44. data/lib/simplecov/source_file/branch.rb +84 -0
  45. data/lib/simplecov/source_file/line.rb +72 -0
  46. data/lib/simplecov/source_file.rb +304 -123
  47. data/lib/simplecov/useless_results_remover.rb +18 -0
  48. data/lib/simplecov/version.rb +4 -2
  49. data/lib/simplecov.rb +396 -49
  50. metadata +81 -242
  51. data/.gitignore +0 -30
  52. data/.rvmrc +0 -1
  53. data/.travis.yml +0 -13
  54. data/Gemfile +0 -9
  55. data/Rakefile +0 -16
  56. data/cucumber.yml +0 -13
  57. data/features/config_adapters.feature +0 -44
  58. data/features/config_autoload.feature +0 -46
  59. data/features/config_command_name.feature +0 -33
  60. data/features/config_coverage_dir.feature +0 -20
  61. data/features/config_deactivate_merging.feature +0 -42
  62. data/features/config_merge_timeout.feature +0 -38
  63. data/features/config_nocov_token.feature +0 -79
  64. data/features/config_project_name.feature +0 -27
  65. data/features/config_styles.feature +0 -93
  66. data/features/cucumber_basic.feature +0 -29
  67. data/features/merging_test_unit_and_rspec.feature +0 -44
  68. data/features/rspec_basic.feature +0 -31
  69. data/features/rspec_fails_on_initialization.feature +0 -14
  70. data/features/rspec_groups_and_filters_basic.feature +0 -29
  71. data/features/rspec_groups_and_filters_complex.feature +0 -35
  72. data/features/rspec_groups_using_filter_class.feature +0 -40
  73. data/features/rspec_without_simplecov.feature +0 -20
  74. data/features/skipping_code_blocks_manually.feature +0 -70
  75. data/features/step_definitions/html_steps.rb +0 -45
  76. data/features/step_definitions/simplecov_steps.rb +0 -61
  77. data/features/step_definitions/transformers.rb +0 -13
  78. data/features/step_definitions/web_steps.rb +0 -64
  79. data/features/support/env.rb +0 -26
  80. data/features/test_unit_basic.feature +0 -34
  81. data/features/test_unit_groups_and_filters_basic.feature +0 -29
  82. data/features/test_unit_groups_and_filters_complex.feature +0 -35
  83. data/features/test_unit_groups_using_filter_class.feature +0 -40
  84. data/features/test_unit_without_simplecov.feature +0 -20
  85. data/features/unicode_compatiblity.feature +0 -67
  86. data/lib/simplecov/adapters.rb +0 -29
  87. data/lib/simplecov/merge_helpers.rb +0 -39
  88. data/lib/simplecov/railtie.rb +0 -7
  89. data/lib/simplecov/railties/tasks.rake +0 -11
  90. data/simplecov.gemspec +0 -28
  91. data/test/faked_project/Gemfile +0 -6
  92. data/test/faked_project/Rakefile +0 -8
  93. data/test/faked_project/cucumber.yml +0 -13
  94. data/test/faked_project/features/step_definitions/my_steps.rb +0 -23
  95. data/test/faked_project/features/support/env.rb +0 -12
  96. data/test/faked_project/features/test_stuff.feature +0 -6
  97. data/test/faked_project/lib/faked_project/framework_specific.rb +0 -18
  98. data/test/faked_project/lib/faked_project/meta_magic.rb +0 -24
  99. data/test/faked_project/lib/faked_project/some_class.rb +0 -29
  100. data/test/faked_project/lib/faked_project.rb +0 -11
  101. data/test/faked_project/spec/faked_spec.rb +0 -11
  102. data/test/faked_project/spec/meta_magic_spec.rb +0 -10
  103. data/test/faked_project/spec/some_class_spec.rb +0 -10
  104. data/test/faked_project/spec/spec_helper.rb +0 -15
  105. data/test/faked_project/test/faked_test.rb +0 -11
  106. data/test/faked_project/test/meta_magic_test.rb +0 -13
  107. data/test/faked_project/test/some_class_test.rb +0 -15
  108. data/test/faked_project/test/test_helper.rb +0 -16
  109. data/test/fixtures/app/controllers/sample_controller.rb +0 -10
  110. data/test/fixtures/app/models/user.rb +0 -10
  111. data/test/fixtures/deleted_source_sample.rb +0 -15
  112. data/test/fixtures/frameworks/rspec_bad.rb +0 -9
  113. data/test/fixtures/frameworks/rspec_good.rb +0 -9
  114. data/test/fixtures/frameworks/testunit_bad.rb +0 -9
  115. data/test/fixtures/frameworks/testunit_good.rb +0 -9
  116. data/test/fixtures/resultset1.rb +0 -4
  117. data/test/fixtures/resultset2.rb +0 -5
  118. data/test/fixtures/sample.rb +0 -16
  119. data/test/fixtures/utf-8.rb +0 -3
  120. data/test/helper.rb +0 -35
  121. data/test/shoulda_macros.rb +0 -29
  122. data/test/test_1_8_fallbacks.rb +0 -33
  123. data/test/test_command_guesser.rb +0 -21
  124. data/test/test_deleted_source.rb +0 -16
  125. data/test/test_file_list.rb +0 -24
  126. data/test/test_filters.rb +0 -80
  127. data/test/test_merge_helpers.rb +0 -107
  128. data/test/test_result.rb +0 -147
  129. data/test/test_return_codes.rb +0 -39
  130. data/test/test_source_file.rb +0 -86
  131. data/test/test_source_file_line.rb +0 -110
data/README.md CHANGED
@@ -1,154 +1,239 @@
1
- SimpleCov [![Build Status](https://secure.travis-ci.org/colszowka/simplecov.png)][Continuous Integration]
1
+ SimpleCov [![Gem Version](https://badge.fury.io/rb/simplecov.svg)](https://badge.fury.io/rb/simplecov) [![Build Status](https://github.com/simplecov-ruby/simplecov/workflows/stable/badge.svg?branch=main)][Continuous Integration] [![Maintainability](https://api.codeclimate.com/v1/badges/c071d197d61953a7e482/maintainability)](https://codeclimate.com/github/simplecov-ruby/simplecov/maintainability) [![Inline docs](http://inch-ci.org/github/simplecov-ruby/simplecov.svg?branch=main)](http://inch-ci.org/github/simplecov-ruby/simplecov)
2
2
  =========
3
- *Code coverage for Ruby 1.9*
3
+
4
+ **Code coverage for Ruby**
4
5
 
5
6
  * [Source Code]
6
7
  * [API documentation]
7
8
  * [Changelog]
8
9
  * [Rubygem]
9
10
  * [Continuous Integration]
10
- * IRC #simplecov on Freenode
11
11
 
12
- [Coverage]: http://www.ruby-doc.org/ruby-1.9/classes/Coverage.html "API doc for Ruby 1.9's Coverage library"
13
- [Source Code]: https://github.com/colszowka/simplecov "Source Code @ GitHub"
12
+ [Coverage]: https://ruby-doc.org/stdlib/libdoc/coverage/rdoc/Coverage.html "API doc for Ruby's Coverage library"
13
+ [Source Code]: https://github.com/simplecov-ruby/simplecov "Source Code @ GitHub"
14
14
  [API documentation]: http://rubydoc.info/gems/simplecov/frames "RDoc API Documentation at Rubydoc.info"
15
15
  [Configuration]: http://rubydoc.info/gems/simplecov/SimpleCov/Configuration "Configuration options API documentation"
16
- [Changelog]: https://github.com/colszowka/simplecov/blob/master/CHANGELOG.md "Project Changelog"
16
+ [Changelog]: https://github.com/simplecov-ruby/simplecov/blob/main/CHANGELOG.md "Project Changelog"
17
17
  [Rubygem]: http://rubygems.org/gems/simplecov "SimpleCov @ rubygems.org"
18
- [Continuous Integration]: http://travis-ci.org/colszowka/simplecov "SimpleCov is built around the clock by travis-ci.org"
19
- [simplecov-html]: https://github.com/colszowka/simplecov-html "SimpleCov HTML Formatter Source Code @ GitHub"
20
-
18
+ [Continuous Integration]: https://github.com/simplecov-ruby/simplecov/actions?query=workflow%3Astable "SimpleCov is built around the clock by github.com"
19
+ [Dependencies]: https://gemnasium.com/simplecov-ruby/simplecov "SimpleCov dependencies on Gemnasium"
20
+ [simplecov-html]: https://github.com/simplecov-ruby/simplecov-html "SimpleCov HTML Formatter Source Code @ GitHub"
21
21
 
22
- SimpleCov is a code coverage analysis tool for Ruby 1.9. It uses [1.9's built-in Coverage][Coverage] library to gather code
23
- coverage data, but makes processing its results much easier by providing a clean API to filter, group, merge, format
24
- and display those results, thus giving you a complete code coverage suite that can be set up with just a couple lines of
22
+ SimpleCov is a code coverage analysis tool for Ruby. It uses [Ruby's built-in Coverage][Coverage] library to gather code
23
+ coverage data, but makes processing its results much easier by providing a clean API to filter, group, merge, format,
24
+ and display those results, giving you a complete code coverage suite that can be set up with just a couple lines of
25
25
  code.
26
+ SimpleCov/Coverage track covered ruby code, gathering coverage for common templating solutions like erb, slim and haml is not supported.
26
27
 
27
- In most cases, you'll want overall coverage results for your projects, including all types of tests, cucumber features
28
- etc. SimpleCov automatically takes care of this by caching and then merging results when generating reports, so your
28
+ In most cases, you'll want overall coverage results for your projects, including all types of tests, Cucumber features,
29
+ etc. SimpleCov automatically takes care of this by caching and merging results when generating reports, so your
29
30
  report actually includes coverage across your test suites and thereby gives you a better picture of blank spots.
30
31
 
31
- The official formatter of SimpleCov is packaged as a separate gem called [simplecov-html] but will be installed and configured
32
- automatically when you launch SimpleCov. If you're curious, you can find it [on Github, too][simplecov-html].
32
+ The official formatter of SimpleCov is packaged as a separate gem called [simplecov-html], but will be installed and
33
+ configured automatically when you launch SimpleCov. If you're curious, you can find it [on GitHub, too][simplecov-html].
34
+
35
+
36
+ ## Contact
33
37
 
38
+ *Code and Bug Reports*
34
39
 
40
+ * [Issue Tracker](https://github.com/simplecov-ruby/simplecov/issues)
41
+ * See [CONTRIBUTING](https://github.com/simplecov-ruby/simplecov/blob/main/CONTRIBUTING.md) for how to contribute along
42
+ with some common problems to check out before creating an issue.
43
+
44
+ *Questions, Problems, Suggestions, etc.*
45
+
46
+ * [Mailing List](https://groups.google.com/forum/#!forum/simplecov) "Open mailing list for discussion and announcements
47
+ on Google Groups"
35
48
 
36
49
  Getting started
37
50
  ---------------
38
-
39
51
  1. Add SimpleCov to your `Gemfile` and `bundle install`:
40
52
 
41
- gem 'simplecov', :require => false, :group => :test
53
+ ```ruby
54
+ gem 'simplecov', require: false, group: :test
55
+ ```
56
+ 2. Load and launch SimpleCov **at the very top** of your `test/test_helper.rb`
57
+ (*or `spec_helper.rb`, `rails_helper`, cucumber `env.rb`, or whatever your preferred test
58
+ framework uses*):
42
59
 
43
- 2. Load and launch SimpleCov **at the very top** of your `test/test_helper.rb` (*or `spec_helper.rb`, cucumber `env.rb`, or whatever
44
- your preferred test framework uses*):
60
+ ```ruby
61
+ require 'simplecov'
62
+ SimpleCov.start
63
+
64
+ # Previous content of test helper now starts here
65
+ ```
66
+
67
+ **Note:** If SimpleCov starts after your application code is already loaded
68
+ (via `require`), it won't be able to track your files and their coverage!
69
+ The `SimpleCov.start` **must** be issued **before any of your application
70
+ code is required!**
71
+
72
+ This is especially true if you use anything that keeps your tests application loaded like spring, check out the **[spring section](#want-to-use-spring-with-simplecov)**.
73
+
74
+ SimpleCov must be running in the process that you want the code coverage
75
+ analysis to happen on. When testing a server process (e.g. a JSON API
76
+ endpoint) via a separate test process (e.g. when using Selenium) where you
77
+ want to see all code executed by the `rails server`, and not just code
78
+ executed in your actual test files, you need to require SimpleCov in the
79
+ server process. For rails for instance, you'll want to add something like this
80
+ to the top of `bin/rails`, but below the "shebang" line (`#! /usr/bin/env
81
+ ruby`) and after config/boot is required:
82
+
83
+ ```ruby
84
+ if ENV['RAILS_ENV'] == 'test'
85
+ require 'simplecov'
86
+ SimpleCov.start 'rails'
87
+ puts "required simplecov"
88
+ end
89
+ ```
45
90
 
46
- require 'simplecov'
47
- SimpleCov.start
91
+ 3. Run your full test suite to see the percent coverage that your application has.
92
+ 4. After running your tests, open `coverage/index.html` in the browser of your choice. For example, in a Mac Terminal,
93
+ run the following command from your application's root directory:
48
94
 
49
- # Previous content of test helper now starts here
95
+ ```
96
+ open coverage/index.html
97
+ ```
98
+ in a debian/ubuntu Terminal,
50
99
 
51
- **Note:** If SimpleCov starts after your application code is already loaded (via `require`), it won't be able to track your files and their coverage!
52
- The `SimpleCov.start` **must** be issued **before any of your application code is required!**
100
+ ```
101
+ xdg-open coverage/index.html
102
+ ```
53
103
 
54
- 3. Run your tests, open up `coverage/index.html` in your browser and check out what you've missed so far.
104
+ **Note:** [This guide](https://dwheeler.com/essays/open-files-urls.html) can help if you're unsure which command your particular
105
+ operating system requires.
55
106
 
56
- 4. Add the following to your `.gitignore` file to ensure that coverage results are not tracked by Git (optional):
107
+ 5. Add the following to your `.gitignore` file to ensure that coverage results
108
+ are not tracked by Git (optional):
57
109
 
58
- coverage
110
+ ```
111
+ echo coverage >> .gitignore
112
+ ```
59
113
 
60
- If you're making a Rails application, SimpleCov comes with a built-in adapter (see below for more information on what adapters are)
61
- which will get you started with groups for your Controllers, Views, Models and Helpers. To use it, the first two lines of
62
- your test_helper should be like this:
114
+ If you're making a Rails application, SimpleCov comes with built-in configurations (see below for information on
115
+ profiles) that will get you started with groups for your Controllers, Models and Helpers. To use it, the
116
+ first two lines of your test_helper should be like this:
63
117
 
64
- require 'simplecov'
65
- SimpleCov.start 'rails'
118
+ ```ruby
119
+ require 'simplecov'
120
+ SimpleCov.start 'rails'
121
+ ```
66
122
 
67
123
  ## Example output
68
124
 
69
125
  **Coverage results report, fully browsable locally with sorting and much more:**
70
126
 
71
- ![SimpleCov coverage report](http://colszowka.github.com/simplecov/devise_result-0.5.3.png)
127
+ ![SimpleCov coverage report](https://cloud.githubusercontent.com/assets/137793/17071162/db6f253e-502d-11e6-9d84-e40c3d75f333.png)
72
128
 
73
129
 
74
130
  **Source file coverage details view:**
75
131
 
76
- ![SimpleCov source file detail view](http://colszowka.github.com/simplecov/devise_source_file-0.5.3.png)
77
-
78
-
132
+ ![SimpleCov source file detail view](https://cloud.githubusercontent.com/assets/137793/17071163/db6f9f0a-502d-11e6-816c-edb2c66fad8d.png)
79
133
 
80
134
  ## Use it with any framework!
81
135
 
82
- Similarily to the usage with Test::Unit described above, the only thing you have to do is to add the simplecov
136
+ Similarly to the usage with Test::Unit described above, the only thing you have to do is to add the SimpleCov
83
137
  config to the very top of your Cucumber/RSpec/whatever setup file.
84
138
 
85
139
  Add the setup code to the **top** of `features/support/env.rb` (for Cucumber) or `spec/spec_helper.rb` (for RSpec).
86
140
  Other test frameworks should work accordingly, whatever their setup file may be:
87
141
 
88
- require 'simplecov'
89
- SimpleCov.start 'rails'
142
+ ```ruby
143
+ require 'simplecov'
144
+ SimpleCov.start 'rails'
145
+ ```
90
146
 
91
- You could even track what kind of code your UI testers are touching if you want to go overboard with things. SimpleCov does not
92
- care what kind of framework it is running in, it just looks at what code is being executed and generates a report about it.
147
+ You could even track what kind of code your UI testers are touching if you want to go overboard with things. SimpleCov
148
+ does not care what kind of framework it is running in; it just looks at what code is being executed and generates a
149
+ report about it.
93
150
 
94
151
  ### Notes on specific frameworks and test utilities
95
152
 
96
- For some frameworks and testing tools there are quirks and problems you might want to know about if you want
153
+ For some frameworks and testing tools there are quirks and problems you might want to know about if you want
97
154
  to use SimpleCov with them. Here's an overview of the known ones:
98
155
 
99
156
  <table>
100
- <tr><th>Framework</th><th>Notes</th><th>Issue #</th></tr>
101
- <tr>
102
- <td>
103
- <b>Test/Unit 2</b>
104
- </td>
105
- <td>
106
- Test Unit 2 used to mess with ARGV, leading to failure to detect the test process name in SimpleCov.
107
- <code>test-unit</code> releases 2.4.3+ (Dec 11th, 2011) should have this problem resolved.
108
- </td>
109
- <td>
110
- <a href="https://github.com/colszowka/simplecov/issues/45">SimpleCov #45</a> &
111
- <a href="https://github.com/test-unit/test-unit/pull/12">Test/Unit #12</a>
112
- </td>
113
- </tr>
114
- <tr>
115
- <td>
116
- <b>Spork</b>
117
- </td>
118
- <td>
119
- SimpleCov currently does not correctly cope with the way Spork (and similar forking testing tools)
120
- works. With the right configuration this might be resolvable.
121
- To make sure your coverage is reported correctly you probably should fall back to running your test suite
122
- without spork when you want a coverage report.
123
- </td>
124
- <td>
125
- <a href="https://github.com/colszowka/simplecov/issues/42">SimpleCov #42</a>
126
- </td>
127
- </tr>
128
- <tr>
129
- <td>
130
- <b>parallel_tests</b>
131
- </td>
132
- <td>
133
- SimpleCov does not detect parallel_test automatically yet but can be taught to do so
134
- with a simple workaround explained at Issue #64.
135
- </td>
136
- <td>
137
- <a href="https://github.com/colszowka/simplecov/issues/64">SimpleCov #64</a>
138
- </td>
139
- </tr>
140
- <tr>
141
- <td>
142
- <b>Riot</b>
143
- </td>
144
- <td>
145
- A user has reported problems with the coverage report using the riot framework. If you experience
146
- similar trouble please follow up on the related Github issue.
147
- </td>
148
- <td>
149
- <a href="https://github.com/colszowka/simplecov/issues/80">SimpleCov #80</a>
150
- </td>
151
- </tr>
157
+ <tr><th>Framework</th><th>Notes</th><th>Issue</th></tr>
158
+ <tr>
159
+ <th>
160
+ parallel_tests
161
+ </th>
162
+ <td>
163
+ As of 0.8.0, SimpleCov should correctly recognize parallel_tests and
164
+ supplement your test suite names with their corresponding test env
165
+ numbers. SimpleCov locks the resultset cache while merging, ensuring no
166
+ race conditions occur when results are merged.
167
+ </td>
168
+ <td>
169
+ <a href="https://github.com/simplecov-ruby/simplecov/issues/64">#64</a> &amp;
170
+ <a href="https://github.com/simplecov-ruby/simplecov/pull/185">#185</a>
171
+ </td>
172
+ </tr>
173
+ <tr>
174
+ <th>
175
+ knapsack_pro
176
+ </th>
177
+ <td>
178
+ To make SimpleCov work with Knapsack Pro Queue Mode to split tests in parallel on CI jobs you need to provide CI node index number to the <code>SimpleCov.command_name</code> in <code>KnapsackPro::Hooks::Queue.before_queue</code> hook.
179
+ </td>
180
+ <td>
181
+ <a href="https://knapsackpro.com/faq/question/how-to-use-simplecov-in-queue-mode">Tip</a>
182
+ </td>
183
+ </tr>
184
+ <tr>
185
+ <th>
186
+ RubyMine
187
+ </th>
188
+ <td>
189
+ The <a href="https://www.jetbrains.com/ruby/">RubyMine IDE</a> has
190
+ built-in support for SimpleCov's coverage reports, though you might need
191
+ to explicitly set the output root using `SimpleCov.root('foo/bar/baz')`
192
+ </td>
193
+ <td>
194
+ <a href="https://github.com/simplecov-ruby/simplecov/issues/95">#95</a>
195
+ </td>
196
+ </tr>
197
+ <tr>
198
+ <th>
199
+ Spork
200
+ </th>
201
+ <td>
202
+ Because of how Spork works internally (using preforking), there used to
203
+ be trouble when using SimpleCov with it, but that has apparently been
204
+ resolved with a specific configuration strategy. See <a
205
+ href="https://github.com/simplecov-ruby/simplecov/issues/42#issuecomment-4440284">this</a>
206
+ comment.
207
+ </td>
208
+ <td>
209
+ <a href="https://github.com/simplecov-ruby/simplecov/issues/42#issuecomment-4440284">#42</a>
210
+ </td>
211
+ </tr>
212
+ <tr>
213
+ <th>
214
+ Spring
215
+ </th>
216
+ <td>
217
+ <a href="#want-to-use-spring-with-simplecov">See section below.</a>
218
+ </td>
219
+ <td>
220
+ <a href="https://github.com/simplecov-ruby/simplecov/issues/381">#381</a>
221
+ </td>
222
+ </tr>
223
+ <tr>
224
+ <th>
225
+ Test/Unit
226
+ </th>
227
+ <td>
228
+ Test Unit 2 used to mess with ARGV, leading to a failure to detect the
229
+ test process name in SimpleCov. <code>test-unit</code> releases 2.4.3+
230
+ (Dec 11th, 2011) should have this problem resolved.
231
+ </td>
232
+ <td>
233
+ <a href="https://github.com/simplecov-ruby/simplecov/issues/45">#45</a> &amp;
234
+ <a href="https://github.com/test-unit/test-unit/pull/12">test-unit/test-unit#12</a>
235
+ </td>
236
+ </tr>
152
237
  </table>
153
238
 
154
239
  ## Configuring SimpleCov
@@ -157,319 +242,733 @@ to use SimpleCov with them. Here's an overview of the known ones:
157
242
 
158
243
  * The most common way is to configure it directly in your start block:
159
244
 
160
- SimpleCov.start do
161
- some_config_option 'foo'
162
- end
163
-
245
+ ```ruby
246
+ SimpleCov.start do
247
+ some_config_option 'foo'
248
+ end
249
+ ```
164
250
  * You can also set all configuration options directly:
165
251
 
166
- SimpleCov.some_config_option 'foo'
167
-
168
- * If you do not want to start coverage immediately after launch or want to add additional configuration later on in a concise way, use:
252
+ ```ruby
253
+ SimpleCov.some_config_option 'foo'
254
+ ```
255
+ * If you do not want to start coverage immediately after launch or want to add additional configuration later on in a
256
+ concise way, use:
169
257
 
170
- SimpleCov.configure do
171
- some_config_option 'foo'
172
- end
258
+ ```ruby
259
+ SimpleCov.configure do
260
+ some_config_option 'foo'
261
+ end
262
+ ```
173
263
 
174
264
  Please check out the [Configuration] API documentation to find out what you can customize.
175
265
 
176
-
177
266
  ## Using .simplecov for centralized config
178
267
 
179
- If you use SimpleCov to merge multiple test suite results (i.e. Test/Unit and Cucumber) into a single report, you'd normally have to
180
- set up all your config options twice, once in `test_helper.rb` and once in `env.rb`.
268
+ If you use SimpleCov to merge multiple test suite results (e.g. Test/Unit and Cucumber) into a single report, you'd
269
+ normally have to set up all your config options twice, once in `test_helper.rb` and once in `env.rb`.
181
270
 
182
- To avoid this, you can place a file called `.simplecov` in your project root. You can then just leave the `require 'simplecov'` in each
183
- test setup helper and move the `SimpleCov.start` code with all your custom config options into `.simplecov`:
271
+ To avoid this, you can place a file called `.simplecov` in your project root. You can then just leave the
272
+ `require 'simplecov'` in each test setup helper (**at the top**) and move the `SimpleCov.start` code with all your
273
+ custom config options into `.simplecov`:
184
274
 
185
- # test/test_helper.rb
186
- require 'simplecov'
275
+ ```ruby
276
+ # test/test_helper.rb
277
+ require 'simplecov'
187
278
 
188
- # features/support/env.rb
189
- require 'simplecov'
279
+ # features/support/env.rb
280
+ require 'simplecov'
190
281
 
191
- # .simplecov
192
- SimpleCov.start 'rails' do
193
- # any custom configs like groups and filters can be here at a central place
194
- end
282
+ # .simplecov
283
+ SimpleCov.start 'rails' do
284
+ # any custom configs like groups and filters can be here at a central place
285
+ end
286
+ ```
287
+
288
+ Using `.simplecov` rather than separately requiring SimpleCov multiple times is recommended if you are merging multiple
289
+ test frameworks like Cucumber and RSpec that rely on each other, as invoking SimpleCov multiple times can cause coverage
290
+ information to be lost.
291
+
292
+ ## Branch coverage (ruby "~> 2.5")
293
+ Add branch coverage measurement statistics to your results. Supported in CRuby versions 2.5+.
294
+
295
+ ```ruby
296
+ SimpleCov.start do
297
+ enable_coverage :branch
298
+ end
299
+ ```
300
+
301
+ Branch coverage is a feature introduced in Ruby 2.5 concerning itself with whether a
302
+ particular branch of a condition had been executed. Line coverage on the other hand
303
+ is only interested in whether a line of code has been executed.
304
+
305
+ This comes in handy for instance for one line conditionals:
306
+
307
+ ```ruby
308
+ number.odd? ? "odd" : "even"
309
+ ```
310
+
311
+ In line coverage this line would always be marked as executed but you'd never know if both
312
+ conditions were met. Guard clauses have a similar story:
313
+
314
+ ```ruby
315
+ return if number.odd?
195
316
 
317
+ # more code
318
+ ```
319
+
320
+ If all the code in that method was covered you'd never know if the guard clause was ever
321
+ triggered! With line coverage as just evaluating the condition marks it as covered.
322
+
323
+ In the HTML report the lines of code will be annotated like `branch_type: hit_count`:
324
+
325
+ * `then: 2` - the then branch (of an `if`) was executed twice
326
+ * `else: 0` - the else branch (of an `if` or `case`) was never executed
327
+
328
+ Not that even if you don't declare an `else` branch it will still show up in the coverage
329
+ reports meaning that the condition of the `if` was not hit or that no `when` of `case`
330
+ was hit during the test runs.
331
+
332
+ **Is branch coverage strictly better?** No. Branch coverage really only concerns itself with
333
+ conditionals - meaning coverage of sequential code is of no interest to it. A file without
334
+ conditional logic will have no branch coverage data and SimpleCov will report 0 of 0
335
+ branches covered as 100% (as everything that can be covered was covered).
336
+
337
+ Hence, we recommend looking at both metrics together. Branch coverage might also be a good
338
+ overall metric to look at - while you might be missing only 10% of your lines that might
339
+ account for 50% of your branches for instance.
340
+
341
+ ## Primary Coverage
342
+
343
+ By default, the primary coverage type is `line`. To set the primary coverage to something else, use the following:
344
+
345
+ ```ruby
346
+ # or in configure SimpleCov.primary_coverage :branch
347
+ SimpleCov.start do
348
+ enable_coverage :branch
349
+ primary_coverage :branch
350
+ end
351
+ ```
352
+
353
+ Primary coverage determines what will come in first all output, and the type of coverage to check if you don't specify the type of coverage when customizing exit behavior (`SimpleCov.minimum_coverage 90`).
354
+
355
+ Note that coverage must first be enabled for non-default coverage types.
356
+
357
+ ## Coverage for eval
358
+
359
+ You can measure coverage for code that is evaluated by `Kernel#eval`. Supported in CRuby versions 3.2+.
360
+
361
+ ```ruby
362
+ SimpleCov.start do
363
+ enable_coverage_for_eval
364
+ end
365
+ ```
366
+
367
+ This is typically useful for ERB. Set `ERB#filename=` to make it possible for SimpleCov to trace the original .erb source file.
196
368
 
197
369
  ## Filters
198
370
 
199
- Filters can be used to remove selected files from your coverage data. By default, a filter is applied that removes all files
200
- OUTSIDE of your project's root directory - otherwise you'd end up with a billion of coverage reports for source files in the
201
- gems you are using.
371
+ Filters can be used to remove selected files from your coverage data. By default, a filter is applied that removes all
372
+ files OUTSIDE of your project's root directory - otherwise you'd end up with billions of coverage reports for source
373
+ files in the gems you are using.
202
374
 
203
- Of course you can define your own to remove things like configuration files, tests or whatever you don't need in your coverage
375
+ You can define your own to remove things like configuration files, tests or whatever you don't need in your coverage
204
376
  report.
205
377
 
206
378
  ### Defining custom filters
207
379
 
208
- You can currently define a filter using either a String (that will then be Regexp-matched against each source file's path),
209
- a block or by passing in your own Filter class.
380
+ You can currently define a filter using either a String or Regexp (that will then be Regexp-matched against each source
381
+ file's path), a block or by passing in your own Filter class.
210
382
 
211
383
  #### String filter
212
384
 
213
- SimpleCov.start do
214
- add_filter "/test/"
215
- end
385
+ ```ruby
386
+ SimpleCov.start do
387
+ add_filter "/test/"
388
+ end
389
+ ```
216
390
 
217
391
  This simple string filter will remove all files that match "/test/" in their path.
218
392
 
393
+ #### Regex filter
394
+
395
+ ```ruby
396
+ SimpleCov.start do
397
+ add_filter %r{^/test/}
398
+ end
399
+ ```
400
+
401
+ This simple regex filter will remove all files that start with /test/ in their path.
402
+
219
403
  #### Block filter
220
404
 
221
- SimpleCov.start do
222
- add_filter do |source_file|
223
- source_file.lines.count < 5
224
- end
225
- end
405
+ ```ruby
406
+ SimpleCov.start do
407
+ add_filter do |source_file|
408
+ source_file.lines.count < 5
409
+ end
410
+ end
411
+ ```
226
412
 
227
- Block filters receive a SimpleCov::SourceFile instance and expect your block to return either true (if the file is to be removed
228
- from the result) or false (if the result should be kept). Please check out the RDoc for SimpleCov::SourceFile to learn about the
229
- methods available to you. In the above example, the filter will remove all files that have less then 5 lines of code.
413
+ Block filters receive a SimpleCov::SourceFile instance and expect your block to return either true (if the file is to be
414
+ removed from the result) or false (if the result should be kept). Please check out the RDoc for SimpleCov::SourceFile to
415
+ learn about the methods available to you. In the above example, the filter will remove all files that have less than 5
416
+ lines of code.
230
417
 
231
418
  #### Custom filter class
232
419
 
233
- class LineFilter < SimpleCov::Filter
234
- def matches?(source_file)
235
- source_file.lines.count < filter_argument
236
- end
237
- end
420
+ ```ruby
421
+ class LineFilter < SimpleCov::Filter
422
+ def matches?(source_file)
423
+ source_file.lines.count < filter_argument
424
+ end
425
+ end
426
+
427
+ SimpleCov.add_filter LineFilter.new(5)
428
+ ```
429
+
430
+ Defining your own filters is pretty easy: Just inherit from SimpleCov::Filter and define a method
431
+ 'matches?(source_file)'. When running the filter, a true return value from this method will result in the removal of the
432
+ given source_file. The filter_argument method is being set in the SimpleCov::Filter initialize method and thus is set to
433
+ 5 in this example.
434
+
435
+ #### Array filter
436
+
437
+ ```ruby
438
+ SimpleCov.start do
439
+ proc = Proc.new { |source_file| false }
440
+ add_filter ["string", /regex/, proc, LineFilter.new(5)]
441
+ end
442
+ ```
443
+
444
+ You can pass in an array containing any of the other filter types.
445
+
446
+ #### Ignoring/skipping code
447
+
448
+ You can exclude code from the coverage report by wrapping it in `# :nocov:`.
238
449
 
239
- SimpleCov.add_filter LineFilter.new(5)
450
+ ```ruby
451
+ # :nocov:
452
+ def skip_this_method
453
+ never_reached
454
+ end
455
+ # :nocov:
456
+ ```
240
457
 
241
- Defining your own filters is pretty easy: Just inherit from SimpleCov::Filter and define a method 'matches?(source_file)'. When running
242
- the filter, a true return value from this method will result in the removal of the given source_file. The filter_argument method
243
- is being set in the SimpleCov::Filter initialize method and thus is set to 5 in this example.
458
+ The name of the token can be changed to your liking. [Learn more about the nocov feature.]( https://github.com/simplecov-ruby/simplecov/blob/main/features/config_nocov_token.feature)
244
459
 
460
+ **Note:** You shouldn't have to use the nocov token to skip private methods that are being included in your coverage. If
461
+ you appropriately test the public interface of your classes and objects you should automatically get full coverage of
462
+ your private methods.
463
+
464
+ ## Default root filter and coverage for things outside of it
465
+
466
+ By default, SimpleCov filters everything outside of the `SimpleCov.root` directory. However, sometimes you may want
467
+ to include coverage reports for things you include as a gem, for example a Rails Engine.
468
+
469
+ Here's an example by [@lsaffie](https://github.com/lsaffie) from [#221](https://github.com/simplecov-ruby/simplecov/issues/221)
470
+ that shows how you can achieve just that:
471
+
472
+ ```ruby
473
+ SimpleCov.start :rails do
474
+ filters.clear # This will remove the :root_filter and :bundler_filter that come via simplecov's defaults
475
+ add_filter do |src|
476
+ !(src.filename =~ /^#{SimpleCov.root}/) unless src.filename =~ /my_engine/
477
+ end
478
+ end
479
+ ```
245
480
 
246
481
  ## Groups
247
482
 
248
- You can separate your source files into groups. For example, in a rails app, you'll want to have separate listings for
249
- Models, Controllers, Helpers, Libs and Plugins. Group definition works similar to Filters (and indeed also accepts custom
250
- filter classes), but source files end up in a group when the filter passes (returns true), as opposed to filtering results,
251
- which exclude files from results when the filter results in a true value.
483
+ You can separate your source files into groups. For example, in a Rails app, you'll want to have separate listings for
484
+ Models, Controllers, Helpers, and Libs. Group definition works similarly to Filters (and also accepts custom
485
+ filter classes), but source files end up in a group when the filter passes (returns true), as opposed to filtering
486
+ results, which exclude files from results when the filter results in a true value.
252
487
 
253
488
  Add your groups with:
254
489
 
255
- SimpleCov.start do
256
- add_group "Models", "app/models"
257
- add_group "Controllers", "app/controllers"
258
- add_group "Long files" do |src_file|
259
- src_file.lines.count > 100
260
- end
261
- add_group "Short files", LineFilter.new(5) # Using the LineFilter class defined in Filters section above
262
- end
490
+ ```ruby
491
+ SimpleCov.start do
492
+ add_group "Models", "app/models"
493
+ add_group "Controllers", "app/controllers"
494
+ add_group "Long files" do |src_file|
495
+ src_file.lines.count > 100
496
+ end
497
+ add_group "Multiple Files", ["app/models", "app/controllers"] # You can also pass in an array
498
+ add_group "Short files", LineFilter.new(5) # Using the LineFilter class defined in Filters section above
499
+ end
500
+ ```
263
501
 
264
502
  ## Merging results
265
503
 
266
- Normally, you want to have your coverage analyzed across ALL of your test suites, right?
504
+ You normally want to have your coverage analyzed across ALL of your test suites, right?
267
505
 
268
- Simplecov automatically caches coverage results in your (coverage_path)/resultset.yml. Those results will then
269
- be automatically merged when generating the result, so when coverage is set up properly for cucumber and your
270
- unit / functional / integration tests, all of those test suites will be taken into account when building the
271
- coverage report.
272
-
273
- There are two things to note here though:
506
+ Simplecov automatically caches coverage results in your
507
+ (coverage_path)/.resultset.json, and will merge or override those with
508
+ subsequent runs, depending on whether simplecov considers those subsequent runs
509
+ as different test suites or as the same test suite as the cached results. To
510
+ make this distinction, simplecov has the concept of "test suite names".
274
511
 
275
512
  ### Test suite names
276
513
 
277
- Simplecov tries to guess the name of the currently running test suite based upon the shell command the tests are running
278
- on. This should work fine for Unit Tests, RSpec and Cucumber. If it fails, it will use the shell command
279
- that invoked the test suite as a command name.
514
+ SimpleCov tries to guess the name of the currently running test suite based upon the shell command the tests
515
+ are running on. This should work fine for Unit Tests, RSpec, and Cucumber. If it fails, it will use the shell
516
+ command that invoked the test suite as a command name.
517
+
518
+ If you have some non-standard setup and still want nicely labeled test suites, you have to give Simplecov a
519
+ cue as to what the name of the currently running test suite is. You can do so by specifying
520
+ `SimpleCov.command_name` in one test file that is part of your specific suite.
280
521
 
281
- If you have some non-standard setup and still want nicely labeled test suites, you have to give Simplecov a cue what the
282
- name of the currently running test suite is. You can do so by specifying SimpleCov.command_name in one test file that is
283
- part of your specific suite.
522
+ To customize the suite names on a Rails app (yeah, sorry for being Rails-biased, but everyone knows what
523
+ the structure of those projects is. You can apply this accordingly to the RSpecs in your
524
+ Outlook-WebDAV-Calendar-Sync gem), you could do something like this:
284
525
 
285
- So, to customize the suite names on a Rails app (yeah, sorry for being Rails biased, but everyone knows what
286
- the structure of those projects is. You can apply this accordingly to the RSpecs in your Outlook-WebDAV-Calendar-Sync gem),
287
- you could do something like this:
526
+ ```ruby
527
+ # test/unit/some_test.rb
528
+ SimpleCov.command_name 'test:units'
288
529
 
289
- # test/unit/some_test.rb
290
- SimpleCov.command_name 'test:units'
530
+ # test/functionals/some_controller_test.rb
531
+ SimpleCov.command_name "test:functionals"
291
532
 
292
- # test/functionals/some_controller_test.rb
293
- SimpleCov.command_name "test:functionals"
533
+ # test/integration/some_integration_test.rb
534
+ SimpleCov.command_name "test:integration"
294
535
 
295
- # test/integration/some_integration_test.rb
296
- SimpleCov.command_name "test:integration"
536
+ # features/support/env.rb
537
+ SimpleCov.command_name "features"
538
+ ```
297
539
 
298
- # features/support/env.rb
299
- SimpleCov.command_name "features"
540
+ Note that this only has to be invoked ONCE PER TEST SUITE, so even if you have 200 unit test files,
541
+ specifying it in `some_test.rb` is enough.
300
542
 
301
- Note that this has only to be invoked ONCE PER TEST SUITE, so even if you have 200 unit test files, specifying it in
302
- some_test.rb is fair enough.
543
+ Last but not least **if multiple suites resolve to the same `command_name`** be aware that the coverage results **will
544
+ clobber each other instead of being merged**. SimpleCov is smart enough to detect unique names for the most common
545
+ setups, but if you have more than one test suite that doesn't follow a common pattern then you will want to manually
546
+ ensure that each suite gets a unique `command_name`.
547
+
548
+ If you are running tests in parallel each process has the potential to clobber results from the other test processes.
549
+ If you are relying on the default `command_name` then SimpleCov will attempt to detect and avoid parallel test suite
550
+ `command_name` collisions based on the presence of `ENV['PARALLEL_TEST_GROUPS']` and `ENV['TEST_ENV_NUMBER']`. If your
551
+ parallel test runner does not set one or both of these then *you must* set a `command_name` and ensure that it is unique
552
+ per process (eg. `command_name "Unit Tests PID #{$$}"`).
553
+
554
+ If you are using parallel_tests, you must incorporate `TEST_ENV_NUMBER` into the command name yourself, in
555
+ order for SimpleCov to merge the results correctly. For example:
556
+
557
+ ```ruby
558
+ # spec/spec_helper.rb
559
+ SimpleCov.command_name "features" + (ENV['TEST_ENV_NUMBER'] || '')
560
+ ```
303
561
 
304
562
  [simplecov-html] prints the used test suites in the footer of the generated coverage report.
305
563
 
306
- ### Timeout for merge
307
564
 
308
- Of course, your cached coverage data is likely to become invalid at some point. Thus, result sets that are older than
309
- SimpleCov.merge_timeout will not be used any more. By default, the timeout is 600 seconds (10 minutes), and you can
310
- raise (or lower) it by specifying `SimpleCov.merge_timeout 3600` (1 hour), or, inside a configure/start block, with
311
- just "merge_timeout 3600".
565
+ ### Merging test runs under the same execution environment
566
+
567
+ Test results are automatically merged with previous runs in the same execution
568
+ environment when generating the result, so when coverage is set up properly for
569
+ Cucumber and your unit / functional / integration tests, all of those test
570
+ suites will be taken into account when building the coverage report.
571
+
572
+ #### Timeout for merge
573
+
574
+ Of course, your cached coverage data is likely to become invalid at some point. Thus, when automatically merging
575
+ subsequent test runs, result sets that are older than `SimpleCov.merge_timeout` will not be used any more. By default,
576
+ the timeout is 600 seconds (10 minutes), and you can raise (or lower) it by specifying `SimpleCov.merge_timeout 3600`
577
+ (1 hour), or, inside a configure/start block, with just `merge_timeout 3600`.
578
+
579
+ You can deactivate this automatic merging altogether with `SimpleCov.use_merging false`.
312
580
 
313
- You can deactivate merging altogether with `SimpleCov.use_merging false`.
581
+ ### Merging test runs under different execution environments
314
582
 
583
+ If your tests are done in parallel across multiple build machines, you can fetch them all and merge them into a single
584
+ result set using the `SimpleCov.collate` method. This can be added to a Rakefile or script file, having downloaded a set of
585
+ `.resultset.json` files from each parallel test run.
586
+
587
+ ```ruby
588
+ # lib/tasks/coverage_report.rake
589
+ namespace :coverage do
590
+ desc "Collates all result sets generated by the different test runners"
591
+ task :report do
592
+ require 'simplecov'
593
+
594
+ SimpleCov.collate Dir["simplecov-resultset-*/.resultset.json"]
595
+ end
596
+ end
597
+ ```
598
+
599
+ `SimpleCov.collate` also takes an optional simplecov profile and an optional
600
+ block for configuration, just the same as `SimpleCov.start` or
601
+ `SimpleCov.configure`. This means you can configure a separate formatter for
602
+ the collated output. For instance, you can make the formatter in
603
+ `SimpleCov.start` the `SimpleCov::Formatter::SimpleFormatter`, and only use more
604
+ complex formatters in the final `SimpleCov.collate` run.
605
+
606
+ ```ruby
607
+ # spec/spec_helper.rb
608
+ require 'simplecov'
609
+
610
+ SimpleCov.start 'rails' do
611
+ # Disambiguates individual test runs
612
+ command_name "Job #{ENV["TEST_ENV_NUMBER"]}" if ENV["TEST_ENV_NUMBER"]
613
+
614
+ if ENV['CI']
615
+ formatter SimpleCov::Formatter::SimpleFormatter
616
+ else
617
+ formatter SimpleCov::Formatter::MultiFormatter.new([
618
+ SimpleCov::Formatter::SimpleFormatter,
619
+ SimpleCov::Formatter::HTMLFormatter
620
+ ])
621
+ end
622
+
623
+ track_files "**/*.rb"
624
+ end
625
+ ```
626
+
627
+ ```ruby
628
+ # lib/tasks/coverage_report.rake
629
+ namespace :coverage do
630
+ task :report do
631
+ require 'simplecov'
632
+
633
+ SimpleCov.collate Dir["simplecov-resultset-*/.resultset.json"], 'rails' do
634
+ formatter SimpleCov::Formatter::MultiFormatter.new([
635
+ SimpleCov::Formatter::SimpleFormatter,
636
+ SimpleCov::Formatter::HTMLFormatter
637
+ ])
638
+ end
639
+ end
640
+ end
641
+ ```
642
+
643
+ ## Running simplecov against subprocesses
644
+
645
+ `SimpleCov.enable_for_subprocesses` will allow SimpleCov to observe subprocesses starting using `Process.fork`.
646
+ This modifies ruby's core Process.fork method so that SimpleCov can see into it, appending `" (subprocess #{pid})"`
647
+ to the `SimpleCov.command_name`, with results that can be merged together using SimpleCov's merging feature.
648
+
649
+ To configure this, use `.at_fork`.
650
+
651
+ ```ruby
652
+ SimpleCov.enable_for_subprocesses true
653
+ SimpleCov.at_fork do |pid|
654
+ # This needs a unique name so it won't be ovewritten
655
+ SimpleCov.command_name "#{SimpleCov.command_name} (subprocess: #{pid})"
656
+ # be quiet, the parent process will be in charge of output and checking coverage totals
657
+ SimpleCov.print_error_status = false
658
+ SimpleCov.formatter SimpleCov::Formatter::SimpleFormatter
659
+ SimpleCov.minimum_coverage 0
660
+ # start
661
+ SimpleCov.start
662
+ end
663
+ ```
664
+
665
+ NOTE: SimpleCov must have already been started before `Process.fork` was called.
666
+
667
+ ### Running simplecov against spawned subprocesses
668
+
669
+ Perhaps you're testing a ruby script with `PTY.spawn` or `Open3.popen`, or `Process.spawn` or etc.
670
+ SimpleCov can cover this too.
671
+
672
+ Add a .simplecov_spawn.rb file to your project root
673
+ ```ruby
674
+ # .simplecov_spawn.rb
675
+ require 'simplecov' # this will also pick up whatever config is in .simplecov
676
+ # so ensure it just contains configuration, and doesn't call SimpleCov.start.
677
+ SimpleCov.command_name 'spawn' # As this is not for a test runner directly, script doesn't have a pre-defined base command_name
678
+ SimpleCov.at_fork.call(Process.pid) # Use the per-process setup described previously
679
+ SimpleCov.start # only now can we start.
680
+ ```
681
+ Then, instead of calling your script directly, like:
682
+ ```ruby
683
+ PTY.spawn('my_script.rb') do # ...
684
+ ```
685
+ Use bin/ruby to require the new .simplecov_spawn file, then your script
686
+ ```ruby
687
+ PTY.spawn('ruby -r./.simplecov_spawn my_script.rb') do # ...
688
+ ```
315
689
 
316
690
  ## Running coverage only on demand
317
691
 
318
- The Ruby STDLIB Coverage library that SimpleCov builds upon is *very* fast (i.e. on a ~10 min Rails test suite, the speed drop was
319
- only a couple seconds for me), and therefore it's SimpleCov's policy to just generate coverage every time you run your tests because
320
- it doesn't do your test speed any harm and you're always equipped with the latest and greatest coverage results.
692
+ The Ruby STDLIB Coverage library that SimpleCov builds upon is *very* fast (on a ~10 min Rails test suite, the speed
693
+ drop was only a couple seconds for me), and therefore it's SimpleCov's policy to just generate coverage every time you
694
+ run your tests because it doesn't do your test speed any harm and you're always equipped with the latest and greatest
695
+ coverage results.
321
696
 
322
697
  Because of this, SimpleCov has no explicit built-in mechanism to run coverage only on demand.
323
698
 
324
- However, you can still accomplish this very easily by introducing a ENV variable conditional into your SimpleCov setup block, like this:
699
+ However, you can still accomplish this very easily by introducing an ENV variable conditional into your SimpleCov setup
700
+ block, like this:
325
701
 
326
- SimpleCov.start if ENV["COVERAGE"]
702
+ ```ruby
703
+ SimpleCov.start if ENV["COVERAGE"]
704
+ ```
327
705
 
328
706
  Then, SimpleCov will only run if you execute your tests like this:
329
707
 
330
- $ COVERAGE=true rake test
708
+ ```shell
709
+ COVERAGE=true rake test
710
+ ```
331
711
 
712
+ ## Errors and exit statuses
332
713
 
333
- ## Adapters
714
+ To aid in debugging issues, if an error is raised, SimpleCov will print a message to `STDERR`
715
+ with the exit status of the error, like:
334
716
 
335
- By default, Simplecov's only config assumption is that you only want coverage reports for files inside your project
336
- root. To save you from repetitive configuration, you can use predefined blocks of configuration, called 'adapters',
337
- or define your own.
717
+ ```
718
+ SimpleCov failed with exit 1
719
+ ```
338
720
 
339
- You can then pass the name of the adapter to be used as the first argument to SimpleCov.start. For example, simplecov
340
- comes bundled with a 'rails' adapter. It looks somewhat like this:
721
+ This `STDERR` message can be disabled with:
341
722
 
342
- SimpleCov.adapters.define 'rails' do
343
- add_filter '/test/'
344
- add_filter '/config/'
723
+ ```
724
+ SimpleCov.print_error_status = false
725
+ ```
345
726
 
346
- add_group 'Controllers', 'app/controllers'
347
- add_group 'Models', 'app/models'
348
- add_group 'Helpers', 'app/helpers'
349
- add_group 'Libraries', 'lib'
350
- add_group 'Plugins', 'vendor/plugins'
351
- end
727
+ ## Profiles
352
728
 
353
- As you can see, it's just a glorified SimpleCov.configure block. In your test_helper.rb, launch simplecov with:
729
+ By default, SimpleCov's only config assumption is that you only want coverage reports for files inside your project
730
+ root. To save yourself from repetitive configuration, you can use predefined blocks of configuration, called 'profiles',
731
+ or define your own.
354
732
 
355
- SimpleCov.start 'rails'
733
+ You can then pass the name of the profile to be used as the first argument to SimpleCov.start. For example, simplecov
734
+ comes bundled with a 'rails' profile. It looks somewhat like this:
356
735
 
357
- **OR**
736
+ ```ruby
737
+ SimpleCov.profiles.define 'rails' do
738
+ add_filter '/test/'
739
+ add_filter '/config/'
358
740
 
359
- SimpleCov.start 'rails' do
360
- # additional config here
361
- end
741
+ add_group 'Controllers', 'app/controllers'
742
+ add_group 'Models', 'app/models'
743
+ add_group 'Helpers', 'app/helpers'
744
+ add_group 'Libraries', 'lib'
745
+ end
746
+ ```
362
747
 
363
- ### Custom adapters
748
+ As you can see, it's just a SimpleCov.configure block. In your test_helper.rb, launch SimpleCov with:
364
749
 
365
- You can load additional adapters with the SimpleCov.load_adapter('xyz') method. This allows you to build upon an existing
366
- adapter and customize it so you can reuse it in unit tests and cucumber features, for example.
750
+ ```ruby
751
+ SimpleCov.start 'rails'
752
+ ```
367
753
 
368
- # lib/simplecov_custom_adapter.rb
369
- require 'simplecov'
370
- SimpleCov.adapters.define 'myadapter' do
371
- load_adapter 'rails'
372
- add_filter 'vendor' # Don't include vendored stuff
373
- end
754
+ or
374
755
 
375
- # features/support/env.rb
376
- require 'simplecov_custom_adapter'
377
- SimpleCov.start 'myadapter'
756
+ ```ruby
757
+ SimpleCov.start 'rails' do
758
+ # additional config here
759
+ end
760
+ ```
378
761
 
379
- # test/test_helper.rb
380
- require 'simplecov_custom_adapter'
381
- SimpleCov.start 'myadapter'
762
+ ### Custom profiles
382
763
 
764
+ You can load additional profiles with the SimpleCov.load_profile('xyz') method. This allows you to build upon an
765
+ existing profile and customize it so you can reuse it in unit tests and Cucumber features. For example:
383
766
 
767
+ ```ruby
768
+ # lib/simplecov_custom_profile.rb
769
+ require 'simplecov'
770
+ SimpleCov.profiles.define 'myprofile' do
771
+ load_profile 'rails'
772
+ add_filter 'vendor' # Don't include vendored stuff
773
+ end
774
+
775
+ # features/support/env.rb
776
+ require 'simplecov_custom_profile'
777
+ SimpleCov.start 'myprofile'
778
+
779
+ # test/test_helper.rb
780
+ require 'simplecov_custom_profile'
781
+ SimpleCov.start 'myprofile'
782
+ ```
384
783
 
385
784
  ## Customizing exit behaviour
386
785
 
387
- You can define what simplecov should do when your test suite finishes by customizing the at_exit hook:
786
+ You can define what SimpleCov should do when your test suite finishes by customizing the at_exit hook:
388
787
 
389
- SimpleCov.at_exit do
390
- SimpleCov.result.format!
391
- end
788
+ ```ruby
789
+ SimpleCov.at_exit do
790
+ SimpleCov.result.format!
791
+ end
792
+ ```
392
793
 
393
794
  Above is the default behaviour. Do whatever you like instead!
394
795
 
796
+ ### Minimum coverage
797
+
798
+ You can define the minimum coverage percentage expected. SimpleCov will return non-zero if unmet.
799
+
800
+ ```ruby
801
+ SimpleCov.minimum_coverage 90
802
+ # same as above (the default is to check line coverage)
803
+ SimpleCov.minimum_coverage line: 90
804
+ # check for a minimum line coverage of 90% and minimum 80% branch coverage
805
+ SimpleCov.minimum_coverage line: 90, branch: 80
806
+ ```
807
+
808
+ ### Minimum coverage by file
809
+
810
+ You can define the minimum coverage by file percentage expected. SimpleCov will return non-zero if unmet. This is useful
811
+ to help ensure coverage is relatively consistent, rather than being skewed by particularly good or bad areas of the code.
812
+
813
+ ```ruby
814
+ SimpleCov.minimum_coverage_by_file 80
815
+ # same as above (the default is to check line coverage by file)
816
+ SimpleCov.minimum_coverage_by_file line: 80
817
+ # check for a minimum line coverage by file of 90% and minimum 80% branch coverage
818
+ SimpleCov.minimum_coverage_by_file line: 90, branch: 80
819
+ ```
820
+
821
+ ### Maximum coverage drop
822
+
823
+ You can define the maximum coverage drop percentage at once. SimpleCov will return non-zero if exceeded.
824
+
825
+ ```ruby
826
+ SimpleCov.maximum_coverage_drop 5
827
+ # same as above (the default is to check line drop)
828
+ SimpleCov.maximum_coverage_drop line: 5
829
+ # check for a maximum line drop of 5% and maximum 10% branch drop
830
+ SimpleCov.maximum_coverage_drop line: 5, branch: 10
831
+ ```
832
+
833
+ ### Refuse dropping coverage
834
+
835
+ You can also entirely refuse dropping coverage between test runs:
836
+
837
+ ```ruby
838
+ SimpleCov.refuse_coverage_drop
839
+ # same as above (the default is to only refuse line drop)
840
+ SimpleCov.refuse_coverage_drop :line
841
+ # refuse drop for line and branch
842
+ SimpleCov.refuse_coverage_drop :line, :branch
843
+ ```
395
844
 
396
845
  ## Using your own formatter
397
846
 
398
847
  You can use your own formatter with:
399
848
 
400
- SimpleCov.formatter = SimpleCov::Formatter::HTMLFormatter
849
+ ```ruby
850
+ SimpleCov.formatter = SimpleCov::Formatter::HTMLFormatter
851
+ ```
401
852
 
402
- When calling SimpleCov.result.format!, it will be invoked with SimpleCov::Formatter::YourFormatter.new.format(result), "result"
403
- being an instance of SimpleCov::Result. Do whatever your wish with that!
853
+ Calling `SimpleCov.result.format!` will be invoked with `SimpleCov::Formatter::YourFormatter.new.format(result)`,
854
+ and `result` is an instance of `SimpleCov::Result`. Do whatever your wish with that!
404
855
 
405
856
 
406
857
  ## Using multiple formatters
407
858
 
408
- There is currently no built-in support for this, but you could help yourself with a wrapper class:
859
+ As of SimpleCov 0.9, you can specify multiple result formats. Formatters besides the default HTML formatter require separate gems, however.
409
860
 
410
- class SimpleCov::Formatter::MergedFormatter
411
- def format(result)
412
- SimpleCov::Formatter::HTMLFormatter.new.format(result)
413
- SimpleCov::Formatter::CSVFormatter.new.format(result)
414
- end
415
- end
861
+ ```ruby
862
+ require "simplecov-html"
416
863
 
417
- Then configure the formatter to use the new merger:
864
+ SimpleCov.formatters = SimpleCov::Formatter::MultiFormatter.new([
865
+ SimpleCov::Formatter::HTMLFormatter,
866
+ SimpleCov::Formatter::CSVFormatter,
867
+ ])
868
+ ```
418
869
 
419
- SimpleCov.formatter = SimpleCov::Formatter::MergedFormatter
870
+ ## JSON formatter
420
871
 
421
- ## Available formatters
872
+ SimpleCov is packaged with a separate gem called [simplecov_json_formatter](https://github.com/codeclimate-community/simplecov_json_formatter) that provides you with a JSON formatter, this formatter could be useful for different use cases, such as for CI consumption or for reporting to external services.
422
873
 
423
- Apart from the direct companion [simplecov-html], there are other formatters
424
- available:
874
+ In order to use it you will need to manually load the installed gem like so:
425
875
 
426
- ### [simplecov-rcov](https://github.com/fguillen/simplecov-rcov)
427
- *by Fernando Guillen*
876
+ ```ruby
877
+ require "simplecov_json_formatter"
878
+ SimpleCov.formatter = SimpleCov::Formatter::JSONFormatter
879
+ ```
428
880
 
429
- "The target of this formatter is to cheat on Hudson so I can use the Ruby metrics plugin with SimpleCov."
881
+ > _Note:_ In case you plan to report your coverage results to CodeClimate services, know that SimpleCov will automatically use the
882
+ > JSON formatter along with the HTML formatter when the `CC_TEST_REPORTER_ID` variable is present in the environment.
430
883
 
431
- #### [simplecov-csv](https://github.com/fguillen/simplecov-csv)
432
- *by Fernando Guillen*
884
+ ## Available formatters, editor integrations and hosted services
433
885
 
434
- CSV formatter for SimpleCov code coverage tool for ruby 1.9+
886
+ * [Open Source formatter and integration plugins for SimpleCov](doc/alternate-formatters.md)
887
+ * [Editor Integration](doc/editor-integration.md)
888
+ * [Hosted (commercial) services](doc/commercial-services.md)
435
889
 
890
+ ## Ruby version compatibility
436
891
 
892
+ SimpleCov is built in [Continuous Integration] on Ruby 2.7+ as well as JRuby 9.3+.
437
893
 
438
- ## Ruby version compatibility
894
+ Note for JRuby => You need to pass JRUBY_OPTS="--debug" or create .jrubyrc and add debug.fullTrace=true
439
895
 
440
- [![Build Status](https://secure.travis-ci.org/colszowka/simplecov.png)](http://travis-ci.org/colszowka/simplecov)
896
+ ## Want to find dead code in production?
441
897
 
442
- Only Ruby 1.9+ ships with the coverage library that SimpleCov depends upon. SimpleCov is built against various other Rubies,
443
- including Rubinius and JRuby, in [Continuous Integration], but this happens only to ensure that SimpleCov does not make your
444
- test suite crash right now. Whether SimpleCov will support JRuby/Rubinius in the future depends solely on whether those Ruby
445
- interpreters add the coverage library.
898
+ Try [Coverband](https://github.com/danmayer/coverband).
446
899
 
447
- SimpleCov is built in [Continuous Integration] on 1.8.7, ree, 1.9.2, 1.9.3.
900
+ ## Want to use Spring with SimpleCov?
448
901
 
902
+ If you're using [Spring](https://github.com/rails/spring) to speed up test suite runs and want to run SimpleCov along
903
+ with them, you'll find that it often misreports coverage with the default config due to some sort of eager loading
904
+ issue. Don't despair!
449
905
 
906
+ One solution is to [explicitly call eager
907
+ load](https://github.com/simplecov-ruby/simplecov/issues/381#issuecomment-347651728)
908
+ in your `test_helper.rb` / `spec_helper.rb` after calling `SimpleCov.start`.
450
909
 
451
- ## Contributions
910
+ ```ruby
911
+ require 'simplecov'
912
+ SimpleCov.start 'rails'
913
+ Rails.application.eager_load!
914
+ ```
452
915
 
453
- To fetch & test the library for development, do:
916
+ Alternatively, you could disable Spring while running SimpleCov:
454
917
 
455
- $ git clone https://github.com/colszowka/simplecov
456
- $ cd simplecov
457
- $ bundle
458
- $ bundle exec rake test && bundle exec cucumber features
918
+ ```
919
+ DISABLE_SPRING=1 rake test
920
+ ```
459
921
 
460
- If you wont to contribute, please:
922
+ Or you could remove `gem 'spring'` from your `Gemfile`.
461
923
 
462
- * Fork the project.
463
- * Make your feature addition or bug fix.
464
- * Add tests for it. This is important so I don't break it in a future version unintentionally.
465
- * Send me a pull request on Github.
924
+ ## Troubleshooting
466
925
 
467
- ## Kudos
926
+ The **most common problem is that simplecov isn't required and started before everything else**. In order to track
927
+ coverage for your whole application **simplecov needs to be the first one** so that it (and the underlying coverage
928
+ library) can subsequently track loaded files and their usage.
468
929
 
469
- Thanks to Aaron Patterson (http://engineering.attinteractive.com/2010/08/code-coverage-in-ruby-1-9/) for the original idea
470
- for this!
930
+ If you are missing coverage for some code a simple trick is to put a puts statement in there and right after
931
+ `SimpleCov.start` so you can see if the file really was loaded after simplecov was started.
932
+
933
+ ```ruby
934
+ # my_code.rb
935
+ class MyCode
936
+
937
+ puts "MyCode is being loaded!"
938
+
939
+ def my_method
940
+ # ...
941
+ end
942
+ end
943
+
944
+ # spec_helper.rb/rails_helper.rb/test_helper.rb/.simplecov whatever
945
+
946
+ SimpleCov.start
947
+ puts "SimpleCov started successfully!"
948
+ ```
949
+
950
+ Now when you run your test suite and you see:
951
+
952
+ ```
953
+ SimpleCov started successfully!
954
+ MyCode is being loaded!
955
+ ```
956
+
957
+ then it's good otherwise you likely have a problem :)
958
+
959
+ ## Code of Conduct
960
+
961
+ Everyone participating in this project's development, issue trackers and other channels is expected to follow our
962
+ [Code of Conduct](./CODE_OF_CONDUCT.md)
963
+
964
+ ## Contributing
965
+
966
+ See the [contributing guide](https://github.com/simplecov-ruby/simplecov/blob/main/CONTRIBUTING.md).
967
+
968
+ ## Kudos
471
969
 
970
+ Thanks to Aaron Patterson for the original idea for this!
472
971
 
473
972
  ## Copyright
474
973
 
475
- Copyright (c) 2010-2012 Christoph Olszowka. See LICENSE for details.
974
+ Copyright (c) 2010-2017 Christoph Olszowka. See MIT-LICENSE for details.