simplecov 0.6.0 → 0.22.0
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +7 -0
- data/CHANGELOG.md +163 -80
- data/LICENSE +1 -1
- data/README.md +776 -277
- data/doc/alternate-formatters.md +71 -0
- data/doc/commercial-services.md +25 -0
- data/doc/editor-integration.md +18 -0
- data/lib/minitest/simplecov_plugin.rb +15 -0
- data/lib/simplecov/combine/branches_combiner.rb +32 -0
- data/lib/simplecov/combine/files_combiner.rb +24 -0
- data/lib/simplecov/combine/lines_combiner.rb +43 -0
- data/lib/simplecov/combine/results_combiner.rb +60 -0
- data/lib/simplecov/combine.rb +30 -0
- data/lib/simplecov/command_guesser.rb +53 -38
- data/lib/simplecov/configuration.rb +478 -193
- data/lib/simplecov/coverage_statistics.rb +56 -0
- data/lib/simplecov/default_formatter.rb +20 -0
- data/lib/simplecov/defaults.rb +40 -44
- data/lib/simplecov/exit_codes/exit_code_handling.rb +29 -0
- data/lib/simplecov/exit_codes/maximum_coverage_drop_check.rb +83 -0
- data/lib/simplecov/exit_codes/minimum_coverage_by_file_check.rb +54 -0
- data/lib/simplecov/exit_codes/minimum_overall_coverage_check.rb +53 -0
- data/lib/simplecov/exit_codes.rb +15 -0
- data/lib/simplecov/file_list.rb +112 -36
- data/lib/simplecov/filter.rb +54 -4
- data/lib/simplecov/formatter/multi_formatter.rb +32 -0
- data/lib/simplecov/formatter/simple_formatter.rb +21 -15
- data/lib/simplecov/formatter.rb +4 -1
- data/lib/simplecov/last_run.rb +28 -0
- data/lib/simplecov/lines_classifier.rb +48 -0
- data/lib/simplecov/load_global_config.rb +8 -0
- data/lib/simplecov/no_defaults.rb +4 -0
- data/lib/simplecov/process.rb +19 -0
- data/lib/simplecov/profiles/bundler_filter.rb +5 -0
- data/lib/simplecov/profiles/hidden_filter.rb +5 -0
- data/lib/simplecov/profiles/rails.rb +18 -0
- data/lib/simplecov/profiles/root_filter.rb +10 -0
- data/lib/simplecov/profiles/test_frameworks.rb +8 -0
- data/lib/simplecov/profiles.rb +35 -0
- data/lib/simplecov/result.rb +33 -64
- data/lib/simplecov/result_adapter.rb +30 -0
- data/lib/simplecov/result_merger.rb +178 -64
- data/lib/simplecov/simulate_coverage.rb +29 -0
- data/lib/simplecov/source_file/branch.rb +84 -0
- data/lib/simplecov/source_file/line.rb +72 -0
- data/lib/simplecov/source_file.rb +304 -123
- data/lib/simplecov/useless_results_remover.rb +18 -0
- data/lib/simplecov/version.rb +4 -2
- data/lib/simplecov.rb +396 -49
- metadata +81 -242
- data/.gitignore +0 -30
- data/.rvmrc +0 -1
- data/.travis.yml +0 -13
- data/Gemfile +0 -9
- data/Rakefile +0 -16
- data/cucumber.yml +0 -13
- data/features/config_adapters.feature +0 -44
- data/features/config_autoload.feature +0 -46
- data/features/config_command_name.feature +0 -33
- data/features/config_coverage_dir.feature +0 -20
- data/features/config_deactivate_merging.feature +0 -42
- data/features/config_merge_timeout.feature +0 -38
- data/features/config_nocov_token.feature +0 -79
- data/features/config_project_name.feature +0 -27
- data/features/config_styles.feature +0 -93
- data/features/cucumber_basic.feature +0 -29
- data/features/merging_test_unit_and_rspec.feature +0 -44
- data/features/rspec_basic.feature +0 -31
- data/features/rspec_fails_on_initialization.feature +0 -14
- data/features/rspec_groups_and_filters_basic.feature +0 -29
- data/features/rspec_groups_and_filters_complex.feature +0 -35
- data/features/rspec_groups_using_filter_class.feature +0 -40
- data/features/rspec_without_simplecov.feature +0 -20
- data/features/skipping_code_blocks_manually.feature +0 -70
- data/features/step_definitions/html_steps.rb +0 -45
- data/features/step_definitions/simplecov_steps.rb +0 -61
- data/features/step_definitions/transformers.rb +0 -13
- data/features/step_definitions/web_steps.rb +0 -64
- data/features/support/env.rb +0 -26
- data/features/test_unit_basic.feature +0 -34
- data/features/test_unit_groups_and_filters_basic.feature +0 -29
- data/features/test_unit_groups_and_filters_complex.feature +0 -35
- data/features/test_unit_groups_using_filter_class.feature +0 -40
- data/features/test_unit_without_simplecov.feature +0 -20
- data/features/unicode_compatiblity.feature +0 -67
- data/lib/simplecov/adapters.rb +0 -29
- data/lib/simplecov/merge_helpers.rb +0 -39
- data/lib/simplecov/railtie.rb +0 -7
- data/lib/simplecov/railties/tasks.rake +0 -11
- data/simplecov.gemspec +0 -28
- data/test/faked_project/Gemfile +0 -6
- data/test/faked_project/Rakefile +0 -8
- data/test/faked_project/cucumber.yml +0 -13
- data/test/faked_project/features/step_definitions/my_steps.rb +0 -23
- data/test/faked_project/features/support/env.rb +0 -12
- data/test/faked_project/features/test_stuff.feature +0 -6
- data/test/faked_project/lib/faked_project/framework_specific.rb +0 -18
- data/test/faked_project/lib/faked_project/meta_magic.rb +0 -24
- data/test/faked_project/lib/faked_project/some_class.rb +0 -29
- data/test/faked_project/lib/faked_project.rb +0 -11
- data/test/faked_project/spec/faked_spec.rb +0 -11
- data/test/faked_project/spec/meta_magic_spec.rb +0 -10
- data/test/faked_project/spec/some_class_spec.rb +0 -10
- data/test/faked_project/spec/spec_helper.rb +0 -15
- data/test/faked_project/test/faked_test.rb +0 -11
- data/test/faked_project/test/meta_magic_test.rb +0 -13
- data/test/faked_project/test/some_class_test.rb +0 -15
- data/test/faked_project/test/test_helper.rb +0 -16
- data/test/fixtures/app/controllers/sample_controller.rb +0 -10
- data/test/fixtures/app/models/user.rb +0 -10
- data/test/fixtures/deleted_source_sample.rb +0 -15
- data/test/fixtures/frameworks/rspec_bad.rb +0 -9
- data/test/fixtures/frameworks/rspec_good.rb +0 -9
- data/test/fixtures/frameworks/testunit_bad.rb +0 -9
- data/test/fixtures/frameworks/testunit_good.rb +0 -9
- data/test/fixtures/resultset1.rb +0 -4
- data/test/fixtures/resultset2.rb +0 -5
- data/test/fixtures/sample.rb +0 -16
- data/test/fixtures/utf-8.rb +0 -3
- data/test/helper.rb +0 -35
- data/test/shoulda_macros.rb +0 -29
- data/test/test_1_8_fallbacks.rb +0 -33
- data/test/test_command_guesser.rb +0 -21
- data/test/test_deleted_source.rb +0 -16
- data/test/test_file_list.rb +0 -24
- data/test/test_filters.rb +0 -80
- data/test/test_merge_helpers.rb +0 -107
- data/test/test_result.rb +0 -147
- data/test/test_return_codes.rb +0 -39
- data/test/test_source_file.rb +0 -86
- data/test/test_source_file_line.rb +0 -110
data/README.md
CHANGED
@@ -1,154 +1,239 @@
|
|
1
|
-
SimpleCov [![Build Status](https://
|
1
|
+
SimpleCov [![Gem Version](https://badge.fury.io/rb/simplecov.svg)](https://badge.fury.io/rb/simplecov) [![Build Status](https://github.com/simplecov-ruby/simplecov/workflows/stable/badge.svg?branch=main)][Continuous Integration] [![Maintainability](https://api.codeclimate.com/v1/badges/c071d197d61953a7e482/maintainability)](https://codeclimate.com/github/simplecov-ruby/simplecov/maintainability) [![Inline docs](http://inch-ci.org/github/simplecov-ruby/simplecov.svg?branch=main)](http://inch-ci.org/github/simplecov-ruby/simplecov)
|
2
2
|
=========
|
3
|
-
|
3
|
+
|
4
|
+
**Code coverage for Ruby**
|
4
5
|
|
5
6
|
* [Source Code]
|
6
7
|
* [API documentation]
|
7
8
|
* [Changelog]
|
8
9
|
* [Rubygem]
|
9
10
|
* [Continuous Integration]
|
10
|
-
* IRC #simplecov on Freenode
|
11
11
|
|
12
|
-
[Coverage]:
|
13
|
-
[Source Code]: https://github.com/
|
12
|
+
[Coverage]: https://ruby-doc.org/stdlib/libdoc/coverage/rdoc/Coverage.html "API doc for Ruby's Coverage library"
|
13
|
+
[Source Code]: https://github.com/simplecov-ruby/simplecov "Source Code @ GitHub"
|
14
14
|
[API documentation]: http://rubydoc.info/gems/simplecov/frames "RDoc API Documentation at Rubydoc.info"
|
15
15
|
[Configuration]: http://rubydoc.info/gems/simplecov/SimpleCov/Configuration "Configuration options API documentation"
|
16
|
-
[Changelog]: https://github.com/
|
16
|
+
[Changelog]: https://github.com/simplecov-ruby/simplecov/blob/main/CHANGELOG.md "Project Changelog"
|
17
17
|
[Rubygem]: http://rubygems.org/gems/simplecov "SimpleCov @ rubygems.org"
|
18
|
-
[Continuous Integration]:
|
19
|
-
[
|
20
|
-
|
18
|
+
[Continuous Integration]: https://github.com/simplecov-ruby/simplecov/actions?query=workflow%3Astable "SimpleCov is built around the clock by github.com"
|
19
|
+
[Dependencies]: https://gemnasium.com/simplecov-ruby/simplecov "SimpleCov dependencies on Gemnasium"
|
20
|
+
[simplecov-html]: https://github.com/simplecov-ruby/simplecov-html "SimpleCov HTML Formatter Source Code @ GitHub"
|
21
21
|
|
22
|
-
SimpleCov is a code coverage analysis tool for Ruby
|
23
|
-
coverage data, but makes processing its results much easier by providing a clean API to filter, group, merge, format
|
24
|
-
and display those results,
|
22
|
+
SimpleCov is a code coverage analysis tool for Ruby. It uses [Ruby's built-in Coverage][Coverage] library to gather code
|
23
|
+
coverage data, but makes processing its results much easier by providing a clean API to filter, group, merge, format,
|
24
|
+
and display those results, giving you a complete code coverage suite that can be set up with just a couple lines of
|
25
25
|
code.
|
26
|
+
SimpleCov/Coverage track covered ruby code, gathering coverage for common templating solutions like erb, slim and haml is not supported.
|
26
27
|
|
27
|
-
In most cases, you'll want overall coverage results for your projects, including all types of tests,
|
28
|
-
etc. SimpleCov automatically takes care of this by caching and
|
28
|
+
In most cases, you'll want overall coverage results for your projects, including all types of tests, Cucumber features,
|
29
|
+
etc. SimpleCov automatically takes care of this by caching and merging results when generating reports, so your
|
29
30
|
report actually includes coverage across your test suites and thereby gives you a better picture of blank spots.
|
30
31
|
|
31
|
-
The official formatter of SimpleCov is packaged as a separate gem called [simplecov-html] but will be installed and
|
32
|
-
automatically when you launch SimpleCov. If you're curious, you can find it [on
|
32
|
+
The official formatter of SimpleCov is packaged as a separate gem called [simplecov-html], but will be installed and
|
33
|
+
configured automatically when you launch SimpleCov. If you're curious, you can find it [on GitHub, too][simplecov-html].
|
34
|
+
|
35
|
+
|
36
|
+
## Contact
|
33
37
|
|
38
|
+
*Code and Bug Reports*
|
34
39
|
|
40
|
+
* [Issue Tracker](https://github.com/simplecov-ruby/simplecov/issues)
|
41
|
+
* See [CONTRIBUTING](https://github.com/simplecov-ruby/simplecov/blob/main/CONTRIBUTING.md) for how to contribute along
|
42
|
+
with some common problems to check out before creating an issue.
|
43
|
+
|
44
|
+
*Questions, Problems, Suggestions, etc.*
|
45
|
+
|
46
|
+
* [Mailing List](https://groups.google.com/forum/#!forum/simplecov) "Open mailing list for discussion and announcements
|
47
|
+
on Google Groups"
|
35
48
|
|
36
49
|
Getting started
|
37
50
|
---------------
|
38
|
-
|
39
51
|
1. Add SimpleCov to your `Gemfile` and `bundle install`:
|
40
52
|
|
41
|
-
|
53
|
+
```ruby
|
54
|
+
gem 'simplecov', require: false, group: :test
|
55
|
+
```
|
56
|
+
2. Load and launch SimpleCov **at the very top** of your `test/test_helper.rb`
|
57
|
+
(*or `spec_helper.rb`, `rails_helper`, cucumber `env.rb`, or whatever your preferred test
|
58
|
+
framework uses*):
|
42
59
|
|
43
|
-
|
44
|
-
|
60
|
+
```ruby
|
61
|
+
require 'simplecov'
|
62
|
+
SimpleCov.start
|
63
|
+
|
64
|
+
# Previous content of test helper now starts here
|
65
|
+
```
|
66
|
+
|
67
|
+
**Note:** If SimpleCov starts after your application code is already loaded
|
68
|
+
(via `require`), it won't be able to track your files and their coverage!
|
69
|
+
The `SimpleCov.start` **must** be issued **before any of your application
|
70
|
+
code is required!**
|
71
|
+
|
72
|
+
This is especially true if you use anything that keeps your tests application loaded like spring, check out the **[spring section](#want-to-use-spring-with-simplecov)**.
|
73
|
+
|
74
|
+
SimpleCov must be running in the process that you want the code coverage
|
75
|
+
analysis to happen on. When testing a server process (e.g. a JSON API
|
76
|
+
endpoint) via a separate test process (e.g. when using Selenium) where you
|
77
|
+
want to see all code executed by the `rails server`, and not just code
|
78
|
+
executed in your actual test files, you need to require SimpleCov in the
|
79
|
+
server process. For rails for instance, you'll want to add something like this
|
80
|
+
to the top of `bin/rails`, but below the "shebang" line (`#! /usr/bin/env
|
81
|
+
ruby`) and after config/boot is required:
|
82
|
+
|
83
|
+
```ruby
|
84
|
+
if ENV['RAILS_ENV'] == 'test'
|
85
|
+
require 'simplecov'
|
86
|
+
SimpleCov.start 'rails'
|
87
|
+
puts "required simplecov"
|
88
|
+
end
|
89
|
+
```
|
45
90
|
|
46
|
-
|
47
|
-
|
91
|
+
3. Run your full test suite to see the percent coverage that your application has.
|
92
|
+
4. After running your tests, open `coverage/index.html` in the browser of your choice. For example, in a Mac Terminal,
|
93
|
+
run the following command from your application's root directory:
|
48
94
|
|
49
|
-
|
95
|
+
```
|
96
|
+
open coverage/index.html
|
97
|
+
```
|
98
|
+
in a debian/ubuntu Terminal,
|
50
99
|
|
51
|
-
|
52
|
-
|
100
|
+
```
|
101
|
+
xdg-open coverage/index.html
|
102
|
+
```
|
53
103
|
|
54
|
-
|
104
|
+
**Note:** [This guide](https://dwheeler.com/essays/open-files-urls.html) can help if you're unsure which command your particular
|
105
|
+
operating system requires.
|
55
106
|
|
56
|
-
|
107
|
+
5. Add the following to your `.gitignore` file to ensure that coverage results
|
108
|
+
are not tracked by Git (optional):
|
57
109
|
|
58
|
-
|
110
|
+
```
|
111
|
+
echo coverage >> .gitignore
|
112
|
+
```
|
59
113
|
|
60
|
-
If you're making a Rails application, SimpleCov comes with
|
61
|
-
|
62
|
-
your test_helper should be like this:
|
114
|
+
If you're making a Rails application, SimpleCov comes with built-in configurations (see below for information on
|
115
|
+
profiles) that will get you started with groups for your Controllers, Models and Helpers. To use it, the
|
116
|
+
first two lines of your test_helper should be like this:
|
63
117
|
|
64
|
-
|
65
|
-
|
118
|
+
```ruby
|
119
|
+
require 'simplecov'
|
120
|
+
SimpleCov.start 'rails'
|
121
|
+
```
|
66
122
|
|
67
123
|
## Example output
|
68
124
|
|
69
125
|
**Coverage results report, fully browsable locally with sorting and much more:**
|
70
126
|
|
71
|
-
![SimpleCov coverage report](
|
127
|
+
![SimpleCov coverage report](https://cloud.githubusercontent.com/assets/137793/17071162/db6f253e-502d-11e6-9d84-e40c3d75f333.png)
|
72
128
|
|
73
129
|
|
74
130
|
**Source file coverage details view:**
|
75
131
|
|
76
|
-
![SimpleCov source file detail view](
|
77
|
-
|
78
|
-
|
132
|
+
![SimpleCov source file detail view](https://cloud.githubusercontent.com/assets/137793/17071163/db6f9f0a-502d-11e6-816c-edb2c66fad8d.png)
|
79
133
|
|
80
134
|
## Use it with any framework!
|
81
135
|
|
82
|
-
|
136
|
+
Similarly to the usage with Test::Unit described above, the only thing you have to do is to add the SimpleCov
|
83
137
|
config to the very top of your Cucumber/RSpec/whatever setup file.
|
84
138
|
|
85
139
|
Add the setup code to the **top** of `features/support/env.rb` (for Cucumber) or `spec/spec_helper.rb` (for RSpec).
|
86
140
|
Other test frameworks should work accordingly, whatever their setup file may be:
|
87
141
|
|
88
|
-
|
89
|
-
|
142
|
+
```ruby
|
143
|
+
require 'simplecov'
|
144
|
+
SimpleCov.start 'rails'
|
145
|
+
```
|
90
146
|
|
91
|
-
You could even track what kind of code your UI testers are touching if you want to go overboard with things. SimpleCov
|
92
|
-
care what kind of framework it is running in
|
147
|
+
You could even track what kind of code your UI testers are touching if you want to go overboard with things. SimpleCov
|
148
|
+
does not care what kind of framework it is running in; it just looks at what code is being executed and generates a
|
149
|
+
report about it.
|
93
150
|
|
94
151
|
### Notes on specific frameworks and test utilities
|
95
152
|
|
96
|
-
For some frameworks and testing tools there are quirks and problems you might want to know about if you want
|
153
|
+
For some frameworks and testing tools there are quirks and problems you might want to know about if you want
|
97
154
|
to use SimpleCov with them. Here's an overview of the known ones:
|
98
155
|
|
99
156
|
<table>
|
100
|
-
<tr><th>Framework</th><th>Notes</th><th>Issue
|
101
|
-
<tr>
|
102
|
-
|
103
|
-
|
104
|
-
|
105
|
-
|
106
|
-
|
107
|
-
|
108
|
-
|
109
|
-
|
110
|
-
|
111
|
-
|
112
|
-
</
|
113
|
-
</
|
114
|
-
|
115
|
-
|
116
|
-
|
117
|
-
|
118
|
-
|
119
|
-
|
120
|
-
|
121
|
-
|
122
|
-
|
123
|
-
|
124
|
-
|
125
|
-
|
126
|
-
|
127
|
-
|
128
|
-
<
|
129
|
-
|
130
|
-
|
131
|
-
|
132
|
-
<
|
133
|
-
|
134
|
-
|
135
|
-
|
136
|
-
|
137
|
-
|
138
|
-
|
139
|
-
</tr>
|
140
|
-
<tr>
|
141
|
-
|
142
|
-
|
143
|
-
|
144
|
-
|
145
|
-
|
146
|
-
|
147
|
-
|
148
|
-
|
149
|
-
|
150
|
-
|
151
|
-
|
157
|
+
<tr><th>Framework</th><th>Notes</th><th>Issue</th></tr>
|
158
|
+
<tr>
|
159
|
+
<th>
|
160
|
+
parallel_tests
|
161
|
+
</th>
|
162
|
+
<td>
|
163
|
+
As of 0.8.0, SimpleCov should correctly recognize parallel_tests and
|
164
|
+
supplement your test suite names with their corresponding test env
|
165
|
+
numbers. SimpleCov locks the resultset cache while merging, ensuring no
|
166
|
+
race conditions occur when results are merged.
|
167
|
+
</td>
|
168
|
+
<td>
|
169
|
+
<a href="https://github.com/simplecov-ruby/simplecov/issues/64">#64</a> &
|
170
|
+
<a href="https://github.com/simplecov-ruby/simplecov/pull/185">#185</a>
|
171
|
+
</td>
|
172
|
+
</tr>
|
173
|
+
<tr>
|
174
|
+
<th>
|
175
|
+
knapsack_pro
|
176
|
+
</th>
|
177
|
+
<td>
|
178
|
+
To make SimpleCov work with Knapsack Pro Queue Mode to split tests in parallel on CI jobs you need to provide CI node index number to the <code>SimpleCov.command_name</code> in <code>KnapsackPro::Hooks::Queue.before_queue</code> hook.
|
179
|
+
</td>
|
180
|
+
<td>
|
181
|
+
<a href="https://knapsackpro.com/faq/question/how-to-use-simplecov-in-queue-mode">Tip</a>
|
182
|
+
</td>
|
183
|
+
</tr>
|
184
|
+
<tr>
|
185
|
+
<th>
|
186
|
+
RubyMine
|
187
|
+
</th>
|
188
|
+
<td>
|
189
|
+
The <a href="https://www.jetbrains.com/ruby/">RubyMine IDE</a> has
|
190
|
+
built-in support for SimpleCov's coverage reports, though you might need
|
191
|
+
to explicitly set the output root using `SimpleCov.root('foo/bar/baz')`
|
192
|
+
</td>
|
193
|
+
<td>
|
194
|
+
<a href="https://github.com/simplecov-ruby/simplecov/issues/95">#95</a>
|
195
|
+
</td>
|
196
|
+
</tr>
|
197
|
+
<tr>
|
198
|
+
<th>
|
199
|
+
Spork
|
200
|
+
</th>
|
201
|
+
<td>
|
202
|
+
Because of how Spork works internally (using preforking), there used to
|
203
|
+
be trouble when using SimpleCov with it, but that has apparently been
|
204
|
+
resolved with a specific configuration strategy. See <a
|
205
|
+
href="https://github.com/simplecov-ruby/simplecov/issues/42#issuecomment-4440284">this</a>
|
206
|
+
comment.
|
207
|
+
</td>
|
208
|
+
<td>
|
209
|
+
<a href="https://github.com/simplecov-ruby/simplecov/issues/42#issuecomment-4440284">#42</a>
|
210
|
+
</td>
|
211
|
+
</tr>
|
212
|
+
<tr>
|
213
|
+
<th>
|
214
|
+
Spring
|
215
|
+
</th>
|
216
|
+
<td>
|
217
|
+
<a href="#want-to-use-spring-with-simplecov">See section below.</a>
|
218
|
+
</td>
|
219
|
+
<td>
|
220
|
+
<a href="https://github.com/simplecov-ruby/simplecov/issues/381">#381</a>
|
221
|
+
</td>
|
222
|
+
</tr>
|
223
|
+
<tr>
|
224
|
+
<th>
|
225
|
+
Test/Unit
|
226
|
+
</th>
|
227
|
+
<td>
|
228
|
+
Test Unit 2 used to mess with ARGV, leading to a failure to detect the
|
229
|
+
test process name in SimpleCov. <code>test-unit</code> releases 2.4.3+
|
230
|
+
(Dec 11th, 2011) should have this problem resolved.
|
231
|
+
</td>
|
232
|
+
<td>
|
233
|
+
<a href="https://github.com/simplecov-ruby/simplecov/issues/45">#45</a> &
|
234
|
+
<a href="https://github.com/test-unit/test-unit/pull/12">test-unit/test-unit#12</a>
|
235
|
+
</td>
|
236
|
+
</tr>
|
152
237
|
</table>
|
153
238
|
|
154
239
|
## Configuring SimpleCov
|
@@ -157,319 +242,733 @@ to use SimpleCov with them. Here's an overview of the known ones:
|
|
157
242
|
|
158
243
|
* The most common way is to configure it directly in your start block:
|
159
244
|
|
160
|
-
|
161
|
-
|
162
|
-
|
163
|
-
|
245
|
+
```ruby
|
246
|
+
SimpleCov.start do
|
247
|
+
some_config_option 'foo'
|
248
|
+
end
|
249
|
+
```
|
164
250
|
* You can also set all configuration options directly:
|
165
251
|
|
166
|
-
|
167
|
-
|
168
|
-
|
252
|
+
```ruby
|
253
|
+
SimpleCov.some_config_option 'foo'
|
254
|
+
```
|
255
|
+
* If you do not want to start coverage immediately after launch or want to add additional configuration later on in a
|
256
|
+
concise way, use:
|
169
257
|
|
170
|
-
|
171
|
-
|
172
|
-
|
258
|
+
```ruby
|
259
|
+
SimpleCov.configure do
|
260
|
+
some_config_option 'foo'
|
261
|
+
end
|
262
|
+
```
|
173
263
|
|
174
264
|
Please check out the [Configuration] API documentation to find out what you can customize.
|
175
265
|
|
176
|
-
|
177
266
|
## Using .simplecov for centralized config
|
178
267
|
|
179
|
-
If you use SimpleCov to merge multiple test suite results (
|
180
|
-
set up all your config options twice, once in `test_helper.rb` and once in `env.rb`.
|
268
|
+
If you use SimpleCov to merge multiple test suite results (e.g. Test/Unit and Cucumber) into a single report, you'd
|
269
|
+
normally have to set up all your config options twice, once in `test_helper.rb` and once in `env.rb`.
|
181
270
|
|
182
|
-
To avoid this, you can place a file called `.simplecov` in your project root. You can then just leave the
|
183
|
-
test setup helper and move the `SimpleCov.start` code with all your
|
271
|
+
To avoid this, you can place a file called `.simplecov` in your project root. You can then just leave the
|
272
|
+
`require 'simplecov'` in each test setup helper (**at the top**) and move the `SimpleCov.start` code with all your
|
273
|
+
custom config options into `.simplecov`:
|
184
274
|
|
185
|
-
|
186
|
-
|
275
|
+
```ruby
|
276
|
+
# test/test_helper.rb
|
277
|
+
require 'simplecov'
|
187
278
|
|
188
|
-
|
189
|
-
|
279
|
+
# features/support/env.rb
|
280
|
+
require 'simplecov'
|
190
281
|
|
191
|
-
|
192
|
-
|
193
|
-
|
194
|
-
|
282
|
+
# .simplecov
|
283
|
+
SimpleCov.start 'rails' do
|
284
|
+
# any custom configs like groups and filters can be here at a central place
|
285
|
+
end
|
286
|
+
```
|
287
|
+
|
288
|
+
Using `.simplecov` rather than separately requiring SimpleCov multiple times is recommended if you are merging multiple
|
289
|
+
test frameworks like Cucumber and RSpec that rely on each other, as invoking SimpleCov multiple times can cause coverage
|
290
|
+
information to be lost.
|
291
|
+
|
292
|
+
## Branch coverage (ruby "~> 2.5")
|
293
|
+
Add branch coverage measurement statistics to your results. Supported in CRuby versions 2.5+.
|
294
|
+
|
295
|
+
```ruby
|
296
|
+
SimpleCov.start do
|
297
|
+
enable_coverage :branch
|
298
|
+
end
|
299
|
+
```
|
300
|
+
|
301
|
+
Branch coverage is a feature introduced in Ruby 2.5 concerning itself with whether a
|
302
|
+
particular branch of a condition had been executed. Line coverage on the other hand
|
303
|
+
is only interested in whether a line of code has been executed.
|
304
|
+
|
305
|
+
This comes in handy for instance for one line conditionals:
|
306
|
+
|
307
|
+
```ruby
|
308
|
+
number.odd? ? "odd" : "even"
|
309
|
+
```
|
310
|
+
|
311
|
+
In line coverage this line would always be marked as executed but you'd never know if both
|
312
|
+
conditions were met. Guard clauses have a similar story:
|
313
|
+
|
314
|
+
```ruby
|
315
|
+
return if number.odd?
|
195
316
|
|
317
|
+
# more code
|
318
|
+
```
|
319
|
+
|
320
|
+
If all the code in that method was covered you'd never know if the guard clause was ever
|
321
|
+
triggered! With line coverage as just evaluating the condition marks it as covered.
|
322
|
+
|
323
|
+
In the HTML report the lines of code will be annotated like `branch_type: hit_count`:
|
324
|
+
|
325
|
+
* `then: 2` - the then branch (of an `if`) was executed twice
|
326
|
+
* `else: 0` - the else branch (of an `if` or `case`) was never executed
|
327
|
+
|
328
|
+
Not that even if you don't declare an `else` branch it will still show up in the coverage
|
329
|
+
reports meaning that the condition of the `if` was not hit or that no `when` of `case`
|
330
|
+
was hit during the test runs.
|
331
|
+
|
332
|
+
**Is branch coverage strictly better?** No. Branch coverage really only concerns itself with
|
333
|
+
conditionals - meaning coverage of sequential code is of no interest to it. A file without
|
334
|
+
conditional logic will have no branch coverage data and SimpleCov will report 0 of 0
|
335
|
+
branches covered as 100% (as everything that can be covered was covered).
|
336
|
+
|
337
|
+
Hence, we recommend looking at both metrics together. Branch coverage might also be a good
|
338
|
+
overall metric to look at - while you might be missing only 10% of your lines that might
|
339
|
+
account for 50% of your branches for instance.
|
340
|
+
|
341
|
+
## Primary Coverage
|
342
|
+
|
343
|
+
By default, the primary coverage type is `line`. To set the primary coverage to something else, use the following:
|
344
|
+
|
345
|
+
```ruby
|
346
|
+
# or in configure SimpleCov.primary_coverage :branch
|
347
|
+
SimpleCov.start do
|
348
|
+
enable_coverage :branch
|
349
|
+
primary_coverage :branch
|
350
|
+
end
|
351
|
+
```
|
352
|
+
|
353
|
+
Primary coverage determines what will come in first all output, and the type of coverage to check if you don't specify the type of coverage when customizing exit behavior (`SimpleCov.minimum_coverage 90`).
|
354
|
+
|
355
|
+
Note that coverage must first be enabled for non-default coverage types.
|
356
|
+
|
357
|
+
## Coverage for eval
|
358
|
+
|
359
|
+
You can measure coverage for code that is evaluated by `Kernel#eval`. Supported in CRuby versions 3.2+.
|
360
|
+
|
361
|
+
```ruby
|
362
|
+
SimpleCov.start do
|
363
|
+
enable_coverage_for_eval
|
364
|
+
end
|
365
|
+
```
|
366
|
+
|
367
|
+
This is typically useful for ERB. Set `ERB#filename=` to make it possible for SimpleCov to trace the original .erb source file.
|
196
368
|
|
197
369
|
## Filters
|
198
370
|
|
199
|
-
Filters can be used to remove selected files from your coverage data. By default, a filter is applied that removes all
|
200
|
-
OUTSIDE of your project's root directory - otherwise you'd end up with
|
201
|
-
gems you are using.
|
371
|
+
Filters can be used to remove selected files from your coverage data. By default, a filter is applied that removes all
|
372
|
+
files OUTSIDE of your project's root directory - otherwise you'd end up with billions of coverage reports for source
|
373
|
+
files in the gems you are using.
|
202
374
|
|
203
|
-
|
375
|
+
You can define your own to remove things like configuration files, tests or whatever you don't need in your coverage
|
204
376
|
report.
|
205
377
|
|
206
378
|
### Defining custom filters
|
207
379
|
|
208
|
-
You can currently define a filter using either a String (that will then be Regexp-matched against each source
|
209
|
-
a block or by passing in your own Filter class.
|
380
|
+
You can currently define a filter using either a String or Regexp (that will then be Regexp-matched against each source
|
381
|
+
file's path), a block or by passing in your own Filter class.
|
210
382
|
|
211
383
|
#### String filter
|
212
384
|
|
213
|
-
|
214
|
-
|
215
|
-
|
385
|
+
```ruby
|
386
|
+
SimpleCov.start do
|
387
|
+
add_filter "/test/"
|
388
|
+
end
|
389
|
+
```
|
216
390
|
|
217
391
|
This simple string filter will remove all files that match "/test/" in their path.
|
218
392
|
|
393
|
+
#### Regex filter
|
394
|
+
|
395
|
+
```ruby
|
396
|
+
SimpleCov.start do
|
397
|
+
add_filter %r{^/test/}
|
398
|
+
end
|
399
|
+
```
|
400
|
+
|
401
|
+
This simple regex filter will remove all files that start with /test/ in their path.
|
402
|
+
|
219
403
|
#### Block filter
|
220
404
|
|
221
|
-
|
222
|
-
|
223
|
-
|
224
|
-
|
225
|
-
|
405
|
+
```ruby
|
406
|
+
SimpleCov.start do
|
407
|
+
add_filter do |source_file|
|
408
|
+
source_file.lines.count < 5
|
409
|
+
end
|
410
|
+
end
|
411
|
+
```
|
226
412
|
|
227
|
-
Block filters receive a SimpleCov::SourceFile instance and expect your block to return either true (if the file is to be
|
228
|
-
from the result) or false (if the result should be kept). Please check out the RDoc for SimpleCov::SourceFile to
|
229
|
-
methods available to you. In the above example, the filter will remove all files that have less
|
413
|
+
Block filters receive a SimpleCov::SourceFile instance and expect your block to return either true (if the file is to be
|
414
|
+
removed from the result) or false (if the result should be kept). Please check out the RDoc for SimpleCov::SourceFile to
|
415
|
+
learn about the methods available to you. In the above example, the filter will remove all files that have less than 5
|
416
|
+
lines of code.
|
230
417
|
|
231
418
|
#### Custom filter class
|
232
419
|
|
233
|
-
|
234
|
-
|
235
|
-
|
236
|
-
|
237
|
-
|
420
|
+
```ruby
|
421
|
+
class LineFilter < SimpleCov::Filter
|
422
|
+
def matches?(source_file)
|
423
|
+
source_file.lines.count < filter_argument
|
424
|
+
end
|
425
|
+
end
|
426
|
+
|
427
|
+
SimpleCov.add_filter LineFilter.new(5)
|
428
|
+
```
|
429
|
+
|
430
|
+
Defining your own filters is pretty easy: Just inherit from SimpleCov::Filter and define a method
|
431
|
+
'matches?(source_file)'. When running the filter, a true return value from this method will result in the removal of the
|
432
|
+
given source_file. The filter_argument method is being set in the SimpleCov::Filter initialize method and thus is set to
|
433
|
+
5 in this example.
|
434
|
+
|
435
|
+
#### Array filter
|
436
|
+
|
437
|
+
```ruby
|
438
|
+
SimpleCov.start do
|
439
|
+
proc = Proc.new { |source_file| false }
|
440
|
+
add_filter ["string", /regex/, proc, LineFilter.new(5)]
|
441
|
+
end
|
442
|
+
```
|
443
|
+
|
444
|
+
You can pass in an array containing any of the other filter types.
|
445
|
+
|
446
|
+
#### Ignoring/skipping code
|
447
|
+
|
448
|
+
You can exclude code from the coverage report by wrapping it in `# :nocov:`.
|
238
449
|
|
239
|
-
|
450
|
+
```ruby
|
451
|
+
# :nocov:
|
452
|
+
def skip_this_method
|
453
|
+
never_reached
|
454
|
+
end
|
455
|
+
# :nocov:
|
456
|
+
```
|
240
457
|
|
241
|
-
|
242
|
-
the filter, a true return value from this method will result in the removal of the given source_file. The filter_argument method
|
243
|
-
is being set in the SimpleCov::Filter initialize method and thus is set to 5 in this example.
|
458
|
+
The name of the token can be changed to your liking. [Learn more about the nocov feature.]( https://github.com/simplecov-ruby/simplecov/blob/main/features/config_nocov_token.feature)
|
244
459
|
|
460
|
+
**Note:** You shouldn't have to use the nocov token to skip private methods that are being included in your coverage. If
|
461
|
+
you appropriately test the public interface of your classes and objects you should automatically get full coverage of
|
462
|
+
your private methods.
|
463
|
+
|
464
|
+
## Default root filter and coverage for things outside of it
|
465
|
+
|
466
|
+
By default, SimpleCov filters everything outside of the `SimpleCov.root` directory. However, sometimes you may want
|
467
|
+
to include coverage reports for things you include as a gem, for example a Rails Engine.
|
468
|
+
|
469
|
+
Here's an example by [@lsaffie](https://github.com/lsaffie) from [#221](https://github.com/simplecov-ruby/simplecov/issues/221)
|
470
|
+
that shows how you can achieve just that:
|
471
|
+
|
472
|
+
```ruby
|
473
|
+
SimpleCov.start :rails do
|
474
|
+
filters.clear # This will remove the :root_filter and :bundler_filter that come via simplecov's defaults
|
475
|
+
add_filter do |src|
|
476
|
+
!(src.filename =~ /^#{SimpleCov.root}/) unless src.filename =~ /my_engine/
|
477
|
+
end
|
478
|
+
end
|
479
|
+
```
|
245
480
|
|
246
481
|
## Groups
|
247
482
|
|
248
|
-
You can separate your source files into groups. For example, in a
|
249
|
-
Models, Controllers, Helpers,
|
250
|
-
filter classes), but source files end up in a group when the filter passes (returns true), as opposed to filtering
|
251
|
-
which exclude files from results when the filter results in a true value.
|
483
|
+
You can separate your source files into groups. For example, in a Rails app, you'll want to have separate listings for
|
484
|
+
Models, Controllers, Helpers, and Libs. Group definition works similarly to Filters (and also accepts custom
|
485
|
+
filter classes), but source files end up in a group when the filter passes (returns true), as opposed to filtering
|
486
|
+
results, which exclude files from results when the filter results in a true value.
|
252
487
|
|
253
488
|
Add your groups with:
|
254
489
|
|
255
|
-
|
256
|
-
|
257
|
-
|
258
|
-
|
259
|
-
|
260
|
-
|
261
|
-
|
262
|
-
|
490
|
+
```ruby
|
491
|
+
SimpleCov.start do
|
492
|
+
add_group "Models", "app/models"
|
493
|
+
add_group "Controllers", "app/controllers"
|
494
|
+
add_group "Long files" do |src_file|
|
495
|
+
src_file.lines.count > 100
|
496
|
+
end
|
497
|
+
add_group "Multiple Files", ["app/models", "app/controllers"] # You can also pass in an array
|
498
|
+
add_group "Short files", LineFilter.new(5) # Using the LineFilter class defined in Filters section above
|
499
|
+
end
|
500
|
+
```
|
263
501
|
|
264
502
|
## Merging results
|
265
503
|
|
266
|
-
|
504
|
+
You normally want to have your coverage analyzed across ALL of your test suites, right?
|
267
505
|
|
268
|
-
Simplecov automatically caches coverage results in your
|
269
|
-
|
270
|
-
|
271
|
-
|
272
|
-
|
273
|
-
There are two things to note here though:
|
506
|
+
Simplecov automatically caches coverage results in your
|
507
|
+
(coverage_path)/.resultset.json, and will merge or override those with
|
508
|
+
subsequent runs, depending on whether simplecov considers those subsequent runs
|
509
|
+
as different test suites or as the same test suite as the cached results. To
|
510
|
+
make this distinction, simplecov has the concept of "test suite names".
|
274
511
|
|
275
512
|
### Test suite names
|
276
513
|
|
277
|
-
|
278
|
-
on. This should work fine for Unit Tests, RSpec and Cucumber. If it fails, it will use the shell
|
279
|
-
that invoked the test suite as a command name.
|
514
|
+
SimpleCov tries to guess the name of the currently running test suite based upon the shell command the tests
|
515
|
+
are running on. This should work fine for Unit Tests, RSpec, and Cucumber. If it fails, it will use the shell
|
516
|
+
command that invoked the test suite as a command name.
|
517
|
+
|
518
|
+
If you have some non-standard setup and still want nicely labeled test suites, you have to give Simplecov a
|
519
|
+
cue as to what the name of the currently running test suite is. You can do so by specifying
|
520
|
+
`SimpleCov.command_name` in one test file that is part of your specific suite.
|
280
521
|
|
281
|
-
|
282
|
-
|
283
|
-
|
522
|
+
To customize the suite names on a Rails app (yeah, sorry for being Rails-biased, but everyone knows what
|
523
|
+
the structure of those projects is. You can apply this accordingly to the RSpecs in your
|
524
|
+
Outlook-WebDAV-Calendar-Sync gem), you could do something like this:
|
284
525
|
|
285
|
-
|
286
|
-
|
287
|
-
|
526
|
+
```ruby
|
527
|
+
# test/unit/some_test.rb
|
528
|
+
SimpleCov.command_name 'test:units'
|
288
529
|
|
289
|
-
|
290
|
-
|
530
|
+
# test/functionals/some_controller_test.rb
|
531
|
+
SimpleCov.command_name "test:functionals"
|
291
532
|
|
292
|
-
|
293
|
-
|
533
|
+
# test/integration/some_integration_test.rb
|
534
|
+
SimpleCov.command_name "test:integration"
|
294
535
|
|
295
|
-
|
296
|
-
|
536
|
+
# features/support/env.rb
|
537
|
+
SimpleCov.command_name "features"
|
538
|
+
```
|
297
539
|
|
298
|
-
|
299
|
-
|
540
|
+
Note that this only has to be invoked ONCE PER TEST SUITE, so even if you have 200 unit test files,
|
541
|
+
specifying it in `some_test.rb` is enough.
|
300
542
|
|
301
|
-
|
302
|
-
|
543
|
+
Last but not least **if multiple suites resolve to the same `command_name`** be aware that the coverage results **will
|
544
|
+
clobber each other instead of being merged**. SimpleCov is smart enough to detect unique names for the most common
|
545
|
+
setups, but if you have more than one test suite that doesn't follow a common pattern then you will want to manually
|
546
|
+
ensure that each suite gets a unique `command_name`.
|
547
|
+
|
548
|
+
If you are running tests in parallel each process has the potential to clobber results from the other test processes.
|
549
|
+
If you are relying on the default `command_name` then SimpleCov will attempt to detect and avoid parallel test suite
|
550
|
+
`command_name` collisions based on the presence of `ENV['PARALLEL_TEST_GROUPS']` and `ENV['TEST_ENV_NUMBER']`. If your
|
551
|
+
parallel test runner does not set one or both of these then *you must* set a `command_name` and ensure that it is unique
|
552
|
+
per process (eg. `command_name "Unit Tests PID #{$$}"`).
|
553
|
+
|
554
|
+
If you are using parallel_tests, you must incorporate `TEST_ENV_NUMBER` into the command name yourself, in
|
555
|
+
order for SimpleCov to merge the results correctly. For example:
|
556
|
+
|
557
|
+
```ruby
|
558
|
+
# spec/spec_helper.rb
|
559
|
+
SimpleCov.command_name "features" + (ENV['TEST_ENV_NUMBER'] || '')
|
560
|
+
```
|
303
561
|
|
304
562
|
[simplecov-html] prints the used test suites in the footer of the generated coverage report.
|
305
563
|
|
306
|
-
### Timeout for merge
|
307
564
|
|
308
|
-
|
309
|
-
|
310
|
-
|
311
|
-
|
565
|
+
### Merging test runs under the same execution environment
|
566
|
+
|
567
|
+
Test results are automatically merged with previous runs in the same execution
|
568
|
+
environment when generating the result, so when coverage is set up properly for
|
569
|
+
Cucumber and your unit / functional / integration tests, all of those test
|
570
|
+
suites will be taken into account when building the coverage report.
|
571
|
+
|
572
|
+
#### Timeout for merge
|
573
|
+
|
574
|
+
Of course, your cached coverage data is likely to become invalid at some point. Thus, when automatically merging
|
575
|
+
subsequent test runs, result sets that are older than `SimpleCov.merge_timeout` will not be used any more. By default,
|
576
|
+
the timeout is 600 seconds (10 minutes), and you can raise (or lower) it by specifying `SimpleCov.merge_timeout 3600`
|
577
|
+
(1 hour), or, inside a configure/start block, with just `merge_timeout 3600`.
|
578
|
+
|
579
|
+
You can deactivate this automatic merging altogether with `SimpleCov.use_merging false`.
|
312
580
|
|
313
|
-
|
581
|
+
### Merging test runs under different execution environments
|
314
582
|
|
583
|
+
If your tests are done in parallel across multiple build machines, you can fetch them all and merge them into a single
|
584
|
+
result set using the `SimpleCov.collate` method. This can be added to a Rakefile or script file, having downloaded a set of
|
585
|
+
`.resultset.json` files from each parallel test run.
|
586
|
+
|
587
|
+
```ruby
|
588
|
+
# lib/tasks/coverage_report.rake
|
589
|
+
namespace :coverage do
|
590
|
+
desc "Collates all result sets generated by the different test runners"
|
591
|
+
task :report do
|
592
|
+
require 'simplecov'
|
593
|
+
|
594
|
+
SimpleCov.collate Dir["simplecov-resultset-*/.resultset.json"]
|
595
|
+
end
|
596
|
+
end
|
597
|
+
```
|
598
|
+
|
599
|
+
`SimpleCov.collate` also takes an optional simplecov profile and an optional
|
600
|
+
block for configuration, just the same as `SimpleCov.start` or
|
601
|
+
`SimpleCov.configure`. This means you can configure a separate formatter for
|
602
|
+
the collated output. For instance, you can make the formatter in
|
603
|
+
`SimpleCov.start` the `SimpleCov::Formatter::SimpleFormatter`, and only use more
|
604
|
+
complex formatters in the final `SimpleCov.collate` run.
|
605
|
+
|
606
|
+
```ruby
|
607
|
+
# spec/spec_helper.rb
|
608
|
+
require 'simplecov'
|
609
|
+
|
610
|
+
SimpleCov.start 'rails' do
|
611
|
+
# Disambiguates individual test runs
|
612
|
+
command_name "Job #{ENV["TEST_ENV_NUMBER"]}" if ENV["TEST_ENV_NUMBER"]
|
613
|
+
|
614
|
+
if ENV['CI']
|
615
|
+
formatter SimpleCov::Formatter::SimpleFormatter
|
616
|
+
else
|
617
|
+
formatter SimpleCov::Formatter::MultiFormatter.new([
|
618
|
+
SimpleCov::Formatter::SimpleFormatter,
|
619
|
+
SimpleCov::Formatter::HTMLFormatter
|
620
|
+
])
|
621
|
+
end
|
622
|
+
|
623
|
+
track_files "**/*.rb"
|
624
|
+
end
|
625
|
+
```
|
626
|
+
|
627
|
+
```ruby
|
628
|
+
# lib/tasks/coverage_report.rake
|
629
|
+
namespace :coverage do
|
630
|
+
task :report do
|
631
|
+
require 'simplecov'
|
632
|
+
|
633
|
+
SimpleCov.collate Dir["simplecov-resultset-*/.resultset.json"], 'rails' do
|
634
|
+
formatter SimpleCov::Formatter::MultiFormatter.new([
|
635
|
+
SimpleCov::Formatter::SimpleFormatter,
|
636
|
+
SimpleCov::Formatter::HTMLFormatter
|
637
|
+
])
|
638
|
+
end
|
639
|
+
end
|
640
|
+
end
|
641
|
+
```
|
642
|
+
|
643
|
+
## Running simplecov against subprocesses
|
644
|
+
|
645
|
+
`SimpleCov.enable_for_subprocesses` will allow SimpleCov to observe subprocesses starting using `Process.fork`.
|
646
|
+
This modifies ruby's core Process.fork method so that SimpleCov can see into it, appending `" (subprocess #{pid})"`
|
647
|
+
to the `SimpleCov.command_name`, with results that can be merged together using SimpleCov's merging feature.
|
648
|
+
|
649
|
+
To configure this, use `.at_fork`.
|
650
|
+
|
651
|
+
```ruby
|
652
|
+
SimpleCov.enable_for_subprocesses true
|
653
|
+
SimpleCov.at_fork do |pid|
|
654
|
+
# This needs a unique name so it won't be ovewritten
|
655
|
+
SimpleCov.command_name "#{SimpleCov.command_name} (subprocess: #{pid})"
|
656
|
+
# be quiet, the parent process will be in charge of output and checking coverage totals
|
657
|
+
SimpleCov.print_error_status = false
|
658
|
+
SimpleCov.formatter SimpleCov::Formatter::SimpleFormatter
|
659
|
+
SimpleCov.minimum_coverage 0
|
660
|
+
# start
|
661
|
+
SimpleCov.start
|
662
|
+
end
|
663
|
+
```
|
664
|
+
|
665
|
+
NOTE: SimpleCov must have already been started before `Process.fork` was called.
|
666
|
+
|
667
|
+
### Running simplecov against spawned subprocesses
|
668
|
+
|
669
|
+
Perhaps you're testing a ruby script with `PTY.spawn` or `Open3.popen`, or `Process.spawn` or etc.
|
670
|
+
SimpleCov can cover this too.
|
671
|
+
|
672
|
+
Add a .simplecov_spawn.rb file to your project root
|
673
|
+
```ruby
|
674
|
+
# .simplecov_spawn.rb
|
675
|
+
require 'simplecov' # this will also pick up whatever config is in .simplecov
|
676
|
+
# so ensure it just contains configuration, and doesn't call SimpleCov.start.
|
677
|
+
SimpleCov.command_name 'spawn' # As this is not for a test runner directly, script doesn't have a pre-defined base command_name
|
678
|
+
SimpleCov.at_fork.call(Process.pid) # Use the per-process setup described previously
|
679
|
+
SimpleCov.start # only now can we start.
|
680
|
+
```
|
681
|
+
Then, instead of calling your script directly, like:
|
682
|
+
```ruby
|
683
|
+
PTY.spawn('my_script.rb') do # ...
|
684
|
+
```
|
685
|
+
Use bin/ruby to require the new .simplecov_spawn file, then your script
|
686
|
+
```ruby
|
687
|
+
PTY.spawn('ruby -r./.simplecov_spawn my_script.rb') do # ...
|
688
|
+
```
|
315
689
|
|
316
690
|
## Running coverage only on demand
|
317
691
|
|
318
|
-
The Ruby STDLIB Coverage library that SimpleCov builds upon is *very* fast (
|
319
|
-
only a couple seconds for me), and therefore it's SimpleCov's policy to just generate coverage every time you
|
320
|
-
it doesn't do your test speed any harm and you're always equipped with the latest and greatest
|
692
|
+
The Ruby STDLIB Coverage library that SimpleCov builds upon is *very* fast (on a ~10 min Rails test suite, the speed
|
693
|
+
drop was only a couple seconds for me), and therefore it's SimpleCov's policy to just generate coverage every time you
|
694
|
+
run your tests because it doesn't do your test speed any harm and you're always equipped with the latest and greatest
|
695
|
+
coverage results.
|
321
696
|
|
322
697
|
Because of this, SimpleCov has no explicit built-in mechanism to run coverage only on demand.
|
323
698
|
|
324
|
-
However, you can still accomplish this very easily by introducing
|
699
|
+
However, you can still accomplish this very easily by introducing an ENV variable conditional into your SimpleCov setup
|
700
|
+
block, like this:
|
325
701
|
|
326
|
-
|
702
|
+
```ruby
|
703
|
+
SimpleCov.start if ENV["COVERAGE"]
|
704
|
+
```
|
327
705
|
|
328
706
|
Then, SimpleCov will only run if you execute your tests like this:
|
329
707
|
|
330
|
-
|
708
|
+
```shell
|
709
|
+
COVERAGE=true rake test
|
710
|
+
```
|
331
711
|
|
712
|
+
## Errors and exit statuses
|
332
713
|
|
333
|
-
|
714
|
+
To aid in debugging issues, if an error is raised, SimpleCov will print a message to `STDERR`
|
715
|
+
with the exit status of the error, like:
|
334
716
|
|
335
|
-
|
336
|
-
|
337
|
-
|
717
|
+
```
|
718
|
+
SimpleCov failed with exit 1
|
719
|
+
```
|
338
720
|
|
339
|
-
|
340
|
-
comes bundled with a 'rails' adapter. It looks somewhat like this:
|
721
|
+
This `STDERR` message can be disabled with:
|
341
722
|
|
342
|
-
|
343
|
-
|
344
|
-
|
723
|
+
```
|
724
|
+
SimpleCov.print_error_status = false
|
725
|
+
```
|
345
726
|
|
346
|
-
|
347
|
-
add_group 'Models', 'app/models'
|
348
|
-
add_group 'Helpers', 'app/helpers'
|
349
|
-
add_group 'Libraries', 'lib'
|
350
|
-
add_group 'Plugins', 'vendor/plugins'
|
351
|
-
end
|
727
|
+
## Profiles
|
352
728
|
|
353
|
-
|
729
|
+
By default, SimpleCov's only config assumption is that you only want coverage reports for files inside your project
|
730
|
+
root. To save yourself from repetitive configuration, you can use predefined blocks of configuration, called 'profiles',
|
731
|
+
or define your own.
|
354
732
|
|
355
|
-
|
733
|
+
You can then pass the name of the profile to be used as the first argument to SimpleCov.start. For example, simplecov
|
734
|
+
comes bundled with a 'rails' profile. It looks somewhat like this:
|
356
735
|
|
357
|
-
|
736
|
+
```ruby
|
737
|
+
SimpleCov.profiles.define 'rails' do
|
738
|
+
add_filter '/test/'
|
739
|
+
add_filter '/config/'
|
358
740
|
|
359
|
-
|
360
|
-
|
361
|
-
|
741
|
+
add_group 'Controllers', 'app/controllers'
|
742
|
+
add_group 'Models', 'app/models'
|
743
|
+
add_group 'Helpers', 'app/helpers'
|
744
|
+
add_group 'Libraries', 'lib'
|
745
|
+
end
|
746
|
+
```
|
362
747
|
|
363
|
-
|
748
|
+
As you can see, it's just a SimpleCov.configure block. In your test_helper.rb, launch SimpleCov with:
|
364
749
|
|
365
|
-
|
366
|
-
|
750
|
+
```ruby
|
751
|
+
SimpleCov.start 'rails'
|
752
|
+
```
|
367
753
|
|
368
|
-
|
369
|
-
require 'simplecov'
|
370
|
-
SimpleCov.adapters.define 'myadapter' do
|
371
|
-
load_adapter 'rails'
|
372
|
-
add_filter 'vendor' # Don't include vendored stuff
|
373
|
-
end
|
754
|
+
or
|
374
755
|
|
375
|
-
|
376
|
-
|
377
|
-
|
756
|
+
```ruby
|
757
|
+
SimpleCov.start 'rails' do
|
758
|
+
# additional config here
|
759
|
+
end
|
760
|
+
```
|
378
761
|
|
379
|
-
|
380
|
-
require 'simplecov_custom_adapter'
|
381
|
-
SimpleCov.start 'myadapter'
|
762
|
+
### Custom profiles
|
382
763
|
|
764
|
+
You can load additional profiles with the SimpleCov.load_profile('xyz') method. This allows you to build upon an
|
765
|
+
existing profile and customize it so you can reuse it in unit tests and Cucumber features. For example:
|
383
766
|
|
767
|
+
```ruby
|
768
|
+
# lib/simplecov_custom_profile.rb
|
769
|
+
require 'simplecov'
|
770
|
+
SimpleCov.profiles.define 'myprofile' do
|
771
|
+
load_profile 'rails'
|
772
|
+
add_filter 'vendor' # Don't include vendored stuff
|
773
|
+
end
|
774
|
+
|
775
|
+
# features/support/env.rb
|
776
|
+
require 'simplecov_custom_profile'
|
777
|
+
SimpleCov.start 'myprofile'
|
778
|
+
|
779
|
+
# test/test_helper.rb
|
780
|
+
require 'simplecov_custom_profile'
|
781
|
+
SimpleCov.start 'myprofile'
|
782
|
+
```
|
384
783
|
|
385
784
|
## Customizing exit behaviour
|
386
785
|
|
387
|
-
You can define what
|
786
|
+
You can define what SimpleCov should do when your test suite finishes by customizing the at_exit hook:
|
388
787
|
|
389
|
-
|
390
|
-
|
391
|
-
|
788
|
+
```ruby
|
789
|
+
SimpleCov.at_exit do
|
790
|
+
SimpleCov.result.format!
|
791
|
+
end
|
792
|
+
```
|
392
793
|
|
393
794
|
Above is the default behaviour. Do whatever you like instead!
|
394
795
|
|
796
|
+
### Minimum coverage
|
797
|
+
|
798
|
+
You can define the minimum coverage percentage expected. SimpleCov will return non-zero if unmet.
|
799
|
+
|
800
|
+
```ruby
|
801
|
+
SimpleCov.minimum_coverage 90
|
802
|
+
# same as above (the default is to check line coverage)
|
803
|
+
SimpleCov.minimum_coverage line: 90
|
804
|
+
# check for a minimum line coverage of 90% and minimum 80% branch coverage
|
805
|
+
SimpleCov.minimum_coverage line: 90, branch: 80
|
806
|
+
```
|
807
|
+
|
808
|
+
### Minimum coverage by file
|
809
|
+
|
810
|
+
You can define the minimum coverage by file percentage expected. SimpleCov will return non-zero if unmet. This is useful
|
811
|
+
to help ensure coverage is relatively consistent, rather than being skewed by particularly good or bad areas of the code.
|
812
|
+
|
813
|
+
```ruby
|
814
|
+
SimpleCov.minimum_coverage_by_file 80
|
815
|
+
# same as above (the default is to check line coverage by file)
|
816
|
+
SimpleCov.minimum_coverage_by_file line: 80
|
817
|
+
# check for a minimum line coverage by file of 90% and minimum 80% branch coverage
|
818
|
+
SimpleCov.minimum_coverage_by_file line: 90, branch: 80
|
819
|
+
```
|
820
|
+
|
821
|
+
### Maximum coverage drop
|
822
|
+
|
823
|
+
You can define the maximum coverage drop percentage at once. SimpleCov will return non-zero if exceeded.
|
824
|
+
|
825
|
+
```ruby
|
826
|
+
SimpleCov.maximum_coverage_drop 5
|
827
|
+
# same as above (the default is to check line drop)
|
828
|
+
SimpleCov.maximum_coverage_drop line: 5
|
829
|
+
# check for a maximum line drop of 5% and maximum 10% branch drop
|
830
|
+
SimpleCov.maximum_coverage_drop line: 5, branch: 10
|
831
|
+
```
|
832
|
+
|
833
|
+
### Refuse dropping coverage
|
834
|
+
|
835
|
+
You can also entirely refuse dropping coverage between test runs:
|
836
|
+
|
837
|
+
```ruby
|
838
|
+
SimpleCov.refuse_coverage_drop
|
839
|
+
# same as above (the default is to only refuse line drop)
|
840
|
+
SimpleCov.refuse_coverage_drop :line
|
841
|
+
# refuse drop for line and branch
|
842
|
+
SimpleCov.refuse_coverage_drop :line, :branch
|
843
|
+
```
|
395
844
|
|
396
845
|
## Using your own formatter
|
397
846
|
|
398
847
|
You can use your own formatter with:
|
399
848
|
|
400
|
-
|
849
|
+
```ruby
|
850
|
+
SimpleCov.formatter = SimpleCov::Formatter::HTMLFormatter
|
851
|
+
```
|
401
852
|
|
402
|
-
|
403
|
-
|
853
|
+
Calling `SimpleCov.result.format!` will be invoked with `SimpleCov::Formatter::YourFormatter.new.format(result)`,
|
854
|
+
and `result` is an instance of `SimpleCov::Result`. Do whatever your wish with that!
|
404
855
|
|
405
856
|
|
406
857
|
## Using multiple formatters
|
407
858
|
|
408
|
-
|
859
|
+
As of SimpleCov 0.9, you can specify multiple result formats. Formatters besides the default HTML formatter require separate gems, however.
|
409
860
|
|
410
|
-
|
411
|
-
|
412
|
-
SimpleCov::Formatter::HTMLFormatter.new.format(result)
|
413
|
-
SimpleCov::Formatter::CSVFormatter.new.format(result)
|
414
|
-
end
|
415
|
-
end
|
861
|
+
```ruby
|
862
|
+
require "simplecov-html"
|
416
863
|
|
417
|
-
|
864
|
+
SimpleCov.formatters = SimpleCov::Formatter::MultiFormatter.new([
|
865
|
+
SimpleCov::Formatter::HTMLFormatter,
|
866
|
+
SimpleCov::Formatter::CSVFormatter,
|
867
|
+
])
|
868
|
+
```
|
418
869
|
|
419
|
-
|
870
|
+
## JSON formatter
|
420
871
|
|
421
|
-
|
872
|
+
SimpleCov is packaged with a separate gem called [simplecov_json_formatter](https://github.com/codeclimate-community/simplecov_json_formatter) that provides you with a JSON formatter, this formatter could be useful for different use cases, such as for CI consumption or for reporting to external services.
|
422
873
|
|
423
|
-
|
424
|
-
available:
|
874
|
+
In order to use it you will need to manually load the installed gem like so:
|
425
875
|
|
426
|
-
|
427
|
-
|
876
|
+
```ruby
|
877
|
+
require "simplecov_json_formatter"
|
878
|
+
SimpleCov.formatter = SimpleCov::Formatter::JSONFormatter
|
879
|
+
```
|
428
880
|
|
429
|
-
|
881
|
+
> _Note:_ In case you plan to report your coverage results to CodeClimate services, know that SimpleCov will automatically use the
|
882
|
+
> JSON formatter along with the HTML formatter when the `CC_TEST_REPORTER_ID` variable is present in the environment.
|
430
883
|
|
431
|
-
|
432
|
-
*by Fernando Guillen*
|
884
|
+
## Available formatters, editor integrations and hosted services
|
433
885
|
|
434
|
-
|
886
|
+
* [Open Source formatter and integration plugins for SimpleCov](doc/alternate-formatters.md)
|
887
|
+
* [Editor Integration](doc/editor-integration.md)
|
888
|
+
* [Hosted (commercial) services](doc/commercial-services.md)
|
435
889
|
|
890
|
+
## Ruby version compatibility
|
436
891
|
|
892
|
+
SimpleCov is built in [Continuous Integration] on Ruby 2.7+ as well as JRuby 9.3+.
|
437
893
|
|
438
|
-
|
894
|
+
Note for JRuby => You need to pass JRUBY_OPTS="--debug" or create .jrubyrc and add debug.fullTrace=true
|
439
895
|
|
440
|
-
|
896
|
+
## Want to find dead code in production?
|
441
897
|
|
442
|
-
|
443
|
-
including Rubinius and JRuby, in [Continuous Integration], but this happens only to ensure that SimpleCov does not make your
|
444
|
-
test suite crash right now. Whether SimpleCov will support JRuby/Rubinius in the future depends solely on whether those Ruby
|
445
|
-
interpreters add the coverage library.
|
898
|
+
Try [Coverband](https://github.com/danmayer/coverband).
|
446
899
|
|
447
|
-
|
900
|
+
## Want to use Spring with SimpleCov?
|
448
901
|
|
902
|
+
If you're using [Spring](https://github.com/rails/spring) to speed up test suite runs and want to run SimpleCov along
|
903
|
+
with them, you'll find that it often misreports coverage with the default config due to some sort of eager loading
|
904
|
+
issue. Don't despair!
|
449
905
|
|
906
|
+
One solution is to [explicitly call eager
|
907
|
+
load](https://github.com/simplecov-ruby/simplecov/issues/381#issuecomment-347651728)
|
908
|
+
in your `test_helper.rb` / `spec_helper.rb` after calling `SimpleCov.start`.
|
450
909
|
|
451
|
-
|
910
|
+
```ruby
|
911
|
+
require 'simplecov'
|
912
|
+
SimpleCov.start 'rails'
|
913
|
+
Rails.application.eager_load!
|
914
|
+
```
|
452
915
|
|
453
|
-
|
916
|
+
Alternatively, you could disable Spring while running SimpleCov:
|
454
917
|
|
455
|
-
|
456
|
-
|
457
|
-
|
458
|
-
$ bundle exec rake test && bundle exec cucumber features
|
918
|
+
```
|
919
|
+
DISABLE_SPRING=1 rake test
|
920
|
+
```
|
459
921
|
|
460
|
-
|
922
|
+
Or you could remove `gem 'spring'` from your `Gemfile`.
|
461
923
|
|
462
|
-
|
463
|
-
* Make your feature addition or bug fix.
|
464
|
-
* Add tests for it. This is important so I don't break it in a future version unintentionally.
|
465
|
-
* Send me a pull request on Github.
|
924
|
+
## Troubleshooting
|
466
925
|
|
467
|
-
|
926
|
+
The **most common problem is that simplecov isn't required and started before everything else**. In order to track
|
927
|
+
coverage for your whole application **simplecov needs to be the first one** so that it (and the underlying coverage
|
928
|
+
library) can subsequently track loaded files and their usage.
|
468
929
|
|
469
|
-
|
470
|
-
|
930
|
+
If you are missing coverage for some code a simple trick is to put a puts statement in there and right after
|
931
|
+
`SimpleCov.start` so you can see if the file really was loaded after simplecov was started.
|
932
|
+
|
933
|
+
```ruby
|
934
|
+
# my_code.rb
|
935
|
+
class MyCode
|
936
|
+
|
937
|
+
puts "MyCode is being loaded!"
|
938
|
+
|
939
|
+
def my_method
|
940
|
+
# ...
|
941
|
+
end
|
942
|
+
end
|
943
|
+
|
944
|
+
# spec_helper.rb/rails_helper.rb/test_helper.rb/.simplecov whatever
|
945
|
+
|
946
|
+
SimpleCov.start
|
947
|
+
puts "SimpleCov started successfully!"
|
948
|
+
```
|
949
|
+
|
950
|
+
Now when you run your test suite and you see:
|
951
|
+
|
952
|
+
```
|
953
|
+
SimpleCov started successfully!
|
954
|
+
MyCode is being loaded!
|
955
|
+
```
|
956
|
+
|
957
|
+
then it's good otherwise you likely have a problem :)
|
958
|
+
|
959
|
+
## Code of Conduct
|
960
|
+
|
961
|
+
Everyone participating in this project's development, issue trackers and other channels is expected to follow our
|
962
|
+
[Code of Conduct](./CODE_OF_CONDUCT.md)
|
963
|
+
|
964
|
+
## Contributing
|
965
|
+
|
966
|
+
See the [contributing guide](https://github.com/simplecov-ruby/simplecov/blob/main/CONTRIBUTING.md).
|
967
|
+
|
968
|
+
## Kudos
|
471
969
|
|
970
|
+
Thanks to Aaron Patterson for the original idea for this!
|
472
971
|
|
473
972
|
## Copyright
|
474
973
|
|
475
|
-
Copyright (c) 2010-
|
974
|
+
Copyright (c) 2010-2017 Christoph Olszowka. See MIT-LICENSE for details.
|