simplecov 0.22.0 → 1.0.0.rc1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +81 -1
- data/LICENSE +1 -1
- data/README.md +1009 -511
- data/doc/alternate-formatters.md +0 -5
- data/doc/commercial-services.md +5 -5
- data/exe/simplecov +11 -0
- data/lib/minitest/simplecov_plugin.rb +13 -5
- data/lib/simplecov/autostart.rb +11 -0
- data/lib/simplecov/cli/clean.rb +47 -0
- data/lib/simplecov/cli/coverage.rb +91 -0
- data/lib/simplecov/cli/diff.rb +151 -0
- data/lib/simplecov/cli/dotfile.rb +100 -0
- data/lib/simplecov/cli/merge.rb +116 -0
- data/lib/simplecov/cli/open.rb +50 -0
- data/lib/simplecov/cli/report.rb +84 -0
- data/lib/simplecov/cli/run.rb +36 -0
- data/lib/simplecov/cli/serve.rb +139 -0
- data/lib/simplecov/cli/uncovered.rb +107 -0
- data/lib/simplecov/cli.rb +150 -0
- data/lib/simplecov/color.rb +74 -0
- data/lib/simplecov/combine/branches_combiner.rb +3 -2
- data/lib/simplecov/combine/files_combiner.rb +7 -1
- data/lib/simplecov/combine/lines_combiner.rb +19 -17
- data/lib/simplecov/combine/methods_combiner.rb +26 -0
- data/lib/simplecov/combine/results_combiner.rb +5 -4
- data/lib/simplecov/command_guesser.rb +46 -32
- data/lib/simplecov/configuration/coverage.rb +171 -0
- data/lib/simplecov/configuration/coverage_criteria.rb +156 -0
- data/lib/simplecov/configuration/filters.rb +195 -0
- data/lib/simplecov/configuration/formatting.rb +119 -0
- data/lib/simplecov/configuration/ignored_entries.rb +63 -0
- data/lib/simplecov/configuration/merging.rb +74 -0
- data/lib/simplecov/configuration/thresholds.rb +174 -0
- data/lib/simplecov/configuration.rb +79 -405
- data/lib/simplecov/coverage_statistics.rb +12 -9
- data/lib/simplecov/coverage_violations.rb +148 -0
- data/lib/simplecov/defaults.rb +27 -20
- data/lib/simplecov/directive.rb +162 -0
- data/lib/simplecov/exit_codes/exit_code_handling.rb +8 -2
- data/lib/simplecov/exit_codes/maximum_coverage_drop_check.rb +19 -57
- data/lib/simplecov/exit_codes/maximum_overall_coverage_check.rb +45 -0
- data/lib/simplecov/exit_codes/minimum_coverage_by_file_check.rb +17 -27
- data/lib/simplecov/exit_codes/minimum_coverage_by_group_check.rb +41 -0
- data/lib/simplecov/exit_codes/minimum_overall_coverage_check.rb +38 -21
- data/lib/simplecov/exit_codes.rb +3 -0
- data/lib/simplecov/exit_handling.rb +158 -0
- data/lib/simplecov/file_list.rb +61 -17
- data/lib/simplecov/filter.rb +69 -24
- data/lib/simplecov/formatter/base.rb +101 -0
- data/lib/simplecov/formatter/html_formatter/public/application.css +1 -0
- data/lib/simplecov/formatter/html_formatter/public/application.js +18 -0
- data/lib/simplecov/formatter/html_formatter/public/favicon_green.png +0 -0
- data/lib/simplecov/formatter/html_formatter/public/favicon_red.png +0 -0
- data/lib/simplecov/formatter/html_formatter/public/favicon_yellow.png +0 -0
- data/lib/simplecov/formatter/html_formatter/public/index.html +56 -0
- data/lib/simplecov/formatter/html_formatter.rb +79 -0
- data/lib/simplecov/formatter/json_formatter/errors_formatter.rb +84 -0
- data/lib/simplecov/formatter/json_formatter/result_hash_formatter.rb +127 -0
- data/lib/simplecov/formatter/json_formatter/source_file_formatter.rb +99 -0
- data/lib/simplecov/formatter/json_formatter.rb +77 -0
- data/lib/simplecov/formatter/multi_formatter.rb +4 -5
- data/lib/simplecov/formatter/simple_formatter.rb +9 -11
- data/lib/simplecov/formatter.rb +4 -0
- data/lib/simplecov/last_run.rb +10 -3
- data/lib/simplecov/lines_classifier.rb +26 -13
- data/lib/simplecov/load_global_config.rb +9 -4
- data/lib/simplecov/parallel_adapters/base.rb +51 -0
- data/lib/simplecov/parallel_adapters/generic.rb +42 -0
- data/lib/simplecov/parallel_adapters/parallel_tests.rb +77 -0
- data/lib/simplecov/parallel_adapters.rb +83 -0
- data/lib/simplecov/parallel_coordination.rb +95 -0
- data/lib/simplecov/process.rb +20 -14
- data/lib/simplecov/profiles/bundler_filter.rb +1 -1
- data/lib/simplecov/profiles/hidden_filter.rb +1 -1
- data/lib/simplecov/profiles/rails.rb +24 -10
- data/lib/simplecov/profiles/root_filter.rb +6 -5
- data/lib/simplecov/profiles/strict.rb +32 -0
- data/lib/simplecov/profiles/test_frameworks.rb +1 -4
- data/lib/simplecov/profiles.rb +32 -3
- data/lib/simplecov/result/missing_source_files_reporter.rb +49 -0
- data/lib/simplecov/result/source_file_builder.rb +51 -0
- data/lib/simplecov/result.rb +97 -19
- data/lib/simplecov/result_adapter.rb +68 -6
- data/lib/simplecov/result_merger/legacy_format_adapter.rb +28 -0
- data/lib/simplecov/result_merger/resultset_file.rb +38 -0
- data/lib/simplecov/result_merger/resultset_store.rb +50 -0
- data/lib/simplecov/result_merger.rb +46 -90
- data/lib/simplecov/result_processing.rb +162 -0
- data/lib/simplecov/simulate_coverage.rb +54 -8
- data/lib/simplecov/source_file/branch.rb +1 -3
- data/lib/simplecov/source_file/branch_builder.rb +114 -0
- data/lib/simplecov/source_file/builder_context.rb +28 -0
- data/lib/simplecov/source_file/line.rb +7 -2
- data/lib/simplecov/source_file/line_builder.rb +43 -0
- data/lib/simplecov/source_file/method.rb +52 -0
- data/lib/simplecov/source_file/method_builder.rb +58 -0
- data/lib/simplecov/source_file/ruby_data_parser.rb +88 -0
- data/lib/simplecov/source_file/skip_chunks.rb +77 -0
- data/lib/simplecov/source_file/source_loader.rb +63 -0
- data/lib/simplecov/source_file/statistics.rb +57 -0
- data/lib/simplecov/source_file.rb +66 -232
- data/lib/simplecov/static_coverage_extractor/visitor.rb +193 -0
- data/lib/simplecov/static_coverage_extractor.rb +111 -0
- data/lib/simplecov/useless_results_remover.rb +16 -7
- data/lib/simplecov/version.rb +1 -1
- data/lib/simplecov-html.rb +4 -0
- data/lib/simplecov.rb +131 -377
- data/lib/simplecov_json_formatter.rb +4 -0
- data/schemas/coverage-v1.0.schema.json +300 -0
- data/schemas/coverage.schema.json +300 -0
- metadata +88 -56
- data/lib/simplecov/default_formatter.rb +0 -20
data/README.md
CHANGED
|
@@ -1,61 +1,50 @@
|
|
|
1
|
-
SimpleCov [](https://badge.fury.io/rb/simplecov) [][Continuous Integration] [](https://badge.fury.io/rb/simplecov) [][Continuous Integration] [](https://github.com/simplecov-ruby/simplecov/actions/workflows/lint.yml) [](https://github.com/simplecov-ruby/simplecov/actions/workflows/typecheck.yml) [](https://codeclimate.com/github/simplecov-ruby/simplecov/maintainability)
|
|
2
2
|
=========
|
|
3
3
|
|
|
4
4
|
**Code coverage for Ruby**
|
|
5
5
|
|
|
6
6
|
* [Source Code]
|
|
7
7
|
* [API documentation]
|
|
8
|
+
* [Configuration]
|
|
8
9
|
* [Changelog]
|
|
9
10
|
* [Rubygem]
|
|
10
11
|
* [Continuous Integration]
|
|
11
12
|
|
|
12
|
-
[Coverage]: https://ruby-
|
|
13
|
+
[Coverage]: https://docs.ruby-lang.org/en/master/Coverage.html "API doc for Ruby's Coverage library"
|
|
13
14
|
[Source Code]: https://github.com/simplecov-ruby/simplecov "Source Code @ GitHub"
|
|
14
15
|
[API documentation]: http://rubydoc.info/gems/simplecov/frames "RDoc API Documentation at Rubydoc.info"
|
|
15
16
|
[Configuration]: http://rubydoc.info/gems/simplecov/SimpleCov/Configuration "Configuration options API documentation"
|
|
16
17
|
[Changelog]: https://github.com/simplecov-ruby/simplecov/blob/main/CHANGELOG.md "Project Changelog"
|
|
17
18
|
[Rubygem]: http://rubygems.org/gems/simplecov "SimpleCov @ rubygems.org"
|
|
18
19
|
[Continuous Integration]: https://github.com/simplecov-ruby/simplecov/actions?query=workflow%3Astable "SimpleCov is built around the clock by github.com"
|
|
19
|
-
[Dependencies]: https://gemnasium.com/simplecov-ruby/simplecov "SimpleCov dependencies on Gemnasium"
|
|
20
|
-
[simplecov-html]: https://github.com/simplecov-ruby/simplecov-html "SimpleCov HTML Formatter Source Code @ GitHub"
|
|
21
20
|
|
|
22
|
-
SimpleCov is a code coverage analysis tool for Ruby. It uses [Ruby's built-in Coverage][Coverage] library to gather
|
|
23
|
-
coverage data, but makes processing
|
|
24
|
-
and display
|
|
25
|
-
code.
|
|
26
|
-
SimpleCov/Coverage track covered ruby code, gathering coverage for common templating solutions like erb, slim and haml is not supported.
|
|
21
|
+
SimpleCov is a code coverage analysis tool for Ruby. It uses [Ruby's built-in Coverage][Coverage] library to gather
|
|
22
|
+
coverage data, but makes processing the results much easier by providing a clean API to filter, group, merge, format,
|
|
23
|
+
and display them — a complete coverage suite you can set up in just a couple of lines.
|
|
27
24
|
|
|
28
|
-
|
|
29
|
-
|
|
30
|
-
report actually includes coverage across your test suites and thereby gives you a better picture of blank spots.
|
|
25
|
+
SimpleCov tracks covered Ruby code; gathering coverage for templating solutions like ERB, Slim, and Haml is not
|
|
26
|
+
supported (though see [Eval coverage](#eval-coverage) for ERB).
|
|
31
27
|
|
|
32
|
-
|
|
33
|
-
|
|
28
|
+
In most cases you'll want overall coverage results spanning all of your tests — unit tests, Cucumber features, and so
|
|
29
|
+
on. SimpleCov handles this automatically by caching and merging results as it generates reports, so a report reflects
|
|
30
|
+
coverage across your whole test suite and gives you a truer picture of your blank spots.
|
|
34
31
|
|
|
32
|
+
SimpleCov bundles two formatters that need no extra gems: the default HTML formatter (which renders the browsable
|
|
33
|
+
report) and a JSON formatter. Both were once separate gems (`simplecov-html` and `simplecov_json_formatter`) but are
|
|
34
|
+
now built into SimpleCov and configured automatically when you launch it.
|
|
35
35
|
|
|
36
|
-
##
|
|
36
|
+
## Getting started
|
|
37
37
|
|
|
38
|
-
*Code and Bug Reports*
|
|
39
|
-
|
|
40
|
-
* [Issue Tracker](https://github.com/simplecov-ruby/simplecov/issues)
|
|
41
|
-
* See [CONTRIBUTING](https://github.com/simplecov-ruby/simplecov/blob/main/CONTRIBUTING.md) for how to contribute along
|
|
42
|
-
with some common problems to check out before creating an issue.
|
|
43
|
-
|
|
44
|
-
*Questions, Problems, Suggestions, etc.*
|
|
45
|
-
|
|
46
|
-
* [Mailing List](https://groups.google.com/forum/#!forum/simplecov) "Open mailing list for discussion and announcements
|
|
47
|
-
on Google Groups"
|
|
48
|
-
|
|
49
|
-
Getting started
|
|
50
|
-
---------------
|
|
51
38
|
1. Add SimpleCov to your `Gemfile` and `bundle install`:
|
|
52
39
|
|
|
53
40
|
```ruby
|
|
54
41
|
gem 'simplecov', require: false, group: :test
|
|
55
42
|
```
|
|
56
|
-
|
|
57
|
-
|
|
58
|
-
framework uses
|
|
43
|
+
|
|
44
|
+
2. Load and launch SimpleCov **at the very top** of your test helper — `test/test_helper.rb`, `spec/spec_helper.rb`,
|
|
45
|
+
`rails_helper.rb`, Cucumber's `features/support/env.rb`, or whatever setup file your framework uses. SimpleCov
|
|
46
|
+
doesn't care which framework you run in; it just watches what code executes and reports on it, so the same two lines
|
|
47
|
+
work everywhere:
|
|
59
48
|
|
|
60
49
|
```ruby
|
|
61
50
|
require 'simplecov'
|
|
@@ -64,196 +53,77 @@ Getting started
|
|
|
64
53
|
# Previous content of test helper now starts here
|
|
65
54
|
```
|
|
66
55
|
|
|
67
|
-
|
|
68
|
-
|
|
69
|
-
|
|
70
|
-
code is required!**
|
|
71
|
-
|
|
72
|
-
This is especially true if you use anything that keeps your tests application loaded like spring, check out the **[spring section](#want-to-use-spring-with-simplecov)**.
|
|
56
|
+
> **Important:** `SimpleCov.start` **must** run **before any of your application code is required** — otherwise
|
|
57
|
+
> SimpleCov (and the underlying Coverage library) can't track those files. This bites hardest with tools that keep
|
|
58
|
+
> your app loaded between runs, like Spring; see the [Spring section](#using-spring-with-simplecov).
|
|
73
59
|
|
|
74
|
-
|
|
75
|
-
|
|
76
|
-
|
|
77
|
-
|
|
78
|
-
executed in your actual test files, you need to require SimpleCov in the
|
|
79
|
-
server process. For rails for instance, you'll want to add something like this
|
|
80
|
-
to the top of `bin/rails`, but below the "shebang" line (`#! /usr/bin/env
|
|
81
|
-
ruby`) and after config/boot is required:
|
|
60
|
+
SimpleCov must run in the process you want to analyze. When you test a server process (e.g. a JSON API) from a
|
|
61
|
+
separate test process (e.g. via Selenium) and want to see all code the `rails server` executes — not just code in
|
|
62
|
+
your test files — require SimpleCov in the server process. For Rails, add this near the top of `bin/rails`, below
|
|
63
|
+
the shebang and after `config/boot` is required:
|
|
82
64
|
|
|
83
65
|
```ruby
|
|
84
66
|
if ENV['RAILS_ENV'] == 'test'
|
|
85
67
|
require 'simplecov'
|
|
86
68
|
SimpleCov.start 'rails'
|
|
87
|
-
puts "required simplecov"
|
|
88
69
|
end
|
|
89
70
|
```
|
|
90
71
|
|
|
91
|
-
3. Run your full test suite to see
|
|
92
|
-
4. After running your tests, open `coverage/index.html` in the browser of your choice. For example, in a Mac Terminal,
|
|
93
|
-
run the following command from your application's root directory:
|
|
94
|
-
|
|
95
|
-
```
|
|
96
|
-
open coverage/index.html
|
|
97
|
-
```
|
|
98
|
-
in a debian/ubuntu Terminal,
|
|
99
|
-
|
|
100
|
-
```
|
|
101
|
-
xdg-open coverage/index.html
|
|
102
|
-
```
|
|
72
|
+
3. Run your full test suite to see your application's coverage.
|
|
103
73
|
|
|
104
|
-
|
|
105
|
-
operating system requires.
|
|
106
|
-
|
|
107
|
-
5. Add the following to your `.gitignore` file to ensure that coverage results
|
|
108
|
-
are not tracked by Git (optional):
|
|
109
|
-
|
|
110
|
-
```
|
|
111
|
-
echo coverage >> .gitignore
|
|
112
|
-
```
|
|
113
|
-
|
|
114
|
-
If you're making a Rails application, SimpleCov comes with built-in configurations (see below for information on
|
|
115
|
-
profiles) that will get you started with groups for your Controllers, Models and Helpers. To use it, the
|
|
116
|
-
first two lines of your test_helper should be like this:
|
|
117
|
-
|
|
118
|
-
```ruby
|
|
119
|
-
require 'simplecov'
|
|
120
|
-
SimpleCov.start 'rails'
|
|
121
|
-
```
|
|
122
|
-
|
|
123
|
-
## Example output
|
|
124
|
-
|
|
125
|
-
**Coverage results report, fully browsable locally with sorting and much more:**
|
|
74
|
+
4. Open the HTML report in your default browser:
|
|
126
75
|
|
|
127
|
-
|
|
128
|
-
|
|
129
|
-
|
|
130
|
-
**Source file coverage details view:**
|
|
76
|
+
```sh
|
|
77
|
+
simplecov open
|
|
78
|
+
```
|
|
131
79
|
|
|
132
|
-
|
|
80
|
+
(The bundled `simplecov` CLI picks the right opener for your platform — `open` on macOS, `xdg-open` on Linux/BSD,
|
|
81
|
+
`start` on Windows. Pass `--report PATH` to open a non-default location. See [Command-line interface](#command-line-interface)
|
|
82
|
+
for the full set of subcommands.)
|
|
133
83
|
|
|
134
|
-
|
|
84
|
+
5. Optionally, keep coverage results out of Git:
|
|
135
85
|
|
|
136
|
-
|
|
137
|
-
|
|
86
|
+
```sh
|
|
87
|
+
echo coverage >> .gitignore
|
|
88
|
+
```
|
|
138
89
|
|
|
139
|
-
|
|
140
|
-
|
|
90
|
+
For Rails applications, SimpleCov ships a built-in `rails` [profile](#profiles) that sets up groups for your
|
|
91
|
+
Controllers, Models, Helpers, and Libraries:
|
|
141
92
|
|
|
142
93
|
```ruby
|
|
143
94
|
require 'simplecov'
|
|
144
95
|
SimpleCov.start 'rails'
|
|
145
96
|
```
|
|
146
97
|
|
|
147
|
-
|
|
148
|
-
does not care what kind of framework it is running in; it just looks at what code is being executed and generates a
|
|
149
|
-
report about it.
|
|
98
|
+
## Example output
|
|
150
99
|
|
|
151
|
-
|
|
100
|
+
**Coverage results report, fully browsable locally with sorting and much more:**
|
|
152
101
|
|
|
153
|
-
|
|
154
|
-
to use SimpleCov with them. Here's an overview of the known ones:
|
|
102
|
+

|
|
155
103
|
|
|
156
|
-
|
|
157
|
-
|
|
158
|
-
|
|
159
|
-
<th>
|
|
160
|
-
parallel_tests
|
|
161
|
-
</th>
|
|
162
|
-
<td>
|
|
163
|
-
As of 0.8.0, SimpleCov should correctly recognize parallel_tests and
|
|
164
|
-
supplement your test suite names with their corresponding test env
|
|
165
|
-
numbers. SimpleCov locks the resultset cache while merging, ensuring no
|
|
166
|
-
race conditions occur when results are merged.
|
|
167
|
-
</td>
|
|
168
|
-
<td>
|
|
169
|
-
<a href="https://github.com/simplecov-ruby/simplecov/issues/64">#64</a> &
|
|
170
|
-
<a href="https://github.com/simplecov-ruby/simplecov/pull/185">#185</a>
|
|
171
|
-
</td>
|
|
172
|
-
</tr>
|
|
173
|
-
<tr>
|
|
174
|
-
<th>
|
|
175
|
-
knapsack_pro
|
|
176
|
-
</th>
|
|
177
|
-
<td>
|
|
178
|
-
To make SimpleCov work with Knapsack Pro Queue Mode to split tests in parallel on CI jobs you need to provide CI node index number to the <code>SimpleCov.command_name</code> in <code>KnapsackPro::Hooks::Queue.before_queue</code> hook.
|
|
179
|
-
</td>
|
|
180
|
-
<td>
|
|
181
|
-
<a href="https://knapsackpro.com/faq/question/how-to-use-simplecov-in-queue-mode">Tip</a>
|
|
182
|
-
</td>
|
|
183
|
-
</tr>
|
|
184
|
-
<tr>
|
|
185
|
-
<th>
|
|
186
|
-
RubyMine
|
|
187
|
-
</th>
|
|
188
|
-
<td>
|
|
189
|
-
The <a href="https://www.jetbrains.com/ruby/">RubyMine IDE</a> has
|
|
190
|
-
built-in support for SimpleCov's coverage reports, though you might need
|
|
191
|
-
to explicitly set the output root using `SimpleCov.root('foo/bar/baz')`
|
|
192
|
-
</td>
|
|
193
|
-
<td>
|
|
194
|
-
<a href="https://github.com/simplecov-ruby/simplecov/issues/95">#95</a>
|
|
195
|
-
</td>
|
|
196
|
-
</tr>
|
|
197
|
-
<tr>
|
|
198
|
-
<th>
|
|
199
|
-
Spork
|
|
200
|
-
</th>
|
|
201
|
-
<td>
|
|
202
|
-
Because of how Spork works internally (using preforking), there used to
|
|
203
|
-
be trouble when using SimpleCov with it, but that has apparently been
|
|
204
|
-
resolved with a specific configuration strategy. See <a
|
|
205
|
-
href="https://github.com/simplecov-ruby/simplecov/issues/42#issuecomment-4440284">this</a>
|
|
206
|
-
comment.
|
|
207
|
-
</td>
|
|
208
|
-
<td>
|
|
209
|
-
<a href="https://github.com/simplecov-ruby/simplecov/issues/42#issuecomment-4440284">#42</a>
|
|
210
|
-
</td>
|
|
211
|
-
</tr>
|
|
212
|
-
<tr>
|
|
213
|
-
<th>
|
|
214
|
-
Spring
|
|
215
|
-
</th>
|
|
216
|
-
<td>
|
|
217
|
-
<a href="#want-to-use-spring-with-simplecov">See section below.</a>
|
|
218
|
-
</td>
|
|
219
|
-
<td>
|
|
220
|
-
<a href="https://github.com/simplecov-ruby/simplecov/issues/381">#381</a>
|
|
221
|
-
</td>
|
|
222
|
-
</tr>
|
|
223
|
-
<tr>
|
|
224
|
-
<th>
|
|
225
|
-
Test/Unit
|
|
226
|
-
</th>
|
|
227
|
-
<td>
|
|
228
|
-
Test Unit 2 used to mess with ARGV, leading to a failure to detect the
|
|
229
|
-
test process name in SimpleCov. <code>test-unit</code> releases 2.4.3+
|
|
230
|
-
(Dec 11th, 2011) should have this problem resolved.
|
|
231
|
-
</td>
|
|
232
|
-
<td>
|
|
233
|
-
<a href="https://github.com/simplecov-ruby/simplecov/issues/45">#45</a> &
|
|
234
|
-
<a href="https://github.com/test-unit/test-unit/pull/12">test-unit/test-unit#12</a>
|
|
235
|
-
</td>
|
|
236
|
-
</tr>
|
|
237
|
-
</table>
|
|
104
|
+
**Source file coverage details view:**
|
|
105
|
+
|
|
106
|
+

|
|
238
107
|
|
|
239
|
-
##
|
|
108
|
+
## Configuration
|
|
240
109
|
|
|
241
|
-
[Configuration] settings can be applied in three formats
|
|
110
|
+
[Configuration] settings can be applied in three equivalent formats:
|
|
242
111
|
|
|
243
|
-
*
|
|
112
|
+
* Directly in your start block (the most common way):
|
|
244
113
|
|
|
245
114
|
```ruby
|
|
246
115
|
SimpleCov.start do
|
|
247
116
|
some_config_option 'foo'
|
|
248
117
|
end
|
|
249
118
|
```
|
|
250
|
-
|
|
119
|
+
|
|
120
|
+
* As direct setters:
|
|
251
121
|
|
|
252
122
|
```ruby
|
|
253
123
|
SimpleCov.some_config_option 'foo'
|
|
254
124
|
```
|
|
255
|
-
|
|
256
|
-
|
|
125
|
+
|
|
126
|
+
* In a `configure` block — useful when you don't want to start coverage immediately, or want to add configuration later:
|
|
257
127
|
|
|
258
128
|
```ruby
|
|
259
129
|
SimpleCov.configure do
|
|
@@ -261,55 +131,159 @@ to use SimpleCov with them. Here's an overview of the known ones:
|
|
|
261
131
|
end
|
|
262
132
|
```
|
|
263
133
|
|
|
264
|
-
|
|
134
|
+
See the [Configuration] API documentation for the full list of options.
|
|
265
135
|
|
|
266
|
-
|
|
136
|
+
### Using `.simplecov` for centralized config
|
|
267
137
|
|
|
268
|
-
If you
|
|
269
|
-
|
|
270
|
-
|
|
271
|
-
To avoid this, you can place a file called `.simplecov` in your project root. You can then just leave the
|
|
272
|
-
`require 'simplecov'` in each test setup helper (**at the top**) and move the `SimpleCov.start` code with all your
|
|
273
|
-
custom config options into `.simplecov`:
|
|
138
|
+
If you merge multiple test-suite results (e.g. RSpec and Cucumber) into a single report, you'd otherwise have to repeat
|
|
139
|
+
your filters / groups / profile in every test helper. To avoid that, place a `.simplecov` file at your project root
|
|
140
|
+
with the shared configuration; each test helper then requires SimpleCov and explicitly starts it:
|
|
274
141
|
|
|
275
142
|
```ruby
|
|
276
|
-
#
|
|
143
|
+
# .simplecov — configuration only
|
|
144
|
+
SimpleCov.load_profile 'rails'
|
|
145
|
+
SimpleCov.skip 'lib/generators'
|
|
146
|
+
SimpleCov.group 'Models', 'app/models'
|
|
147
|
+
|
|
148
|
+
# spec/spec_helper.rb
|
|
277
149
|
require 'simplecov'
|
|
150
|
+
SimpleCov.start
|
|
278
151
|
|
|
279
152
|
# features/support/env.rb
|
|
280
153
|
require 'simplecov'
|
|
154
|
+
SimpleCov.start
|
|
155
|
+
```
|
|
281
156
|
|
|
282
|
-
|
|
283
|
-
|
|
284
|
-
|
|
157
|
+
This is recommended whenever you merge frameworks that rely on each other, like Cucumber and RSpec.
|
|
158
|
+
|
|
159
|
+
> [!NOTE]
|
|
160
|
+
> Calling `SimpleCov.start` directly from `.simplecov` is deprecated. Tracking still begins for backward
|
|
161
|
+
> compatibility, but a one-time deprecation warning fires; a future release will require the explicit `SimpleCov.start`
|
|
162
|
+
> from a test helper. Migrating prevents a long-standing bug where `.simplecov` auto-loaded in a Rakefile or Rails'
|
|
163
|
+
> `Bundler.require` would leave an empty parent-process report that overwrites the test subprocess's good one. See #581.
|
|
164
|
+
|
|
165
|
+
### Changing the report location
|
|
166
|
+
|
|
167
|
+
By default the report ends up in `SimpleCov.root / SimpleCov.coverage_dir`. For out-of-tree build setups
|
|
168
|
+
(CMake/CTest, Bazel, etc.) — where the build directory is elsewhere on the filesystem and you don't want the report
|
|
169
|
+
under the source root — set `SimpleCov.coverage_path` directly:
|
|
170
|
+
|
|
171
|
+
```ruby
|
|
172
|
+
SimpleCov.start do
|
|
173
|
+
root '/source/checkout'
|
|
174
|
+
coverage_path '/tmp/build/coverage'
|
|
175
|
+
end
|
|
176
|
+
```
|
|
177
|
+
|
|
178
|
+
Setting `coverage_path` explicitly pins the destination — subsequent changes to `root` or `coverage_dir` don't move
|
|
179
|
+
it. The directory is created if it doesn't already exist.
|
|
180
|
+
|
|
181
|
+
### Running coverage only on demand
|
|
182
|
+
|
|
183
|
+
The Ruby STDLIB Coverage library is *very* fast (on a ~10-minute Rails suite the slowdown is only a couple of seconds),
|
|
184
|
+
so SimpleCov's policy is to generate coverage on every run — it costs you almost nothing and you always have the latest
|
|
185
|
+
results. There's therefore no built-in on-demand switch, but you can add one with an `ENV` conditional:
|
|
186
|
+
|
|
187
|
+
```ruby
|
|
188
|
+
SimpleCov.start if ENV["COVERAGE"]
|
|
189
|
+
```
|
|
190
|
+
|
|
191
|
+
Then coverage runs only when you ask for it:
|
|
192
|
+
|
|
193
|
+
```sh
|
|
194
|
+
COVERAGE=true rake test
|
|
195
|
+
```
|
|
196
|
+
|
|
197
|
+
### Migrating from the legacy configuration API
|
|
198
|
+
|
|
199
|
+
The configuration API was redesigned to use a smaller set of consistent verbs. The legacy methods continue to work but
|
|
200
|
+
emit deprecation warnings that name their replacement; the table below is the canonical migration map.
|
|
201
|
+
|
|
202
|
+
| Legacy | New | Notes |
|
|
203
|
+
|-------------------------------------|----------------------------------|------------------------------------------------------------------------------------------------------------------------|
|
|
204
|
+
| `add_filter "lib/legacy"` | `skip "lib/legacy"` | Identical matcher grammar (string = path-segment substring; Regexp; block; Array). No behavior change. |
|
|
205
|
+
| `add_group "Models", "app/models"` | `group "Models", "app/models"` | Identical matcher grammar. No behavior change. |
|
|
206
|
+
| `track_files "lib/**/*.rb"` | `cover "lib/**/*.rb"` | `cover` includes unloaded files (the legacy `track_files` behavior) **and** restricts the report to the matching set. To keep the old additive-only behavior, pass every directory you want reported: `cover "lib/**/*.rb", "app/**/*.rb"`. |
|
|
207
|
+
| `use_merging false` | `merging false` | Same value, same behavior. |
|
|
208
|
+
| `enable_for_subprocesses true` | `merge_subprocesses true` | Same value, same behavior. |
|
|
209
|
+
| `enable_coverage_for_eval` | `enable_coverage :eval` | Eval coverage now folds into the same call you use to enable `:line`/`:branch`/`:method`: `enable_coverage :branch, :eval`. |
|
|
210
|
+
| `print_error_status` (reader) | `print_errors` | Reader only. The `print_error_status=` writer still works without a warning, but `print_errors true`/`print_errors false` is the new spelling. |
|
|
211
|
+
| `minimum_coverage_by_file line: 70, 'app/x.rb' => 100` | `coverage(:line) { minimum_per_file 70; minimum_per_file 100, only: 'app/x.rb' }` | The `coverage` block fixes the criterion, so per-path overrides are plain percentages with an `only:` target instead of a hash mixing Symbol / String / Regexp keys. See [Per-criterion thresholds](#per-criterion-thresholds-with-coverage). |
|
|
212
|
+
| `minimum_coverage_by_group 'Models' => { line: 90 }` | `coverage(:line) { minimum_per_group 90, only: 'Models' }` | Same uniform shape as `minimum_per_file`. |
|
|
213
|
+
|
|
214
|
+
Brand-new in the redesigned API (no legacy method to migrate from):
|
|
215
|
+
|
|
216
|
+
| Method | Purpose |
|
|
217
|
+
|-------------------------------------|--------------------------------------------------------------------------------------------------------------------------|
|
|
218
|
+
| `cover "lib/**/*.rb"` | Positive scope (allowlist). Multiple calls union; strings are globs. See above for the relationship with `track_files`. |
|
|
219
|
+
| `no_default_skips` | Clear every previously-installed filter — defaults and anything earlier in the block — so subsequent `skip`s start clean.|
|
|
220
|
+
| `formatter false` / `formatters []` | Opt out of formatting entirely. Workers in big parallel CI runs only need their `.resultset.json` for a final `SimpleCov.collate` step; skipping the formatter saves the per-job HTML / multi-formatter overhead. See #964. |
|
|
221
|
+
| `parallel_tests true` / `false` | Force on / off the auto-require of the `parallel_tests` gem. Default (unset) auto-detects from `TEST_ENV_NUMBER` / `PARALLEL_TEST_GROUPS` and silently skips if the gem isn't installed. Set explicitly when you use those env vars for unrelated subprocess coordination. See #1018. |
|
|
222
|
+
|
|
223
|
+
Example before/after:
|
|
224
|
+
|
|
225
|
+
```ruby
|
|
226
|
+
# Before
|
|
227
|
+
SimpleCov.start do
|
|
228
|
+
add_filter "/test/"
|
|
229
|
+
add_filter %r{\Aconfig/}
|
|
230
|
+
add_group "Models", "app/models"
|
|
231
|
+
track_files "lib/**/*.rb"
|
|
232
|
+
enable_coverage_for_eval
|
|
233
|
+
use_merging true
|
|
234
|
+
enable_for_subprocesses true
|
|
235
|
+
end
|
|
236
|
+
|
|
237
|
+
# After
|
|
238
|
+
SimpleCov.start do
|
|
239
|
+
skip "/test/"
|
|
240
|
+
skip %r{\Aconfig/}
|
|
241
|
+
group "Models", "app/models"
|
|
242
|
+
cover "lib/**/*.rb"
|
|
243
|
+
enable_coverage :eval
|
|
244
|
+
merging true
|
|
245
|
+
merge_subprocesses true
|
|
285
246
|
end
|
|
286
247
|
```
|
|
287
248
|
|
|
288
|
-
|
|
289
|
-
|
|
290
|
-
|
|
249
|
+
## Coverage criteria
|
|
250
|
+
|
|
251
|
+
Line coverage is on by default. You can additionally enable branch, method, oneshot-line, and eval coverage, and choose
|
|
252
|
+
which criterion leads the report.
|
|
291
253
|
|
|
292
|
-
|
|
293
|
-
|
|
254
|
+
### Disabling line coverage
|
|
255
|
+
|
|
256
|
+
If you want a branch-only or method-only run (e.g. you find the line numbers noisy in CI and only care about whether
|
|
257
|
+
each conditional was exercised), enable the criterion you want and then disable line coverage:
|
|
294
258
|
|
|
295
259
|
```ruby
|
|
296
260
|
SimpleCov.start do
|
|
297
261
|
enable_coverage :branch
|
|
262
|
+
disable_coverage :line
|
|
298
263
|
end
|
|
299
264
|
```
|
|
300
265
|
|
|
301
|
-
|
|
302
|
-
|
|
303
|
-
|
|
266
|
+
If you disable every criterion, `SimpleCov.start` raises `SimpleCov::ConfigurationError` — at least one of `:line`,
|
|
267
|
+
`:branch`, or `:method` must remain enabled.
|
|
268
|
+
|
|
269
|
+
### Branch coverage
|
|
270
|
+
|
|
271
|
+
Branch coverage records whether each branch of a condition executed, not just whether a line ran.
|
|
304
272
|
|
|
305
|
-
|
|
273
|
+
```ruby
|
|
274
|
+
SimpleCov.start do
|
|
275
|
+
enable_coverage :branch
|
|
276
|
+
end
|
|
277
|
+
```
|
|
278
|
+
|
|
279
|
+
It's handy for one-line conditionals:
|
|
306
280
|
|
|
307
281
|
```ruby
|
|
308
282
|
number.odd? ? "odd" : "even"
|
|
309
283
|
```
|
|
310
284
|
|
|
311
|
-
|
|
312
|
-
|
|
285
|
+
Line coverage always marks this line as executed, but never tells you whether both arms were taken. Guard clauses have
|
|
286
|
+
the same story:
|
|
313
287
|
|
|
314
288
|
```ruby
|
|
315
289
|
return if number.odd?
|
|
@@ -317,103 +291,178 @@ return if number.odd?
|
|
|
317
291
|
# more code
|
|
318
292
|
```
|
|
319
293
|
|
|
320
|
-
If
|
|
321
|
-
|
|
294
|
+
If the whole method is covered you still won't know whether the guard ever triggered — line coverage just sees the
|
|
295
|
+
condition evaluated.
|
|
296
|
+
|
|
297
|
+
In the HTML report, lines are annotated as `branch_type: hit_count`:
|
|
298
|
+
|
|
299
|
+
* `then: 2` — the `then` branch (of an `if`) was executed twice
|
|
300
|
+
* `else: 0` — the `else` branch (of an `if` or `case`) was never executed
|
|
322
301
|
|
|
323
|
-
|
|
302
|
+
Even if you don't write an `else` branch, it still shows up: a missed implicit `else` means the `if` condition was
|
|
303
|
+
never false, or no `when` of a `case` matched.
|
|
324
304
|
|
|
325
|
-
|
|
326
|
-
|
|
305
|
+
**Is branch coverage strictly better?** No. Branch coverage only concerns itself with conditionals — coverage of
|
|
306
|
+
sequential code is of no interest to it. A file with no conditional logic has no branch data, and SimpleCov reports its
|
|
307
|
+
0-of-0 branches as 100% (everything coverable was covered). So look at both metrics together: missing 10% of your lines
|
|
308
|
+
might account for 50% of your branches.
|
|
327
309
|
|
|
328
|
-
|
|
329
|
-
reports meaning that the condition of the `if` was not hit or that no `when` of `case`
|
|
330
|
-
was hit during the test runs.
|
|
310
|
+
#### Ignoring implicit `else` branches
|
|
331
311
|
|
|
332
|
-
|
|
333
|
-
|
|
334
|
-
|
|
335
|
-
|
|
312
|
+
Ruby's `Coverage` library reports an `:else` branch for several constructs even when the source has no literal `else`
|
|
313
|
+
keyword — exhaustive `case/in` pattern matches, `case/when` without an `else` arm, `||=` / `&&=`, and `if` / `unless`
|
|
314
|
+
without an `else`. Those synthetic branches show up as missed and depress the branch-coverage percentage despite there
|
|
315
|
+
being no code to test. If your style relies on exhaustive pattern matching (or you just want `||=` to stop tanking
|
|
316
|
+
coverage), opt out:
|
|
336
317
|
|
|
337
|
-
|
|
338
|
-
|
|
339
|
-
|
|
318
|
+
```ruby
|
|
319
|
+
SimpleCov.start do
|
|
320
|
+
enable_coverage :branch
|
|
321
|
+
ignore_branches :implicit_else
|
|
322
|
+
end
|
|
323
|
+
```
|
|
324
|
+
|
|
325
|
+
`ignore_branches` is variadic; `:implicit_else` and `:eval_generated` (below) are the supported tokens. Calling it
|
|
326
|
+
before (or without) `enable_coverage :branch` is harmless: the setting is stored and applies once branch coverage is
|
|
327
|
+
enabled. Explicit `else` arms still count.
|
|
340
328
|
|
|
341
|
-
|
|
329
|
+
#### Ignoring eval-generated branches and methods
|
|
342
330
|
|
|
343
|
-
|
|
331
|
+
Rails' `delegate` (and other macros that call `module_eval(body, __FILE__, __LINE__)`) make Ruby's `Coverage` library
|
|
332
|
+
attribute the eval'd code to the macro's source line. The result is a `delegate :foo, to: :bar` line that surfaces in
|
|
333
|
+
the report as if it had its own `def foo` and an `if` branch — both reported as missed when the delegated method isn't
|
|
334
|
+
called from the suite. Drop those synthetic entries:
|
|
344
335
|
|
|
345
336
|
```ruby
|
|
346
|
-
# or in configure SimpleCov.primary_coverage :branch
|
|
347
337
|
SimpleCov.start do
|
|
348
338
|
enable_coverage :branch
|
|
349
|
-
|
|
339
|
+
enable_coverage :method
|
|
340
|
+
ignore_branches :eval_generated
|
|
341
|
+
ignore_methods :eval_generated
|
|
350
342
|
end
|
|
351
343
|
```
|
|
352
344
|
|
|
353
|
-
|
|
345
|
+
`ignore_methods` is variadic; `:eval_generated` is the only supported token. Both filters detect eval-generated entries
|
|
346
|
+
by walking the static source with [Prism](https://github.com/ruby/prism) and dropping any Coverage entry whose start
|
|
347
|
+
line lacks a real `def` keyword (for methods) or branch construct (for branches). Prism is bundled with Ruby 3.3+; on
|
|
348
|
+
older Rubies `gem install prism` enables the filter, otherwise it's a silent no-op. Real `def`s and branches that share
|
|
349
|
+
a line with an eval-generated entry are kept (line-presence is the matcher).
|
|
354
350
|
|
|
355
|
-
|
|
351
|
+
### Oneshot lines coverage
|
|
356
352
|
|
|
357
|
-
|
|
353
|
+
Oneshot lines coverage is a faster alternative to line coverage.
|
|
358
354
|
|
|
359
|
-
|
|
355
|
+
Traditional coverage records *how many times* each line ran. Often it's enough to know *whether* each line ran at
|
|
356
|
+
least once — and the counting just adds overhead. Oneshot coverage records only the first execution of each line; the
|
|
357
|
+
hook for each line fires once, after which the program runs with zero overhead.
|
|
360
358
|
|
|
361
359
|
```ruby
|
|
362
360
|
SimpleCov.start do
|
|
363
|
-
|
|
361
|
+
enable_coverage :oneshot_line
|
|
362
|
+
primary_coverage :oneshot_line
|
|
363
|
+
end
|
|
364
|
+
```
|
|
365
|
+
|
|
366
|
+
### Eval coverage
|
|
367
|
+
|
|
368
|
+
You can measure coverage for code evaluated by `Kernel#eval`. Supported in CRuby 3.2+.
|
|
369
|
+
|
|
370
|
+
```ruby
|
|
371
|
+
SimpleCov.start do
|
|
372
|
+
enable_coverage :eval
|
|
373
|
+
end
|
|
374
|
+
```
|
|
375
|
+
|
|
376
|
+
This is typically useful for ERB. Set `ERB#filename=` so SimpleCov can trace the original `.erb` source file.
|
|
377
|
+
|
|
378
|
+
### Primary coverage
|
|
379
|
+
|
|
380
|
+
By default the primary coverage type is `line`. The primary type determines what comes first in all output, and which
|
|
381
|
+
type is checked when you customize exit behavior without naming a type (e.g. `SimpleCov.minimum_coverage 90`). To change
|
|
382
|
+
it:
|
|
383
|
+
|
|
384
|
+
```ruby
|
|
385
|
+
SimpleCov.start do
|
|
386
|
+
enable_coverage :branch
|
|
387
|
+
primary_coverage :branch
|
|
364
388
|
end
|
|
389
|
+
|
|
390
|
+
# or, outside a block:
|
|
391
|
+
SimpleCov.primary_coverage :branch
|
|
365
392
|
```
|
|
366
393
|
|
|
367
|
-
|
|
394
|
+
Coverage must first be enabled for non-default types.
|
|
368
395
|
|
|
369
396
|
## Filters
|
|
370
397
|
|
|
371
|
-
Filters
|
|
372
|
-
files OUTSIDE of your project's root directory - otherwise you'd end up with billions of coverage reports for source
|
|
373
|
-
files in the gems you are using.
|
|
398
|
+
Filters remove selected files from your coverage data.
|
|
374
399
|
|
|
375
|
-
|
|
376
|
-
|
|
400
|
+
### Default filters
|
|
401
|
+
|
|
402
|
+
`SimpleCov.start` loads four filters out of the box:
|
|
403
|
+
|
|
404
|
+
* **`root_filter`** — drops every file outside of `SimpleCov.root`, so you don't end up with coverage reports for the
|
|
405
|
+
source files of every gem in your bundle. (See [Covering files outside the root](#covering-files-outside-the-root).)
|
|
406
|
+
* **`bundler_filter`** — drops `/vendor/bundle/` (in case a project keeps its gems checked into the repo).
|
|
407
|
+
* **`hidden_filter`** — drops any path that starts with a dot, matching the regex `/\A\..*/`. This is what hides
|
|
408
|
+
`.bundle/`, `.semaphore-cache/`, and similar dotfile directories — but it also hides legitimate top-level directories
|
|
409
|
+
like `.scripts/`. If you keep code in such a directory, remove this filter (see below).
|
|
410
|
+
* **`test_frameworks`** — drops common test directories (`test/`, `spec/`, `features/`, `autotest/`), matching the
|
|
411
|
+
regex `%r{\A(test|features|spec|autotest)/}`. Running the test suite always executes 100% of the test files
|
|
412
|
+
themselves, which inflates the overall percentage and obscures application coverage. Remove this filter if you
|
|
413
|
+
prefer to see test files in the report (e.g. to surface dead helpers).
|
|
414
|
+
|
|
415
|
+
For a clean slate (no defaults at all), `require 'simplecov/no_defaults'` *before* `require 'simplecov'`, or call
|
|
416
|
+
`SimpleCov.clear_filters` from your config block. To drop a specific default while keeping the others, use
|
|
417
|
+
`remove_filter`:
|
|
418
|
+
|
|
419
|
+
```ruby
|
|
420
|
+
SimpleCov.start do
|
|
421
|
+
remove_filter(/\A\..*/) # restore coverage for .scripts/, .tooling/, etc.
|
|
422
|
+
end
|
|
423
|
+
```
|
|
424
|
+
|
|
425
|
+
`remove_filter` matches by value, so pass back the same `String` or `Regexp` the default profile used. For filters
|
|
426
|
+
added with a block, pass the same `Proc` object you originally handed to `skip`.
|
|
377
427
|
|
|
378
428
|
### Defining custom filters
|
|
379
429
|
|
|
380
|
-
|
|
381
|
-
file's path), a block
|
|
430
|
+
Define your own filters to remove configuration files, tests, or anything else you don't need in the report. A filter
|
|
431
|
+
can be a String or Regexp (Regexp-matched against each source file's path), a block, your own Filter class, or an array
|
|
432
|
+
of any of these.
|
|
382
433
|
|
|
383
434
|
#### String filter
|
|
384
435
|
|
|
385
436
|
```ruby
|
|
386
437
|
SimpleCov.start do
|
|
387
|
-
|
|
438
|
+
skip "/test/"
|
|
388
439
|
end
|
|
389
440
|
```
|
|
390
441
|
|
|
391
|
-
|
|
442
|
+
Removes all files whose path matches "/test/".
|
|
392
443
|
|
|
393
444
|
#### Regex filter
|
|
394
445
|
|
|
395
446
|
```ruby
|
|
396
447
|
SimpleCov.start do
|
|
397
|
-
|
|
448
|
+
skip %r{^/test/}
|
|
398
449
|
end
|
|
399
450
|
```
|
|
400
451
|
|
|
401
|
-
|
|
452
|
+
Removes all files whose path starts with /test/.
|
|
402
453
|
|
|
403
454
|
#### Block filter
|
|
404
455
|
|
|
405
456
|
```ruby
|
|
406
457
|
SimpleCov.start do
|
|
407
|
-
|
|
458
|
+
skip do |source_file|
|
|
408
459
|
source_file.lines.count < 5
|
|
409
460
|
end
|
|
410
461
|
end
|
|
411
462
|
```
|
|
412
463
|
|
|
413
|
-
Block filters receive a SimpleCov::SourceFile
|
|
414
|
-
|
|
415
|
-
learn about the methods available to you. In the above example, the filter will remove all files that have less than 5
|
|
416
|
-
lines of code.
|
|
464
|
+
Block filters receive a `SimpleCov::SourceFile` and return `true` to remove the file or `false` to keep it. (See the
|
|
465
|
+
RDoc for `SimpleCov::SourceFile` for the available methods.) The example above removes files with fewer than 5 lines.
|
|
417
466
|
|
|
418
467
|
#### Custom filter class
|
|
419
468
|
|
|
@@ -424,104 +473,234 @@ class LineFilter < SimpleCov::Filter
|
|
|
424
473
|
end
|
|
425
474
|
end
|
|
426
475
|
|
|
427
|
-
SimpleCov.
|
|
476
|
+
SimpleCov.skip LineFilter.new(5)
|
|
428
477
|
```
|
|
429
478
|
|
|
430
|
-
|
|
431
|
-
|
|
432
|
-
given source_file. The filter_argument method is being set in the SimpleCov::Filter initialize method and thus is set to
|
|
433
|
-
5 in this example.
|
|
479
|
+
Inherit from `SimpleCov::Filter` and define `matches?(source_file)`; a `true` return removes the file. The
|
|
480
|
+
`filter_argument` is set in the `SimpleCov::Filter` initializer — `5` in this example.
|
|
434
481
|
|
|
435
482
|
#### Array filter
|
|
436
483
|
|
|
437
484
|
```ruby
|
|
438
485
|
SimpleCov.start do
|
|
439
486
|
proc = Proc.new { |source_file| false }
|
|
440
|
-
|
|
487
|
+
skip ["string", /regex/, proc, LineFilter.new(5)]
|
|
441
488
|
end
|
|
442
489
|
```
|
|
443
490
|
|
|
444
|
-
|
|
491
|
+
Pass an array containing any of the other filter types.
|
|
445
492
|
|
|
446
|
-
|
|
493
|
+
### Ignoring/skipping code
|
|
447
494
|
|
|
448
|
-
|
|
495
|
+
Disable coverage for a span of code with `# simplecov:disable` and `# simplecov:enable` comments. The available
|
|
496
|
+
categories are `line`, `branch`, and `method`; combine them with commas, and omit them to target all three. Anything
|
|
497
|
+
trailing the directive is treated as a free-form reason and ignored — no separator is required, though `--` or any
|
|
498
|
+
other marker is fine if you prefer one.
|
|
449
499
|
|
|
450
500
|
```ruby
|
|
451
|
-
# :
|
|
452
|
-
def
|
|
501
|
+
# simplecov:disable line
|
|
502
|
+
def skipped_lines
|
|
453
503
|
never_reached
|
|
454
504
|
end
|
|
455
|
-
# :
|
|
505
|
+
# simplecov:enable line
|
|
506
|
+
|
|
507
|
+
# simplecov:disable branch, method legacy adapter, scheduled for removal
|
|
508
|
+
class LegacyAdapter
|
|
509
|
+
def call(value)
|
|
510
|
+
value ? :yes : :no
|
|
511
|
+
end
|
|
512
|
+
end
|
|
513
|
+
# simplecov:enable
|
|
514
|
+
|
|
515
|
+
raise "absurd" # simplecov:disable
|
|
456
516
|
```
|
|
457
517
|
|
|
458
|
-
|
|
518
|
+
Inline directives (trailing real code) only affect the line they sit on. Block directives sit on their own line and
|
|
519
|
+
remain in effect until the matching `# simplecov:enable` for the same category — or end of file if never closed.
|
|
520
|
+
Directive markers inside string literals or heredocs are ignored.
|
|
459
521
|
|
|
460
|
-
|
|
461
|
-
|
|
462
|
-
|
|
522
|
+
> [!WARNING]
|
|
523
|
+
> The older `# :nocov:` toggle still works but is **deprecated** and will be removed in a future release. Each file
|
|
524
|
+
> that uses it emits a one-time deprecation warning pointing at the recommended `# simplecov:disable` /
|
|
525
|
+
> `# simplecov:enable` replacement. The configurable token name (`SimpleCov.nocov_token`) is similarly deprecated.
|
|
463
526
|
|
|
464
|
-
|
|
527
|
+
> [!NOTE]
|
|
528
|
+
> You shouldn't have to skip private methods that are included in your coverage. If you appropriately test the public
|
|
529
|
+
> interface of your classes and objects, you should automatically get full coverage of your private methods.
|
|
465
530
|
|
|
466
|
-
|
|
467
|
-
to include coverage reports for things you include as a gem, for example a Rails Engine.
|
|
531
|
+
### How `cover` and `skip` interact
|
|
468
532
|
|
|
469
|
-
|
|
470
|
-
that
|
|
533
|
+
`cover` and `skip` operate on different sides of the same chain. `skip` (and its deprecated `add_filter` alias) drops
|
|
534
|
+
matching files from the report. `cover` declares a positive scope that restricts the final report to files matching at
|
|
535
|
+
least one `cover` matcher.
|
|
536
|
+
|
|
537
|
+
Order: `skip` runs first, then `cover`. A file matched by any `skip` filter is dropped before `cover` is consulted, so
|
|
538
|
+
a file matched by both is dropped, not kept. The two are not commutative.
|
|
471
539
|
|
|
472
540
|
```ruby
|
|
541
|
+
SimpleCov.start do
|
|
542
|
+
cover "{app,lib}/**/*.rb"
|
|
543
|
+
skip "app/legacy"
|
|
544
|
+
end
|
|
545
|
+
```
|
|
546
|
+
|
|
547
|
+
That config covers `app/` and `lib/`, then drops `app/legacy/`. With only `cover` and no overlapping `skip`, every
|
|
548
|
+
configured default filter (hidden files, vendored gems, test directories) still applies — `cover` doesn't bypass them.
|
|
549
|
+
Use `no_default_skips` to opt out of the defaults wholesale before adding your own.
|
|
550
|
+
|
|
551
|
+
`cover` also expands string-glob matchers on disk so files that exist but were never `require`'d during the run still
|
|
552
|
+
appear in the report (at 0% coverage). Regexp and Proc cover matchers don't trigger disk discovery — they only filter
|
|
553
|
+
the universe of files that Ruby's `Coverage` library reported.
|
|
554
|
+
|
|
555
|
+
### Covering files outside the root
|
|
556
|
+
|
|
557
|
+
The `root_filter` drops every file outside of `SimpleCov.root` from the raw coverage data before any other filters or
|
|
558
|
+
groups run, so paths you might want to track (a Rails Engine installed as a gem, sibling directories in a Docker
|
|
559
|
+
layout, etc.) never reach your filter chain. To include them, widen `SimpleCov.root` to a directory that contains both
|
|
560
|
+
the project and the extra paths — `'/'` works when there's no useful common ancestor — and then express the
|
|
561
|
+
inclusion/exclusion as filters or groups:
|
|
562
|
+
|
|
563
|
+
```ruby
|
|
564
|
+
SimpleCov.root '/'
|
|
473
565
|
SimpleCov.start :rails do
|
|
474
|
-
|
|
475
|
-
add_filter do |src|
|
|
476
|
-
!(src.filename =~ /^#{SimpleCov.root}/) unless src.filename =~ /my_engine/
|
|
477
|
-
end
|
|
566
|
+
skip { |src| !src.filename.start_with?(Rails.root.to_s, '/path/to/my_engine') }
|
|
478
567
|
end
|
|
479
568
|
```
|
|
480
569
|
|
|
481
570
|
## Groups
|
|
482
571
|
|
|
483
|
-
|
|
484
|
-
|
|
485
|
-
|
|
486
|
-
|
|
487
|
-
|
|
488
|
-
Add your groups with:
|
|
572
|
+
Separate your source files into groups — for example, a Rails app might list Models, Controllers, Helpers, and Libs
|
|
573
|
+
separately. Group definition works like filters (and also accepts custom filter classes), but a source file ends up in
|
|
574
|
+
a group when the filter *passes* (returns `true`), as opposed to being excluded from results when a filter returns
|
|
575
|
+
`true`.
|
|
489
576
|
|
|
490
577
|
```ruby
|
|
491
578
|
SimpleCov.start do
|
|
492
|
-
|
|
493
|
-
|
|
494
|
-
|
|
579
|
+
group "Models", "app/models"
|
|
580
|
+
group "Controllers", "app/controllers"
|
|
581
|
+
group "Long files" do |src_file|
|
|
495
582
|
src_file.lines.count > 100
|
|
496
583
|
end
|
|
497
|
-
|
|
498
|
-
|
|
584
|
+
group "Multiple Files", ["app/models", "app/controllers"] # You can also pass in an array
|
|
585
|
+
group "Short files", LineFilter.new(5) # Using the LineFilter class defined in the Filters section above
|
|
499
586
|
end
|
|
500
587
|
```
|
|
501
588
|
|
|
502
|
-
##
|
|
589
|
+
## Profiles
|
|
503
590
|
|
|
504
|
-
|
|
591
|
+
By default, SimpleCov's only assumption is that you want coverage for files inside your project root. To avoid
|
|
592
|
+
repetitive configuration, you can use predefined blocks of configuration called 'profiles', or define your own. Pass a
|
|
593
|
+
profile's name as the first argument to `SimpleCov.start`.
|
|
505
594
|
|
|
506
|
-
|
|
507
|
-
(coverage_path)/.resultset.json, and will merge or override those with
|
|
508
|
-
subsequent runs, depending on whether simplecov considers those subsequent runs
|
|
509
|
-
as different test suites or as the same test suite as the cached results. To
|
|
510
|
-
make this distinction, simplecov has the concept of "test suite names".
|
|
595
|
+
SimpleCov bundles a `rails` profile that looks roughly like this:
|
|
511
596
|
|
|
512
|
-
|
|
597
|
+
```ruby
|
|
598
|
+
SimpleCov.profiles.define 'rails' do
|
|
599
|
+
skip '/test/'
|
|
600
|
+
skip '/config/'
|
|
601
|
+
|
|
602
|
+
group 'Controllers', 'app/controllers'
|
|
603
|
+
group 'Models', 'app/models'
|
|
604
|
+
group 'Helpers', 'app/helpers'
|
|
605
|
+
group 'Libraries', 'lib'
|
|
606
|
+
end
|
|
607
|
+
```
|
|
608
|
+
|
|
609
|
+
It's just a `SimpleCov.configure` block. Launch it from your test helper, optionally adding more config:
|
|
513
610
|
|
|
514
|
-
|
|
515
|
-
|
|
516
|
-
|
|
611
|
+
```ruby
|
|
612
|
+
SimpleCov.start 'rails'
|
|
613
|
+
|
|
614
|
+
# or
|
|
615
|
+
|
|
616
|
+
SimpleCov.start 'rails' do
|
|
617
|
+
# additional config here
|
|
618
|
+
end
|
|
619
|
+
```
|
|
517
620
|
|
|
518
|
-
|
|
519
|
-
cue as to what the name of the currently running test suite is. You can do so by specifying
|
|
520
|
-
`SimpleCov.command_name` in one test file that is part of your specific suite.
|
|
621
|
+
### The `strict` profile
|
|
521
622
|
|
|
522
|
-
|
|
523
|
-
|
|
524
|
-
|
|
623
|
+
For projects that have already reached full coverage (or want to ratchet up to it), the bundled `strict` profile
|
|
624
|
+
enables line, branch, and method coverage and pins each minimum threshold at 100%:
|
|
625
|
+
|
|
626
|
+
```ruby
|
|
627
|
+
SimpleCov.start 'strict'
|
|
628
|
+
```
|
|
629
|
+
|
|
630
|
+
That's equivalent to:
|
|
631
|
+
|
|
632
|
+
```ruby
|
|
633
|
+
SimpleCov.start do
|
|
634
|
+
enable_coverage :branch
|
|
635
|
+
enable_coverage :method
|
|
636
|
+
enable_coverage :eval if Coverage.respond_to?(:supported?) && Coverage.supported?(:eval)
|
|
637
|
+
minimum_coverage line: 100, branch: 100, method: 100
|
|
638
|
+
end
|
|
639
|
+
```
|
|
640
|
+
|
|
641
|
+
The profile drops the branch / method clauses on engines that don't support those criteria (JRuby), so it still loads
|
|
642
|
+
cleanly there, enforcing line coverage at 100%. `:eval` is included on Ruby 3.2+ (where the runtime supports it), so
|
|
643
|
+
any code reached through `Kernel#eval` — typically ERB templates with `ERB#filename=` set — is held to the same 100%
|
|
644
|
+
bar. On older Rubies, the `:eval` clause is silently skipped.
|
|
645
|
+
|
|
646
|
+
### Custom profiles
|
|
647
|
+
|
|
648
|
+
Load additional profiles with `SimpleCov.load_profile('xyz')`. This lets you build on an existing profile and reuse
|
|
649
|
+
it across unit tests and Cucumber features:
|
|
650
|
+
|
|
651
|
+
```ruby
|
|
652
|
+
# lib/simplecov_custom_profile.rb
|
|
653
|
+
require 'simplecov'
|
|
654
|
+
SimpleCov.profiles.define 'myprofile' do
|
|
655
|
+
load_profile 'rails'
|
|
656
|
+
skip 'vendor' # Don't include vendored stuff
|
|
657
|
+
end
|
|
658
|
+
|
|
659
|
+
# features/support/env.rb
|
|
660
|
+
require 'simplecov_custom_profile'
|
|
661
|
+
SimpleCov.start 'myprofile'
|
|
662
|
+
|
|
663
|
+
# test/test_helper.rb
|
|
664
|
+
require 'simplecov_custom_profile'
|
|
665
|
+
SimpleCov.start 'myprofile'
|
|
666
|
+
```
|
|
667
|
+
|
|
668
|
+
### Profile plugin gems
|
|
669
|
+
|
|
670
|
+
If `SimpleCov.start "<name>"` doesn't find a profile registered under `<name>`, the bundled profile loader tries to
|
|
671
|
+
autoload one in two steps: first `require "simplecov/profiles/<name>"` (where bundled profiles like `rails` and
|
|
672
|
+
`strict` live), then `require "simplecov-profile-<name>"` (the conventional name for a third-party plugin gem). Either
|
|
673
|
+
require is expected to call `SimpleCov.profiles.define "<name>" do ... end` so the registered block can be applied. If
|
|
674
|
+
both requires fail or neither registers the profile, `SimpleCov.start` raises `SimpleCov::ConfigurationError`.
|
|
675
|
+
|
|
676
|
+
To publish your own profile as a gem, name it `simplecov-profile-<name>` and have its main file call
|
|
677
|
+
`SimpleCov.profiles.define`:
|
|
678
|
+
|
|
679
|
+
```ruby
|
|
680
|
+
# In a gem named simplecov-profile-myteam
|
|
681
|
+
SimpleCov.profiles.define "myteam" do
|
|
682
|
+
enable_coverage :branch
|
|
683
|
+
cover "{app,lib}/**/*.rb"
|
|
684
|
+
skip "app/legacy"
|
|
685
|
+
end
|
|
686
|
+
```
|
|
687
|
+
|
|
688
|
+
A user who adds the gem to their Gemfile can then `SimpleCov.start "myteam"` without explicitly requiring it.
|
|
689
|
+
|
|
690
|
+
## Merging results and parallel tests
|
|
691
|
+
|
|
692
|
+
You normally want coverage analyzed across ALL of your test suites at once. SimpleCov automatically caches results in
|
|
693
|
+
`(coverage_path)/.resultset.json` and merges them with subsequent runs — or overrides them, depending on whether it
|
|
694
|
+
considers a subsequent run a different test suite or the same one. To make that distinction, SimpleCov uses the concept
|
|
695
|
+
of **test suite names**.
|
|
696
|
+
|
|
697
|
+
### Test suite names
|
|
698
|
+
|
|
699
|
+
SimpleCov guesses the running suite's name from the shell command that started the tests. This works fine for Test::Unit,
|
|
700
|
+
RSpec, and Cucumber; if it fails, it falls back to the invoking shell command as the command name.
|
|
701
|
+
|
|
702
|
+
For a non-standard setup, give SimpleCov a cue with `SimpleCov.command_name` in one test file per suite (you only need
|
|
703
|
+
it once per suite — even with 200 unit test files, setting it in one is enough):
|
|
525
704
|
|
|
526
705
|
```ruby
|
|
527
706
|
# test/unit/some_test.rb
|
|
@@ -537,52 +716,36 @@ SimpleCov.command_name "test:integration"
|
|
|
537
716
|
SimpleCov.command_name "features"
|
|
538
717
|
```
|
|
539
718
|
|
|
540
|
-
|
|
541
|
-
|
|
542
|
-
|
|
543
|
-
Last but not least **if multiple suites resolve to the same `command_name`** be aware that the coverage results **will
|
|
544
|
-
clobber each other instead of being merged**. SimpleCov is smart enough to detect unique names for the most common
|
|
545
|
-
setups, but if you have more than one test suite that doesn't follow a common pattern then you will want to manually
|
|
546
|
-
ensure that each suite gets a unique `command_name`.
|
|
719
|
+
**If multiple suites resolve to the same `command_name`, their results clobber each other instead of merging.**
|
|
720
|
+
SimpleCov detects unique names for the most common setups, but if you have more than one suite that doesn't follow a
|
|
721
|
+
common pattern, ensure each gets a unique `command_name`.
|
|
547
722
|
|
|
548
|
-
|
|
549
|
-
|
|
550
|
-
`command_name`
|
|
551
|
-
|
|
552
|
-
per process (eg. `command_name "Unit Tests PID #{$$}"`).
|
|
553
|
-
|
|
554
|
-
If you are using parallel_tests, you must incorporate `TEST_ENV_NUMBER` into the command name yourself, in
|
|
555
|
-
order for SimpleCov to merge the results correctly. For example:
|
|
723
|
+
When running tests in parallel, each process can clobber the others' results. With the default `command_name`,
|
|
724
|
+
SimpleCov detects and avoids collisions based on `ENV['PARALLEL_TEST_GROUPS']` and `ENV['TEST_ENV_NUMBER']`. If your
|
|
725
|
+
runner sets neither, *you must* set a `command_name` that's unique per process (e.g. `command_name "Unit Tests PID #{$$}"`).
|
|
726
|
+
With parallel_tests specifically, incorporate `TEST_ENV_NUMBER` into the name yourself so results merge correctly:
|
|
556
727
|
|
|
557
728
|
```ruby
|
|
558
729
|
# spec/spec_helper.rb
|
|
559
730
|
SimpleCov.command_name "features" + (ENV['TEST_ENV_NUMBER'] || '')
|
|
560
731
|
```
|
|
561
732
|
|
|
562
|
-
|
|
563
|
-
|
|
564
|
-
|
|
565
|
-
### Merging test runs under the same execution environment
|
|
566
|
-
|
|
567
|
-
Test results are automatically merged with previous runs in the same execution
|
|
568
|
-
environment when generating the result, so when coverage is set up properly for
|
|
569
|
-
Cucumber and your unit / functional / integration tests, all of those test
|
|
570
|
-
suites will be taken into account when building the coverage report.
|
|
733
|
+
The HTML report prints the test suites it used in its footer.
|
|
571
734
|
|
|
572
|
-
|
|
735
|
+
### Merging within one execution environment
|
|
573
736
|
|
|
574
|
-
|
|
575
|
-
|
|
576
|
-
|
|
577
|
-
(1 hour), or, inside a configure/start block, with just `merge_timeout 3600`.
|
|
737
|
+
Results are automatically merged with previous runs in the same execution environment when the report is generated, so
|
|
738
|
+
once coverage is set up for Cucumber and your unit / functional / integration tests, all of those suites feed into one
|
|
739
|
+
report.
|
|
578
740
|
|
|
579
|
-
|
|
741
|
+
Cached coverage data eventually goes stale, so result sets older than `SimpleCov.merge_timeout` are dropped from the
|
|
742
|
+
merge. The default is 600 seconds (10 minutes); raise or lower it with `SimpleCov.merge_timeout 3600` (1 hour), or
|
|
743
|
+
`merge_timeout 3600` inside a configure/start block. Deactivate automatic merging entirely with `SimpleCov.merging false`.
|
|
580
744
|
|
|
581
|
-
### Merging
|
|
745
|
+
### Merging across execution environments
|
|
582
746
|
|
|
583
|
-
If your tests
|
|
584
|
-
result set
|
|
585
|
-
`.resultset.json` files from each parallel test run.
|
|
747
|
+
If your tests run in parallel across multiple build machines, download each run's `.resultset.json` and merge them into
|
|
748
|
+
a single result set with `SimpleCov.collate`:
|
|
586
749
|
|
|
587
750
|
```ruby
|
|
588
751
|
# lib/tasks/coverage_report.rake
|
|
@@ -596,12 +759,10 @@ namespace :coverage do
|
|
|
596
759
|
end
|
|
597
760
|
```
|
|
598
761
|
|
|
599
|
-
`SimpleCov.collate` also takes an optional
|
|
600
|
-
|
|
601
|
-
`SimpleCov
|
|
602
|
-
|
|
603
|
-
`SimpleCov.start` the `SimpleCov::Formatter::SimpleFormatter`, and only use more
|
|
604
|
-
complex formatters in the final `SimpleCov.collate` run.
|
|
762
|
+
`SimpleCov.collate` also takes an optional profile and an optional configuration block, just like `SimpleCov.start` or
|
|
763
|
+
`SimpleCov.configure`. This means you can configure a separate formatter for the collated output — for instance, use the
|
|
764
|
+
plain `SimpleCov::Formatter::SimpleFormatter` in each worker's `SimpleCov.start` and reserve heavier formatters for the
|
|
765
|
+
final `SimpleCov.collate` run:
|
|
605
766
|
|
|
606
767
|
```ruby
|
|
607
768
|
# spec/spec_helper.rb
|
|
@@ -620,7 +781,7 @@ SimpleCov.start 'rails' do
|
|
|
620
781
|
])
|
|
621
782
|
end
|
|
622
783
|
|
|
623
|
-
|
|
784
|
+
cover "{app,lib}/**/*.rb"
|
|
624
785
|
end
|
|
625
786
|
```
|
|
626
787
|
|
|
@@ -640,21 +801,19 @@ namespace :coverage do
|
|
|
640
801
|
end
|
|
641
802
|
```
|
|
642
803
|
|
|
643
|
-
|
|
644
|
-
|
|
645
|
-
`SimpleCov.enable_for_subprocesses` will allow SimpleCov to observe subprocesses starting using `Process.fork`.
|
|
646
|
-
This modifies ruby's core Process.fork method so that SimpleCov can see into it, appending `" (subprocess #{pid})"`
|
|
647
|
-
to the `SimpleCov.command_name`, with results that can be merged together using SimpleCov's merging feature.
|
|
804
|
+
### Forked subprocesses
|
|
648
805
|
|
|
649
|
-
|
|
806
|
+
`SimpleCov.merge_subprocesses true` lets SimpleCov observe subprocesses started with `Process.fork`. It wraps Ruby's
|
|
807
|
+
`Process.fork` so SimpleCov can see into the child, appending `" (subprocess #{pid})"` to the `command_name`, with
|
|
808
|
+
results that merge back together. Configure the child with `.at_fork`:
|
|
650
809
|
|
|
651
810
|
```ruby
|
|
652
|
-
SimpleCov.
|
|
811
|
+
SimpleCov.merge_subprocesses true
|
|
653
812
|
SimpleCov.at_fork do |pid|
|
|
654
|
-
# This needs a unique name so it won't be
|
|
813
|
+
# This needs a unique name so it won't be overwritten
|
|
655
814
|
SimpleCov.command_name "#{SimpleCov.command_name} (subprocess: #{pid})"
|
|
656
815
|
# be quiet, the parent process will be in charge of output and checking coverage totals
|
|
657
|
-
SimpleCov.
|
|
816
|
+
SimpleCov.print_errors false
|
|
658
817
|
SimpleCov.formatter SimpleCov::Formatter::SimpleFormatter
|
|
659
818
|
SimpleCov.minimum_coverage 0
|
|
660
819
|
# start
|
|
@@ -662,250 +821,555 @@ SimpleCov.at_fork do |pid|
|
|
|
662
821
|
end
|
|
663
822
|
```
|
|
664
823
|
|
|
665
|
-
|
|
824
|
+
SimpleCov must already be started before `Process.fork` is called.
|
|
666
825
|
|
|
667
|
-
|
|
826
|
+
> [!NOTE]
|
|
827
|
+
> The bundled `rails` profile turns this on automatically, so `ActiveSupport::TestCase.parallelize(workers: ...)`
|
|
828
|
+
> worker forks contribute to the merged report instead of being silently dropped.
|
|
668
829
|
|
|
669
|
-
|
|
670
|
-
|
|
830
|
+
#### Spawned subprocesses
|
|
831
|
+
|
|
832
|
+
You can also cover a Ruby script you launch with `PTY.spawn`, `Open3.popen`, `Process.spawn`, and the like. Add a
|
|
833
|
+
`.simplecov_spawn.rb` file to your project root:
|
|
671
834
|
|
|
672
|
-
Add a .simplecov_spawn.rb file to your project root
|
|
673
835
|
```ruby
|
|
674
836
|
# .simplecov_spawn.rb
|
|
675
|
-
require 'simplecov' # this will also pick up whatever config is in .simplecov
|
|
676
|
-
# so ensure it just contains configuration
|
|
677
|
-
SimpleCov.command_name 'spawn' # As this
|
|
678
|
-
SimpleCov.at_fork.call(Process.pid) # Use the per-process setup described
|
|
679
|
-
SimpleCov.start # only now can we start
|
|
837
|
+
require 'simplecov' # this will also pick up whatever config is in .simplecov,
|
|
838
|
+
# so ensure it just contains configuration and doesn't call SimpleCov.start.
|
|
839
|
+
SimpleCov.command_name 'spawn' # As this isn't for a test runner directly, the script has no pre-defined base command_name
|
|
840
|
+
SimpleCov.at_fork.call(Process.pid) # Use the per-process setup described above
|
|
841
|
+
SimpleCov.start # only now can we start
|
|
680
842
|
```
|
|
681
|
-
|
|
843
|
+
|
|
844
|
+
Then, instead of spawning your script directly:
|
|
845
|
+
|
|
682
846
|
```ruby
|
|
683
847
|
PTY.spawn('my_script.rb') do # ...
|
|
684
848
|
```
|
|
685
|
-
|
|
849
|
+
|
|
850
|
+
use `ruby -r` to require the spawn file first:
|
|
851
|
+
|
|
686
852
|
```ruby
|
|
687
853
|
PTY.spawn('ruby -r./.simplecov_spawn my_script.rb') do # ...
|
|
688
854
|
```
|
|
689
855
|
|
|
690
|
-
|
|
856
|
+
### Parallel-test-runner adapters
|
|
691
857
|
|
|
692
|
-
|
|
693
|
-
|
|
694
|
-
run your tests because it doesn't do your test speed any harm and you're always equipped with the latest and greatest
|
|
695
|
-
coverage results.
|
|
858
|
+
SimpleCov coordinates with parallel test runners through a small pluggable adapter interface
|
|
859
|
+
(`SimpleCov::ParallelAdapters`). Two adapters ship out of the box:
|
|
696
860
|
|
|
697
|
-
|
|
861
|
+
- **`ParallelTestsAdapter`** — wraps the [grosser/parallel_tests](https://github.com/grosser/parallel_tests) gem and
|
|
862
|
+
uses its `ParallelTests.first_process?` / `ParallelTests.wait_for_other_processes_to_finish` APIs for precise worker
|
|
863
|
+
coordination.
|
|
864
|
+
- **`GenericAdapter`** — catch-all for any runner that follows the `TEST_ENV_NUMBER` / `PARALLEL_TEST_GROUPS` env-var
|
|
865
|
+
convention but doesn't ship a Ruby API (parallel_rspec, knapsack-style splitters, custom CI sharding scripts).
|
|
866
|
+
Activates when `TEST_ENV_NUMBER` is set and no more-specific adapter is.
|
|
698
867
|
|
|
699
|
-
|
|
700
|
-
|
|
868
|
+
Adapters are tried in registration order; the first whose `active?` returns `true` is chosen. With both built-ins, this
|
|
869
|
+
means parallel_tests users get the precise gem-based path and parallel_rspec (or any env-var-only runner) gets the
|
|
870
|
+
polling-based fallback without any configuration change. See #1065.
|
|
701
871
|
|
|
702
|
-
|
|
703
|
-
SimpleCov.start if ENV["COVERAGE"]
|
|
704
|
-
```
|
|
872
|
+
#### Registering a custom adapter
|
|
705
873
|
|
|
706
|
-
|
|
874
|
+
If you use a parallel runner with different env vars or its own synchronization API, define a class that inherits from
|
|
875
|
+
`SimpleCov::ParallelAdapters::Base` and register it:
|
|
707
876
|
|
|
708
|
-
```
|
|
709
|
-
|
|
710
|
-
|
|
877
|
+
```ruby
|
|
878
|
+
# In your spec_helper.rb / test_helper.rb (before SimpleCov.start)
|
|
879
|
+
class MyRunnerAdapter < SimpleCov::ParallelAdapters::Base
|
|
880
|
+
def self.active?
|
|
881
|
+
!ENV["MY_RUNNER_PID"].nil?
|
|
882
|
+
end
|
|
711
883
|
|
|
712
|
-
|
|
884
|
+
def self.first_worker?
|
|
885
|
+
ENV["MY_RUNNER_PID"].to_i == 1
|
|
886
|
+
end
|
|
713
887
|
|
|
714
|
-
|
|
715
|
-
|
|
888
|
+
def self.wait_for_siblings
|
|
889
|
+
MyRunner.barrier! # if your runner provides a sync primitive
|
|
890
|
+
end
|
|
716
891
|
|
|
717
|
-
|
|
718
|
-
|
|
892
|
+
def self.expected_worker_count
|
|
893
|
+
ENV["MY_RUNNER_WORKERS"].to_i
|
|
894
|
+
end
|
|
895
|
+
end
|
|
896
|
+
|
|
897
|
+
SimpleCov::ParallelAdapters.register MyRunnerAdapter
|
|
719
898
|
```
|
|
720
899
|
|
|
721
|
-
|
|
900
|
+
Custom adapters are inserted at the front of the selection chain, so they take precedence over the built-ins. `Base`
|
|
901
|
+
provides safe no-op defaults for any method you don't override (single-process semantics: `active?` returns `false`,
|
|
902
|
+
`first_worker?` returns `true`, etc.).
|
|
722
903
|
|
|
723
|
-
|
|
724
|
-
|
|
904
|
+
## Coverage thresholds and exit behavior
|
|
905
|
+
|
|
906
|
+
Define what SimpleCov does when your test suite finishes by customizing the `at_exit` hook. The default is shown below;
|
|
907
|
+
do whatever you like instead:
|
|
908
|
+
|
|
909
|
+
```ruby
|
|
910
|
+
SimpleCov.at_exit do
|
|
911
|
+
SimpleCov.result.format!
|
|
912
|
+
end
|
|
725
913
|
```
|
|
726
914
|
|
|
727
|
-
|
|
915
|
+
The threshold settings below make SimpleCov exit non-zero when coverage doesn't meet your expectations, so they double
|
|
916
|
+
as CI gates.
|
|
728
917
|
|
|
729
|
-
|
|
730
|
-
root. To save yourself from repetitive configuration, you can use predefined blocks of configuration, called 'profiles',
|
|
731
|
-
or define your own.
|
|
918
|
+
### Per-criterion thresholds with `coverage`
|
|
732
919
|
|
|
733
|
-
|
|
734
|
-
|
|
920
|
+
The `coverage` block configures each criterion (line, branch, method) the same way: because the criterion is fixed by
|
|
921
|
+
the enclosing block, every threshold value is a plain percentage, so line, branch, and method coverage read identically.
|
|
922
|
+
Naming a criterion also enables it (line is enabled by default).
|
|
735
923
|
|
|
736
924
|
```ruby
|
|
737
|
-
SimpleCov.
|
|
738
|
-
|
|
739
|
-
|
|
925
|
+
SimpleCov.start do
|
|
926
|
+
coverage :line do
|
|
927
|
+
minimum 90 # suite-wide minimum; SimpleCov exits non-zero if unmet
|
|
928
|
+
minimum_per_file 80 # per-file minimum
|
|
929
|
+
minimum_per_file 100, only: "app/mailers/request_mailer.rb" # per-path override (String path or Regexp)
|
|
930
|
+
minimum_per_group 95, only: "Models" # minimum for a named group
|
|
931
|
+
maximum_drop 5 # exit non-zero if coverage drops more than 5% between runs
|
|
932
|
+
end
|
|
740
933
|
|
|
741
|
-
|
|
742
|
-
|
|
743
|
-
add_group 'Helpers', 'app/helpers'
|
|
744
|
-
add_group 'Libraries', 'lib'
|
|
934
|
+
coverage :branch, minimum: 80 # one-liner form for a single setting
|
|
935
|
+
coverage :method, minimum: 100
|
|
745
936
|
end
|
|
746
937
|
```
|
|
747
938
|
|
|
748
|
-
|
|
939
|
+
| Verb | Effect |
|
|
940
|
+
|------|--------|
|
|
941
|
+
| `minimum N` | Suite-wide minimum for this criterion. |
|
|
942
|
+
| `maximum N` | Suite-wide maximum: fails if coverage rises above N. Pairs with `minimum` to pin coverage so an unexpected jump fails instead of being silently absorbed. |
|
|
943
|
+
| `exact N` | Pins coverage by setting both `minimum` and `maximum` to N. |
|
|
944
|
+
| `maximum_drop N` | Maximum allowed drop between runs (`maximum_drop 0` refuses any drop). |
|
|
945
|
+
| `minimum_per_file N` | Per-file minimum. Add `only: "path"` / `only: %r{regexp}` to override it for matching files (later, more specific overrides win). |
|
|
946
|
+
| `minimum_per_group N, only: "Name"` | Minimum for a named [group](#groups). |
|
|
947
|
+
|
|
948
|
+
Every verb is also a keyword on the one-liner form (`coverage :branch, minimum: 80, maximum_drop: 5`). Two more options:
|
|
949
|
+
`coverage :line, oneshot: true` selects the faster [oneshot-lines mode](#oneshot-lines-coverage), and
|
|
950
|
+
`coverage :branch, primary: true` makes branch the report's leading criterion (the one a bare `minimum_coverage 90`
|
|
951
|
+
targets). `coverage :eval` enables [eval coverage](#eval-coverage).
|
|
952
|
+
|
|
953
|
+
### Suite-wide shortcuts
|
|
954
|
+
|
|
955
|
+
For the common case of a single suite-wide threshold, the flat helpers are convenient sugar over the block above. A bare
|
|
956
|
+
number targets the primary criterion (line by default); a Hash sets per-criterion values:
|
|
749
957
|
|
|
750
958
|
```ruby
|
|
751
|
-
SimpleCov.
|
|
959
|
+
SimpleCov.minimum_coverage 90 # primary criterion (line)
|
|
960
|
+
SimpleCov.minimum_coverage line: 90, branch: 80
|
|
961
|
+
SimpleCov.maximum_coverage line: 90
|
|
962
|
+
SimpleCov.maximum_coverage_drop line: 5, branch: 10
|
|
963
|
+
SimpleCov.expected_coverage 95.42 # pins minimum == maximum
|
|
964
|
+
SimpleCov.refuse_coverage_drop :line, :branch # maximum drop of 0
|
|
752
965
|
```
|
|
753
966
|
|
|
754
|
-
|
|
967
|
+
`expected_coverage` floors the actual percentage to two decimal places, so an actual of 95.4287 still passes at
|
|
968
|
+
`expected_coverage 95.42`.
|
|
969
|
+
|
|
970
|
+
> [!NOTE]
|
|
971
|
+
> `minimum_coverage_by_file` and `minimum_coverage_by_group` are **deprecated** in favor of the `coverage` block's
|
|
972
|
+
> `minimum_per_file` / `minimum_per_group`. They still work but emit a deprecation warning. For example, replace
|
|
973
|
+
> `minimum_coverage_by_file line: 70, 'app/x.rb' => 100` with:
|
|
974
|
+
>
|
|
975
|
+
> ```ruby
|
|
976
|
+
> coverage :line do
|
|
977
|
+
> minimum_per_file 70
|
|
978
|
+
> minimum_per_file 100, only: "app/x.rb"
|
|
979
|
+
> end
|
|
980
|
+
> ```
|
|
981
|
+
|
|
982
|
+
## Formatters
|
|
983
|
+
|
|
984
|
+
### Using your own formatter
|
|
755
985
|
|
|
756
986
|
```ruby
|
|
757
|
-
SimpleCov.
|
|
758
|
-
# additional config here
|
|
759
|
-
end
|
|
987
|
+
SimpleCov.formatter = SimpleCov::Formatter::HTMLFormatter
|
|
760
988
|
```
|
|
761
989
|
|
|
762
|
-
|
|
990
|
+
`SimpleCov.result.format!` then invokes `SimpleCov::Formatter::YourFormatter.new.format(result)`, where `result` is a
|
|
991
|
+
`SimpleCov::Result`. Do whatever you wish with it.
|
|
763
992
|
|
|
764
|
-
|
|
765
|
-
existing profile and customize it so you can reuse it in unit tests and Cucumber features. For example:
|
|
993
|
+
### Using multiple formatters
|
|
766
994
|
|
|
767
|
-
|
|
768
|
-
|
|
769
|
-
|
|
770
|
-
|
|
771
|
-
load_profile 'rails'
|
|
772
|
-
add_filter 'vendor' # Don't include vendored stuff
|
|
773
|
-
end
|
|
995
|
+
As of SimpleCov 0.9 you can specify multiple result formats. The HTML and JSON formatters are built in; other
|
|
996
|
+
formatters ship as separate gems you'll need to add and require — for example,
|
|
997
|
+
[simplecov-cobertura](https://github.com/dashingrocket/simplecov-cobertura) for the Cobertura XML that many CI services
|
|
998
|
+
consume.
|
|
774
999
|
|
|
775
|
-
|
|
776
|
-
require
|
|
777
|
-
SimpleCov.start 'myprofile'
|
|
1000
|
+
```ruby
|
|
1001
|
+
require "simplecov-cobertura"
|
|
778
1002
|
|
|
779
|
-
|
|
780
|
-
|
|
781
|
-
SimpleCov
|
|
1003
|
+
SimpleCov.formatters = [
|
|
1004
|
+
SimpleCov::Formatter::HTMLFormatter,
|
|
1005
|
+
SimpleCov::Formatter::CoberturaFormatter,
|
|
1006
|
+
]
|
|
782
1007
|
```
|
|
783
1008
|
|
|
784
|
-
|
|
1009
|
+
### JSON formatter
|
|
785
1010
|
|
|
786
|
-
|
|
1011
|
+
`SimpleCov::Formatter::JSONFormatter` emits JSON — useful for CI consumption or reporting to external services.
|
|
787
1012
|
|
|
788
1013
|
```ruby
|
|
789
|
-
SimpleCov.
|
|
790
|
-
|
|
1014
|
+
SimpleCov.formatter = SimpleCov::Formatter::JSONFormatter
|
|
1015
|
+
```
|
|
1016
|
+
|
|
1017
|
+
By default `coverage.json` carries the full source-text array for every file, which makes the payload self-contained
|
|
1018
|
+
but dominates the file size on larger projects. Tools that read the project's source files directly from disk can opt
|
|
1019
|
+
out of that field with:
|
|
1020
|
+
|
|
1021
|
+
```ruby
|
|
1022
|
+
SimpleCov.start do
|
|
1023
|
+
source_in_json false
|
|
791
1024
|
end
|
|
792
1025
|
```
|
|
793
1026
|
|
|
794
|
-
|
|
1027
|
+
The HTML report's `coverage_data.js` always retains the source array — the client-side viewer renders source from
|
|
1028
|
+
there. The setting only affects the side-file `coverage.json`. When the source is omitted, `meta.commit` (the git
|
|
1029
|
+
commit SHA the report was generated against) lets tools recover the exact source lines from repository history.
|
|
1030
|
+
|
|
1031
|
+
> The JSON formatter was originally a separate gem,
|
|
1032
|
+
> [simplecov_json_formatter](https://github.com/codeclimate-community/simplecov_json_formatter). It is now built in and
|
|
1033
|
+
> loaded by default; existing code that does `require "simplecov_json_formatter"` will continue to work.
|
|
1034
|
+
|
|
1035
|
+
### JSON Schema for `coverage.json`
|
|
1036
|
+
|
|
1037
|
+
`coverage.json` is a public contract, described by a JSON Schema (2020-12) so downstream tools can validate it,
|
|
1038
|
+
generate types, or pin to a known shape. Every emitted document carries a top-level `$schema` URL pointing at the
|
|
1039
|
+
versioned canonical, plus a human-readable `meta.schema_version` (`"major.minor"`).
|
|
1040
|
+
|
|
1041
|
+
The **versioned canonical** lives at [`schemas/coverage-v1.0.schema.json`](schemas/coverage-v1.0.schema.json) and
|
|
1042
|
+
long-lived integrations should pin to it. Once a SimpleCov release ships with a given versioned schema file, that file
|
|
1043
|
+
is immutable: bug fixes, additions, or shape changes ship as a new versioned file (a minor or major bump), never as a
|
|
1044
|
+
silent rewrite of an already-released one. Schemas may still be corrected in-place between gem releases — i.e., the
|
|
1045
|
+
schema file as it currently exists on `main` may change before the next gem release, but the schema for any published
|
|
1046
|
+
gem version stays frozen. A convenience alias at [`schemas/coverage.schema.json`](schemas/coverage.schema.json) always
|
|
1047
|
+
tracks the latest and may shift when a new SimpleCov release bumps the schema.
|
|
1048
|
+
|
|
1049
|
+
The schema version is independent of the gem version:
|
|
1050
|
+
|
|
1051
|
+
- Additive changes (new fields) bump the **minor** segment. Existing consumers keep working.
|
|
1052
|
+
- Removals or shape changes bump the **major** segment, and ship as a new `schemas/coverage-vX.0.schema.json` file so
|
|
1053
|
+
v1.x consumers stay valid.
|
|
1054
|
+
|
|
1055
|
+
The current version is **1.0**. Top-level structure:
|
|
1056
|
+
|
|
1057
|
+
```jsonc
|
|
1058
|
+
{
|
|
1059
|
+
"$schema": "https://raw.githubusercontent.com/simplecov-ruby/simplecov/main/schemas/coverage-v1.0.schema.json",
|
|
1060
|
+
"meta": { /* schema_version, simplecov_version, command_name, project_name, timestamp, root, commit, line_coverage, branch_coverage, method_coverage */ },
|
|
1061
|
+
"total": { /* aggregate stats for lines (and branches / methods when enabled) */ },
|
|
1062
|
+
"coverage": { "<project-relative path>": { /* per-file lines, source, branches, methods, etc. */ } },
|
|
1063
|
+
"groups": { "<group name>": { /* per-group stats + files */ } },
|
|
1064
|
+
"errors": { /* minimum_coverage, minimum_coverage_by_file, minimum_coverage_by_group, maximum_coverage, maximum_coverage_drop violations */ }
|
|
1065
|
+
}
|
|
1066
|
+
```
|
|
1067
|
+
|
|
1068
|
+
The `.resultset.json` file is **not** schema'd — it's SimpleCov-internal and may change shape across releases. Build
|
|
1069
|
+
integrations on top of `coverage.json`.
|
|
795
1070
|
|
|
796
|
-
###
|
|
1071
|
+
### More formatters, editor integrations, and hosted services
|
|
1072
|
+
|
|
1073
|
+
* [Open Source formatter and integration plugins for SimpleCov](doc/alternate-formatters.md)
|
|
1074
|
+
* [Editor Integration](doc/editor-integration.md)
|
|
1075
|
+
* [Hosted (commercial) services](doc/commercial-services.md)
|
|
797
1076
|
|
|
798
|
-
|
|
1077
|
+
## Output and diagnostics
|
|
1078
|
+
|
|
1079
|
+
### Errors and exit statuses
|
|
1080
|
+
|
|
1081
|
+
If an error is raised, SimpleCov prints a message to `STDERR` with the exit status, to aid debugging:
|
|
1082
|
+
|
|
1083
|
+
```
|
|
1084
|
+
SimpleCov failed with exit 1
|
|
1085
|
+
```
|
|
1086
|
+
|
|
1087
|
+
Disable this message with:
|
|
799
1088
|
|
|
800
1089
|
```ruby
|
|
801
|
-
SimpleCov.
|
|
802
|
-
# same as above (the default is to check line coverage)
|
|
803
|
-
SimpleCov.minimum_coverage line: 90
|
|
804
|
-
# check for a minimum line coverage of 90% and minimum 80% branch coverage
|
|
805
|
-
SimpleCov.minimum_coverage line: 90, branch: 80
|
|
1090
|
+
SimpleCov.print_errors false
|
|
806
1091
|
```
|
|
807
1092
|
|
|
808
|
-
###
|
|
1093
|
+
### Color output
|
|
809
1094
|
|
|
810
|
-
|
|
811
|
-
|
|
1095
|
+
When color is enabled, SimpleCov highlights coverage percentages in its `STDERR` diagnostics by band (green for
|
|
1096
|
+
`>= 90%`, yellow for `>= 75%`, red below) and prints the "SimpleCov failed with exit ..." summary in red. By default,
|
|
1097
|
+
color is on only when `STDERR` is a TTY. Two environment variables override that:
|
|
1098
|
+
|
|
1099
|
+
- `NO_COLOR=1` (any non-empty value) disables color even when stderr is a TTY. Honors the
|
|
1100
|
+
[no-color.org](https://no-color.org) convention.
|
|
1101
|
+
- `FORCE_COLOR=1` (any non-empty value) enables color even when stderr is not a TTY. Useful when stderr is piped through
|
|
1102
|
+
a wrapper that itself renders ANSI in a terminal (`parallel_tests --combine-stderr`, log multiplexers, some CI runners).
|
|
1103
|
+
|
|
1104
|
+
`NO_COLOR` wins if both are set.
|
|
1105
|
+
|
|
1106
|
+
For programmatic control, use `SimpleCov.color`. An explicit `true` or `false` wins over the env vars and TTY detection:
|
|
812
1107
|
|
|
813
1108
|
```ruby
|
|
814
|
-
SimpleCov.
|
|
815
|
-
#
|
|
816
|
-
SimpleCov.
|
|
817
|
-
# check for a minimum line coverage by file of 90% and minimum 80% branch coverage
|
|
818
|
-
SimpleCov.minimum_coverage_by_file line: 90, branch: 80
|
|
1109
|
+
SimpleCov.color true # always on
|
|
1110
|
+
SimpleCov.color false # always off
|
|
1111
|
+
SimpleCov.color :auto # default behavior: NO_COLOR/FORCE_COLOR/TTY
|
|
819
1112
|
```
|
|
820
1113
|
|
|
821
|
-
|
|
1114
|
+
## Command-line interface
|
|
822
1115
|
|
|
823
|
-
|
|
1116
|
+
The bundled `simplecov` CLI provides a set of subcommands. The read-only reporting commands consume the JSONFormatter's
|
|
1117
|
+
`coverage.json` output, so you don't need to re-run your suite — any prior run that emitted JSON suffices. Paths default
|
|
1118
|
+
to `SimpleCov.coverage_dir` from your project's `.simplecov` when one is present.
|
|
824
1119
|
|
|
825
|
-
|
|
826
|
-
|
|
827
|
-
|
|
828
|
-
|
|
829
|
-
|
|
830
|
-
|
|
1120
|
+
| Command | Description |
|
|
1121
|
+
|--------------------|---------------------------------------------------------------------|
|
|
1122
|
+
| `run <command…>` | Execute `<command>` with simplecov pre-loaded (no `test_helper` hook needed) |
|
|
1123
|
+
| `coverage <path>` | Print coverage stats for a single file |
|
|
1124
|
+
| `report` | Print the overall summary and per-group totals |
|
|
1125
|
+
| `uncovered` | List the lowest-coverage files |
|
|
1126
|
+
| `merge <files…>` | Merge multiple `.resultset.json` files |
|
|
1127
|
+
| `diff <baseline>` | Show per-file coverage delta vs a baseline |
|
|
1128
|
+
| `open` | Open the HTML report in the default browser |
|
|
1129
|
+
| `serve` | Serve the coverage report over HTTP |
|
|
1130
|
+
| `clean` | Remove the coverage report directory |
|
|
1131
|
+
|
|
1132
|
+
Run `simplecov help` for the full option listing.
|
|
1133
|
+
|
|
1134
|
+
### `run` — run a suite with coverage
|
|
1135
|
+
|
|
1136
|
+
If your project has no `test_helper.rb` hook that calls `SimpleCov.start` (or you don't want to add one), `simplecov run`
|
|
1137
|
+
execs your test command with simplecov pre-loaded so a report drops into `coverage/` at the end:
|
|
1138
|
+
|
|
1139
|
+
```sh
|
|
1140
|
+
$ simplecov run bundle exec rspec
|
|
1141
|
+
$ simplecov run -- bundle exec rake test
|
|
1142
|
+
$ simplecov run ruby my_test.rb
|
|
831
1143
|
```
|
|
832
1144
|
|
|
833
|
-
|
|
1145
|
+
Internally this just sets `RUBYOPT=-rsimplecov/autostart` for the child process, so any spawned subprocess (parallel
|
|
1146
|
+
test workers, integration test forks, etc.) also picks up the autostart shim. If your project already has a `.simplecov`
|
|
1147
|
+
config that calls `SimpleCov.start`, the autostart shim defers to it and won't double-start Coverage.
|
|
1148
|
+
|
|
1149
|
+
### `coverage` — per-file lookup
|
|
1150
|
+
|
|
1151
|
+
For editor / TDD inner-loop integrations and tools that want one file's coverage without re-parsing the full report:
|
|
834
1152
|
|
|
835
|
-
|
|
1153
|
+
```sh
|
|
1154
|
+
$ simplecov coverage app/models/user.rb
|
|
1155
|
+
/abs/path/app/models/user.rb
|
|
1156
|
+
Line: 100.00% (12 / 12)
|
|
1157
|
+
Branch: 100.00% (4 / 4)
|
|
1158
|
+
Method: 100.00% (3 / 3)
|
|
1159
|
+
|
|
1160
|
+
$ simplecov coverage --json app/models/user.rb # raw JSON entry
|
|
1161
|
+
$ simplecov coverage --input path/to/coverage.json … # non-default location
|
|
1162
|
+
```
|
|
1163
|
+
|
|
1164
|
+
The same lookup is available in Ruby, with paths resolved relative to `SimpleCov.root` (absolute or project-relative):
|
|
836
1165
|
|
|
837
1166
|
```ruby
|
|
838
|
-
SimpleCov.
|
|
839
|
-
|
|
840
|
-
|
|
841
|
-
|
|
842
|
-
|
|
1167
|
+
result = SimpleCov.result # or SimpleCov::Result.from_hash(...).first
|
|
1168
|
+
result.coverage_for("app/models/user.rb")
|
|
1169
|
+
# => {line: <CoverageStatistics>, branch: <CoverageStatistics>, method: <CoverageStatistics>}
|
|
1170
|
+
|
|
1171
|
+
result.source_file_for("app/models/user.rb")
|
|
1172
|
+
# => <SimpleCov::SourceFile>
|
|
843
1173
|
```
|
|
844
1174
|
|
|
845
|
-
|
|
1175
|
+
### `report` — quick terminal report
|
|
846
1176
|
|
|
847
|
-
|
|
1177
|
+
For CI logs, ssh sessions, or any terminal-only workflow, `simplecov report` prints the same totals row the HTML report
|
|
1178
|
+
renders at the top, plus per-group totals:
|
|
848
1179
|
|
|
849
|
-
```
|
|
850
|
-
|
|
1180
|
+
```sh
|
|
1181
|
+
$ simplecov report
|
|
1182
|
+
All Files
|
|
1183
|
+
Line: 99.75% (1638 / 1642)
|
|
1184
|
+
Branch: 98.50% (396 / 402)
|
|
1185
|
+
Method: 99.73% (372 / 373)
|
|
851
1186
|
```
|
|
852
1187
|
|
|
853
|
-
|
|
854
|
-
|
|
1188
|
+
Pass `--input PATH` to read a non-default `coverage.json`. `--json` emits the totals as a JSON object keyed by section
|
|
1189
|
+
name (`"All Files"` plus each group), useful when a CI step needs to act on the numbers rather than display them.
|
|
855
1190
|
|
|
1191
|
+
### `uncovered` — list lowest-coverage files
|
|
856
1192
|
|
|
857
|
-
|
|
1193
|
+
`simplecov uncovered` prints the lowest-coverage files (by line coverage, worst-first) so you can find where to add
|
|
1194
|
+
tests next without opening the HTML report:
|
|
858
1195
|
|
|
859
|
-
|
|
1196
|
+
```sh
|
|
1197
|
+
$ simplecov uncovered
|
|
1198
|
+
50.00% 5/10 lib/foo.rb
|
|
1199
|
+
80.00% 8/10 lib/bar.rb
|
|
860
1200
|
|
|
861
|
-
|
|
862
|
-
|
|
1201
|
+
$ simplecov uncovered --threshold 90 --top 5
|
|
1202
|
+
$ simplecov uncovered --criterion branch
|
|
1203
|
+
```
|
|
863
1204
|
|
|
864
|
-
|
|
865
|
-
|
|
866
|
-
|
|
867
|
-
|
|
1205
|
+
`--threshold N` filters to files below N% coverage (default `100`); `--top N` caps the list at N entries (default
|
|
1206
|
+
`10`); `--criterion line|branch|method` chooses which coverage to rank by (default `line`). `--json` emits the rows as
|
|
1207
|
+
a JSON array (empty when nothing is below the threshold), useful for piping into a CI gate.
|
|
1208
|
+
|
|
1209
|
+
### `merge` — combine resultsets from parallel CI workers
|
|
1210
|
+
|
|
1211
|
+
CI matrices that produce one `.resultset.json` per worker can stitch them together with `simplecov merge` instead of
|
|
1212
|
+
hand-rolling a Rake task in every project:
|
|
1213
|
+
|
|
1214
|
+
```sh
|
|
1215
|
+
$ simplecov merge worker-*/coverage/.resultset.json --output coverage/.resultset.json
|
|
868
1216
|
```
|
|
869
1217
|
|
|
870
|
-
|
|
1218
|
+
By default `simplecov merge` ignores `merge_timeout`; pass `--honor-timeout` to drop entries older than the configured
|
|
1219
|
+
timeout. Pass `--dry-run` to preview the output path without writing, or `-q` / `--quiet` to suppress the success status
|
|
1220
|
+
line for cleaner CI logs. After merging, run `simplecov report` against the combined data.
|
|
871
1221
|
|
|
872
|
-
|
|
1222
|
+
### `diff` — coverage delta vs a baseline
|
|
873
1223
|
|
|
874
|
-
|
|
1224
|
+
`simplecov diff <baseline>` reads two `coverage.json` files (current plus a baseline checked into the repo, or produced
|
|
1225
|
+
by a previous CI run) and prints the files whose coverage moved on any enabled criterion. When branch or method coverage
|
|
1226
|
+
is enabled, those deltas appear alongside the line delta on the same row:
|
|
875
1227
|
|
|
876
|
-
```
|
|
877
|
-
|
|
878
|
-
|
|
1228
|
+
```sh
|
|
1229
|
+
$ simplecov diff coverage/baseline.json
|
|
1230
|
+
-20.00% lines -10.00% branches lib/foo.rb
|
|
1231
|
+
+ 5.00% lines lib/bar.rb
|
|
1232
|
+
+60.00% lines lib/new.rb (new file)
|
|
1233
|
+
-95.00% lines lib/gone.rb (removed)
|
|
879
1234
|
```
|
|
880
1235
|
|
|
881
|
-
|
|
882
|
-
|
|
1236
|
+
Regressions are listed first. Pass `--fail-on-drop` to exit non-zero when any file's line coverage slipped, so this
|
|
1237
|
+
composes with CI as a "coverage of this PR didn't drop" gate even when overall thresholds are still satisfied.
|
|
1238
|
+
`--threshold N` filters out deltas below N% in absolute value, useful when a baseline is noisy. `--json` emits the rows
|
|
1239
|
+
as a JSON array for programmatic consumption:
|
|
1240
|
+
|
|
1241
|
+
```sh
|
|
1242
|
+
$ simplecov diff --json coverage/baseline.json
|
|
1243
|
+
[
|
|
1244
|
+
{"file":"lib/foo.rb","status":"changed","line_delta":-20.0,"branch_delta":-10.0,"method_delta":0.0},
|
|
1245
|
+
{"file":"lib/bar.rb","status":"changed","line_delta":5.0,"branch_delta":0.0,"method_delta":0.0}
|
|
1246
|
+
]
|
|
1247
|
+
```
|
|
883
1248
|
|
|
884
|
-
|
|
1249
|
+
Coverage keys with a leading `/` (from `coverage.json` files emitted before the `SourceFile#project_filename` change)
|
|
1250
|
+
are normalized, so a baseline from an older SimpleCov still diffs cleanly against newer reports.
|
|
885
1251
|
|
|
886
|
-
|
|
887
|
-
* [Editor Integration](doc/editor-integration.md)
|
|
888
|
-
* [Hosted (commercial) services](doc/commercial-services.md)
|
|
1252
|
+
### `serve` and `clean`
|
|
889
1253
|
|
|
890
|
-
|
|
1254
|
+
`simplecov serve` serves the coverage report over HTTP — handy on a remote box where you can't open files directly.
|
|
1255
|
+
`--port N` binds to a specific port (default: a random open port) and `--host HOST` to a specific host (default
|
|
1256
|
+
`127.0.0.1`).
|
|
891
1257
|
|
|
892
|
-
|
|
1258
|
+
`simplecov clean` removes the coverage report directory. `--dry-run` prints what would be removed without deleting
|
|
1259
|
+
anything; `-q` / `--quiet` suppresses status lines.
|
|
893
1260
|
|
|
894
|
-
|
|
1261
|
+
## Compatibility and troubleshooting
|
|
895
1262
|
|
|
896
|
-
|
|
1263
|
+
### Ruby version compatibility
|
|
897
1264
|
|
|
898
|
-
|
|
1265
|
+
SimpleCov is built in [Continuous Integration] on Ruby 3.1+ and JRuby 9.4+. On CRuby, every coverage criterion
|
|
1266
|
+
described above is available on the supported versions, with one exception: [eval coverage](#eval-coverage) requires
|
|
1267
|
+
CRuby 3.2+.
|
|
899
1268
|
|
|
900
|
-
|
|
1269
|
+
### JRuby
|
|
901
1270
|
|
|
902
|
-
|
|
903
|
-
|
|
904
|
-
|
|
1271
|
+
On JRuby, only **line coverage** is available — branch, method, oneshot-line, and eval coverage rely on features of
|
|
1272
|
+
CRuby's `Coverage` library that JRuby doesn't implement. SimpleCov detects this automatically: the bundled `strict`
|
|
1273
|
+
profile, for instance, enforces only line coverage at 100% on JRuby instead of failing to load.
|
|
1274
|
+
|
|
1275
|
+
To get accurate line numbers in coverage results, JRuby needs its full backtrace enabled. Pass `JRUBY_OPTS="--debug"`,
|
|
1276
|
+
or create a `.jrubyrc` with `debug.fullTrace=true`.
|
|
1277
|
+
|
|
1278
|
+
### Notes on specific frameworks and test utilities
|
|
1279
|
+
|
|
1280
|
+
Some frameworks and tools have quirks worth knowing about when using SimpleCov:
|
|
1281
|
+
|
|
1282
|
+
<table>
|
|
1283
|
+
<tr><th>Framework</th><th>Notes</th><th>Issue</th></tr>
|
|
1284
|
+
<tr>
|
|
1285
|
+
<th>
|
|
1286
|
+
parallel_tests
|
|
1287
|
+
</th>
|
|
1288
|
+
<td>
|
|
1289
|
+
As of 0.8.0, SimpleCov should correctly recognize parallel_tests and
|
|
1290
|
+
supplement your test suite names with their corresponding test env
|
|
1291
|
+
numbers. SimpleCov locks the resultset cache while merging, ensuring no
|
|
1292
|
+
race conditions occur when results are merged.
|
|
1293
|
+
</td>
|
|
1294
|
+
<td>
|
|
1295
|
+
<a href="https://github.com/simplecov-ruby/simplecov/issues/64">#64</a> &
|
|
1296
|
+
<a href="https://github.com/simplecov-ruby/simplecov/pull/185">#185</a>
|
|
1297
|
+
</td>
|
|
1298
|
+
</tr>
|
|
1299
|
+
<tr>
|
|
1300
|
+
<th>
|
|
1301
|
+
knapsack_pro
|
|
1302
|
+
</th>
|
|
1303
|
+
<td>
|
|
1304
|
+
To make SimpleCov work with Knapsack Pro Queue Mode to split tests in parallel on CI jobs you need to provide CI node index number to the <code>SimpleCov.command_name</code> in <code>KnapsackPro::Hooks::Queue.before_queue</code> hook.
|
|
1305
|
+
</td>
|
|
1306
|
+
<td>
|
|
1307
|
+
<a href="https://knapsackpro.com/faq/question/how-to-use-simplecov-in-queue-mode">Tip</a>
|
|
1308
|
+
</td>
|
|
1309
|
+
</tr>
|
|
1310
|
+
<tr>
|
|
1311
|
+
<th>
|
|
1312
|
+
RubyMine
|
|
1313
|
+
</th>
|
|
1314
|
+
<td>
|
|
1315
|
+
The <a href="https://www.jetbrains.com/ruby/">RubyMine IDE</a> has
|
|
1316
|
+
built-in support for SimpleCov's coverage reports, though you might need
|
|
1317
|
+
to explicitly set the output root using `SimpleCov.root('foo/bar/baz')`
|
|
1318
|
+
</td>
|
|
1319
|
+
<td>
|
|
1320
|
+
<a href="https://github.com/simplecov-ruby/simplecov/issues/95">#95</a>
|
|
1321
|
+
</td>
|
|
1322
|
+
</tr>
|
|
1323
|
+
<tr>
|
|
1324
|
+
<th>
|
|
1325
|
+
Spork
|
|
1326
|
+
</th>
|
|
1327
|
+
<td>
|
|
1328
|
+
Because of how Spork works internally (using preforking), there used to
|
|
1329
|
+
be trouble when using SimpleCov with it, but that has apparently been
|
|
1330
|
+
resolved with a specific configuration strategy. See <a
|
|
1331
|
+
href="https://github.com/simplecov-ruby/simplecov/issues/42#issuecomment-4440284">this</a>
|
|
1332
|
+
comment.
|
|
1333
|
+
</td>
|
|
1334
|
+
<td>
|
|
1335
|
+
<a href="https://github.com/simplecov-ruby/simplecov/issues/42#issuecomment-4440284">#42</a>
|
|
1336
|
+
</td>
|
|
1337
|
+
</tr>
|
|
1338
|
+
<tr>
|
|
1339
|
+
<th>
|
|
1340
|
+
Spring
|
|
1341
|
+
</th>
|
|
1342
|
+
<td>
|
|
1343
|
+
<a href="#using-spring-with-simplecov">See section below.</a>
|
|
1344
|
+
</td>
|
|
1345
|
+
<td>
|
|
1346
|
+
<a href="https://github.com/simplecov-ruby/simplecov/issues/381">#381</a>
|
|
1347
|
+
</td>
|
|
1348
|
+
</tr>
|
|
1349
|
+
<tr>
|
|
1350
|
+
<th>
|
|
1351
|
+
Test/Unit
|
|
1352
|
+
</th>
|
|
1353
|
+
<td>
|
|
1354
|
+
Test Unit 2 used to mess with ARGV, leading to a failure to detect the
|
|
1355
|
+
test process name in SimpleCov. <code>test-unit</code> releases 2.4.3+
|
|
1356
|
+
(Dec 11th, 2011) should have this problem resolved.
|
|
1357
|
+
</td>
|
|
1358
|
+
<td>
|
|
1359
|
+
<a href="https://github.com/simplecov-ruby/simplecov/issues/45">#45</a> &
|
|
1360
|
+
<a href="https://github.com/test-unit/test-unit/pull/12">test-unit/test-unit#12</a>
|
|
1361
|
+
</td>
|
|
1362
|
+
</tr>
|
|
1363
|
+
</table>
|
|
1364
|
+
|
|
1365
|
+
### Using Spring with SimpleCov
|
|
1366
|
+
|
|
1367
|
+
If you use [Spring](https://github.com/rails/spring) to speed up test runs, SimpleCov often misreports coverage with the
|
|
1368
|
+
default config due to an eager-loading issue. There are a few fixes.
|
|
905
1369
|
|
|
906
1370
|
One solution is to [explicitly call eager
|
|
907
|
-
load](https://github.com/simplecov-ruby/simplecov/issues/381#issuecomment-347651728)
|
|
908
|
-
|
|
1371
|
+
load](https://github.com/simplecov-ruby/simplecov/issues/381#issuecomment-347651728) in your `test_helper.rb` /
|
|
1372
|
+
`spec_helper.rb` after calling `SimpleCov.start`:
|
|
909
1373
|
|
|
910
1374
|
```ruby
|
|
911
1375
|
require 'simplecov'
|
|
@@ -913,22 +1377,36 @@ SimpleCov.start 'rails'
|
|
|
913
1377
|
Rails.application.eager_load!
|
|
914
1378
|
```
|
|
915
1379
|
|
|
916
|
-
Alternatively,
|
|
1380
|
+
Alternatively, disable Spring while running SimpleCov:
|
|
917
1381
|
|
|
918
|
-
```
|
|
1382
|
+
```sh
|
|
919
1383
|
DISABLE_SPRING=1 rake test
|
|
920
1384
|
```
|
|
921
1385
|
|
|
922
|
-
Or
|
|
1386
|
+
Or remove `gem 'spring'` from your `Gemfile`.
|
|
1387
|
+
|
|
1388
|
+
### Different coverage between local and CI
|
|
923
1389
|
|
|
924
|
-
|
|
1390
|
+
Rails generates `config/environments/test.rb` with `config.eager_load = ENV["CI"].present?` (Rails 7+), so **CI eagerly
|
|
1391
|
+
loads every file in `app/` while your local run does not**. The two environments then report different file sets and
|
|
1392
|
+
different totals from the same suite. Two ways to make the report deterministic:
|
|
925
1393
|
|
|
926
|
-
|
|
927
|
-
|
|
928
|
-
|
|
1394
|
+
- Set `config.eager_load = true` everywhere in `test.rb` (slower locally, but matches CI — and matches what users
|
|
1395
|
+
actually see in production).
|
|
1396
|
+
- Stick with the `rails` profile, which folds `{app,lib}/**/*.rb` into the report at 0% on every run regardless of
|
|
1397
|
+
`eager_load`. (The profile resolves the glob relative to `SimpleCov.root`, not the test runner's cwd.) Outside the
|
|
1398
|
+
profile, the equivalent is `cover "{app,lib}/**/*.rb"` — see the
|
|
1399
|
+
[legacy-API migration table](#migrating-from-the-legacy-configuration-api) for the relationship with the older
|
|
1400
|
+
`track_files`.
|
|
929
1401
|
|
|
930
|
-
|
|
931
|
-
|
|
1402
|
+
### Missing coverage
|
|
1403
|
+
|
|
1404
|
+
The **most common problem is that SimpleCov isn't required and started before everything else**. To track coverage for
|
|
1405
|
+
your whole application, **SimpleCov must come first** so that it (and the underlying Coverage library) can track files
|
|
1406
|
+
as they're loaded and used.
|
|
1407
|
+
|
|
1408
|
+
If coverage is missing for some code, a simple trick is to add a `puts` inside that file and another right after
|
|
1409
|
+
`SimpleCov.start`, then check the order they print in:
|
|
932
1410
|
|
|
933
1411
|
```ruby
|
|
934
1412
|
# my_code.rb
|
|
@@ -941,29 +1419,49 @@ class MyCode
|
|
|
941
1419
|
end
|
|
942
1420
|
end
|
|
943
1421
|
|
|
944
|
-
# spec_helper.rb/rails_helper.rb/test_helper.rb
|
|
945
|
-
|
|
1422
|
+
# spec_helper.rb / rails_helper.rb / test_helper.rb / .simplecov — whatever
|
|
946
1423
|
SimpleCov.start
|
|
947
1424
|
puts "SimpleCov started successfully!"
|
|
948
1425
|
```
|
|
949
1426
|
|
|
950
|
-
|
|
1427
|
+
If you see this order, you're good:
|
|
951
1428
|
|
|
952
1429
|
```
|
|
953
1430
|
SimpleCov started successfully!
|
|
954
1431
|
MyCode is being loaded!
|
|
955
1432
|
```
|
|
956
1433
|
|
|
957
|
-
|
|
1434
|
+
If `MyCode is being loaded!` prints first, the file was loaded before SimpleCov started — that's your problem.
|
|
958
1435
|
|
|
959
|
-
|
|
1436
|
+
### Upgrading from 0.x
|
|
960
1437
|
|
|
961
|
-
|
|
962
|
-
|
|
1438
|
+
Four methods that had been deprecated for a decade or more were removed in 1.0. Each had a one-to-one rename:
|
|
1439
|
+
|
|
1440
|
+
| Removed | Use instead |
|
|
1441
|
+
| ---------------------------------------- | ------------------------------------------ |
|
|
1442
|
+
| `SimpleCov::Filter#passes?` | `SimpleCov::Filter#matches?` |
|
|
1443
|
+
| `SimpleCov.adapters` | `SimpleCov.profiles` |
|
|
1444
|
+
| `SimpleCov.load_adapter('rails')` | `SimpleCov.load_profile('rails')` |
|
|
1445
|
+
| `SimpleCov::Formatter::MultiFormatter[]` | `SimpleCov::Formatter::MultiFormatter.new` |
|
|
1446
|
+
|
|
1447
|
+
If a custom filter still defines `passes?`, rename the method to `matches?` — the signature and semantics are identical.
|
|
1448
|
+
|
|
1449
|
+
## Related projects
|
|
1450
|
+
|
|
1451
|
+
Want to find dead code in production? Try [Coverband](https://github.com/danmayer/coverband).
|
|
963
1452
|
|
|
964
1453
|
## Contributing
|
|
965
1454
|
|
|
966
|
-
|
|
1455
|
+
* [Issue Tracker](https://github.com/simplecov-ruby/simplecov/issues) — for code and bug reports. See
|
|
1456
|
+
[CONTRIBUTING](https://github.com/simplecov-ruby/simplecov/blob/main/CONTRIBUTING.md) for how to contribute, along
|
|
1457
|
+
with common problems to check before creating an issue.
|
|
1458
|
+
* [Mailing List](https://groups.google.com/forum/#!forum/simplecov) — open list for discussion and announcements on
|
|
1459
|
+
Google Groups.
|
|
1460
|
+
|
|
1461
|
+
## Code of Conduct
|
|
1462
|
+
|
|
1463
|
+
Everyone participating in this project's development, issue trackers, and other channels is expected to follow our
|
|
1464
|
+
[Code of Conduct](./CODE_OF_CONDUCT.md).
|
|
967
1465
|
|
|
968
1466
|
## Kudos
|
|
969
1467
|
|
|
@@ -971,4 +1469,4 @@ Thanks to Aaron Patterson for the original idea for this!
|
|
|
971
1469
|
|
|
972
1470
|
## Copyright
|
|
973
1471
|
|
|
974
|
-
Copyright (c) 2010-
|
|
1472
|
+
Copyright (c) 2010-2026 Erik Berlin, Benjamin Fleischer, Akira Matsuda, Christoph Olszowka, Tobias Pfeiffer, David Rodríguez, and Xavier Shay. See MIT-LICENSE for details.
|