path_coverage 0.4.0 → 0.5.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/README.md +286 -199
- data/exe/path_coverage_report +4 -0
- data/lib/path_coverage/call_trace.rb +88 -0
- data/lib/path_coverage/calls.rb +49 -43
- data/lib/path_coverage/card_context.rb +104 -0
- data/lib/path_coverage/card_forks.rb +44 -0
- data/lib/path_coverage/card_gaps.rb +125 -0
- data/lib/path_coverage/card_scopes.rb +37 -0
- data/lib/path_coverage/card_tree.rb +79 -66
- data/lib/path_coverage/card_way_lists.rb +72 -0
- data/lib/path_coverage/card_ways.rb +123 -67
- data/lib/path_coverage/{html_syntax.rb → code_tokens.rb} +35 -33
- data/lib/path_coverage/condition.rb +14 -14
- data/lib/path_coverage/configuration.rb +7 -4
- data/lib/path_coverage/configuration_limits.rb +52 -0
- data/lib/path_coverage/configuration_paths.rb +7 -7
- data/lib/path_coverage/console_summary.rb +24 -22
- data/lib/path_coverage/{site.rb → decision.rb} +2 -2
- data/lib/path_coverage/decision_locations.rb +76 -0
- data/lib/path_coverage/{site_paths.rb → decision_paths.rb} +13 -11
- data/lib/path_coverage/{site_tree.rb → decision_tree.rb} +25 -16
- data/lib/path_coverage/fold_branches.rb +18 -0
- data/lib/path_coverage/fold_cells.rb +42 -0
- data/lib/path_coverage/{html_tree.rb → fold_rows.rb} +42 -71
- data/lib/path_coverage/gaps.rb +33 -15
- data/lib/path_coverage/json_reader.rb +54 -19
- data/lib/path_coverage/json_reports.rb +102 -0
- data/lib/path_coverage/json_writer.rb +27 -12
- data/lib/path_coverage/kernel.rb +56 -58
- data/lib/path_coverage/kinds/base.rb +16 -16
- data/lib/path_coverage/kinds/block.rb +16 -15
- data/lib/path_coverage/kinds/case.rb +14 -12
- data/lib/path_coverage/kinds/chain.rb +40 -11
- data/lib/path_coverage/kinds/choice.rb +4 -3
- data/lib/path_coverage/kinds/contains.rb +40 -32
- data/lib/path_coverage/kinds/safe.rb +38 -26
- data/lib/path_coverage/kinds/write.rb +6 -6
- data/lib/path_coverage/kinds.rb +2 -2
- data/lib/path_coverage/leaving.rb +44 -0
- data/lib/path_coverage/loop_forks.rb +57 -0
- data/lib/path_coverage/loop_nodes.rb +55 -0
- data/lib/path_coverage/manifest.rb +92 -58
- data/lib/path_coverage/manifest_blocks.rb +5 -26
- data/lib/path_coverage/manifest_case.rb +2 -2
- data/lib/path_coverage/manifest_chains.rb +8 -8
- data/lib/path_coverage/manifest_choices.rb +23 -20
- data/lib/path_coverage/manifest_disabled.rb +97 -0
- data/lib/path_coverage/manifest_folding.rb +43 -40
- data/lib/path_coverage/manifest_ignoring.rb +6 -6
- data/lib/path_coverage/manifest_writes.rb +11 -10
- data/lib/path_coverage/measured_by.rb +20 -0
- data/lib/path_coverage/merge.rb +7 -7
- data/lib/path_coverage/method_loops.rb +128 -0
- data/lib/path_coverage/method_statements.rb +130 -127
- data/lib/path_coverage/method_ways.rb +142 -0
- data/lib/path_coverage/paths_report.rb +16 -16
- data/lib/path_coverage/recorder.rb +57 -36
- data/lib/path_coverage/recorder_blocks.rb +17 -17
- data/lib/path_coverage/recorder_cases.rb +12 -11
- data/lib/path_coverage/recorder_chains.rb +12 -12
- data/lib/path_coverage/recorder_writes.rb +9 -9
- data/lib/path_coverage/render.rb +58 -51
- data/lib/path_coverage/result.rb +32 -31
- data/lib/path_coverage/rewriter.rb +39 -27
- data/lib/path_coverage/rewriter_nested.rb +13 -13
- data/lib/path_coverage/runs_report.rb +84 -0
- data/lib/path_coverage/source_lines.rb +16 -16
- data/lib/path_coverage/source_report.rb +100 -0
- data/lib/path_coverage/stable_leaves.rb +162 -0
- data/lib/path_coverage/statement_arms.rb +31 -30
- data/lib/path_coverage/summary.rb +4 -62
- data/lib/path_coverage/totals.rb +10 -8
- data/lib/path_coverage/version.rb +1 -1
- data/lib/path_coverage/walk.rb +106 -0
- data/lib/path_coverage/way_algebras.rb +127 -0
- data/lib/path_coverage/ways_report.rb +91 -0
- data/lib/path_coverage/ways_totals.rb +46 -0
- data/lib/path_coverage/within_methods.rb +135 -80
- metadata +33 -28
- data/lib/path_coverage/card.rb +0 -14
- data/lib/path_coverage/html/file.html +0 -33
- data/lib/path_coverage/html/index.html +0 -29
- data/lib/path_coverage/html/settings.js +0 -50
- data/lib/path_coverage/html/side.js +0 -599
- data/lib/path_coverage/html/sort.js +0 -34
- data/lib/path_coverage/html/style.css +0 -484
- data/lib/path_coverage/html/tabs.js +0 -15
- data/lib/path_coverage/html/theme.js +0 -27
- data/lib/path_coverage/html_branch.rb +0 -30
- data/lib/path_coverage/html_cards.rb +0 -88
- data/lib/path_coverage/html_cells.rb +0 -71
- data/lib/path_coverage/html_file.rb +0 -123
- data/lib/path_coverage/html_index.rb +0 -128
- data/lib/path_coverage/html_index_head.rb +0 -29
- data/lib/path_coverage/html_labels.rb +0 -16
- data/lib/path_coverage/html_methods.rb +0 -66
- data/lib/path_coverage/html_nav.rb +0 -67
- data/lib/path_coverage/html_report.rb +0 -131
- data/lib/path_coverage/html_statements.rb +0 -186
- data/lib/path_coverage/html_tabs.rb +0 -26
- data/lib/path_coverage/html_template.rb +0 -26
data/README.md
CHANGED
|
@@ -1,60 +1,102 @@
|
|
|
1
|
-
#
|
|
2
|
-
|
|
3
|
-
[](https://rubygems.org/gems/path_coverage)
|
|
4
|
-
[](https://rubygems.org/gems/path_coverage)
|
|
5
|
-
[](LICENSE.md)
|
|
1
|
+
# <img src="https://avatars.githubusercontent.com/u/353243247?v=4" alt="" width="32" height="32"> path-coverage/ruby
|
|
6
2
|
|
|
7
3
|
Path coverage for Ruby, to run alongside SimpleCov's line and branch
|
|
8
4
|
coverage.
|
|
9
5
|
|
|
10
|
-
|
|
11
|
-
|
|
12
|
-
|
|
13
|
-
`detect` `find_index` `take_while` `select` `filter` `reject`
|
|
14
|
-
`partition` `count` `filter_map`, the writes `||=` and `&&=`, and `&.`
|
|
15
|
-
safe-navigation chains. Method paths cover every site at a method's
|
|
16
|
-
top level.
|
|
17
|
-
More decisions hidden inside calls come next (see Roadmap).
|
|
6
|
+
[](https://rubygems.org/gems/path_coverage)
|
|
7
|
+
[](https://rubygems.org/gems/path_coverage)
|
|
8
|
+
[](LICENSE.md)
|
|
18
9
|
|
|
19
|
-
|
|
20
|
-
|
|
21
|
-
|
|
10
|
+
- [Overview](#overview)
|
|
11
|
+
- [Getting started](#getting-started)
|
|
12
|
+
- [Reading the HTML](#reading-the-html)
|
|
13
|
+
- [Ignoring and treating](#ignoring-and-treating)
|
|
14
|
+
- [Use when you cannot edit the code!](#use-when-you-cannot-edit-the-code)
|
|
15
|
+
- [What a run reports](#what-a-run-reports)
|
|
16
|
+
- [How it works](#how-it-works)
|
|
17
|
+
- [Roadmap](#roadmap)
|
|
18
|
+
- [Constraints](#constraints)
|
|
19
|
+
- [Development](#development)
|
|
22
20
|
|
|
23
21
|
## Overview
|
|
24
22
|
|
|
25
|
-
A *
|
|
26
|
-
`case`,
|
|
27
|
-
Each way it can go is a *path*. For example
|
|
23
|
+
A *decision* is a choice in your code, eg an `&&`, `||`, `if`,
|
|
24
|
+
`case`, etc. Each way it can go is a *path*. For example
|
|
28
25
|
|
|
29
26
|
```ruby
|
|
30
|
-
|
|
27
|
+
def discount(member, coupon)
|
|
28
|
+
member || coupon ? 10 : 0
|
|
29
|
+
end
|
|
31
30
|
```
|
|
32
31
|
|
|
33
|
-
has three paths:
|
|
34
|
-
|
|
32
|
+
has three paths: a member, a non-member with a coupon, and
|
|
33
|
+
neither. Tests calling `discount(true, nil)` and
|
|
34
|
+
`discount(false, nil)` cover every line and both branches of the
|
|
35
|
+
`?:`, yet no test ever tried a coupon.
|
|
35
36
|
|
|
36
37
|
Line and branch coverage can be 100% while a path was never
|
|
37
38
|
taken. path_coverage counts how often each path was taken:
|
|
38
|
-
|
|
39
|
+
alone, each decision's paths on their own, and combined, through
|
|
40
|
+
each method's decisions.
|
|
39
41
|
|
|
40
|
-
As the run exits it writes its report:
|
|
42
|
+
As the run exits it writes its report as two JSON files:
|
|
41
43
|
- `path_coverage.json`, for AIs and other tools.
|
|
42
|
-
- `
|
|
44
|
+
- `path_coverage.source.json`, what drawing each file's code needs.
|
|
45
|
+
|
|
46
|
+
## Getting started
|
|
47
|
+
|
|
48
|
+
```ruby
|
|
49
|
+
# test coverage config, before any application code is required
|
|
50
|
+
require 'simplecov'
|
|
51
|
+
require 'path_coverage'
|
|
52
|
+
SimpleCov.start do
|
|
53
|
+
# ...
|
|
54
|
+
end
|
|
55
|
+
PathCoverage.start do
|
|
56
|
+
root APP_DIR # cover globs and decision names are relative to it
|
|
57
|
+
cover 'source/**/*.rb' # each matching file is instrumented as it loads
|
|
58
|
+
cover 'test/**/*.rb' # more than one cover glob may be given
|
|
59
|
+
skip 'source/vendor/**/*.rb' # except these, though a cover glob matches them
|
|
60
|
+
coverage_dir '/tmp/coverage' # where path_coverage.json is written
|
|
61
|
+
report :gaps, :summary # the reporters run at exit (default :gaps);
|
|
62
|
+
# also :ways and :post
|
|
63
|
+
group 'code', 'source/**/*.rb' # each group its own totals
|
|
64
|
+
group 'test', 'test/**/*.rb'
|
|
65
|
+
ignore '&&=' # operators whose decisions are not measured
|
|
66
|
+
max_paths 4096 # list each method's paths up to this many
|
|
67
|
+
# (the default; at most 65536)
|
|
68
|
+
loops # count each loop as a decision: 0 items
|
|
69
|
+
# or 1+ items (off by default)
|
|
70
|
+
end
|
|
71
|
+
```
|
|
43
72
|
|
|
44
73
|
## Reading the HTML
|
|
45
74
|
|
|
75
|
+
https://path-coverage.github.io/ruby/ shows a demo HTML report
|
|
76
|
+
for the mock source tree in `demo/`. `make demo` builds and opens the
|
|
77
|
+
same report locally.
|
|
78
|
+
|
|
79
|
+
The HTML report, for people, is drawn from the two JSON files, by
|
|
80
|
+
[html-reporter](https://github.com/path-coverage/html-reporter).
|
|
81
|
+
|
|
82
|
+
```sh
|
|
83
|
+
docker run --rm -v "$PWD/coverage:/paths" ghcr.io/path-coverage/html-reporter /paths
|
|
84
|
+
```
|
|
85
|
+
|
|
46
86
|
Clicking a `def` line on a file's HTML page opens that method's
|
|
47
|
-
*card* to its left.
|
|
87
|
+
*card* to its left.
|
|
48
88
|
|
|
49
|
-
A card has a row for each
|
|
89
|
+
A card has a row for each decision in the method, by `line:col`,
|
|
50
90
|
with its counts of paths not taken and taken.
|
|
51
91
|
|
|
52
92
|
Clicking a row's `▼` opens it:
|
|
53
|
-
- on the left
|
|
93
|
+
- on the left are the decision's parts, one per row: its
|
|
94
|
+
operator, eg `&&`, then each operand it tests, nested
|
|
95
|
+
operators indented under it.
|
|
54
96
|
- on the right is one column for each possible path through it.
|
|
55
97
|
- the column's head is `≀0` for a path never taken, and `≀*`
|
|
56
98
|
for one taken (hover for its count). Look for zeros!
|
|
57
|
-
- the symbols in a column line up with
|
|
99
|
+
- the symbols in a column line up with those rows, and
|
|
58
100
|
are explained below.
|
|
59
101
|
|
|
60
102
|
Hovering over code in the open card's method opens the tree holding
|
|
@@ -63,7 +105,7 @@ A long piece of code scrolls within its row.
|
|
|
63
105
|
|
|
64
106
|
### Boolean: `T` `F`
|
|
65
107
|
|
|
66
|
-
Many parts of a
|
|
108
|
+
Many parts of a decision, such as `name.size > 2`, are true or false.
|
|
67
109
|
|
|
68
110
|

|
|
69
111
|
|
|
@@ -75,7 +117,7 @@ In this example, there was no test which exercised the path where `name.size > 2
|
|
|
75
117
|
|
|
76
118
|
### Unreachable: blank
|
|
77
119
|
|
|
78
|
-
Some parts of a
|
|
120
|
+
Some parts of a decision are not always evaluated.
|
|
79
121
|
|
|
80
122
|

|
|
81
123
|
|
|
@@ -145,32 +187,82 @@ Ruby code can be very compact so short expressions can generate lots of paths!
|
|
|
145
187
|
|
|
146
188
|

|
|
147
189
|
|
|
190
|
+
## Ignoring and treating
|
|
148
191
|
|
|
149
|
-
|
|
192
|
+
`ignore '&&=', '||='` stops `&&=` and `||=` writes being measured.
|
|
193
|
+
A decision inside such a write is still measured, eg the `a && b` in
|
|
194
|
+
`@m ||= a && b`. Ignoring `&&` does not ignore `and`, nor does ignoring
|
|
195
|
+
`||` ignore `or`. The summary and the JSON report list what is
|
|
196
|
+
ignored.
|
|
197
|
+
|
|
198
|
+
`treat` tells path_coverage what you know about a method, so paths that
|
|
199
|
+
cannot happen are not listed as gaps. Add `in: 'source/models/**'` to
|
|
200
|
+
limit a treatment to the files that glob matches.
|
|
201
|
+
|
|
202
|
+
- `treat 'name', as: :present`: a call to `name` is never nil, so
|
|
203
|
+
`user.name&.size` has no path with `user.name` nil.
|
|
204
|
+
- `treat 'any?', as: :nonempty`: `x.any?` is true exactly when `x` has
|
|
205
|
+
items. When `x` is a call to `select`, `reject`, `filter`,
|
|
206
|
+
`filter_map`, `partition` or `take_while`, the path that call took
|
|
207
|
+
often decides whether it kept any items. Paths where `any?` disagrees
|
|
208
|
+
with that are not listed, eg `ids.select { |i| i.odd? }.any?` true
|
|
209
|
+
with `ids` empty.
|
|
210
|
+
|
|
211
|
+
A run where a treatment turns out to be wrong is reported:
|
|
212
|
+
|
|
213
|
+
```text
|
|
214
|
+
source/user.rb:4:9 path taken that a treatment rules out: `user.name` nil
|
|
215
|
+
```
|
|
216
|
+
|
|
217
|
+
### Not analysing a block
|
|
218
|
+
|
|
219
|
+
Sometimes a block of code is not worth testing path by path, for example
|
|
220
|
+
logging. Put `# path: disable` before the block and `# path: enable` after
|
|
221
|
+
it. The lines between them still run, but they are not analysed: any
|
|
222
|
+
decision in them is ignored, and they add no paths to the method they
|
|
223
|
+
are in.
|
|
224
|
+
|
|
225
|
+
The line straight after `# path: disable` must start with `# reason:` and
|
|
226
|
+
say why the block is not analysed:
|
|
150
227
|
|
|
151
228
|
```ruby
|
|
152
|
-
|
|
153
|
-
|
|
154
|
-
|
|
155
|
-
|
|
156
|
-
#
|
|
157
|
-
|
|
158
|
-
|
|
159
|
-
|
|
160
|
-
|
|
161
|
-
|
|
162
|
-
|
|
163
|
-
coverage_dir '/tmp/coverage' # where path_coverage.json is written
|
|
164
|
-
report :gaps, :html # the reporters run at exit (default :gaps);
|
|
165
|
-
# also :summary and :post
|
|
166
|
-
group 'code', 'source/**/*.rb' # each group a tab of its own on index.html
|
|
167
|
-
group 'test', 'test/**/*.rb'
|
|
168
|
-
ignore '&&=' # operators whose sites are not measured
|
|
169
|
-
treat 'name', as: :present # a call to name is never nil
|
|
170
|
-
treat 'any?', as: :nonempty # any? is true when its receiver has items
|
|
229
|
+
def ship(order, log)
|
|
230
|
+
if order[:express]
|
|
231
|
+
order[:days] = 1
|
|
232
|
+
end
|
|
233
|
+
# path: disable
|
|
234
|
+
# reason: logging is not under test
|
|
235
|
+
if log.verbose?
|
|
236
|
+
log.lines << order[:id]
|
|
237
|
+
end
|
|
238
|
+
# path: enable
|
|
239
|
+
order
|
|
171
240
|
end
|
|
172
241
|
```
|
|
173
242
|
|
|
243
|
+
Without the `# path: disable` and `# path: enable` comments, `ship()` has
|
|
244
|
+
4 paths: each `if` can be true or false. With them, the second `if` is
|
|
245
|
+
ignored, so `ship()` has 2 paths: `order[:express]` true, and
|
|
246
|
+
`order[:express]` false.
|
|
247
|
+
|
|
248
|
+
Only a real comment that is the whole of its line is a marker. The same
|
|
249
|
+
text inside a string, eg a heredoc, is just text: it disables nothing,
|
|
250
|
+
and is never refused.
|
|
251
|
+
|
|
252
|
+
```ruby
|
|
253
|
+
HELP = <<~TEXT
|
|
254
|
+
# path: disable
|
|
255
|
+
TEXT
|
|
256
|
+
# a heredoc, not a comment: the methods below are still analysed
|
|
257
|
+
```
|
|
258
|
+
|
|
259
|
+
Each of these is refused, with the file and line it is on:
|
|
260
|
+
|
|
261
|
+
- a `# path: disable` without a `# reason:` line straight after it
|
|
262
|
+
- a `# path: disable` without a `# path: enable` after it
|
|
263
|
+
- a statement that starts between the two comments and ends after the
|
|
264
|
+
`# path: enable`
|
|
265
|
+
|
|
174
266
|
## Use when you cannot edit the code!
|
|
175
267
|
|
|
176
268
|
A run whose code cannot be changed, such as a script in a `docker build`
|
|
@@ -190,8 +282,8 @@ starts nothing and prints nothing, so an image can set `RUBYOPT` for every
|
|
|
190
282
|
run and measure only those given a collector.
|
|
191
283
|
|
|
192
284
|
The gem's `path_coverage_collector` command is that collector. It saves
|
|
193
|
-
each report POSTed to it as a numbered file
|
|
194
|
-
|
|
285
|
+
each report POSTed to it as a numbered file; once every run is done,
|
|
286
|
+
report them as one with `path_coverage_report`, below.
|
|
195
287
|
|
|
196
288
|
```sh
|
|
197
289
|
path_coverage_collector 4570 /tmp/paths/runs # until sent TERM
|
|
@@ -206,36 +298,78 @@ path_coverage_collector 4570 /tmp/paths/runs # until sent TERM
|
|
|
206
298
|
- On TERM it stops accepting connections, finishes the report in hand,
|
|
207
299
|
and exits 0.
|
|
208
300
|
|
|
301
|
+
### Reporting saved runs
|
|
302
|
+
|
|
303
|
+
Once every run is done, report them as one with `path_coverage_report`:
|
|
304
|
+
|
|
305
|
+
```text
|
|
306
|
+
path_coverage_report [--loops] [--max-paths N] SOURCE_ROOT GLOBS REPORT_DIR RUN_JSON...
|
|
307
|
+
```
|
|
308
|
+
|
|
309
|
+
- SOURCE_ROOT the directory holding the covered source
|
|
310
|
+
- GLOBS comma-separated globs of the covered files, relative to SOURCE_ROOT
|
|
311
|
+
- REPORT_DIR the directory it writes its report into
|
|
312
|
+
- RUN_JSON... one or more saved runs' JSON files, space-separated, eg runs/*.json
|
|
313
|
+
- --loops count each loop as a decision: 0 items or 1+ items
|
|
314
|
+
- --max-paths fail each method with more than N paths (default 4096, at most 65536)
|
|
315
|
+
|
|
316
|
+
With `--loops`, a loop counts as a decision only when its body has a
|
|
317
|
+
statement that runs first on each pass. These loops never do:
|
|
318
|
+
|
|
209
319
|
```ruby
|
|
210
|
-
|
|
211
|
-
|
|
320
|
+
x += 1 while x < 9 # a modifier loop: its body is written first
|
|
321
|
+
lines.each { } # an empty loop
|
|
322
|
+
urls.each do |url| # a loop whose only statements are rescued
|
|
323
|
+
rescue Timeout::Error
|
|
324
|
+
retry_later(url)
|
|
325
|
+
end
|
|
212
326
|
```
|
|
213
327
|
|
|
328
|
+
It exits 0 when every path is taken, and 1 when any is not.
|
|
329
|
+
|
|
330
|
+
Every path taken:
|
|
331
|
+
|
|
332
|
+
```text
|
|
333
|
+
$ path_coverage_report /app 'lib/**/*.rb' report runs/*.json
|
|
334
|
+
Paths: 2 / 2 (100.00%)
|
|
335
|
+
Functions over --max-paths=4096: 0
|
|
336
|
+
```
|
|
337
|
+
|
|
338
|
+
Some paths untested, and a method over --max-paths:
|
|
339
|
+
|
|
340
|
+
```text
|
|
341
|
+
$ path_coverage_report --max-paths 8 /app 'lib/**/*.rb' report runs/*.json
|
|
342
|
+
lib/fees.rb:2 fee() has 16 paths, more than --max-paths allows.
|
|
343
|
+
Split it into smaller functions.
|
|
344
|
+
lib/orders.rb:2 total() has 1 untested path.
|
|
345
|
+
Add tests that take each of these paths:
|
|
346
|
+
1. line 5 `order[:paid]` false
|
|
347
|
+
Paths: 1 / 2 (50.00%)
|
|
348
|
+
Functions over --max-paths=8: 1
|
|
349
|
+
```
|
|
350
|
+
|
|
351
|
+
Each run's JSON names the kinds of decision it measured. A version that
|
|
352
|
+
measures other kinds lists a decision's paths differently, so a run whose
|
|
353
|
+
kinds differ from those of the installed version is refused. So is a run
|
|
354
|
+
made with `ignore` or `treat`, as the command takes neither.
|
|
355
|
+
|
|
214
356
|
## What a run reports
|
|
215
357
|
|
|
216
358
|
As the process exits:
|
|
217
359
|
|
|
218
|
-
- `path_coverage.json`
|
|
219
|
-
|
|
220
|
-
|
|
221
|
-
`
|
|
222
|
-
reached under `reached`, and the paths taken
|
|
223
|
-
of all possible under `totals`, as [taken, possible]: `decisions`,
|
|
224
|
-
`methods`, and, with groups, each group's
|
|
225
|
-
own under `groups`, eg `"groups": {"code": {"decisions": [481, 567]}}`.
|
|
226
|
-
It also names the kinds of site measured under `kinds`, any `ignored`
|
|
227
|
-
operators and `treats`, and any paths taken that a treatment rules
|
|
228
|
-
out under `outside`.
|
|
229
|
-
It is the base the other reporters read, so they can also be run
|
|
360
|
+
- `path_coverage.json` and `path_coverage.source.json` are always
|
|
361
|
+
written to `coverage_dir`. Their keys, with examples, are in
|
|
362
|
+
[path-coverage/schemas](https://github.com/path-coverage/schemas#path_coveragejson).
|
|
363
|
+
`path_coverage.json` is the base the other reporters read, so they can also be run
|
|
230
364
|
later, in another process, from
|
|
231
365
|
it alone: `PathCoverage::Render.from_json(config)`, where `config` is a
|
|
232
366
|
`PathCoverage::Configuration` built from the same settings block.
|
|
233
367
|
Given several runs' reports, `Render.from_json(config, paths)` adds
|
|
234
368
|
their counts together first, so the report shows only the gaps that
|
|
235
369
|
every run left. A report whose kinds, `ignored` or `treats` differ
|
|
236
|
-
from `config`'s is refused, as its
|
|
370
|
+
from `config`'s is refused, as its decisions list their paths
|
|
237
371
|
differently.
|
|
238
|
-
- `:gaps` prints one line per
|
|
372
|
+
- `:gaps` prints one line per decision path never taken, in manifest
|
|
239
373
|
order, naming the outcome of each leaf on it:
|
|
240
374
|
|
|
241
375
|
```text
|
|
@@ -245,65 +379,52 @@ source/kata.rb:4:24 never reached
|
|
|
245
379
|
|
|
246
380
|
### Decisions
|
|
247
381
|
|
|
248
|
-
A
|
|
382
|
+
A decision is named by the `path:line:column` where it starts, the path
|
|
249
383
|
relative to `root`; but an `if` or `unless` at its keyword, a `?:` at
|
|
250
|
-
its `?`, and a block method's call at its method, so no two
|
|
251
|
-
share a name. A
|
|
384
|
+
its `?`, and a block method's call at its method, so no two decisions
|
|
385
|
+
share a name. A decision in a covered file the run never loaded is still
|
|
252
386
|
reported, as `never reached`.
|
|
253
387
|
|
|
254
|
-
A path is the outcome of each leaf one evaluation of the
|
|
388
|
+
A path is the outcome of each leaf one evaluation of the decision reached, in
|
|
255
389
|
order; a leaf that short-circuiting skipped is not on it. Each leaf is
|
|
256
390
|
quoted as a Markdown code span, on one line, its fence one backtick longer
|
|
257
391
|
than any run of backticks inside it. The gap lines are for people; a tool
|
|
258
|
-
should read the JSON, whose `paths` give each
|
|
259
|
-
`line:column` spans, and each possible path with its count
|
|
260
|
-
|
|
261
|
-
```json
|
|
262
|
-
"paths": {
|
|
263
|
-
"source/kata.rb:3:23": {
|
|
264
|
-
"leaves": [["3:23", "3:30"], ["3:34", "3:42"]],
|
|
265
|
-
"taken": [
|
|
266
|
-
{ "path": [true, true], "count": 2 },
|
|
267
|
-
{ "path": [true, false], "count": 0 },
|
|
268
|
-
{ "path": [false, null], "count": 1 }
|
|
269
|
-
]
|
|
270
|
-
}
|
|
271
|
-
}
|
|
272
|
-
```
|
|
392
|
+
should read the JSON, whose `paths` give each decision's leaves as
|
|
393
|
+
`line:column` spans, and each possible path with its count.
|
|
273
394
|
|
|
274
|
-
A
|
|
395
|
+
A decision is an outermost condition. Its leaves are numbered left to right
|
|
275
396
|
across the whole of it, nested conditions included. A condition inside a
|
|
276
|
-
block is a
|
|
397
|
+
block is a decision of its own.
|
|
277
398
|
|
|
278
|
-
A ternary is a
|
|
399
|
+
A ternary is a decision even when its predicate is a single leaf. An arm
|
|
279
400
|
holding a condition of `&&`, `||`, `and`, `or` or `?:` is part of the
|
|
280
|
-
ternary's
|
|
401
|
+
ternary's decision, its leaves numbered after the predicate's, so the decision's
|
|
281
402
|
paths run through whichever arm the predicate picks. Any other arm, like a
|
|
282
|
-
block, is not part of it, though a condition inside it is a
|
|
403
|
+
block, is not part of it, though a condition inside it is a decision of its
|
|
283
404
|
own.
|
|
284
405
|
|
|
285
|
-
An `if` or an `unless`, a modifier included, is a
|
|
406
|
+
An `if` or an `unless`, a modifier included, is a decision too, even when
|
|
286
407
|
its predicate is a single leaf: its predicate nests in it, an `&&` there
|
|
287
|
-
being no
|
|
408
|
+
being no decision of its own, and its `then` and `else` arms are plain, as
|
|
288
409
|
they are statements; an `unless` takes its `then` arm when its predicate
|
|
289
410
|
is false. It is named at its keyword, and its extent is its keyword to
|
|
290
|
-
the end of its predicate, so a
|
|
291
|
-
An `elsif` is a
|
|
411
|
+
the end of its predicate, so a decision inside an arm keeps its own button.
|
|
412
|
+
An `elsif` is a decision of its own.
|
|
292
413
|
|
|
293
|
-
A `case` with a subject is a
|
|
414
|
+
A `case` with a subject is a decision whose leaves are its `when` patterns,
|
|
294
415
|
tried in order until one matches. Each pattern is wrapped in a stand-in
|
|
295
416
|
whose `===` calls the pattern's `===` once, exactly as the `case` would,
|
|
296
417
|
and notes whether it matched. A `case` with no subject, or with a splat
|
|
297
|
-
among its patterns, is no
|
|
418
|
+
among its patterns, is no decision.
|
|
298
419
|
|
|
299
420
|
A call to `any?` `all?` `none?` `find` `detect` `find_index`
|
|
300
421
|
`take_while` `select` `filter` `reject` `partition` `count` or
|
|
301
|
-
`filter_map`, with no arguments and a block of one expression, is a
|
|
422
|
+
`filter_map`, with no arguments and a block of one expression, is a decision
|
|
302
423
|
named at its method. Its item count is noted from the block's own calls,
|
|
303
424
|
so the receiver is never asked its size, which could consume a sequence
|
|
304
425
|
that can be read only once.
|
|
305
426
|
|
|
306
|
-
A `||=` or `&&=` write is a
|
|
427
|
+
A `||=` or `&&=` write is a decision, whatever its target: a variable of any
|
|
307
428
|
kind, a constant, an attribute (`self.size ||= n`) or an index
|
|
308
429
|
(`store[key] ||= key.upcase`). Its one leaf is its target, and its right
|
|
309
430
|
side is an arm, run when the target is false for `||=`, true for `&&=`,
|
|
@@ -312,14 +433,14 @@ since reading `store[key]` could call a method twice: its outcome follows
|
|
|
312
433
|
from whether the right side ran. So `@clock ||= Clock.now` called only
|
|
313
434
|
once, as a fresh object, leaves the path with `@clock` true a gap.
|
|
314
435
|
|
|
315
|
-
A `&.` chain is a
|
|
436
|
+
A `&.` chain is a decision, named where it starts. Its leaves are the
|
|
316
437
|
receivers of its `&.` calls. `false` counts as not nil, as `&.` calls on
|
|
317
438
|
it.
|
|
318
439
|
|
|
319
440
|
A chain inside a condition, eg `ok?(id) && user&.admin?` or
|
|
320
|
-
`if user&.admin?`, folds into its
|
|
441
|
+
`if user&.admin?`, folds into its decision, as one expression: its value is
|
|
321
442
|
one more leaf, true or false, reached when every receiver is not nil. A
|
|
322
|
-
write through `&.`, eg `box&.size ||= 3`, is no
|
|
443
|
+
write through `&.`, eg `box&.size ||= 3`, is no decision, as a nil `box`
|
|
323
444
|
skips it whole and its target is never looked at.
|
|
324
445
|
|
|
325
446
|
A chain of one operator is flat; a change of operator, by precedence or by
|
|
@@ -353,75 +474,57 @@ cannot be closed is the lesser error than hiding one that can.
|
|
|
353
474
|
### Methods
|
|
354
475
|
|
|
355
476
|
Each method has a card, named by the `path:line:column` of its `def`,
|
|
356
|
-
with a row for each statement that shapes its paths: each
|
|
477
|
+
with a row for each statement that shapes its paths: each decision, and
|
|
357
478
|
each `return` or `raise`, which ends a path. Any other statement, eg
|
|
358
|
-
`x = 1`, runs exactly when what holds it does, so is no row. A
|
|
479
|
+
`x = 1`, runs exactly when what holds it does, so is no row. A decision's
|
|
359
480
|
row counts its paths; a `return` or `raise` row is one path, taken
|
|
360
481
|
when it is reached. A method's total adds up its rows. Its title says
|
|
361
482
|
how many times it was called, so a method never called is a gap,
|
|
362
483
|
even one with no rows. A file's code outside any method gets a card
|
|
363
484
|
of its own.
|
|
364
485
|
|
|
365
|
-
The JSON report's `cards` hold each card's rows, its `ways`
|
|
366
|
-
|
|
367
|
-
|
|
368
|
-
```json
|
|
369
|
-
"cards": {
|
|
370
|
-
"label.rb:1:0": {
|
|
371
|
-
"name": "label", "first": 1, "last": 7, "ways": 2, "calls": 1,
|
|
372
|
-
"rows": [
|
|
373
|
-
{ "at": "2:02", "top": "if", "paths": 2, "taken": 1, "site": "label.rb:2:2" },
|
|
374
|
-
{ "at": "3:04", "top": "return", "paths": 1, "taken": 1, "site": null },
|
|
375
|
-
{ "at": "5:04", "top": "return", "paths": 1, "taken": 0, "site": null }
|
|
376
|
-
]
|
|
377
|
-
}
|
|
378
|
-
}
|
|
379
|
-
```
|
|
380
|
-
|
|
381
|
-
Its `tree` says how those rows nest, by file, then card, then row.
|
|
382
|
-
A row inside a site names that site as its `parent`, the arm it is
|
|
383
|
-
`in`, and `on`, the indices of the parent's paths that run it, in the
|
|
384
|
-
order `paths` lists them. A row at the method's top has none of these.
|
|
385
|
-
|
|
386
|
-
```json
|
|
387
|
-
"tree": {
|
|
388
|
-
"label.rb": {
|
|
389
|
-
"1:0": {
|
|
390
|
-
"2:2": { "parent": null, "in": null, "on": null, "leaf": null, "gives": null },
|
|
391
|
-
"3:4": { "parent": "2:2", "in": "then", "on": [0], "leaf": null, "gives": null },
|
|
392
|
-
"5:4": { "parent": "2:2", "in": "else", "on": [1], "leaf": null, "gives": null }
|
|
393
|
-
}
|
|
394
|
-
}
|
|
395
|
-
}
|
|
396
|
-
```
|
|
486
|
+
The JSON report's `cards` hold each card's rows, its `ways` and its
|
|
487
|
+
`calls`, and its `tree` says how those rows nest.
|
|
397
488
|
|
|
398
|
-
A
|
|
399
|
-
|
|
400
|
-
|
|
401
|
-
known:
|
|
489
|
+
A method's card counts only the paths its method can take. A
|
|
490
|
+
local variable the method never writes, eg a parameter, read as a
|
|
491
|
+
leaf outside any block, has one outcome all through a call, so
|
|
402
492
|
|
|
403
493
|
```ruby
|
|
404
|
-
def
|
|
405
|
-
|
|
494
|
+
def fee(member, big)
|
|
495
|
+
return 0 if member
|
|
496
|
+
total = member ? 90 : 100 # its true path: none, member is false here
|
|
497
|
+
big ? total - 10 : total
|
|
406
498
|
end
|
|
407
499
|
```
|
|
408
500
|
|
|
409
|
-
|
|
410
|
-
|
|
411
|
-
|
|
412
|
-
|
|
501
|
+
the `?:` row has 1 path, not 2, its ways are 3, not 5, and Paths
|
|
502
|
+
combined counts 5, not 6. Paths alone, and the gap lines, still
|
|
503
|
+
list each decision's paths on its own. An instance variable or a
|
|
504
|
+
call may change between two reads, so is never taken to be the
|
|
505
|
+
same; nor is any local of a method calling `binding`, `eval`,
|
|
506
|
+
`send` or the like. A path taken that the method rules out is
|
|
507
|
+
counted, and printed:
|
|
413
508
|
|
|
414
|
-
|
|
415
|
-
|
|
509
|
+
```text
|
|
510
|
+
flip.rb:5:11 path taken that its method rules out: `member` true
|
|
511
|
+
```
|
|
416
512
|
|
|
417
513
|
### Reporters
|
|
418
514
|
|
|
419
|
-
- `:
|
|
420
|
-
|
|
421
|
-
|
|
422
|
-
|
|
423
|
-
|
|
424
|
-
|
|
515
|
+
- `:ways` prints, for each method with a path through it that no run
|
|
516
|
+
took, the choices a test must make to take each one, every leaf by its
|
|
517
|
+
source text; a loop is named by its line and how many items it ran:
|
|
518
|
+
|
|
519
|
+
```text
|
|
520
|
+
lib/orders.rb:2 total() has 1 untested path.
|
|
521
|
+
Add tests that take each of these paths:
|
|
522
|
+
1. line 5 `order[:paid]` false
|
|
523
|
+
```
|
|
524
|
+
|
|
525
|
+
Every run writes these lines to `path_coverage.ways.txt` in the
|
|
526
|
+
coverage dir, whichever reporters print, so a later check can show them.
|
|
527
|
+
|
|
425
528
|
- `:post` sends the JSON report, as an HTTP POST, to the url set by
|
|
426
529
|
`collector 'http://...'`. It is for a run whose own files are thrown
|
|
427
530
|
away, such as a failing `docker build` step. It prints nothing, never
|
|
@@ -432,8 +535,8 @@ answer true or false.
|
|
|
432
535
|
coverage:
|
|
433
536
|
|
|
434
537
|
```text
|
|
435
|
-
|
|
436
|
-
|
|
538
|
+
Paths alone: 527 / 631 (83.52%)
|
|
539
|
+
Paths combined: 44 / 52 (84.62%)
|
|
437
540
|
```
|
|
438
541
|
|
|
439
542
|
With groups, each group is a heading, its own totals indented under it,
|
|
@@ -441,19 +544,19 @@ Path coverage within methods: 44 / 52 (84.62%)
|
|
|
441
544
|
|
|
442
545
|
```text
|
|
443
546
|
code:
|
|
444
|
-
|
|
445
|
-
|
|
547
|
+
Paths alone: 481 / 567 (84.83%)
|
|
548
|
+
Paths combined: 41 / 49 (83.67%)
|
|
446
549
|
test:
|
|
447
|
-
|
|
448
|
-
|
|
550
|
+
Paths alone: 65 / 84 (77.38%)
|
|
551
|
+
Paths combined: 3 / 3 (100.00%)
|
|
449
552
|
```
|
|
450
553
|
|
|
451
554
|
## How it works
|
|
452
555
|
|
|
453
556
|
At load time each covered file is read and parsed once, with Prism. The
|
|
454
|
-
parse gives the manifest: each
|
|
557
|
+
parse gives the manifest: each decision's condition tree and the byte range of
|
|
455
558
|
each of its leaves. The rewriter then inserts a record call around each
|
|
456
|
-
leaf, and a leave call around the whole
|
|
559
|
+
leaf, and a leave call around the whole decision, whose first argument enters
|
|
457
560
|
it, without parsing again, and the result is compiled in place of the
|
|
458
561
|
file:
|
|
459
562
|
|
|
@@ -463,9 +566,9 @@ ok?(id) && new?(id)
|
|
|
463
566
|
PathCoverage.leave(PathCoverage.enter("k.rb:2:2", 2), PathCoverage.record("k.rb:2:2", 0, ok?(id)) && PathCoverage.record("k.rb:2:2", 1, new?(id)))
|
|
464
567
|
```
|
|
465
568
|
|
|
466
|
-
An `if`
|
|
569
|
+
An `if` decision's leave call wraps its predicate alone, as its path is
|
|
467
570
|
decided there, so an arm that returns or raises still counts it.
|
|
468
|
-
Each statement that is no
|
|
571
|
+
Each statement that is no decision gets a reach call before it, counting
|
|
469
572
|
its run, for its card's row.
|
|
470
573
|
|
|
471
574
|
A command call such as `ok? id` (after `and`) is not valid as a bare
|
|
@@ -477,7 +580,7 @@ totals. So a command call, or an `and` or `or`, written over several lines
|
|
|
477
580
|
can add to SimpleCov's relevant lines; nothing else does.
|
|
478
581
|
|
|
479
582
|
`record` counts whether its value was truthy or falsy, and notes it on the
|
|
480
|
-
|
|
583
|
+
decision's current path; `enter` starts that path, and `leave` counts it.
|
|
481
584
|
Each returns its value unchanged, so short-circuiting and the value of
|
|
482
585
|
the expression are preserved. Paths are kept per fiber, so threads never
|
|
483
586
|
mix their outcomes; an evaluation cut short by a raise counts no path,
|
|
@@ -488,33 +591,17 @@ rewrite happens in memory; no file on disk changes.
|
|
|
488
591
|
At exit the manifests parsed during the run are reused; only covered files
|
|
489
592
|
the run never loaded are read and parsed then.
|
|
490
593
|
|
|
491
|
-
##
|
|
492
|
-
|
|
493
|
-
`ignore '&&=', '||='` stops `&&=` and `||=` writes being measured.
|
|
494
|
-
A site inside such a write is still measured, eg the `a && b` in
|
|
495
|
-
`@m ||= a && b`. Ignoring `&&` does not ignore `and`, nor does ignoring
|
|
496
|
-
`||` ignore `or`. The summary and `index.html` list what is ignored.
|
|
497
|
-
|
|
498
|
-
`treat` tells path_coverage what you know about a method, so paths that
|
|
499
|
-
cannot happen are not listed as gaps. Add `in: 'source/models/**'` to
|
|
500
|
-
limit a treatment to the files that glob matches.
|
|
501
|
-
|
|
502
|
-
- `treat 'name', as: :present`: a call to `name` is never nil, so
|
|
503
|
-
`user.name&.size` has no path with `user.name` nil.
|
|
504
|
-
- `treat 'any?', as: :nonempty`: `x.any?` is true exactly when `x` has
|
|
505
|
-
items. When `x` is a call to `select`, `reject`, `filter`,
|
|
506
|
-
`filter_map`, `partition` or `take_while`, the path that call took
|
|
507
|
-
often decides whether it kept any items. Paths where `any?` disagrees
|
|
508
|
-
with that are not listed, eg `ids.select { |i| i.odd? }.any?` true
|
|
509
|
-
with `ids` empty.
|
|
510
|
-
|
|
511
|
-
A run where a treatment turns out to be wrong is reported:
|
|
594
|
+
## Roadmap
|
|
512
595
|
|
|
513
|
-
|
|
514
|
-
|
|
515
|
-
|
|
596
|
+
Decisions covered now: `&&`, `||`, `and`, `or`, the ternary `?:`,
|
|
597
|
+
`if`/`elsif`/`else` and `unless` (modifiers included), `case`/`when`
|
|
598
|
+
with a subject, and the block methods `any?` `all?` `none?` `find`
|
|
599
|
+
`detect` `find_index` `take_while` `select` `filter` `reject`
|
|
600
|
+
`partition` `count` `filter_map`, the writes `||=` and `&&=`, and `&.`
|
|
601
|
+
safe-navigation chains. Method paths cover every decision at a method's
|
|
602
|
+
top level.
|
|
516
603
|
|
|
517
|
-
|
|
604
|
+
Next:
|
|
518
605
|
|
|
519
606
|
- decisions hidden inside calls, each with its own outcome classes:
|
|
520
607
|
- `one?`, whose last answer does not decide it, and block methods
|
|
@@ -529,12 +616,12 @@ source/user.rb:4:9 path taken that a treatment rules out: `user.name` nil
|
|
|
529
616
|
nil or false is reported. Later, treatments could come from the
|
|
530
617
|
project's RBS signatures (`sig/*.rbs`) too.
|
|
531
618
|
- exceptions the code itself names: a `begin` or `def` with `rescue`
|
|
532
|
-
clauses as a
|
|
619
|
+
clauses as a decision, like a `case`, whose paths are no raise and each
|
|
533
620
|
clause that rescued, eg `rescue IOError` never taken. Branch coverage
|
|
534
621
|
ignores `rescue`, and tests reach these paths with real input. A raise
|
|
535
622
|
that nothing names, from any call, stays out, as every leaf would gain
|
|
536
623
|
a third outcome that only fault injection could take.
|
|
537
|
-
- an evaluation that raised part way through a
|
|
624
|
+
- an evaluation that raised part way through a decision, eg
|
|
538
625
|
`ok?(id) && fetch(id)` where `fetch` raises, reported as a path that
|
|
539
626
|
raised rather than dropped as unfinished.
|
|
540
627
|
- complexity maximums, reported and gated, since the possible paths
|
|
@@ -565,7 +652,7 @@ source/user.rb:4:9 path taken that a treatment rules out: `user.name` nil
|
|
|
565
652
|
branches in both the code and the tests, and that every path within the
|
|
566
653
|
tests' methods was taken, within the bounds in
|
|
567
654
|
`test/coverage_metrics_params.json` (a path bound is eg
|
|
568
|
-
`test.paths.
|
|
655
|
+
`test.paths.combined.missed`, and a missing path report denies it)
|
|
569
656
|
(`make test tids='6D1K'` runs only matching tests, and skips the check)
|
|
570
657
|
- `make test_check`: shunit2 tests showing that a breached bound fails the
|
|
571
658
|
check with a non-zero exit
|
|
@@ -582,13 +669,13 @@ is, each with a code and a test tab, then the check's verdict:
|
|
|
582
669
|
code:
|
|
583
670
|
Line coverage: 1270 / 1270 (100.00%)
|
|
584
671
|
Branch coverage: 321 / 321 (100.00%)
|
|
585
|
-
|
|
586
|
-
|
|
672
|
+
Paths alone: 481 / 567 (84.83%)
|
|
673
|
+
Paths combined: 41 / 49 (83.67%)
|
|
587
674
|
test:
|
|
588
675
|
Line coverage: 2250 / 2250 (100.00%)
|
|
589
676
|
Branch coverage: 43 / 43 (100.00%)
|
|
590
|
-
|
|
591
|
-
|
|
677
|
+
Paths alone: 65 / 84 (77.38%)
|
|
678
|
+
Paths combined: 3 / 3 (100.00%)
|
|
592
679
|
SimpleCov report: .../reports/index.html
|
|
593
680
|
PathCov report: .../reports/paths/index.html
|
|
594
681
|
coverage: ALLOWED
|