path_coverage 0.1.0 → 0.2.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/README.md +287 -78
- data/lib/path_coverage/calls.rb +54 -15
- data/lib/path_coverage/condition.rb +25 -2
- data/lib/path_coverage/configuration.rb +44 -7
- data/lib/path_coverage/configuration_paths.rb +56 -0
- data/lib/path_coverage/console_summary.rb +77 -0
- data/lib/path_coverage/gaps.rb +25 -9
- data/lib/path_coverage/html/index.html +3 -17
- data/lib/path_coverage/html/side.js +21 -2
- data/lib/path_coverage/html/sort.js +24 -21
- data/lib/path_coverage/html/style.css +28 -1
- data/lib/path_coverage/html/tabs.js +15 -0
- data/lib/path_coverage/html_index.rb +86 -31
- data/lib/path_coverage/html_index_head.rb +49 -0
- data/lib/path_coverage/html_report.rb +30 -13
- data/lib/path_coverage/html_tabs.rb +26 -0
- data/lib/path_coverage/html_tree.rb +41 -20
- data/lib/path_coverage/json_reader.rb +94 -0
- data/lib/path_coverage/json_writer.rb +41 -9
- data/lib/path_coverage/kernel.rb +42 -20
- data/lib/path_coverage/kinds/base.rb +71 -0
- data/lib/path_coverage/kinds/block.rb +136 -0
- data/lib/path_coverage/kinds/case.rb +89 -0
- data/lib/path_coverage/kinds/chain.rb +45 -0
- data/lib/path_coverage/kinds/choice.rb +68 -0
- data/lib/path_coverage/kinds/contains.rb +111 -0
- data/lib/path_coverage/kinds/safe.rb +151 -0
- data/lib/path_coverage/kinds/write.rb +54 -0
- data/lib/path_coverage/kinds.rb +37 -0
- data/lib/path_coverage/manifest.rb +58 -19
- data/lib/path_coverage/manifest_blocks.rb +49 -10
- data/lib/path_coverage/manifest_chains.rb +76 -0
- data/lib/path_coverage/manifest_choices.rb +30 -17
- data/lib/path_coverage/manifest_folding.rb +91 -0
- data/lib/path_coverage/manifest_ignoring.rb +35 -0
- data/lib/path_coverage/manifest_writes.rb +67 -0
- data/lib/path_coverage/merge.rb +19 -3
- data/lib/path_coverage/post.rb +8 -8
- data/lib/path_coverage/recorder.rb +9 -49
- data/lib/path_coverage/recorder_blocks.rb +68 -0
- data/lib/path_coverage/recorder_cases.rb +23 -0
- data/lib/path_coverage/recorder_chains.rb +40 -0
- data/lib/path_coverage/recorder_writes.rb +26 -0
- data/lib/path_coverage/render.rb +58 -46
- data/lib/path_coverage/result.rb +19 -0
- data/lib/path_coverage/rewriter.rb +58 -77
- data/lib/path_coverage/rewriter_nested.rb +62 -0
- data/lib/path_coverage/site_paths.rb +9 -58
- data/lib/path_coverage/site_tree.rb +12 -57
- data/lib/path_coverage/summary.rb +32 -4
- data/lib/path_coverage/totals.rb +1 -1
- data/lib/path_coverage/version.rb +1 -1
- metadata +30 -6
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: c68ba875b18ba000eb6be70f5c7bcb294ab4d50915868b913825c3f53bc60a93
|
|
4
|
+
data.tar.gz: 2beaba1e32cd4110ff9d04d3ab06e8c689395748566502a092175d8a339c9a5d
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: 6539c5ce97bd892ee616d0b1f780c79f18624381a404d3e6d2c60f6245957243b5ac7c395e130496fbbb2ba0ac6d69b4e9daf8e4fd235cb124d184aa3f8e5ad2
|
|
7
|
+
data.tar.gz: f72f62b796ad07d0290f56a88bc920ef1088b81abc04c6667ae1ee612e97499cc2f40ca6689acb1e8df35ebff607b463e8ec5ef271de11227bcf01017e98bb42
|
data/README.md
CHANGED
|
@@ -7,58 +7,19 @@
|
|
|
7
7
|
Path coverage for Ruby, to run alongside SimpleCov's line and branch
|
|
8
8
|
coverage.
|
|
9
9
|
|
|
10
|
-
Gem: https://rubygems.org/gems/path_coverage (which shows the latest
|
|
11
|
-
version).
|
|
12
|
-
|
|
13
10
|
Status: sites cover `&&`, `||`, `and`, `or`, the ternary `?:`,
|
|
14
11
|
`if`/`elsif`/`else` and `unless` (modifiers included), `case`/`when`
|
|
15
12
|
with a subject, and the block methods `any?` `all?` `none?` `find`
|
|
16
13
|
`detect` `find_index` `take_while` `select` `filter` `reject`
|
|
17
|
-
`partition` `count` `filter_map
|
|
14
|
+
`partition` `count` `filter_map`, the writes `||=` and `&&=`, and `&.`
|
|
15
|
+
safe-navigation chains. Method paths cover `if` decisions.
|
|
18
16
|
More decisions hidden inside calls come next (see Roadmap).
|
|
19
17
|
|
|
20
18
|
Demo: https://cyber-dojo.github.io/path_coverage/ shows the HTML report
|
|
21
19
|
for the mock source tree in `demo/`. `make demo` builds and opens the
|
|
22
20
|
same report locally.
|
|
23
21
|
|
|
24
|
-
|
|
25
|
-
|
|
26
|
-
```ruby
|
|
27
|
-
def fee(user, order)
|
|
28
|
-
total = 100
|
|
29
|
-
if user.member? && order.paid?
|
|
30
|
-
total -= 5
|
|
31
|
-
end
|
|
32
|
-
if order.big?
|
|
33
|
-
total -= 10
|
|
34
|
-
end
|
|
35
|
-
total
|
|
36
|
-
end
|
|
37
|
-
```
|
|
38
|
-
|
|
39
|
-
Tests that run both `if`s both ways reach every line and every branch,
|
|
40
|
-
so SimpleCov, which reads Ruby's built-in `Coverage`, reports 100%. Yet
|
|
41
|
-
they can still miss paths, at two scopes, and path_coverage lists each
|
|
42
|
-
path no test took.
|
|
43
|
-
|
|
44
|
-
## Within each decision
|
|
45
|
-
|
|
46
|
-
The first `if` can go three ways: member and paid, member and not
|
|
47
|
-
paid, not a member. A member who has not paid may never have been
|
|
48
|
-
tried.
|
|
49
|
-
|
|
50
|
-

|
|
52
|
-
|
|
53
|
-
## Across a method's decisions
|
|
54
|
-
|
|
55
|
-
The two `if`s can go four ways together. A paying member with a small
|
|
56
|
-
order may never have been tried.
|
|
57
|
-
|
|
58
|
-

|
|
60
|
-
|
|
61
|
-
## Use
|
|
22
|
+
## Getting started
|
|
62
23
|
|
|
63
24
|
```ruby
|
|
64
25
|
# test coverage config, before any application code is required
|
|
@@ -70,14 +31,101 @@ end
|
|
|
70
31
|
PathCoverage.start do
|
|
71
32
|
root APP_DIR # cover globs and site names are relative to it
|
|
72
33
|
cover 'source/**/*.rb' # each matching file is instrumented as it loads
|
|
34
|
+
cover 'test/**/*.rb' # more than one cover glob may be given
|
|
35
|
+
skip 'source/vendor/**/*.rb' # except these, though a cover glob matches them
|
|
73
36
|
coverage_dir '/tmp/coverage' # where path_coverage.json is written
|
|
74
37
|
measure :decisions, :methods # :decisions alone unless :methods is named
|
|
75
|
-
report :gaps, :html # the reporters run at exit (default :gaps)
|
|
38
|
+
report :gaps, :html # the reporters run at exit (default :gaps);
|
|
39
|
+
# also :summary and :post
|
|
40
|
+
group 'code', 'source/**/*.rb' # each group a tab of its own on index.html
|
|
41
|
+
group 'test', 'test/**/*.rb'
|
|
42
|
+
ignore '&&=' # operators whose sites are not measured
|
|
43
|
+
treat 'name', as: :present # a call to name is never nil
|
|
44
|
+
treat 'any?', as: :nonempty # any? is true when its receiver has items
|
|
76
45
|
end
|
|
77
46
|
```
|
|
78
47
|
|
|
79
|
-
|
|
80
|
-
|
|
48
|
+
|
|
49
|
+
## Reading the HTML
|
|
50
|
+
|
|
51
|
+
Clicking a line number on a file's HTML page opens that line's popout.
|
|
52
|
+
|
|
53
|
+
Each popout has two sections:
|
|
54
|
+
- on the left is a hierarchical representation of the code fragment.
|
|
55
|
+
- on the right is one column for each possible path through that fragment.
|
|
56
|
+
- at the top of the column is the count of that path. Look for zeros!
|
|
57
|
+
- the symbols in the column align with their fragment and are explained below.
|
|
58
|
+
|
|
59
|
+
### Boolean: `T` `F`
|
|
60
|
+
|
|
61
|
+

|
|
62
|
+
|
|
63
|
+
- `T` the expression was true.
|
|
64
|
+
- `F` the expression was false.
|
|
65
|
+
|
|
66
|
+
In this example, there was no test which exercised the path where `name.size > 2` was `true`, and
|
|
67
|
+
`name.ascii_only?` was `false`.
|
|
68
|
+
|
|
69
|
+
### Unreachable: blank
|
|
70
|
+
|
|
71
|
+

|
|
72
|
+
|
|
73
|
+
- "blank" the expression is never evaluated.
|
|
74
|
+
|
|
75
|
+
In the second column `dee` was `T`, so because of `||`'s short-circuiting, `dum` is never evaluated.
|
|
76
|
+
|
|
77
|
+
### Taken: `✓`
|
|
78
|
+
|
|
79
|
+

|
|
80
|
+
|
|
81
|
+
- `✓` marks the column whose path took that "arm"
|
|
82
|
+
|
|
83
|
+
The `paid` row is the `if`'s predicate, `T` or `F`. Below it is one row
|
|
84
|
+
per arm, named `then` and `else`. The first column, `paid` false, took `else`; the
|
|
85
|
+
second, `paid` true, took `then`.
|
|
86
|
+
|
|
87
|
+
### Navigation: `∅` `●`
|
|
88
|
+
|
|
89
|
+

|
|
90
|
+
|
|
91
|
+
- `∅` the receiver was nil, so `&.` skipped the rest of the chain.
|
|
92
|
+
- `●` the receiver was not nil, so `&.` made the call.
|
|
93
|
+
|
|
94
|
+
In this example, there was no test which exercised the path where
|
|
95
|
+
`user` was nil.
|
|
96
|
+
|
|
97
|
+
### Count: `0` `1` `*`
|
|
98
|
+
|
|
99
|
+

|
|
100
|
+
|
|
101
|
+
- `0` the block was called with no items.
|
|
102
|
+
- `1` the block was called with one item.
|
|
103
|
+
- `*` the block was called with two or more items.
|
|
104
|
+
|
|
105
|
+
In this example, there was no test which exercised the path where
|
|
106
|
+
the block was called with two or more items and every answer was
|
|
107
|
+
`false`.
|
|
108
|
+
|
|
109
|
+
### Both: `TF`
|
|
110
|
+
|
|
111
|
+

|
|
112
|
+
|
|
113
|
+
- `T` every answer of the block was true.
|
|
114
|
+
- `F` every answer of the block was false.
|
|
115
|
+
- `TF` the block's answers were both true and false.
|
|
116
|
+
|
|
117
|
+
In this example, there was no test which exercised the path where
|
|
118
|
+
the block was called with two or more items and every answer was
|
|
119
|
+
`false`.
|
|
120
|
+
|
|
121
|
+
### Lots of paths!
|
|
122
|
+
|
|
123
|
+
Ruby code can be very compact so short expressions can generate lots of paths!
|
|
124
|
+
|
|
125
|
+

|
|
126
|
+
|
|
127
|
+
|
|
128
|
+
## Use when you cannot edit the code!
|
|
81
129
|
|
|
82
130
|
A run whose code cannot be changed, such as a script in a `docker build`
|
|
83
131
|
step, can be covered from its environment alone:
|
|
@@ -86,6 +134,7 @@ step, can be covered from its environment alone:
|
|
|
86
134
|
RUBYOPT=-rpath_coverage/auto
|
|
87
135
|
PATH_COVERAGE_ROOT=/app # root
|
|
88
136
|
PATH_COVERAGE_COVER=src/**/*.rb,lib/*.rb # cover, comma-separated
|
|
137
|
+
PATH_COVERAGE_SKIP=src/vendor/**/*.rb # skip, comma-separated, optional
|
|
89
138
|
PATH_COVERAGE_DIR=/tmp/paths # coverage_dir
|
|
90
139
|
PATH_COVERAGE_COLLECTOR=http://host.docker.internal:4570/runs
|
|
91
140
|
```
|
|
@@ -122,13 +171,23 @@ As the process exits:
|
|
|
122
171
|
|
|
123
172
|
- `path_coverage.json` is always written to `coverage_dir`: every site
|
|
124
173
|
of every covered file, with each operand's true and false counts under
|
|
125
|
-
`sites`,
|
|
126
|
-
|
|
174
|
+
`sites`, each site's paths under `paths`, each method's paths taken
|
|
175
|
+
under `method_paths` when `:methods` is measured, and the paths taken
|
|
176
|
+
of all possible under `totals`, as [taken, possible]: `decisions`,
|
|
177
|
+
`methods` when `:methods` is measured, and, with groups, each group's
|
|
178
|
+
own under `groups`, eg `"groups": {"code": {"decisions": [481, 567]}}`.
|
|
179
|
+
It also names the kinds of site measured under `kinds`, any `ignored`
|
|
180
|
+
operators and `treats`, and any paths taken that a treatment rules
|
|
181
|
+
out under `outside`.
|
|
182
|
+
It is the base the other reporters read, so they can also be run
|
|
183
|
+
later, in another process, from
|
|
127
184
|
it alone: `PathCoverage::Render.from_json(config)`, where `config` is a
|
|
128
185
|
`PathCoverage::Configuration` built from the same settings block.
|
|
129
186
|
Given several runs' reports, `Render.from_json(config, paths)` adds
|
|
130
187
|
their counts together first, so the report shows only the gaps that
|
|
131
|
-
every run left.
|
|
188
|
+
every run left. A report whose kinds, `ignored` or `treats` differ
|
|
189
|
+
from `config`'s is refused, as its sites list their paths
|
|
190
|
+
differently.
|
|
132
191
|
- `:gaps` prints one line per path never taken, in manifest order: a
|
|
133
192
|
site's naming the outcome of each leaf on it, a method's the outcome
|
|
134
193
|
of each decision on it, by line:
|
|
@@ -141,7 +200,8 @@ source/fee.rb:2:2 fee path never taken: line 4 true, line 7 false
|
|
|
141
200
|
|
|
142
201
|
### Decisions
|
|
143
202
|
|
|
144
|
-
|
|
203
|
+
The paths within each decision are always measured, whatever `measure`
|
|
204
|
+
names.
|
|
145
205
|
|
|
146
206
|
A site is named by the `path:line:column` where it starts, the path
|
|
147
207
|
relative to `root`. A site in a covered file the run never loaded is
|
|
@@ -168,17 +228,8 @@ should read the JSON, whose `paths` give each site's leaves as
|
|
|
168
228
|
```
|
|
169
229
|
|
|
170
230
|
A site is an outermost condition. Its leaves are numbered left to right
|
|
171
|
-
across the whole of it, nested conditions included
|
|
172
|
-
|
|
173
|
-
```ruby
|
|
174
|
-
ok?(id) && new?(id) && id.size == 6 # one && site, leaves 0 1 2
|
|
175
|
-
ok?(id) && (new?(id) || id.size == 6) # an || nested in an && site, 0 1 2
|
|
176
|
-
ok?(id) && new?(id) || id.size == 6 # an && nested in an || site, 0 1 2
|
|
177
|
-
ids.map { |k| k && k.ok? } && ok?(id) # two sites: the block's is its own
|
|
178
|
-
ok?(id) ? :ok : :no # a ?: site, its predicate leaf 0
|
|
179
|
-
ok?(id) && new?(id) ? :ok : :no # an && nested in a ?: site, 0 1
|
|
180
|
-
ok?(id) ? new?(id) && id.size == 6 : :no # one ?: site, its then arm 1 2
|
|
181
|
-
```
|
|
231
|
+
across the whole of it, nested conditions included. A condition inside a
|
|
232
|
+
block is a site of its own.
|
|
182
233
|
|
|
183
234
|
A ternary is a site even when its predicate is a single leaf. An arm
|
|
184
235
|
holding a condition of `&&`, `||`, `and`, `or` or `?:` is part of the
|
|
@@ -201,16 +252,31 @@ whose `===` calls the pattern's `===` once, exactly as the `case` would,
|
|
|
201
252
|
and notes whether it matched. A `case` with no subject, or with a splat
|
|
202
253
|
among its patterns, is no site.
|
|
203
254
|
|
|
204
|
-
A call to
|
|
205
|
-
|
|
206
|
-
|
|
207
|
-
|
|
208
|
-
|
|
209
|
-
|
|
210
|
-
|
|
211
|
-
|
|
212
|
-
|
|
213
|
-
|
|
255
|
+
A call to `any?` `all?` `none?` `find` `detect` `find_index`
|
|
256
|
+
`take_while` `select` `filter` `reject` `partition` `count` or
|
|
257
|
+
`filter_map`, with no arguments and a block of one expression, is a site
|
|
258
|
+
named at its method. Its item count is noted from the block's own calls,
|
|
259
|
+
so the receiver is never asked its size, which could consume a sequence
|
|
260
|
+
that can be read only once.
|
|
261
|
+
|
|
262
|
+
A `||=` or `&&=` write is a site, whatever its target: a variable of any
|
|
263
|
+
kind, a constant, an attribute (`self.size ||= n`) or an index
|
|
264
|
+
(`store[key] ||= key.upcase`). Its one leaf is its target, and its right
|
|
265
|
+
side is an arm, run when the target is false for `||=`, true for `&&=`,
|
|
266
|
+
as an `unless` or an `if` runs its arm. The target is never read again,
|
|
267
|
+
since reading `store[key]` could call a method twice: its outcome follows
|
|
268
|
+
from whether the right side ran. So `@clock ||= Clock.now` called only
|
|
269
|
+
once, as a fresh object, leaves the path with `@clock` true a gap.
|
|
270
|
+
|
|
271
|
+
A `&.` chain is a site, named where it starts. Its leaves are the
|
|
272
|
+
receivers of its `&.` calls. `false` counts as not nil, as `&.` calls on
|
|
273
|
+
it.
|
|
274
|
+
|
|
275
|
+
A chain inside a condition, eg `ok?(id) && user&.admin?` or
|
|
276
|
+
`if user&.admin?`, folds into its site, as one expression: its value is
|
|
277
|
+
one more leaf, true or false, reached when every receiver is not nil. A
|
|
278
|
+
write through `&.`, eg `box&.size ||= 3`, is no site, as a nil `box`
|
|
279
|
+
skips it whole and its target is never looked at.
|
|
214
280
|
|
|
215
281
|
A chain of one operator is flat; a change of operator, by precedence or by
|
|
216
282
|
parentheses, nests. Parentheses are not part of the tree, since they decide
|
|
@@ -228,6 +294,18 @@ constants, since one can hold nil or false (`VERBOSE = false`). `nil` and
|
|
|
228
294
|
@clock = options[:clock] || MonotonicClock.new # leaf 1: never evaluated, or no gap
|
|
229
295
|
```
|
|
230
296
|
|
|
297
|
+
Any other leaf, a call among them, is taken to have either outcome, as its
|
|
298
|
+
form alone cannot say which it returns. So a call that never returns nil
|
|
299
|
+
or false still leaves a path needing it falsy, which no test can take:
|
|
300
|
+
|
|
301
|
+
```ruby
|
|
302
|
+
methods: paths && methods_of(path, methods, paths) # [true, false] impossible
|
|
303
|
+
methods: paths ? methods_of(path, methods, paths) : nil # the same, no such path
|
|
304
|
+
```
|
|
305
|
+
|
|
306
|
+
Written as a choice, the same code has no such path. Reporting a gap that
|
|
307
|
+
cannot be closed is the lesser error than hiding one that can.
|
|
308
|
+
|
|
231
309
|
### Methods
|
|
232
310
|
|
|
233
311
|
With `measure :decisions, :methods`, each method with two or more
|
|
@@ -239,11 +317,24 @@ site's columns stay its own leaves' combinations, and a method's its own
|
|
|
239
317
|
decisions', rather than multiplying together. A decision whose arm always
|
|
240
318
|
leaves the method ends the path there.
|
|
241
319
|
|
|
320
|
+
The JSON report's `method_paths` lists each method's paths taken, with
|
|
321
|
+
their counts, so a later render, or several runs' reports merged, still
|
|
322
|
+
shows them. A report with none, eg from a run that did not measure
|
|
323
|
+
`:methods`, renders as no method path taken:
|
|
324
|
+
|
|
325
|
+
```json
|
|
326
|
+
"method_paths": {
|
|
327
|
+
"source/fee.rb:2:2": [
|
|
328
|
+
{ "path": [true, false], "count": 1 }
|
|
329
|
+
]
|
|
330
|
+
}
|
|
331
|
+
```
|
|
332
|
+
|
|
242
333
|
### Reporters
|
|
243
334
|
|
|
244
335
|
- `:html` writes `index.html` to `coverage_dir`: the covered files with
|
|
245
336
|
their sites and their paths taken of all possible, within each decision
|
|
246
|
-
and, when `:methods` is measured,
|
|
337
|
+
and, when `:methods` is measured, within each method (eg
|
|
247
338
|
`9 / 28 (32%)`, or `-` for a file with no method of two or more
|
|
248
339
|
decisions), each column sortable, and a Total row for every file
|
|
249
340
|
together. Each file name opens `files/<path>.html`, the file's
|
|
@@ -268,6 +359,25 @@ leaves the method ends the path there.
|
|
|
268
359
|
goes through an `http_proxy`, and gives up in silence on a collector
|
|
269
360
|
that cannot be reached or does not answer within a second, so the
|
|
270
361
|
run's output and exit status are unchanged.
|
|
362
|
+
- `:summary` prints each total as SimpleCov prints its line and branch
|
|
363
|
+
coverage, the methods line only when `:methods` is measured:
|
|
364
|
+
|
|
365
|
+
```text
|
|
366
|
+
Path coverage within decisions: 527 / 631 (83.52%)
|
|
367
|
+
Path coverage within methods: 44 / 52 (84.62%)
|
|
368
|
+
```
|
|
369
|
+
|
|
370
|
+
With groups, each group is a heading, its own totals indented under it,
|
|
371
|
+
and there is no total over every group.
|
|
372
|
+
|
|
373
|
+
```text
|
|
374
|
+
code:
|
|
375
|
+
Path coverage within decisions: 481 / 567 (84.83%)
|
|
376
|
+
Path coverage within methods: 41 / 49 (83.67%)
|
|
377
|
+
test:
|
|
378
|
+
Path coverage within decisions: 65 / 84 (77.38%)
|
|
379
|
+
Path coverage within methods: 3 / 3 (100.00%)
|
|
380
|
+
```
|
|
271
381
|
|
|
272
382
|
## How it works
|
|
273
383
|
|
|
@@ -310,19 +420,65 @@ rewrite happens in memory; no file on disk changes.
|
|
|
310
420
|
At exit the manifests parsed during the run are reused; only covered files
|
|
311
421
|
the run never loaded are read and parsed then.
|
|
312
422
|
|
|
423
|
+
## Ignoring and treating
|
|
424
|
+
|
|
425
|
+
`ignore '&&=', '||='` stops `&&=` and `||=` writes being measured.
|
|
426
|
+
A site inside such a write is still measured, eg the `a && b` in
|
|
427
|
+
`@m ||= a && b`. Ignoring `&&` does not ignore `and`, nor does ignoring
|
|
428
|
+
`||` ignore `or`. The summary and `index.html` list what is ignored.
|
|
429
|
+
|
|
430
|
+
`treat` tells path_coverage what you know about a method, so paths that
|
|
431
|
+
cannot happen are not listed as gaps. Add `in: 'source/models/**'` to
|
|
432
|
+
limit a treatment to the files that glob matches.
|
|
433
|
+
|
|
434
|
+
- `treat 'name', as: :present`: a call to `name` is never nil, so
|
|
435
|
+
`user.name&.size` has no path with `user.name` nil.
|
|
436
|
+
- `treat 'any?', as: :nonempty`: `x.any?` is true exactly when `x` has
|
|
437
|
+
items. When `x` is a call to `select`, `reject`, `filter`,
|
|
438
|
+
`filter_map`, `partition` or `take_while`, the path that call took
|
|
439
|
+
often decides whether it kept any items. Paths where `any?` disagrees
|
|
440
|
+
with that are not listed, eg `ids.select { |i| i.odd? }.any?` true
|
|
441
|
+
with `ids` empty.
|
|
442
|
+
|
|
443
|
+
A run where a treatment turns out to be wrong is reported:
|
|
444
|
+
|
|
445
|
+
```text
|
|
446
|
+
source/user.rb:4:9 path taken that a treatment rules out: `user.name` nil
|
|
447
|
+
```
|
|
448
|
+
|
|
313
449
|
## Roadmap
|
|
314
450
|
|
|
315
451
|
- decisions hidden inside calls, each with its own outcome classes:
|
|
316
452
|
- `one?`, whose last answer does not decide it, and block methods
|
|
317
453
|
with blocks of more than one expression
|
|
318
454
|
- `fetch(k, default)`, `dig`, `Hash#[]` with a default: present vs default
|
|
319
|
-
- `x || default` in value context
|
|
320
|
-
- `||=` and `&&=`, on variables, attributes and index targets: the
|
|
321
|
-
right side evaluated vs skipped, counted without evaluating the
|
|
322
|
-
target twice
|
|
455
|
+
- `x || default` in value context: nil vs not nil
|
|
323
456
|
- `x rescue y` modifier: raised vs not raised
|
|
324
457
|
- `case` with no subject, and `case`/`in` patterns
|
|
325
458
|
- regex matches: nil vs MatchData
|
|
459
|
+
- `treat 'methods_of', as: :truthy`, so each call to it counts as a
|
|
460
|
+
leaf that is only truthy, as `Foo.new` is, and a run where it returns
|
|
461
|
+
nil or false is reported. Later, treatments could come from the
|
|
462
|
+
project's RBS signatures (`sig/*.rbs`) too.
|
|
463
|
+
- exceptions the code itself names: a `begin` or `def` with `rescue`
|
|
464
|
+
clauses as a site, like a `case`, whose paths are no raise and each
|
|
465
|
+
clause that rescued, eg `rescue IOError` never taken. Branch coverage
|
|
466
|
+
ignores `rescue`, and tests reach these paths with real input. A raise
|
|
467
|
+
that nothing names, from any call, stays out, as every leaf would gain
|
|
468
|
+
a third outcome that only fault injection could take.
|
|
469
|
+
- an evaluation that raised part way through a site, eg
|
|
470
|
+
`ok?(id) && fetch(id)` where `fetch` raises, reported as a path that
|
|
471
|
+
raised rather than dropped as unfinished.
|
|
472
|
+
- complexity maximums, reported and gated, since the possible paths
|
|
473
|
+
count how much there is to test, not only how much was tested. A
|
|
474
|
+
file's total measures its size, so each is a maximum over smaller
|
|
475
|
+
parts:
|
|
476
|
+
- paths in one decision, eg 6 for a `select` block
|
|
477
|
+
- paths through one method's decisions, eg 2 top-level ifs give 4
|
|
478
|
+
- paths within each decision, summed over every decision in one
|
|
479
|
+
method, nested ones included, eg decisions of 2, 3 and 15 paths
|
|
480
|
+
sum to 20. Every method has one, and it grows as the method gains
|
|
481
|
+
conditions, where the paths through a method multiply.
|
|
326
482
|
|
|
327
483
|
## Constraints
|
|
328
484
|
|
|
@@ -336,13 +492,48 @@ the run never loaded are read and parsed then.
|
|
|
336
492
|
`make all` runs, in a Docker image built from the repo's `Dockerfile`:
|
|
337
493
|
|
|
338
494
|
- `make lint`: rubocop over `lib/`, `test/` and `bin/` alike
|
|
339
|
-
- `make test`: the tests,
|
|
340
|
-
|
|
341
|
-
|
|
495
|
+
- `make test`: the tests, with path_coverage measuring its own `lib/` and
|
|
496
|
+
`test/`, then a check that SimpleCov's report has zero missed lines and
|
|
497
|
+
branches in both the code and the tests, and that every path within the
|
|
498
|
+
tests' methods was taken, within the bounds in
|
|
499
|
+
`test/coverage_metrics_params.json` (a path bound is eg
|
|
500
|
+
`test.paths.methods.missed`, and a missing path report denies it)
|
|
342
501
|
(`make test tids='6D1K'` runs only matching tests, and skips the check)
|
|
343
502
|
- `make test_check`: shunit2 tests showing that a breached bound fails the
|
|
344
503
|
check with a non-zero exit
|
|
345
504
|
|
|
505
|
+
Each run prints its seed; `SEED=n make test` replays that run's order of
|
|
506
|
+
files and tests. When the coverage check fails, it names each line and
|
|
507
|
+
branch missed.
|
|
508
|
+
|
|
509
|
+
A run ends with each group, code and test, as a heading over its four
|
|
510
|
+
figures, SimpleCov's and path_coverage's together, then where each report
|
|
511
|
+
is, each with a code and a test tab, then the check's verdict:
|
|
512
|
+
|
|
513
|
+
```text
|
|
514
|
+
code:
|
|
515
|
+
Line coverage: 1270 / 1270 (100.00%)
|
|
516
|
+
Branch coverage: 321 / 321 (100.00%)
|
|
517
|
+
Path coverage within decisions: 481 / 567 (84.83%)
|
|
518
|
+
Path coverage within methods: 41 / 49 (83.67%)
|
|
519
|
+
test:
|
|
520
|
+
Line coverage: 2250 / 2250 (100.00%)
|
|
521
|
+
Branch coverage: 43 / 43 (100.00%)
|
|
522
|
+
Path coverage within decisions: 65 / 84 (77.38%)
|
|
523
|
+
Path coverage within methods: 3 / 3 (100.00%)
|
|
524
|
+
Coverage report: .../reports/index.html
|
|
525
|
+
Path coverage report: .../reports/paths/index.html
|
|
526
|
+
coverage: ALLOWED
|
|
527
|
+
```
|
|
528
|
+
|
|
529
|
+
`bin/check_coverage_metrics.rb` prints the groups, reading SimpleCov's
|
|
530
|
+
figures from `reports/coverage_metrics.json` and path_coverage's from the
|
|
531
|
+
`totals` in `reports/paths/path_coverage.json`; SimpleCov's own console
|
|
532
|
+
lines, totals over every group, are silenced. The tool measuring paths is
|
|
533
|
+
a copy of `lib/` renamed `PathCoverageSelf` (`test/self_coverage.rb`), so
|
|
534
|
+
the tests start and restart `PathCoverage` as always. SimpleCov's figures
|
|
535
|
+
for `lib/` are the same with it as without.
|
|
536
|
+
|
|
346
537
|
`make demo` (not part of `make all`) runs `demo/run.rb` through
|
|
347
538
|
`bin/build_demo.sh`, writing the report to `site/` and opening it. The
|
|
348
539
|
Pages workflow (`.github/workflows/pages.yml`) runs the same script on
|
|
@@ -351,3 +542,21 @@ each push to `main` and publishes `site/`.
|
|
|
351
542
|
`bin/capture_readme_images.sh` rebuilds the demo and saves this README's
|
|
352
543
|
two screenshots, `docs/decision.png` and `docs/method.png`, from it with
|
|
353
544
|
headless Chrome. Run it again whenever the report's look changes.
|
|
545
|
+
|
|
546
|
+
|
|
547
|
+
## Within each decision
|
|
548
|
+
|
|
549
|
+
The first `if` can go three ways: member and paid, member and not
|
|
550
|
+
paid, not a member. A member who has not paid may never have been
|
|
551
|
+
tried.
|
|
552
|
+
|
|
553
|
+

|
|
555
|
+
|
|
556
|
+
## Within each method
|
|
557
|
+
|
|
558
|
+
The two `if`s can go four ways together. A paying member with a small
|
|
559
|
+
order may never have been tried.
|
|
560
|
+
|
|
561
|
+

|
data/lib/path_coverage/calls.rb
CHANGED
|
@@ -26,30 +26,69 @@ module PathCoverage
|
|
|
26
26
|
@recorder.when_match(site, index, size, pattern)
|
|
27
27
|
end
|
|
28
28
|
|
|
29
|
+
# The call rewritten code makes around the pattern of a case
|
|
30
|
+
# nested in site, the site's index-th leaf; returns the
|
|
31
|
+
# pattern's stand-in.
|
|
32
|
+
def self.pattern(site, index, pattern)
|
|
33
|
+
@recorder.pattern(site, index, pattern)
|
|
34
|
+
end
|
|
35
|
+
|
|
36
|
+
# The call rewritten code makes around the right side of a
|
|
37
|
+
# write nested in site, its target the site's at-th leaf,
|
|
38
|
+
# outcome being the target's when it runs; returns value
|
|
39
|
+
# unchanged.
|
|
40
|
+
def self.written_at(site, at, outcome, value)
|
|
41
|
+
@recorder.written_at(site, at, outcome, value)
|
|
42
|
+
end
|
|
43
|
+
|
|
44
|
+
# The call rewritten code makes around a write nested in site,
|
|
45
|
+
# its target the site's at-th leaf, skipped being the target's
|
|
46
|
+
# outcome when its right side never ran; returns value
|
|
47
|
+
# unchanged.
|
|
48
|
+
def self.settle(site, at, skipped, value)
|
|
49
|
+
@recorder.settle(site, at, skipped, value)
|
|
50
|
+
end
|
|
51
|
+
|
|
29
52
|
# The call rewritten code makes before a block method's call
|
|
30
|
-
#
|
|
31
|
-
|
|
32
|
-
|
|
53
|
+
# nested in site runs, its receiver the site's at-th leaf;
|
|
54
|
+
# returns the site's trail.
|
|
55
|
+
def self.begin_items(site, at)
|
|
56
|
+
@recorder.begin_items(site, at)
|
|
33
57
|
end
|
|
34
58
|
|
|
35
|
-
# The call rewritten code makes around
|
|
36
|
-
# block
|
|
37
|
-
|
|
38
|
-
|
|
59
|
+
# The call rewritten code makes around the block's expression
|
|
60
|
+
# of a stop-early block method nested in site, its receiver the
|
|
61
|
+
# site's at-th leaf; returns value unchanged.
|
|
62
|
+
def self.item_at(site, at, value)
|
|
63
|
+
@recorder.item_at(site, at, value)
|
|
39
64
|
end
|
|
40
65
|
|
|
41
66
|
# The call rewritten code makes around the block's expression
|
|
42
|
-
# of
|
|
43
|
-
# value unchanged.
|
|
44
|
-
def self.
|
|
45
|
-
@recorder.
|
|
67
|
+
# of an every-item block method nested in site, its receiver
|
|
68
|
+
# the site's at-th leaf; returns value unchanged.
|
|
69
|
+
def self.answer_at(site, at, value)
|
|
70
|
+
@recorder.answer_at(site, at, value)
|
|
46
71
|
end
|
|
47
72
|
|
|
48
73
|
# The call rewritten code makes around a block method's call
|
|
49
|
-
#
|
|
50
|
-
# value unchanged.
|
|
51
|
-
def self.
|
|
52
|
-
@recorder.
|
|
74
|
+
# nested in site, its receiver the site's at-th leaf, with the
|
|
75
|
+
# trail begin_items gave it; returns value unchanged.
|
|
76
|
+
def self.end_items(site, at, trail, value)
|
|
77
|
+
@recorder.end_items(site, at, trail, value)
|
|
78
|
+
end
|
|
79
|
+
|
|
80
|
+
# The call rewritten code makes around a &. receiver, the
|
|
81
|
+
# index-th leaf of site, its chain's leaves starting at first;
|
|
82
|
+
# returns value unchanged.
|
|
83
|
+
def self.present(site, index, first, value)
|
|
84
|
+
@recorder.present(site, index, first, value)
|
|
85
|
+
end
|
|
86
|
+
|
|
87
|
+
# The call rewritten code makes around a &. chain's value, the
|
|
88
|
+
# index-th leaf of site, its chain's leaves starting at first;
|
|
89
|
+
# returns value unchanged.
|
|
90
|
+
def self.chained(site, index, first, value)
|
|
91
|
+
@recorder.chained(site, index, first, value)
|
|
53
92
|
end
|
|
54
93
|
|
|
55
94
|
# The call rewritten code makes as a call reaches its
|
|
@@ -6,9 +6,28 @@ module PathCoverage
|
|
|
6
6
|
# decided by their predicate alone.
|
|
7
7
|
STATEMENT_CHOICES = %w[if unless].freeze
|
|
8
8
|
|
|
9
|
+
# The operator of a safe-navigation chain, eg user&.account.
|
|
10
|
+
SAFE = '&.'.freeze
|
|
11
|
+
|
|
12
|
+
# The operator of a leaf with sites inside it, eg
|
|
13
|
+
# ids.select { |i| i.odd? }.any? as an operand of an &&: its
|
|
14
|
+
# operands are those sites' Conditions, then the leaf's own byte
|
|
15
|
+
# range, whose outcome is the leaf's value.
|
|
16
|
+
CONTAINS = 'contains'.freeze
|
|
17
|
+
|
|
18
|
+
# A &. receiver's outcome: not nil, so the call is made, false
|
|
19
|
+
# included, or nil, so the rest of the chain is skipped.
|
|
20
|
+
PRESENCE = ['not nil', 'nil'].freeze
|
|
21
|
+
|
|
22
|
+
# The writes whose target decides whether their right side
|
|
23
|
+
# runs, as an unless or an if decides its arm: x ||= v runs v
|
|
24
|
+
# when x is false, x &&= v when x is true.
|
|
25
|
+
WRITES = %w[||= &&=].freeze
|
|
26
|
+
|
|
9
27
|
# The predicate outcome on which each choice takes its then
|
|
10
|
-
# arm: an unless's is false.
|
|
11
|
-
THEN_ON = Hash.new(true).merge('unless' => false)
|
|
28
|
+
# arm: an unless's is false, as is a ||= write's.
|
|
29
|
+
THEN_ON = Hash.new(true).merge('unless' => false, '||=' => false)
|
|
30
|
+
.freeze
|
|
12
31
|
|
|
13
32
|
# The methods that call their block an item at a time until
|
|
14
33
|
# it decides, its last answer deciding the result.
|
|
@@ -47,6 +66,10 @@ module PathCoverage
|
|
|
47
66
|
[ITEMS[2], ANSWERS[2]]
|
|
48
67
|
].freeze
|
|
49
68
|
|
|
69
|
+
# The block methods whose value follows from their path alone:
|
|
70
|
+
# all but find and detect, whose found item may itself be nil.
|
|
71
|
+
VALUED = [*STOP_EARLY - %w[find detect], *EVERY_ITEM].freeze
|
|
72
|
+
|
|
50
73
|
# Every block method whose call is a site, by its paths.
|
|
51
74
|
BLOCK_PATHS = [
|
|
52
75
|
*STOP_EARLY.map { |name| [name, STOP_EARLY_PATHS] },
|