path_coverage 0.1.0 → 0.2.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (54) hide show
  1. checksums.yaml +4 -4
  2. data/README.md +287 -78
  3. data/lib/path_coverage/calls.rb +54 -15
  4. data/lib/path_coverage/condition.rb +25 -2
  5. data/lib/path_coverage/configuration.rb +44 -7
  6. data/lib/path_coverage/configuration_paths.rb +56 -0
  7. data/lib/path_coverage/console_summary.rb +77 -0
  8. data/lib/path_coverage/gaps.rb +25 -9
  9. data/lib/path_coverage/html/index.html +3 -17
  10. data/lib/path_coverage/html/side.js +21 -2
  11. data/lib/path_coverage/html/sort.js +24 -21
  12. data/lib/path_coverage/html/style.css +28 -1
  13. data/lib/path_coverage/html/tabs.js +15 -0
  14. data/lib/path_coverage/html_index.rb +86 -31
  15. data/lib/path_coverage/html_index_head.rb +49 -0
  16. data/lib/path_coverage/html_report.rb +30 -13
  17. data/lib/path_coverage/html_tabs.rb +26 -0
  18. data/lib/path_coverage/html_tree.rb +41 -20
  19. data/lib/path_coverage/json_reader.rb +94 -0
  20. data/lib/path_coverage/json_writer.rb +41 -9
  21. data/lib/path_coverage/kernel.rb +42 -20
  22. data/lib/path_coverage/kinds/base.rb +71 -0
  23. data/lib/path_coverage/kinds/block.rb +136 -0
  24. data/lib/path_coverage/kinds/case.rb +89 -0
  25. data/lib/path_coverage/kinds/chain.rb +45 -0
  26. data/lib/path_coverage/kinds/choice.rb +68 -0
  27. data/lib/path_coverage/kinds/contains.rb +111 -0
  28. data/lib/path_coverage/kinds/safe.rb +151 -0
  29. data/lib/path_coverage/kinds/write.rb +54 -0
  30. data/lib/path_coverage/kinds.rb +37 -0
  31. data/lib/path_coverage/manifest.rb +58 -19
  32. data/lib/path_coverage/manifest_blocks.rb +49 -10
  33. data/lib/path_coverage/manifest_chains.rb +76 -0
  34. data/lib/path_coverage/manifest_choices.rb +30 -17
  35. data/lib/path_coverage/manifest_folding.rb +91 -0
  36. data/lib/path_coverage/manifest_ignoring.rb +35 -0
  37. data/lib/path_coverage/manifest_writes.rb +67 -0
  38. data/lib/path_coverage/merge.rb +19 -3
  39. data/lib/path_coverage/post.rb +8 -8
  40. data/lib/path_coverage/recorder.rb +9 -49
  41. data/lib/path_coverage/recorder_blocks.rb +68 -0
  42. data/lib/path_coverage/recorder_cases.rb +23 -0
  43. data/lib/path_coverage/recorder_chains.rb +40 -0
  44. data/lib/path_coverage/recorder_writes.rb +26 -0
  45. data/lib/path_coverage/render.rb +58 -46
  46. data/lib/path_coverage/result.rb +19 -0
  47. data/lib/path_coverage/rewriter.rb +58 -77
  48. data/lib/path_coverage/rewriter_nested.rb +62 -0
  49. data/lib/path_coverage/site_paths.rb +9 -58
  50. data/lib/path_coverage/site_tree.rb +12 -57
  51. data/lib/path_coverage/summary.rb +32 -4
  52. data/lib/path_coverage/totals.rb +1 -1
  53. data/lib/path_coverage/version.rb +1 -1
  54. metadata +30 -6
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 7146cc322217826aadc2bd040b560b5e77d6fad7b6de4d2ae43e8bdfaa893940
4
- data.tar.gz: 29135bab12a1cc92d1a15dd8d383d3e79d75b3b8716fff7efa95fbbb6f8c3b34
3
+ metadata.gz: c68ba875b18ba000eb6be70f5c7bcb294ab4d50915868b913825c3f53bc60a93
4
+ data.tar.gz: 2beaba1e32cd4110ff9d04d3ab06e8c689395748566502a092175d8a339c9a5d
5
5
  SHA512:
6
- metadata.gz: 4ebf32a5b048eb5d3d1b1cbbca0e2ed9c217e429481d1bd77a242f0a3baff83ee7b83a2b4a4cd99f6a6611885d1c165a56b6bd6f814be28c4cb6a92cffbc8282
7
- data.tar.gz: 9ee6e1f2aea70b34ffb38ec5fa16a52f92ae5daf0a680727903db858fcdb557c244d0ae61a048e1a7bc09d99128c0b7068dd408686b2aca683b9f037afe3f6f3
6
+ metadata.gz: 6539c5ce97bd892ee616d0b1f780c79f18624381a404d3e6d2c60f6245957243b5ac7c395e130496fbbb2ba0ac6d69b4e9daf8e4fd235cb124d184aa3f8e5ad2
7
+ data.tar.gz: f72f62b796ad07d0290f56a88bc920ef1088b81abc04c6667ae1ee612e97499cc2f40ca6689acb1e8df35ebff607b463e8ec5ef271de11227bcf01017e98bb42
data/README.md CHANGED
@@ -7,58 +7,19 @@
7
7
  Path coverage for Ruby, to run alongside SimpleCov's line and branch
8
8
  coverage.
9
9
 
10
- Gem: https://rubygems.org/gems/path_coverage (which shows the latest
11
- version).
12
-
13
10
  Status: sites cover `&&`, `||`, `and`, `or`, the ternary `?:`,
14
11
  `if`/`elsif`/`else` and `unless` (modifiers included), `case`/`when`
15
12
  with a subject, and the block methods `any?` `all?` `none?` `find`
16
13
  `detect` `find_index` `take_while` `select` `filter` `reject`
17
- `partition` `count` `filter_map`. Method paths cover `if` decisions.
14
+ `partition` `count` `filter_map`, the writes `||=` and `&&=`, and `&.`
15
+ safe-navigation chains. Method paths cover `if` decisions.
18
16
  More decisions hidden inside calls come next (see Roadmap).
19
17
 
20
18
  Demo: https://cyber-dojo.github.io/path_coverage/ shows the HTML report
21
19
  for the mock source tree in `demo/`. `make demo` builds and opens the
22
20
  same report locally.
23
21
 
24
- Take this method:
25
-
26
- ```ruby
27
- def fee(user, order)
28
- total = 100
29
- if user.member? && order.paid?
30
- total -= 5
31
- end
32
- if order.big?
33
- total -= 10
34
- end
35
- total
36
- end
37
- ```
38
-
39
- Tests that run both `if`s both ways reach every line and every branch,
40
- so SimpleCov, which reads Ruby's built-in `Coverage`, reports 100%. Yet
41
- they can still miss paths, at two scopes, and path_coverage lists each
42
- path no test took.
43
-
44
- ## Within each decision
45
-
46
- The first `if` can go three ways: member and paid, member and not
47
- paid, not a member. A member who has not paid may never have been
48
- tried.
49
-
50
- ![The first if's grid: a column per way it can go, each headed by
51
- its count, the member who has not paid at 0](docs/decision.png)
52
-
53
- ## Across a method's decisions
54
-
55
- The two `if`s can go four ways together. A paying member with a small
56
- order may never have been tried.
57
-
58
- ![fee's grid: a column per way its two ifs can go together, each
59
- headed by its count, true then false at 0 and selected](docs/method.png)
60
-
61
- ## Use
22
+ ## Getting started
62
23
 
63
24
  ```ruby
64
25
  # test coverage config, before any application code is required
@@ -70,14 +31,101 @@ end
70
31
  PathCoverage.start do
71
32
  root APP_DIR # cover globs and site names are relative to it
72
33
  cover 'source/**/*.rb' # each matching file is instrumented as it loads
34
+ cover 'test/**/*.rb' # more than one cover glob may be given
35
+ skip 'source/vendor/**/*.rb' # except these, though a cover glob matches them
73
36
  coverage_dir '/tmp/coverage' # where path_coverage.json is written
74
37
  measure :decisions, :methods # :decisions alone unless :methods is named
75
- report :gaps, :html # the reporters run at exit (default :gaps)
38
+ report :gaps, :html # the reporters run at exit (default :gaps);
39
+ # also :summary and :post
40
+ group 'code', 'source/**/*.rb' # each group a tab of its own on index.html
41
+ group 'test', 'test/**/*.rb'
42
+ ignore '&&=' # operators whose sites are not measured
43
+ treat 'name', as: :present # a call to name is never nil
44
+ treat 'any?', as: :nonempty # any? is true when its receiver has items
76
45
  end
77
46
  ```
78
47
 
79
- A misspelt setting or an unknown reporter is refused by name, with the list
80
- of the ones there are. Calling `start` again replaces the configuration.
48
+
49
+ ## Reading the HTML
50
+
51
+ Clicking a line number on a file's HTML page opens that line's popout.
52
+
53
+ Each popout has two sections:
54
+ - on the left is a hierarchical representation of the code fragment.
55
+ - on the right is one column for each possible path through that fragment.
56
+ - at the top of the column is the count of that path. Look for zeros!
57
+ - the symbols in the column align with their fragment and are explained below.
58
+
59
+ ### Boolean: `T` `F`
60
+
61
+ ![A file's page: line 2, name.size > 2 && name.ascii_only?, its button pressed, its popout open beside it](docs/example-outcome.png)
62
+
63
+ - `T` the expression was true.
64
+ - `F` the expression was false.
65
+
66
+ In this example, there was no test which exercised the path where `name.size > 2` was `true`, and
67
+ `name.ascii_only?` was `false`.
68
+
69
+ ### Unreachable: blank
70
+
71
+ ![A file's page: line 2, @x = dee || dum, its button pressed, its popout open beside it](docs/example-unreachable.png)
72
+
73
+ - "blank" the expression is never evaluated.
74
+
75
+ In the second column `dee` was `T`, so because of `||`'s short-circuiting, `dum` is never evaluated.
76
+
77
+ ### Taken: `✓`
78
+
79
+ ![A file's page: line 2, if paid, its button pressed, its popout open beside it](docs/example-taken.png)
80
+
81
+ - `✓` marks the column whose path took that "arm"
82
+
83
+ The `paid` row is the `if`'s predicate, `T` or `F`. Below it is one row
84
+ per arm, named `then` and `else`. The first column, `paid` false, took `else`; the
85
+ second, `paid` true, took `then`.
86
+
87
+ ### Navigation: `∅` `●`
88
+
89
+ ![A file's page: line 2, user&.account&.owner, its button pressed, its popout open beside it](docs/example-nil.png)
90
+
91
+ - `∅` the receiver was nil, so `&.` skipped the rest of the chain.
92
+ - `●` the receiver was not nil, so `&.` made the call.
93
+
94
+ In this example, there was no test which exercised the path where
95
+ `user` was nil.
96
+
97
+ ### Count: `0` `1` `*`
98
+
99
+ ![A file's page: line 2, ids.any? { |id| id.odd? }, its button pressed, its popout open beside it](docs/example-count.png)
100
+
101
+ - `0` the block was called with no items.
102
+ - `1` the block was called with one item.
103
+ - `*` the block was called with two or more items.
104
+
105
+ In this example, there was no test which exercised the path where
106
+ the block was called with two or more items and every answer was
107
+ `false`.
108
+
109
+ ### Both: `TF`
110
+
111
+ ![A file's page: line 2, ids.select { |id| id.odd? }, its button pressed, its popout open beside it](docs/example-both.png)
112
+
113
+ - `T` every answer of the block was true.
114
+ - `F` every answer of the block was false.
115
+ - `TF` the block's answers were both true and false.
116
+
117
+ In this example, there was no test which exercised the path where
118
+ the block was called with two or more items and every answer was
119
+ `false`.
120
+
121
+ ### Lots of paths!
122
+
123
+ Ruby code can be very compact so short expressions can generate lots of paths!
124
+
125
+ ![A file's page: line 2, if user&.admin? && ids.select { |id| id.odd? }.any?, its button pressed, its popout open beside it](docs/example-overview.png)
126
+
127
+
128
+ ## Use when you cannot edit the code!
81
129
 
82
130
  A run whose code cannot be changed, such as a script in a `docker build`
83
131
  step, can be covered from its environment alone:
@@ -86,6 +134,7 @@ step, can be covered from its environment alone:
86
134
  RUBYOPT=-rpath_coverage/auto
87
135
  PATH_COVERAGE_ROOT=/app # root
88
136
  PATH_COVERAGE_COVER=src/**/*.rb,lib/*.rb # cover, comma-separated
137
+ PATH_COVERAGE_SKIP=src/vendor/**/*.rb # skip, comma-separated, optional
89
138
  PATH_COVERAGE_DIR=/tmp/paths # coverage_dir
90
139
  PATH_COVERAGE_COLLECTOR=http://host.docker.internal:4570/runs
91
140
  ```
@@ -122,13 +171,23 @@ As the process exits:
122
171
 
123
172
  - `path_coverage.json` is always written to `coverage_dir`: every site
124
173
  of every covered file, with each operand's true and false counts under
125
- `sites`, and each site's paths under `paths`. It is the base the other
126
- reporters read, so they can also be run later, in another process, from
174
+ `sites`, each site's paths under `paths`, each method's paths taken
175
+ under `method_paths` when `:methods` is measured, and the paths taken
176
+ of all possible under `totals`, as [taken, possible]: `decisions`,
177
+ `methods` when `:methods` is measured, and, with groups, each group's
178
+ own under `groups`, eg `"groups": {"code": {"decisions": [481, 567]}}`.
179
+ It also names the kinds of site measured under `kinds`, any `ignored`
180
+ operators and `treats`, and any paths taken that a treatment rules
181
+ out under `outside`.
182
+ It is the base the other reporters read, so they can also be run
183
+ later, in another process, from
127
184
  it alone: `PathCoverage::Render.from_json(config)`, where `config` is a
128
185
  `PathCoverage::Configuration` built from the same settings block.
129
186
  Given several runs' reports, `Render.from_json(config, paths)` adds
130
187
  their counts together first, so the report shows only the gaps that
131
- every run left.
188
+ every run left. A report whose kinds, `ignored` or `treats` differ
189
+ from `config`'s is refused, as its sites list their paths
190
+ differently.
132
191
  - `:gaps` prints one line per path never taken, in manifest order: a
133
192
  site's naming the outcome of each leaf on it, a method's the outcome
134
193
  of each decision on it, by line:
@@ -141,7 +200,8 @@ source/fee.rb:2:2 fee path never taken: line 4 true, line 7 false
141
200
 
142
201
  ### Decisions
143
202
 
144
- Measured always, as `:decisions`: the paths within each decision.
203
+ The paths within each decision are always measured, whatever `measure`
204
+ names.
145
205
 
146
206
  A site is named by the `path:line:column` where it starts, the path
147
207
  relative to `root`. A site in a covered file the run never loaded is
@@ -168,17 +228,8 @@ should read the JSON, whose `paths` give each site's leaves as
168
228
  ```
169
229
 
170
230
  A site is an outermost condition. Its leaves are numbered left to right
171
- across the whole of it, nested conditions included:
172
-
173
- ```ruby
174
- ok?(id) && new?(id) && id.size == 6 # one && site, leaves 0 1 2
175
- ok?(id) && (new?(id) || id.size == 6) # an || nested in an && site, 0 1 2
176
- ok?(id) && new?(id) || id.size == 6 # an && nested in an || site, 0 1 2
177
- ids.map { |k| k && k.ok? } && ok?(id) # two sites: the block's is its own
178
- ok?(id) ? :ok : :no # a ?: site, its predicate leaf 0
179
- ok?(id) && new?(id) ? :ok : :no # an && nested in a ?: site, 0 1
180
- ok?(id) ? new?(id) && id.size == 6 : :no # one ?: site, its then arm 1 2
181
- ```
231
+ across the whole of it, nested conditions included. A condition inside a
232
+ block is a site of its own.
182
233
 
183
234
  A ternary is a site even when its predicate is a single leaf. An arm
184
235
  holding a condition of `&&`, `||`, `and`, `or` or `?:` is part of the
@@ -201,16 +252,31 @@ whose `===` calls the pattern's `===` once, exactly as the `case` would,
201
252
  and notes whether it matched. A `case` with no subject, or with a splat
202
253
  among its patterns, is no site.
203
254
 
204
- A call to a block method, with no arguments and a block of one
205
- expression, is a site named at its method. Its two leaves are its
206
- receiver, showing how many items its block was called with (`0 items`,
207
- `1 item` or `2+ items`), and its block's expression: for `any?` `all?`
208
- `none?` `find` `detect` `find_index` and `take_while`, which stop once
209
- their block decides, its last answer; for `select` `filter` `reject`
210
- `partition` `count` and `filter_map`, which call it with every item,
211
- whether its answers were `all true`, `all false` or `both`. Both are
212
- noted from the block's own calls, so the receiver is never asked for its
213
- size, which could consume a sequence that can be read only once.
255
+ A call to `any?` `all?` `none?` `find` `detect` `find_index`
256
+ `take_while` `select` `filter` `reject` `partition` `count` or
257
+ `filter_map`, with no arguments and a block of one expression, is a site
258
+ named at its method. Its item count is noted from the block's own calls,
259
+ so the receiver is never asked its size, which could consume a sequence
260
+ that can be read only once.
261
+
262
+ A `||=` or `&&=` write is a site, whatever its target: a variable of any
263
+ kind, a constant, an attribute (`self.size ||= n`) or an index
264
+ (`store[key] ||= key.upcase`). Its one leaf is its target, and its right
265
+ side is an arm, run when the target is false for `||=`, true for `&&=`,
266
+ as an `unless` or an `if` runs its arm. The target is never read again,
267
+ since reading `store[key]` could call a method twice: its outcome follows
268
+ from whether the right side ran. So `@clock ||= Clock.now` called only
269
+ once, as a fresh object, leaves the path with `@clock` true a gap.
270
+
271
+ A `&.` chain is a site, named where it starts. Its leaves are the
272
+ receivers of its `&.` calls. `false` counts as not nil, as `&.` calls on
273
+ it.
274
+
275
+ A chain inside a condition, eg `ok?(id) && user&.admin?` or
276
+ `if user&.admin?`, folds into its site, as one expression: its value is
277
+ one more leaf, true or false, reached when every receiver is not nil. A
278
+ write through `&.`, eg `box&.size ||= 3`, is no site, as a nil `box`
279
+ skips it whole and its target is never looked at.
214
280
 
215
281
  A chain of one operator is flat; a change of operator, by precedence or by
216
282
  parentheses, nests. Parentheses are not part of the tree, since they decide
@@ -228,6 +294,18 @@ constants, since one can hold nil or false (`VERBOSE = false`). `nil` and
228
294
  @clock = options[:clock] || MonotonicClock.new # leaf 1: never evaluated, or no gap
229
295
  ```
230
296
 
297
+ Any other leaf, a call among them, is taken to have either outcome, as its
298
+ form alone cannot say which it returns. So a call that never returns nil
299
+ or false still leaves a path needing it falsy, which no test can take:
300
+
301
+ ```ruby
302
+ methods: paths && methods_of(path, methods, paths) # [true, false] impossible
303
+ methods: paths ? methods_of(path, methods, paths) : nil # the same, no such path
304
+ ```
305
+
306
+ Written as a choice, the same code has no such path. Reporting a gap that
307
+ cannot be closed is the lesser error than hiding one that can.
308
+
231
309
  ### Methods
232
310
 
233
311
  With `measure :decisions, :methods`, each method with two or more
@@ -239,11 +317,24 @@ site's columns stay its own leaves' combinations, and a method's its own
239
317
  decisions', rather than multiplying together. A decision whose arm always
240
318
  leaves the method ends the path there.
241
319
 
320
+ The JSON report's `method_paths` lists each method's paths taken, with
321
+ their counts, so a later render, or several runs' reports merged, still
322
+ shows them. A report with none, eg from a run that did not measure
323
+ `:methods`, renders as no method path taken:
324
+
325
+ ```json
326
+ "method_paths": {
327
+ "source/fee.rb:2:2": [
328
+ { "path": [true, false], "count": 1 }
329
+ ]
330
+ }
331
+ ```
332
+
242
333
  ### Reporters
243
334
 
244
335
  - `:html` writes `index.html` to `coverage_dir`: the covered files with
245
336
  their sites and their paths taken of all possible, within each decision
246
- and, when `:methods` is measured, across a method's decisions (eg
337
+ and, when `:methods` is measured, within each method (eg
247
338
  `9 / 28 (32%)`, or `-` for a file with no method of two or more
248
339
  decisions), each column sortable, and a Total row for every file
249
340
  together. Each file name opens `files/<path>.html`, the file's
@@ -268,6 +359,25 @@ leaves the method ends the path there.
268
359
  goes through an `http_proxy`, and gives up in silence on a collector
269
360
  that cannot be reached or does not answer within a second, so the
270
361
  run's output and exit status are unchanged.
362
+ - `:summary` prints each total as SimpleCov prints its line and branch
363
+ coverage, the methods line only when `:methods` is measured:
364
+
365
+ ```text
366
+ Path coverage within decisions: 527 / 631 (83.52%)
367
+ Path coverage within methods: 44 / 52 (84.62%)
368
+ ```
369
+
370
+ With groups, each group is a heading, its own totals indented under it,
371
+ and there is no total over every group.
372
+
373
+ ```text
374
+ code:
375
+ Path coverage within decisions: 481 / 567 (84.83%)
376
+ Path coverage within methods: 41 / 49 (83.67%)
377
+ test:
378
+ Path coverage within decisions: 65 / 84 (77.38%)
379
+ Path coverage within methods: 3 / 3 (100.00%)
380
+ ```
271
381
 
272
382
  ## How it works
273
383
 
@@ -310,19 +420,65 @@ rewrite happens in memory; no file on disk changes.
310
420
  At exit the manifests parsed during the run are reused; only covered files
311
421
  the run never loaded are read and parsed then.
312
422
 
423
+ ## Ignoring and treating
424
+
425
+ `ignore '&&=', '||='` stops `&&=` and `||=` writes being measured.
426
+ A site inside such a write is still measured, eg the `a && b` in
427
+ `@m ||= a && b`. Ignoring `&&` does not ignore `and`, nor does ignoring
428
+ `||` ignore `or`. The summary and `index.html` list what is ignored.
429
+
430
+ `treat` tells path_coverage what you know about a method, so paths that
431
+ cannot happen are not listed as gaps. Add `in: 'source/models/**'` to
432
+ limit a treatment to the files that glob matches.
433
+
434
+ - `treat 'name', as: :present`: a call to `name` is never nil, so
435
+ `user.name&.size` has no path with `user.name` nil.
436
+ - `treat 'any?', as: :nonempty`: `x.any?` is true exactly when `x` has
437
+ items. When `x` is a call to `select`, `reject`, `filter`,
438
+ `filter_map`, `partition` or `take_while`, the path that call took
439
+ often decides whether it kept any items. Paths where `any?` disagrees
440
+ with that are not listed, eg `ids.select { |i| i.odd? }.any?` true
441
+ with `ids` empty.
442
+
443
+ A run where a treatment turns out to be wrong is reported:
444
+
445
+ ```text
446
+ source/user.rb:4:9 path taken that a treatment rules out: `user.name` nil
447
+ ```
448
+
313
449
  ## Roadmap
314
450
 
315
451
  - decisions hidden inside calls, each with its own outcome classes:
316
452
  - `one?`, whose last answer does not decide it, and block methods
317
453
  with blocks of more than one expression
318
454
  - `fetch(k, default)`, `dig`, `Hash#[]` with a default: present vs default
319
- - `x || default` in value context, `&.` safe navigation: nil vs not nil
320
- - `||=` and `&&=`, on variables, attributes and index targets: the
321
- right side evaluated vs skipped, counted without evaluating the
322
- target twice
455
+ - `x || default` in value context: nil vs not nil
323
456
  - `x rescue y` modifier: raised vs not raised
324
457
  - `case` with no subject, and `case`/`in` patterns
325
458
  - regex matches: nil vs MatchData
459
+ - `treat 'methods_of', as: :truthy`, so each call to it counts as a
460
+ leaf that is only truthy, as `Foo.new` is, and a run where it returns
461
+ nil or false is reported. Later, treatments could come from the
462
+ project's RBS signatures (`sig/*.rbs`) too.
463
+ - exceptions the code itself names: a `begin` or `def` with `rescue`
464
+ clauses as a site, like a `case`, whose paths are no raise and each
465
+ clause that rescued, eg `rescue IOError` never taken. Branch coverage
466
+ ignores `rescue`, and tests reach these paths with real input. A raise
467
+ that nothing names, from any call, stays out, as every leaf would gain
468
+ a third outcome that only fault injection could take.
469
+ - an evaluation that raised part way through a site, eg
470
+ `ok?(id) && fetch(id)` where `fetch` raises, reported as a path that
471
+ raised rather than dropped as unfinished.
472
+ - complexity maximums, reported and gated, since the possible paths
473
+ count how much there is to test, not only how much was tested. A
474
+ file's total measures its size, so each is a maximum over smaller
475
+ parts:
476
+ - paths in one decision, eg 6 for a `select` block
477
+ - paths through one method's decisions, eg 2 top-level ifs give 4
478
+ - paths within each decision, summed over every decision in one
479
+ method, nested ones included, eg decisions of 2, 3 and 15 paths
480
+ sum to 20. Every method has one, and it grows as the method gains
481
+ conditions, where the paths through a method multiply.
326
482
 
327
483
  ## Constraints
328
484
 
@@ -336,13 +492,48 @@ the run never loaded are read and parsed then.
336
492
  `make all` runs, in a Docker image built from the repo's `Dockerfile`:
337
493
 
338
494
  - `make lint`: rubocop over `lib/`, `test/` and `bin/` alike
339
- - `make test`: the tests, then a check that the coverage report has zero
340
- missed lines and branches in both the code and the tests, within the
341
- bounds in `test/coverage_metrics_params.json`
495
+ - `make test`: the tests, with path_coverage measuring its own `lib/` and
496
+ `test/`, then a check that SimpleCov's report has zero missed lines and
497
+ branches in both the code and the tests, and that every path within the
498
+ tests' methods was taken, within the bounds in
499
+ `test/coverage_metrics_params.json` (a path bound is eg
500
+ `test.paths.methods.missed`, and a missing path report denies it)
342
501
  (`make test tids='6D1K'` runs only matching tests, and skips the check)
343
502
  - `make test_check`: shunit2 tests showing that a breached bound fails the
344
503
  check with a non-zero exit
345
504
 
505
+ Each run prints its seed; `SEED=n make test` replays that run's order of
506
+ files and tests. When the coverage check fails, it names each line and
507
+ branch missed.
508
+
509
+ A run ends with each group, code and test, as a heading over its four
510
+ figures, SimpleCov's and path_coverage's together, then where each report
511
+ is, each with a code and a test tab, then the check's verdict:
512
+
513
+ ```text
514
+ code:
515
+ Line coverage: 1270 / 1270 (100.00%)
516
+ Branch coverage: 321 / 321 (100.00%)
517
+ Path coverage within decisions: 481 / 567 (84.83%)
518
+ Path coverage within methods: 41 / 49 (83.67%)
519
+ test:
520
+ Line coverage: 2250 / 2250 (100.00%)
521
+ Branch coverage: 43 / 43 (100.00%)
522
+ Path coverage within decisions: 65 / 84 (77.38%)
523
+ Path coverage within methods: 3 / 3 (100.00%)
524
+ Coverage report: .../reports/index.html
525
+ Path coverage report: .../reports/paths/index.html
526
+ coverage: ALLOWED
527
+ ```
528
+
529
+ `bin/check_coverage_metrics.rb` prints the groups, reading SimpleCov's
530
+ figures from `reports/coverage_metrics.json` and path_coverage's from the
531
+ `totals` in `reports/paths/path_coverage.json`; SimpleCov's own console
532
+ lines, totals over every group, are silenced. The tool measuring paths is
533
+ a copy of `lib/` renamed `PathCoverageSelf` (`test/self_coverage.rb`), so
534
+ the tests start and restart `PathCoverage` as always. SimpleCov's figures
535
+ for `lib/` are the same with it as without.
536
+
346
537
  `make demo` (not part of `make all`) runs `demo/run.rb` through
347
538
  `bin/build_demo.sh`, writing the report to `site/` and opening it. The
348
539
  Pages workflow (`.github/workflows/pages.yml`) runs the same script on
@@ -351,3 +542,21 @@ each push to `main` and publishes `site/`.
351
542
  `bin/capture_readme_images.sh` rebuilds the demo and saves this README's
352
543
  two screenshots, `docs/decision.png` and `docs/method.png`, from it with
353
544
  headless Chrome. Run it again whenever the report's look changes.
545
+
546
+
547
+ ## Within each decision
548
+
549
+ The first `if` can go three ways: member and paid, member and not
550
+ paid, not a member. A member who has not paid may never have been
551
+ tried.
552
+
553
+ ![The first if's grid: a column per way it can go, each headed by
554
+ its count, the member who has not paid at 0](docs/decision.png)
555
+
556
+ ## Within each method
557
+
558
+ The two `if`s can go four ways together. A paying member with a small
559
+ order may never have been tried.
560
+
561
+ ![fee's grid: a column per way its two ifs can go together, each
562
+ headed by its count, true then false at 0 and selected](docs/method.png)
@@ -26,30 +26,69 @@ module PathCoverage
26
26
  @recorder.when_match(site, index, size, pattern)
27
27
  end
28
28
 
29
+ # The call rewritten code makes around the pattern of a case
30
+ # nested in site, the site's index-th leaf; returns the
31
+ # pattern's stand-in.
32
+ def self.pattern(site, index, pattern)
33
+ @recorder.pattern(site, index, pattern)
34
+ end
35
+
36
+ # The call rewritten code makes around the right side of a
37
+ # write nested in site, its target the site's at-th leaf,
38
+ # outcome being the target's when it runs; returns value
39
+ # unchanged.
40
+ def self.written_at(site, at, outcome, value)
41
+ @recorder.written_at(site, at, outcome, value)
42
+ end
43
+
44
+ # The call rewritten code makes around a write nested in site,
45
+ # its target the site's at-th leaf, skipped being the target's
46
+ # outcome when its right side never ran; returns value
47
+ # unchanged.
48
+ def self.settle(site, at, skipped, value)
49
+ @recorder.settle(site, at, skipped, value)
50
+ end
51
+
29
52
  # The call rewritten code makes before a block method's call
30
- # at site runs; returns the evaluation's trail.
31
- def self.enter_items(site)
32
- @recorder.enter_items(site)
53
+ # nested in site runs, its receiver the site's at-th leaf;
54
+ # returns the site's trail.
55
+ def self.begin_items(site, at)
56
+ @recorder.begin_items(site, at)
33
57
  end
34
58
 
35
- # The call rewritten code makes around a block method's
36
- # block's expression at site; returns value unchanged.
37
- def self.item(site, value)
38
- @recorder.item(site, value)
59
+ # The call rewritten code makes around the block's expression
60
+ # of a stop-early block method nested in site, its receiver the
61
+ # site's at-th leaf; returns value unchanged.
62
+ def self.item_at(site, at, value)
63
+ @recorder.item_at(site, at, value)
39
64
  end
40
65
 
41
66
  # The call rewritten code makes around the block's expression
42
- # of a block method that calls it with every item; returns
43
- # value unchanged.
44
- def self.answer(site, value)
45
- @recorder.answer(site, value)
67
+ # of an every-item block method nested in site, its receiver
68
+ # the site's at-th leaf; returns value unchanged.
69
+ def self.answer_at(site, at, value)
70
+ @recorder.answer_at(site, at, value)
46
71
  end
47
72
 
48
73
  # The call rewritten code makes around a block method's call
49
- # as a whole, with the trail enter_items gave it; returns
50
- # value unchanged.
51
- def self.leave_items(trail, value)
52
- @recorder.leave_items(trail, value)
74
+ # nested in site, its receiver the site's at-th leaf, with the
75
+ # trail begin_items gave it; returns value unchanged.
76
+ def self.end_items(site, at, trail, value)
77
+ @recorder.end_items(site, at, trail, value)
78
+ end
79
+
80
+ # The call rewritten code makes around a &. receiver, the
81
+ # index-th leaf of site, its chain's leaves starting at first;
82
+ # returns value unchanged.
83
+ def self.present(site, index, first, value)
84
+ @recorder.present(site, index, first, value)
85
+ end
86
+
87
+ # The call rewritten code makes around a &. chain's value, the
88
+ # index-th leaf of site, its chain's leaves starting at first;
89
+ # returns value unchanged.
90
+ def self.chained(site, index, first, value)
91
+ @recorder.chained(site, index, first, value)
53
92
  end
54
93
 
55
94
  # The call rewritten code makes as a call reaches its
@@ -6,9 +6,28 @@ module PathCoverage
6
6
  # decided by their predicate alone.
7
7
  STATEMENT_CHOICES = %w[if unless].freeze
8
8
 
9
+ # The operator of a safe-navigation chain, eg user&.account.
10
+ SAFE = '&.'.freeze
11
+
12
+ # The operator of a leaf with sites inside it, eg
13
+ # ids.select { |i| i.odd? }.any? as an operand of an &&: its
14
+ # operands are those sites' Conditions, then the leaf's own byte
15
+ # range, whose outcome is the leaf's value.
16
+ CONTAINS = 'contains'.freeze
17
+
18
+ # A &. receiver's outcome: not nil, so the call is made, false
19
+ # included, or nil, so the rest of the chain is skipped.
20
+ PRESENCE = ['not nil', 'nil'].freeze
21
+
22
+ # The writes whose target decides whether their right side
23
+ # runs, as an unless or an if decides its arm: x ||= v runs v
24
+ # when x is false, x &&= v when x is true.
25
+ WRITES = %w[||= &&=].freeze
26
+
9
27
  # The predicate outcome on which each choice takes its then
10
- # arm: an unless's is false.
11
- THEN_ON = Hash.new(true).merge('unless' => false).freeze
28
+ # arm: an unless's is false, as is a ||= write's.
29
+ THEN_ON = Hash.new(true).merge('unless' => false, '||=' => false)
30
+ .freeze
12
31
 
13
32
  # The methods that call their block an item at a time until
14
33
  # it decides, its last answer deciding the result.
@@ -47,6 +66,10 @@ module PathCoverage
47
66
  [ITEMS[2], ANSWERS[2]]
48
67
  ].freeze
49
68
 
69
+ # The block methods whose value follows from their path alone:
70
+ # all but find and detect, whose found item may itself be nil.
71
+ VALUED = [*STOP_EARLY - %w[find detect], *EVERY_ITEM].freeze
72
+
50
73
  # Every block method whose call is a site, by its paths.
51
74
  BLOCK_PATHS = [
52
75
  *STOP_EARLY.map { |name| [name, STOP_EARLY_PATHS] },