padded_cell 0.1.0-aarch64-linux-gnu → 0.2.0-aarch64-linux-gnu

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 9e61ac6aa1468930e0eb7feb94901aba611f11f2eeda222beef54917c37bb4b3
4
- data.tar.gz: 5046cf840a35e01dca0c36e37772129cfe1454e6805f05119f3476b67f8bc450
3
+ metadata.gz: a7198972d01002adff236f510d794f334d872cc484287b327f1ecee9d63045da
4
+ data.tar.gz: eacd77c69b26d4d3c60ea5ecb87ff41a33d6eba511c665309c58753362eeacfc
5
5
  SHA512:
6
- metadata.gz: ce4abe5735a7bb637f871a7dd47b0712ac6f49c3700940add46374c8ded101ef85309f7fde9fc3876304404b31b3de05db680cd5cd55b43049e10845125e98b9
7
- data.tar.gz: 420514f129aeef7f48be23ba4204710c6b4975682308c12506732a7fa7fb72875d25f169cbe42a759988d18c26fd0a3e8052e7b744574cd883aa3c25402a5686
6
+ metadata.gz: 801d616e45e9039dc9a49671b730e1e2f666d527e50c9da79137e926755ae711d8fa1efd729285719d4ae0fd712b36f6724472d1d428f11b70a46eaf631554b5
7
+ data.tar.gz: 50464d7fa9738cf3f6b6312c9bc2537af5a61be58204cfd8477802f2f36704548473ad16f33c2aa9d163aabb5677a4121bed5e82ab95963995c47f3e2c30c20a
data/CHANGELOG.md CHANGED
@@ -1,5 +1,30 @@
1
1
  # Changelog
2
2
 
3
+ ## 0.2.0 (2026-10-11)
4
+
5
+ - Built-in coverage: `--coverage[=lines,branches,methods,eval]` (or
6
+ `PADDED_CELL_COVERAGE`) measures the runner, every worker and every cell,
7
+ and sums them exactly. Configured with `PaddedCell.coverage` in the test
8
+ helper: tracked files, filters, groups, the `:rails` profile. Honors
9
+ `# :nocov:` blocks and `# simplecov:disable` comments. Writes
10
+ `coverage.md` (the code that never ran, for agents, reviews and CI
11
+ summaries), `lcov.info`, and `coverage.json` / `.resultset.json`.
12
+ `minimum`, `minimum_by_file`, `minimum_by_group` and `maximum_drop` fail a
13
+ whole passing run with exit 3. `--coverage-merge` merges shards' reports.
14
+ - `--seed` must be 0 to 2**64 - 1; others crashed the planner.
15
+ - `--asylum[=N]`: retry failed tests (N times, default 2) after the run,
16
+ alone and in fresh cells. Tests that pass on retry count as passed and are
17
+ reported as flaky, with the likely cause (timing and shared state, or
18
+ randomness and order) and their first failure; JUnit gets
19
+ `<flakyFailure>`, JSON a `flaky` field.
20
+ - A `cleanup` hook that raises fails the run (exit 1), `--asylum` retries
21
+ included. It exited 0 when the tests passed.
22
+ - A passing run filtered with `--name` exits 0: it exited 1, reporting that
23
+ nothing ran.
24
+ - `--list` applies `--name` and `--exclude`, before `--shard`, as a run does.
25
+ - An invalid `--name` or `--exclude` regexp is a usage error (exit 2), not a
26
+ crash.
27
+
3
28
  ## 0.1.0 (2026-10-11)
4
29
 
5
30
  First release, for CRuby 4.0+ and Minitest 6 on Linux, macOS and FreeBSD: a
data/README.md CHANGED
@@ -70,13 +70,15 @@ $ padded_cell -n /refund/ -e test_slow # Minitest's --include / --exclude
70
70
  $ padded_cell -j 8 --seed 1234 # 8 workers, reproducible order
71
71
  $ padded_cell --timeout 30 # kill and fail anything running 30s
72
72
  $ padded_cell --fail-fast
73
+ $ padded_cell --asylum # retry failures; report the flaky ones (see below)
73
74
  $ padded_cell --list # what would run, without loading anything
74
75
  $ padded_cell --shard 2/4 # CI: the second quarter of the classes
75
76
  $ padded_cell --junit tmp/junit.xml --json tmp/results.jsonl
77
+ $ padded_cell --coverage # line and branch coverage (see Coverage)
76
78
  ```
77
79
 
78
80
  Exit status: 0 when everything passed, 1 otherwise, 2 for usage or load
79
- errors, 130 when interrupted.
81
+ errors, 3 when only coverage is below a threshold, 130 when interrupted.
80
82
 
81
83
  ### How a run works
82
84
 
@@ -113,6 +115,38 @@ discards Active Record's connection pool, and every test would reload its
113
115
  fixtures, so those suites default to `worker`. The `Run options:` line shows
114
116
  which one a run uses; `--isolation` overrides it.
115
117
 
118
+ ### Flaky tests
119
+
120
+ `--asylum[=N]` gives failed tests up to N more attempts (default 2) once the
121
+ run is over: one test at a time, each in a fresh cell. A test that passes on
122
+ retry counts as passed, but padded_cell shows every attempt, its seed, and
123
+ what made the difference:
124
+
125
+ ```console
126
+ 4 runs, 4 assertions, 1 failures, 0 errors, 0 skips
127
+
128
+ You're not crazy, the world is. 2 tests failed, then passed:
129
+
130
+ FlakyTest#test_passes_when_retried
131
+ seed 1 lost the race
132
+ seed 1 passed, alone → timing, contention or state left by other tests
133
+
134
+ FlakyTest#test_depends_on_the_seed
135
+ seed 1 unlucky seed.
136
+ seed 1 unlucky seed. (alone)
137
+ seed 15472 passed, alone → randomness or test order
138
+
139
+ It's not the world, it's the test:
140
+ FlakyTest#test_always_fails failed all 3 attempts
141
+ ```
142
+
143
+ The first retry keeps the run's seed and runs alone, so passing there points
144
+ at timing, contention or leftover state; later retries change the seed, so
145
+ passing only there points at randomness or order. A test that fails every
146
+ attempt keeps its first failure, and the run fails. JUnit reports carry a
147
+ `<flakyFailure>` per flaky test; JSON lines carry the attempts in a `flaky`
148
+ field, or an `asylum` field for a test that never passed.
149
+
116
150
  ### Hooks
117
151
 
118
152
  For anything that must not be shared across forks:
@@ -125,7 +159,115 @@ PaddedCell.cleanup { |worker| Database.drop("app_test_#{worker}") }
125
159
  ```
126
160
 
127
161
  `before_fork` runs in the runner, `after_fork` and `cleanup` once per worker
128
- (`0...jobs`). Cells run no hooks: they inherit their worker's state.
162
+ (`0...jobs`). Cells run no hooks: they inherit their worker's state. A
163
+ failing `after_fork` fails every test of its worker; a failing `cleanup`
164
+ fails the run, even when every test passed.
165
+
166
+ ## Coverage
167
+
168
+ Ruby's `Coverage`, measured in the runner, in every worker and in every
169
+ cell, and summed exactly: a parallel run reports the counts of a serial one.
170
+
171
+ ```console
172
+ $ padded_cell --coverage test/ # lines and branches
173
+ $ padded_cell --coverage=lines,branches,methods test/
174
+ $ PADDED_CELL_COVERAGE=1 bin/rails padded_cell
175
+ ```
176
+
177
+ Measuring starts before the suite loads, so what is measured (`lines`,
178
+ `branches`, `methods`, and `eval` for templates, experimental) comes from the
179
+ command line. The report is configured in the test helper:
180
+
181
+ ```ruby
182
+ # test/test_helper.rb
183
+ PaddedCell.coverage(:rails) do
184
+ track "{app,lib}/**/*.rb" # report files never loaded, at 0%
185
+ skip "app/legacy/" # also `filter`
186
+ group "Operations", "app/concepts" # string, regexp, array or block
187
+ group("Contracts") { |file| file.filename.include?("contract") }
188
+ minimum line: 90, branch: 80
189
+ minimum_by_file line: 50
190
+ minimum_by_group "Operations", line: 95
191
+ maximum_drop line: 1 # since the previous whole run
192
+ formats :markdown, :lcov, :json
193
+ dir "coverage" # --coverage-dir wins
194
+ end
195
+ ```
196
+
197
+ - Paths are matched relative to the project root. A string matches whole
198
+ path segments (`"app/models"` takes `app/models/user.rb`, not
199
+ `app/models_old/`), `"lib/"` a directory anywhere, `"contract.rb"` a file
200
+ name ending; a regexp matches the path; a block gets the file
201
+ (`filename`, `project_filename`, `src`).
202
+ - Left out by default: `vendor/bundle`, hidden files, `test/`, `spec/`,
203
+ `features/`. `clear_filters` drops these.
204
+ - `:rails` skips `config/` and `db/`, groups `app/channels`,
205
+ `app/controllers`, `app/helpers`, `app/jobs` and `app/workers`, `lib/`,
206
+ `app/mailers`, `app/models` and `app/views`, and tracks
207
+ `{app,lib}/**/*.rb`. A file in no group is Ungrouped.
208
+ - Thresholds compare percents floored to 2 decimals.
209
+
210
+ `PaddedCell.coverage` configures only; without `--coverage` nothing is
211
+ measured. Outside padded_cell, guard it with `if defined?(PaddedCell.coverage)`.
212
+
213
+ Into `coverage/` go, by format:
214
+
215
+ - `markdown`: `coverage.md`, for agents, reviews and CI summaries
216
+ (`cat coverage/coverage.md >> $GITHUB_STEP_SUMMARY`): the totals and
217
+ groups, then every file with code that never ran, the most missed first,
218
+ with its missed lines, branches and methods, and that code. It stops
219
+ listing files at 1 MB, the size of a GitHub job summary.
220
+ - `lcov`: `lcov.info` (`SF:./path`), for CI and editors.
221
+ - `json`: `coverage.json` and `.resultset.json`, in SimpleCov's formats, for
222
+ the tools reading them.
223
+
224
+ Also, every run: `.padded_cell.json` (what the report is made of, and what
225
+ `--coverage-merge` reads), and after a whole passing run `.last_run.json`.
226
+
227
+ Lines and branches inside `# :nocov:` blocks, and under
228
+ `# simplecov:disable [line|branch|method]` / `# simplecov:enable` comments,
229
+ are skipped. A directive after code skips its own line.
230
+
231
+ A run below a threshold exits 3. Thresholds hold for a whole, passing,
232
+ complete run only: they are not checked when tests failed, when tests were
233
+ picked by name or line, under `--shard`, or when a worker or cell died
234
+ before handing over what it measured (the report then says it is partial).
235
+ Reports are written either way, failing runs included.
236
+
237
+ A suite that starts SimpleCov as well is refused (exit 2): two tools cannot
238
+ share Ruby's `Coverage`.
239
+
240
+ ### Shards
241
+
242
+ Each shard reports on its own; merge them, at the same commit, without
243
+ running anything:
244
+
245
+ ```console
246
+ $ padded_cell --coverage --coverage-dir coverage/1 --shard 1/2 test/ # machine 1
247
+ $ padded_cell --coverage --coverage-dir coverage/2 --shard 2/2 test/ # machine 2
248
+ $ padded_cell --coverage-merge coverage/1,coverage/2 --coverage-dir coverage
249
+ ```
250
+
251
+ The merge refuses files that differ between shards or from the checkout,
252
+ and checks thresholds when every shard of the run is there. Lines that run
253
+ while the suite loads count once per shard.
254
+
255
+ ### Cost
256
+
257
+ Measuring is opt-in: Ruby's instrumentation slows tests down. On a Rails
258
+ suite of 6,300 tests, lines and branches took the run from 465 s to 638 s of
259
+ CPU.
260
+
261
+ A worker hands over what it measured every 10 seconds or more: each time,
262
+ Ruby copies the coverage of every file it loaded, gems included (a few
263
+ milliseconds in a small app, a few hundred in a big Rails one), so the
264
+ interval stretches to keep this under 1% of the worker's time. A worker that
265
+ dies loses what it measured since. With `--isolation test` every cell pays
266
+ one such copy, at its end.
267
+
268
+ Files never loaded report their branches in the order Ruby compiles them.
269
+ With `eval`, code that `class_eval` runs from a heredoc counts as lines of
270
+ its file; directives in templates are not read.
129
271
 
130
272
  ## Rails
131
273
 
@@ -144,8 +286,7 @@ $ bin/rails padded_cell ARGS="-j 8 test/models"
144
286
  Rails suites with fixtures run with `--isolation worker` unless told
145
287
  otherwise (see [Isolation](#isolation)).
146
288
 
147
- Coverage tools that write their results from an `at_exit` hook (SimpleCov)
148
- see only the runner, as with Rails' process parallelization.
289
+ For coverage, see [Coverage](#coverage).
149
290
 
150
291
  ## Library
151
292
 
@@ -173,9 +314,9 @@ Needs Rust 1.99+ (`mise install` reads `mise.toml`).
173
314
 
174
315
  ```console
175
316
  bundle install
176
- rake test:all # cargo test, then the Ruby suite (compiles the extension first)
177
- rake test # the Ruby suite only
178
- rake dogfood # this suite, run by padded_cell
317
+ rake test:all # cargo test, then this suite run by padded_cell itself
318
+ rake dogfood # this suite, run by padded_cell (compiles the extension first)
319
+ rake test # this suite on plain Minitest, for when padded_cell itself is broken
179
320
  rake lint # dictator, cargo fmt --check, clippy
180
321
  ```
181
322
 
@@ -0,0 +1,91 @@
1
+ # frozen_string_literal: true
2
+
3
+ module PaddedCell
4
+ # --asylum: tests that failed get more attempts once the run is over, one
5
+ # at a time, each in a fresh cell. A test that passes on retry counts as
6
+ # passed but is reported as flaky, with every attempt and what changed:
7
+ #
8
+ # attempt 1, same seed, run alone: timing, contention, or state left by
9
+ # the tests that ran around it
10
+ # attempt 2+, another seed: randomness or test order
11
+ #
12
+ # A test that fails every attempt keeps its first failure.
13
+ class Asylum
14
+ ALONE = 'timing, contention or state left by other tests'
15
+ RESEEDED = 'randomness or test order'
16
+
17
+ # The seed of retry attempt, derived from the run's seed so a report can
18
+ # be replayed. Attempt 1 keeps the run's seed.
19
+ def self.retry_seed(seed, attempt)
20
+ attempt == 1 ? seed : (seed + (attempt * 0x9e37)) % 0xffff
21
+ end
22
+
23
+ def initialize(tests, attempts:, seed:, timeout: nil)
24
+ @tests = tests
25
+ @attempts = attempts
26
+ @seed = seed
27
+ @timeout = timeout
28
+ @success = true
29
+ end
30
+
31
+ # False when a retry's worker failed (its cleanup hook).
32
+ def success? = @success
33
+
34
+ # failures: {test index => first failing result}. Yields (index, result)
35
+ # once per test, with its final result. Its metadata holds the attempts:
36
+ # :flaky when it passed on retry, :asylum when it never did. Both are
37
+ # plain data, for Marshal and the JUnit and JSON reports.
38
+ def call(failures)
39
+ history = failures.to_h { |index, result| [index, [attempt_entry(@seed, false, result)]] }
40
+ pending = failures.keys
41
+ 1.upto(@attempts) do |attempt|
42
+ break if pending.empty?
43
+
44
+ pending -= retry_round(pending, history, attempt) do |index, result|
45
+ result.metadata[:flaky] = flaky(failures[index], history[index], attempt)
46
+ yield index, result
47
+ end
48
+ end
49
+ pending.each do |index|
50
+ failures[index].metadata[:asylum] = { attempts: history[index].size, history: history[index] }
51
+ yield index, failures[index]
52
+ end
53
+ end
54
+
55
+ private
56
+
57
+ # Runs the pending tests once more, alone; returns the ones that passed.
58
+ def retry_round(pending, history, attempt)
59
+ seed = self.class.retry_seed(@seed, attempt)
60
+ passed = []
61
+ pool = Pool.new(@tests, jobs: 1, isolation: :test, seed:, timeout: @timeout)
62
+ pool.run([pending.dup]) do |index, result|
63
+ recovered = result.passed? && !result.skipped?
64
+ history[index] << attempt_entry(seed, true, recovered ? nil : result)
65
+ next unless recovered
66
+
67
+ passed << index
68
+ yield index, result
69
+ end
70
+ @success &&= pool.success?
71
+ passed
72
+ end
73
+
74
+ # One attempt: its seed, whether it ran alone, and how it ended.
75
+ def attempt_entry(seed, alone, failed)
76
+ { seed:, alone:, outcome: failed ? summary(failed.failure)[:message] : 'passed' }
77
+ end
78
+
79
+ def flaky(first, history, attempt)
80
+ {
81
+ attempts: history.size, seed: history.last[:seed], cause: attempt == 1 ? ALONE : RESEEDED,
82
+ first_failure: summary(first.failure), history:
83
+ }
84
+ end
85
+
86
+ def summary(failure)
87
+ type = failure.is_a?(Minitest::UnexpectedError) ? failure.error.class.name : failure.class.name
88
+ { type:, message: failure.message.lines.first.to_s.strip }
89
+ end
90
+ end
91
+ end
@@ -12,15 +12,18 @@ module PaddedCell
12
12
  # result when the cell died or ran past timeout seconds.
13
13
  #
14
14
  # close: descriptors the cell must not keep open (the worker's socket)
15
- def run(klass, name, seed:, timeout: nil, close: [])
15
+ # probe: the worker's Chart::Probe, which gets the cell's coverage
16
+ def run(klass, name, seed:, timeout: nil, close: [], probe: nil)
16
17
  # Binary: the result is Marshal data, and with Encoding.default_internal
17
18
  # set (Rails sets it) a text-mode pipe transcodes what is written to it.
18
19
  reader, writer = IO.pipe
19
20
  [reader, writer].each(&:binmode)
20
21
  started = Minitest.clock_time
21
- pid = fork_cell(klass, name, seed, reader, writer, close)
22
+ pid = fork_cell(klass, name, seed, reader, writer, close, probe)
22
23
  writer.close
23
- collect(pid, reader, klass, name, started, timeout)
24
+ delta, result = collect(pid, reader, klass, name, started, timeout)
25
+ delta ? probe.absorb(delta) : probe&.lost
26
+ result
24
27
  ensure
25
28
  reader&.close
26
29
  writer&.close unless writer&.closed?
@@ -34,7 +37,7 @@ module PaddedCell
34
37
  Results.crashed(klass, name, "exit(#{e.status}) called during the test")
35
38
  end
36
39
 
37
- def fork_cell(klass, name, seed, reader, writer, close)
40
+ def fork_cell(klass, name, seed, reader, writer, close, probe)
38
41
  $stdout.flush
39
42
  $stderr.flush
40
43
  Process.fork do
@@ -43,7 +46,9 @@ module PaddedCell
43
46
  reader.close
44
47
  close.each(&:close)
45
48
  srand(seed)
46
- writer.write(Results.dump(execute(klass, name)))
49
+ probe&.start
50
+ result = execute(klass, name)
51
+ writer.write(Results.envelope(probe&.capture, result))
47
52
  status = 0
48
53
  rescue Exception => e # anything: report it, then exit!
49
54
  warn "padded_cell: cell #{klass}##{name} failed to report: #{e.class}: #{e.message}"
@@ -54,19 +59,20 @@ module PaddedCell
54
59
  end
55
60
  end
56
61
 
62
+ # [coverage delta or nil, result]
57
63
  def collect(pid, reader, klass, name, started, timeout)
58
64
  data = read(reader, timeout && (started + timeout))
59
65
  elapsed = Minitest.clock_time - started
60
66
  if data.nil?
61
67
  Process.kill(:KILL, pid)
62
68
  Process.wait(pid)
63
- return Results.crashed(klass, name, "timed out after #{timeout}s", time: elapsed)
69
+ return [nil, Results.crashed(klass, name, "timed out after #{timeout}s", time: elapsed)]
64
70
  end
65
71
 
66
72
  _, status = Process.wait2(pid)
67
- return Results.load(data, klass, name) unless data.empty?
73
+ return Results.unwrap(data, klass, name) unless data.empty?
68
74
 
69
- Results.crashed(klass, name, Results.describe(status, 'cell'), time: elapsed)
75
+ [nil, Results.crashed(klass, name, Results.describe(status, 'cell'), time: elapsed)]
70
76
  end
71
77
 
72
78
  # Everything until EOF, or nil when the deadline passes first.
@@ -0,0 +1,103 @@
1
+ # frozen_string_literal: true
2
+
3
+ require 'coverage'
4
+ require 'fileutils'
5
+ require 'tmpdir'
6
+
7
+ module PaddedCell
8
+ module Chart
9
+ # The runner's side. Starts Ruby's Coverage before the suite loads, hands
10
+ # each worker a Probe, then sums its own measure (loading the suite) with
11
+ # every frame the workers wrote.
12
+ class Accumulator
13
+ attr_reader :criteria
14
+
15
+ # criteria: what Ruby's Coverage measures, :lines always among them
16
+ def initialize(criteria)
17
+ @criteria = criteria
18
+ @root = File.realpath(Dir.pwd)
19
+ @roots = [Dir.pwd, @root].uniq.map { "#{it}/" }
20
+ @kept = {}
21
+ @lost = 0
22
+ @known = Set.new
23
+ end
24
+
25
+ # Nothing loaded before is measured.
26
+ def start
27
+ ::Coverage.setup(**@criteria.to_h { [it, true] })
28
+ ::Coverage.resume
29
+ @frames = Dir.mktmpdir('padded_cell-coverage')
30
+ self
31
+ end
32
+
33
+ # After the suite loaded, before the first fork: the files whose
34
+ # structure the runner will report itself. Forks inherit the set.
35
+ def snapshot
36
+ ::Coverage.peek_result.each_key { @known << it if keep?(it) }
37
+ end
38
+
39
+ # For a worker, right after its fork.
40
+ def probe
41
+ Probe.new(file: File.join(@frames, "#{Process.pid}.cov"), known: @known, keep: method(:keep?),
42
+ methods: @criteria.include?(:methods), interval: Chart.config.interval)
43
+ end
44
+
45
+ # A worker died: what it measured since its last flush is gone.
46
+ def lost_worker = @lost += 1
47
+
48
+ # Sums everything (see Raw); nil for an interrupted run.
49
+ def finish(interrupted: false)
50
+ return if interrupted
51
+
52
+ total = Delta.capture(::Coverage.peek_result, Set.new, method(:keep?))
53
+ cells = 0
54
+ frames.each do |frame|
55
+ Delta.merge!(total, frame[:delta])
56
+ cells += frame[:lost]
57
+ end
58
+ Raw.build(normalize(total), root: @root, criteria: @criteria, lost: { workers: @lost, cells: })
59
+ ensure
60
+ FileUtils.rm_rf(@frames) if @frames
61
+ end
62
+
63
+ private
64
+
65
+ # Files under the project root, which exist (not eval'd code).
66
+ def keep?(path)
67
+ @kept.fetch(path) { @kept[path] = @roots.any? { path.start_with?(it) } && File.file?(path) }
68
+ end
69
+
70
+ # Every complete frame of every worker. A worker killed while writing
71
+ # leaves a truncated last frame, skipped.
72
+ def frames
73
+ Dir[File.join(@frames, '*.cov')].flat_map do |file|
74
+ data = File.binread(file)
75
+ offset = 0
76
+ frames = []
77
+ while offset + 4 <= data.bytesize
78
+ size = data.unpack1('N', offset:)
79
+ break if offset + 4 + size > data.bytesize
80
+
81
+ frames << Marshal.load(data.byteslice(offset + 4, size)) # written by our own workers
82
+ offset += 4 + size
83
+ end
84
+ frames
85
+ end
86
+ end
87
+
88
+ # Keys by real path relative to the root: a file required through a
89
+ # symlink and through its real path is one file. A file deleted during
90
+ # the run is not reported.
91
+ def normalize(total)
92
+ total.each_with_object({}) do |(path, entry), files|
93
+ relative = File.realpath(path).delete_prefix("#{@root}/")
94
+ next if relative.start_with?('/')
95
+
96
+ files[relative] ? Delta.merge!(files, { relative => entry }) : files[relative] = entry
97
+ rescue SystemCallError
98
+ next
99
+ end
100
+ end
101
+ end
102
+ end
103
+ end
@@ -0,0 +1,129 @@
1
+ # frozen_string_literal: true
2
+
3
+ module PaddedCell
4
+ module Chart
5
+ # A PaddedCell.coverage block, in the test helper:
6
+ #
7
+ # PaddedCell.coverage(:rails) do
8
+ # track "{app,lib}/**/*.rb"
9
+ # skip "app/legacy/"
10
+ # group "Operations", "app/concepts"
11
+ # minimum line: 90, branch: 80
12
+ # minimum_by_file line: 50
13
+ # minimum_by_group "Models", line: 95
14
+ # maximum_drop line: 1
15
+ # formats :markdown, :lcov, :json
16
+ # end
17
+ #
18
+ # Filters and groups match paths (see Filter). What is
19
+ # measured is decided before the suite loads, on the command line:
20
+ # --coverage=lines,branches,methods.
21
+ class Config
22
+ FORMATS = %i[markdown lcov json].freeze
23
+ # Left out unless cleared: bundled gems, hidden files, test files.
24
+ DEFAULT_FILTERS = ['/vendor/bundle/', /\A\..*/, %r{\A(test|features|spec|autotest)/}].freeze
25
+ RAILS_GROUPS = {
26
+ 'Channels' => 'app/channels', 'Controllers' => 'app/controllers', 'Helpers' => 'app/helpers',
27
+ 'Jobs' => %w[app/jobs app/workers], 'Libraries' => 'lib/', 'Mailers' => 'app/mailers',
28
+ 'Models' => 'app/models', 'Views' => 'app/views'
29
+ }.freeze
30
+
31
+ attr_reader :filters, :groups, :tracked, :output_formats, :directory, :token, :interval, :minimums,
32
+ :file_minimums, :group_minimums, :maximum_drops
33
+
34
+ def initialize
35
+ @filters = DEFAULT_FILTERS.map { Filter.build(it) }
36
+ @groups = {}
37
+ @tracked = []
38
+ @minimums = {}
39
+ @file_minimums = {}
40
+ @group_minimums = {}
41
+ @maximum_drops = {}
42
+ @output_formats = %i[markdown lcov json]
43
+ @directory = 'coverage'
44
+ @token = 'nocov'
45
+ @interval = Probe::INTERVAL
46
+ end
47
+
48
+ # :rails skips config/ and db/, groups the
49
+ # app's directories, tracks {app,lib}/**/*.rb.
50
+ def profile(name)
51
+ raise ArgumentError, "unknown coverage profile #{name.inspect} (known: :rails)" unless name.to_s == 'rails'
52
+
53
+ skip(%r{\Aconfig/}, %r{\Adb/})
54
+ RAILS_GROUPS.each { |group_name, filter| group(group_name, filter) }
55
+ track('{app,lib}/**/*.rb')
56
+ end
57
+
58
+ # Files to report even when the suite never loads them, at 0%.
59
+ def track(*globs) = @tracked.concat(globs)
60
+
61
+ # Files to leave out of the report.
62
+ def skip(*filters, &block)
63
+ filters.each { @filters << Filter.build(it) }
64
+ @filters << Filter.build(&block) if block
65
+ end
66
+ alias filter skip
67
+
68
+ # Removes every filter, the defaults included.
69
+ def clear_filters = @filters.clear
70
+
71
+ # A file can be in several groups; files in none are Ungrouped.
72
+ # Defining a group again replaces it.
73
+ def group(name, filter = nil, &) = @groups[name.to_s] = Filter.build(filter, &)
74
+
75
+ # Thresholds compare percents floored to 2 decimals; a whole, passing
76
+ # run that misses one exits 3.
77
+ def minimum(line: nil, branch: nil, method: nil) = @minimums.merge!({ line:, branch:, method: }.compact)
78
+
79
+ # For every file.
80
+ def minimum_by_file(line: nil, branch: nil, method: nil)
81
+ @file_minimums.merge!({ line:, branch:, method: }.compact)
82
+ end
83
+
84
+ def minimum_by_group(name, line: nil, branch: nil, method: nil)
85
+ (@group_minimums[name.to_s] ||= {}).merge!({ line:, branch:, method: }.compact)
86
+ end
87
+
88
+ # How many points coverage may lose since the previous whole run.
89
+ def maximum_drop(line: nil, branch: nil, method: nil)
90
+ @maximum_drops.merge!({ line:, branch:, method: }.compact)
91
+ end
92
+
93
+ def formats(*names)
94
+ unknown = names.map(&:to_sym) - FORMATS
95
+ unless unknown.empty?
96
+ raise ArgumentError, "unknown coverage formats: #{unknown.join(', ')} (known: #{FORMATS.join(', ')})"
97
+ end
98
+
99
+ @output_formats = names.map(&:to_sym)
100
+ end
101
+
102
+ # Where reports go (default: coverage). --coverage-dir wins.
103
+ def dir(path) = @directory = path
104
+
105
+ # `# :nocov:` opens and closes blocks of lines coverage ignores.
106
+ def nocov_token(token) = @token = token
107
+
108
+ # How often, at least, a worker hands over what it measured (seconds,
109
+ # default 10, stretched to keep it under 1% of the worker's time; 0:
110
+ # after every test). A worker that dies loses what it measured since.
111
+ def flush_interval(seconds) = @interval = seconds
112
+
113
+ # What a report needs once files are filtered and grouped, as stored
114
+ # in its Document.
115
+ def settings
116
+ {
117
+ formats: @output_formats.map(&:to_s), nocov_token: @token, minimum: @minimums,
118
+ minimum_by_file: @file_minimums, minimum_by_group: @group_minimums, maximum_drop: @maximum_drops
119
+ }
120
+ end
121
+
122
+ # Criteria come from the command line, before the suite (and this
123
+ # block) loads.
124
+ def enable_coverage(*)
125
+ raise ArgumentError, 'coverage criteria come from --coverage=lines,branches,methods, not the test helper'
126
+ end
127
+ end
128
+ end
129
+ end