active_mutator 0.6.1 → 0.7.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/README.md +37 -9
- data/lib/active_mutator/abort_flag.rb +53 -0
- data/lib/active_mutator/baseline.rb +132 -58
- data/lib/active_mutator/cli.rb +16 -1
- data/lib/active_mutator/config.rb +8 -1
- data/lib/active_mutator/config_file.rb +17 -1
- data/lib/active_mutator/coverage_map.rb +4 -1
- data/lib/active_mutator/diagnostics/ndjson.rb +25 -0
- data/lib/active_mutator/diagnostics/text.rb +110 -0
- data/lib/active_mutator/events.rb +55 -0
- data/lib/active_mutator/memory_ceiling.rb +59 -0
- data/lib/active_mutator/memory_probe.rb +82 -0
- data/lib/active_mutator/reporter/github.rb +13 -2
- data/lib/active_mutator/reporter/json.rb +19 -5
- data/lib/active_mutator/reporter/stryker_json.rb +7 -1
- data/lib/active_mutator/reporter/terminal.rb +31 -2
- data/lib/active_mutator/result.rb +7 -1
- data/lib/active_mutator/runner.rb +198 -63
- data/lib/active_mutator/sampler.rb +69 -0
- data/lib/active_mutator/scheduler.rb +105 -61
- data/lib/active_mutator/since_filter.rb +30 -7
- data/lib/active_mutator/version.rb +1 -1
- data/lib/active_mutator.rb +7 -0
- metadata +8 -1
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: cf0b7bed1f6b35d6ab688affd01cb2ccc362944019ae5c18bff2e1205cccb53e
|
|
4
|
+
data.tar.gz: 9b4d7cac2fc3f8e3a6d737db4aa7f3b8f3f46c00baa275669fecbc5752e50910
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: f2ed986856e22c4881936ae20f66bac4ecdc57b31239b5786942be6c2128516db64542be9185da5f9e8bc06ad1acd893d9e231602e1cf553ad7c74d60af9ef61
|
|
7
|
+
data.tar.gz: 8f8bd75ed8572b8e1eee883644b412bc9e718eeb0b85f35e22d8f27b6f937113c505bccadca869fd33546895bbf68126a30b9931fc755c3df8d71e33b8168bb0
|
data/README.md
CHANGED
|
@@ -127,12 +127,26 @@ Each character on the progress line is one mutant, printed as it finishes:
|
|
|
127
127
|
| `A` | `accepted` | matches a known-equivalent entry in the acceptance ledger. Excluded from the score |
|
|
128
128
|
|
|
129
129
|
`invalid` mutants (edits that don't even re-parse as valid Ruby) are
|
|
130
|
-
discarded before scheduling and reported as a count only.
|
|
131
|
-
|
|
132
|
-
|
|
133
|
-
|
|
134
|
-
|
|
135
|
-
|
|
130
|
+
discarded before scheduling and reported as a count only.
|
|
131
|
+
|
|
132
|
+
| Exit | When |
|
|
133
|
+
|---|---|
|
|
134
|
+
| `0` | no unaccepted survivors or errors (only `uncovered` or `accepted` results count as clean), or the score meets `--fail-at` |
|
|
135
|
+
| `1` | unaccepted survivors or errors (with `--fail-at`, only when the score is below it), or an empty plan (below) |
|
|
136
|
+
| `2` | a usage or setup problem: a bad flag or config file, or a red baseline suite |
|
|
137
|
+
| `3` | the run reached `--max-rss` and stopped |
|
|
138
|
+
| `130` | the run was stopped by SIGINT (Ctrl-C) |
|
|
139
|
+
| `143` | the run was stopped by SIGTERM (130 before 0.7.0) |
|
|
140
|
+
|
|
141
|
+
A stopped run (`3`, `130`, `143`) kills its child processes, prints the
|
|
142
|
+
summary for the mutants that finished with a `Partial mutation score:`
|
|
143
|
+
line in place of `Mutation score:`, and never passes, whatever `--fail-at`
|
|
144
|
+
says. See [Run diagnostics](docs/guides/diagnostics.md#aborted-runs).
|
|
145
|
+
|
|
146
|
+
The JSON report's `exit_reason` field (`unaccepted_survivors`,
|
|
147
|
+
`worker_errors`, `clean`, `empty_plan`, and for a stopped run `interrupted`
|
|
148
|
+
or `memory_ceiling`) is independent of the `--fail-at` gate. `complete` is
|
|
149
|
+
`false` only for a stopped run. A `--since` or `--subject` run that
|
|
136
150
|
plans zero mutants skips the baseline, prints the usual count block with
|
|
137
151
|
every status at `0` and no `Mutation score:` line (JSON: `score` is `null`,
|
|
138
152
|
`exit_reason` is `empty_plan`), then warns with the cause and exits `1`
|
|
@@ -267,9 +281,12 @@ itself whether that emptiness is fine:
|
|
|
267
281
|
blank lines, added or deleted. A magic comment such as
|
|
268
282
|
`# frozen_string_literal: true` counts as code, and so does a file that
|
|
269
283
|
fails to parse.
|
|
284
|
+
- **Exit 0** when a file only lost code and no method spans the spot, e.g.
|
|
285
|
+
a whole method was removed. A deletion inside a method makes `--since`
|
|
286
|
+
mutate that method, like any other edit to it, so it never reaches this
|
|
287
|
+
rule unless the method has nothing left to mutate.
|
|
270
288
|
- **Exit 1** otherwise. A candidate file's code changed but produced no
|
|
271
|
-
mutants, and the warning lists the file(s).
|
|
272
|
-
counts as a real change here.
|
|
289
|
+
mutants, and the warning lists the file(s).
|
|
273
290
|
- With `--subject` and no `--since` there is no diff to judge, so
|
|
274
291
|
`--allow-empty` exits `0` unconditionally.
|
|
275
292
|
|
|
@@ -301,6 +318,10 @@ projects used to tell a docs-only PR from a broken `--since` range.
|
|
|
301
318
|
| `--require FILE` | none | preload files (repeatable) |
|
|
302
319
|
| `--operator FILE` | none | load a custom operator file before analysis (repeatable) |
|
|
303
320
|
| `--[no-]class-level` | on | mutate class-level code (macros, constants, DSL/scope lambdas) via class-body subjects |
|
|
321
|
+
| `--diagnostics` | off | print timestamped phase, mutant, and memory lines to stderr, for finding out where a CI run died (see [Run diagnostics](docs/guides/diagnostics.md)) |
|
|
322
|
+
| `--events FILE` | none | write the same events as NDJSON, one object per line, flushed as they happen |
|
|
323
|
+
| `--sample-interval S` | 5 | seconds between memory samples with `--diagnostics`, `--events`, or `--max-rss` |
|
|
324
|
+
| `--max-rss SIZE` | none | stop the run and exit 3 when the gem's total memory reaches SIZE (`6G`, `6144M`, or plain MB); warns once at 90% |
|
|
304
325
|
| `--fail-at SCORE` | none (strict) | exit 0 if score >= SCORE even with survivors (opt-in relaxation for gradual adoption; 0 = report-only) |
|
|
305
326
|
|
|
306
327
|
`--spec-path` tells active_mutator where spec files live (coverage
|
|
@@ -338,7 +359,10 @@ replaces the default `spec`),
|
|
|
338
359
|
`adaptive_timeout` (`true`/`false`),
|
|
339
360
|
`class_level` (`true`/`false`, default `true` — mutate class-level code),
|
|
340
361
|
`class_level_closure_cap` (integer, default `10` — max constants a
|
|
341
|
-
class-body mutant may reload before it is `skipped`)
|
|
362
|
+
class-body mutant may reload before it is `skipped`),
|
|
363
|
+
`diagnostics` (`true`/`false`, default `false`),
|
|
364
|
+
`events_file` (a path), `sample_interval` (seconds, default `5`),
|
|
365
|
+
`max_rss` (a size like `6G`, `6144M`, or plain MB).
|
|
342
366
|
Unknown keys and wrong types are errors, not silent no-ops.
|
|
343
367
|
|
|
344
368
|
```yaml
|
|
@@ -425,6 +449,10 @@ remaining limits are:
|
|
|
425
449
|
- [Operator reference](docs/guides/operators.md): every mutation
|
|
426
450
|
active_mutator can generate, with before/after examples and what a
|
|
427
451
|
survivor of each one means.
|
|
452
|
+
- [Run diagnostics](docs/guides/diagnostics.md): `--diagnostics`,
|
|
453
|
+
`--events`, and `--max-rss`. Find the phase and mutants a dying CI run
|
|
454
|
+
was in, stop a run before it runs out of memory, and the `--events`
|
|
455
|
+
NDJSON schema.
|
|
428
456
|
- [Custom operators](docs/guides/custom-operators.md): write and load your
|
|
429
457
|
own mutation operators with `--operator` / the `operators:` config key.
|
|
430
458
|
- [Mutation-check skill](docs/skills/mutation-check.md): the agent-facing
|
|
@@ -0,0 +1,53 @@
|
|
|
1
|
+
module ActiveMutator
|
|
2
|
+
# A run stopped early by SIGINT, SIGTERM, or the memory ceiling. Carries
|
|
3
|
+
# what finished (`results`) and what was still running (`in_flight`).
|
|
4
|
+
class Aborted < StandardError
|
|
5
|
+
attr_reader :reason, :results, :in_flight
|
|
6
|
+
|
|
7
|
+
def initialize(reason, results: [], in_flight: [])
|
|
8
|
+
super("run aborted: #{reason}")
|
|
9
|
+
@reason = reason
|
|
10
|
+
@results = results
|
|
11
|
+
@in_flight = in_flight
|
|
12
|
+
end
|
|
13
|
+
|
|
14
|
+
# The same abort, with the finished results a caller up the stack knows.
|
|
15
|
+
def with_results(results) = self.class.new(reason, results: results, in_flight: in_flight)
|
|
16
|
+
end
|
|
17
|
+
|
|
18
|
+
# Where a signal or a memory breach lands. `trip!` is safe in a trap
|
|
19
|
+
# handler: no I/O and no locks (a Mutex in a trap raises ThreadError).
|
|
20
|
+
#
|
|
21
|
+
# Outside a `deferred` block, a trip raises Aborted on the main thread at
|
|
22
|
+
# once: boot and planning own no child processes, so there is nothing to
|
|
23
|
+
# clean up. Inside one (the baseline child's wait, the fork pool) it only
|
|
24
|
+
# records the reason; the owner's poll loop kills its children and then
|
|
25
|
+
# raises with what it knows. The first reason wins, and later trips are
|
|
26
|
+
# ignored while the abort is under way.
|
|
27
|
+
class AbortFlag
|
|
28
|
+
attr_reader :reason
|
|
29
|
+
|
|
30
|
+
def trip!(reason)
|
|
31
|
+
return if @reason
|
|
32
|
+
|
|
33
|
+
@reason = reason
|
|
34
|
+
Thread.main.raise(Aborted.new(reason)) unless @deferred
|
|
35
|
+
end
|
|
36
|
+
|
|
37
|
+
# Not an endless def: its one line runs at load, so coverage could
|
|
38
|
+
# never tie it to the specs that call it.
|
|
39
|
+
def tripped?
|
|
40
|
+
!@reason.nil?
|
|
41
|
+
end
|
|
42
|
+
|
|
43
|
+
def deferred
|
|
44
|
+
outer = @deferred
|
|
45
|
+
@deferred = true
|
|
46
|
+
raise Aborted, @reason if @reason
|
|
47
|
+
|
|
48
|
+
yield
|
|
49
|
+
ensure
|
|
50
|
+
@deferred = outer
|
|
51
|
+
end
|
|
52
|
+
end
|
|
53
|
+
end
|
|
@@ -7,48 +7,69 @@ module ActiveMutator
|
|
|
7
7
|
# caches the CoverageMap. Invalidation is coarse: any digest change in
|
|
8
8
|
# {app,lib}/**/*.rb or the configured spec paths triggers a full re-run.
|
|
9
9
|
class Baseline
|
|
10
|
-
def initialize(root:, spec_paths: ["spec"], cache_dir: File.join(root, ".active_mutator")
|
|
10
|
+
def initialize(root:, spec_paths: ["spec"], cache_dir: File.join(root, ".active_mutator"), events: Events.new,
|
|
11
|
+
abort: AbortFlag.new)
|
|
12
|
+
@abort = abort
|
|
11
13
|
@root = root
|
|
12
14
|
@spec_paths = spec_paths
|
|
13
15
|
@cache_dir = cache_dir
|
|
14
16
|
@out_path = File.join(cache_dir, "coverage.json")
|
|
17
|
+
@events = events
|
|
15
18
|
end
|
|
16
19
|
|
|
17
|
-
|
|
20
|
+
POLL_SECONDS = 0.05
|
|
21
|
+
|
|
22
|
+
# child_pid: the running baseline child, nil between runs. Read by the
|
|
23
|
+
# memory sampler from its own thread.
|
|
24
|
+
attr_reader :last_refresh, :child_pid
|
|
18
25
|
|
|
19
26
|
def coverage_map(force: false)
|
|
20
27
|
digests = current_digests
|
|
21
|
-
|
|
22
|
-
map =
|
|
23
|
-
|
|
24
|
-
# under a removed spec path just vanish from the digest scan, so
|
|
25
|
-
# BaselineDelta treats their stale example records as untouched
|
|
26
|
-
# source coverage instead of dropping them. Force a full rebuild
|
|
27
|
-
# whenever the configured spec_paths differ from what the cache was
|
|
28
|
-
# stamped with.
|
|
29
|
-
if stored_spec_paths(map) == @spec_paths && map.fresh?(digests)
|
|
30
|
-
@last_refresh = :cached
|
|
31
|
-
return map
|
|
32
|
-
end
|
|
33
|
-
if stored_spec_paths(map) == @spec_paths && map.version == 2
|
|
34
|
-
delta = BaselineDelta.compute(old_digests: stored_digests(map), new_digests: digests,
|
|
35
|
-
coverage_map: map, root: @root, spec_paths: @spec_paths)
|
|
36
|
-
unless delta.full?
|
|
37
|
-
run_partial!(delta)
|
|
38
|
-
stamp_digests(digests)
|
|
39
|
-
@last_refresh = :partial
|
|
40
|
-
return CoverageMap.load(@out_path)
|
|
41
|
-
end
|
|
42
|
-
end
|
|
28
|
+
unless force
|
|
29
|
+
map = reuse_cache(digests)
|
|
30
|
+
return map if map
|
|
43
31
|
end
|
|
44
|
-
run_baseline!
|
|
45
|
-
stamp_digests(digests)
|
|
32
|
+
data = run_baseline!
|
|
46
33
|
@last_refresh = :full
|
|
47
|
-
|
|
34
|
+
stamp(data, digests)
|
|
48
35
|
end
|
|
49
36
|
|
|
50
37
|
private
|
|
51
38
|
|
|
39
|
+
# The cached map, refreshed in place when a delta allows it; nil when only
|
|
40
|
+
# a full rebuild will do. The old map (records plus its inverted index)
|
|
41
|
+
# lives only in this method, so it is garbage before the full rebuild's
|
|
42
|
+
# child starts.
|
|
43
|
+
def reuse_cache(digests)
|
|
44
|
+
return unless File.exist?(@out_path)
|
|
45
|
+
|
|
46
|
+
data = load_payload
|
|
47
|
+
map = CoverageMap.new(data)
|
|
48
|
+
# A spec_paths change silently degrades the delta classifier: files
|
|
49
|
+
# under a removed spec path just vanish from the digest scan, so
|
|
50
|
+
# BaselineDelta treats their stale example records as untouched
|
|
51
|
+
# source coverage instead of dropping them. Force a full rebuild
|
|
52
|
+
# whenever the configured spec_paths differ from what the cache was
|
|
53
|
+
# stamped with.
|
|
54
|
+
return unless map.spec_paths == @spec_paths
|
|
55
|
+
|
|
56
|
+
if map.fresh?(digests)
|
|
57
|
+
@last_refresh = :cached
|
|
58
|
+
return map
|
|
59
|
+
end
|
|
60
|
+
return unless map.version == 2
|
|
61
|
+
|
|
62
|
+
delta = BaselineDelta.compute(old_digests: map.digests, new_digests: digests,
|
|
63
|
+
coverage_map: map, root: @root, spec_paths: @spec_paths)
|
|
64
|
+
return if delta.full?
|
|
65
|
+
|
|
66
|
+
# The map shares data["records"], and the merge edits it in place: the
|
|
67
|
+
# old map must not be read past this point.
|
|
68
|
+
run_partial!(delta, data)
|
|
69
|
+
@last_refresh = :partial
|
|
70
|
+
stamp(data, digests)
|
|
71
|
+
end
|
|
72
|
+
|
|
52
73
|
# The cache is disposable and must never be committed. Host projects
|
|
53
74
|
# rarely gitignore it themselves, so the directory ignores its own
|
|
54
75
|
# contents (the node_modules trick).
|
|
@@ -60,22 +81,31 @@ module ActiveMutator
|
|
|
60
81
|
|
|
61
82
|
def run_baseline!
|
|
62
83
|
prepare_cache_dir
|
|
63
|
-
|
|
64
|
-
# out: :err: the subprocess suite's progress output must not pollute
|
|
65
|
-
# our stdout (breaks `--format json` consumers).
|
|
66
|
-
ok = system(env, "bundle", "exec", "rspec", chdir: @root, out: :err)
|
|
84
|
+
ok = run_rspec(@out_path)
|
|
67
85
|
raise BaselineFailed, "baseline suite failed, fix the suite before mutating" unless ok
|
|
68
86
|
raise BaselineFailed, "baseline produced no coverage output" unless File.exist?(@out_path)
|
|
69
87
|
|
|
70
|
-
|
|
88
|
+
data = load_payload
|
|
89
|
+
verify_complete!(data)
|
|
90
|
+
data
|
|
91
|
+
end
|
|
92
|
+
|
|
93
|
+
# Its own phase, apart from the child's run: 0.6.0 died here, in the
|
|
94
|
+
# parent reading a huge file back. The size goes out BEFORE the parse, so
|
|
95
|
+
# the log names the cause even if nothing runs after it, and --max-rss
|
|
96
|
+
# can stop the run before the parse.
|
|
97
|
+
def load_payload(path = @out_path)
|
|
98
|
+
@events.emit(:phase_start, phase: :coverage_load, bytes: File.size(path))
|
|
99
|
+
data = JSON.parse(File.read(path))
|
|
100
|
+
@events.emit(:phase_end, phase: :coverage_load, examples: data.fetch("records", {}).size)
|
|
101
|
+
data
|
|
71
102
|
end
|
|
72
103
|
|
|
73
104
|
# An aborted subprocess can still exit 0 with a partial map (RSpec
|
|
74
105
|
# rescues Errno::EPIPE and runs after(:suite)); stamping that as fresh
|
|
75
106
|
# silently reports every mutant uncovered. Payloads without the count
|
|
76
107
|
# predate this check and are accepted as-is.
|
|
77
|
-
def verify_complete!(
|
|
78
|
-
payload = JSON.parse(File.read(out_path))
|
|
108
|
+
def verify_complete!(payload)
|
|
79
109
|
expected = payload["expected_examples"]
|
|
80
110
|
return unless expected
|
|
81
111
|
|
|
@@ -87,6 +117,60 @@ module ActiveMutator
|
|
|
87
117
|
"re-run without interrupting the suite"
|
|
88
118
|
end
|
|
89
119
|
|
|
120
|
+
# Spawned and polled, not `system`, so the parent knows the child's pid
|
|
121
|
+
# and keeps control while it runs. out: :err: the subprocess suite's
|
|
122
|
+
# progress output must not pollute our stdout (breaks `--format json`
|
|
123
|
+
# consumers).
|
|
124
|
+
#
|
|
125
|
+
# The phase starts once the child exists, so it carries the pid the
|
|
126
|
+
# memory sampler follows. The child gets its own process group so an
|
|
127
|
+
# abort can kill the whole suite (browsers, app servers) in one signal;
|
|
128
|
+
# a Ctrl-C reaches it through the Runner's trap instead of the terminal.
|
|
129
|
+
# So does nothing else: a hangup from a closed terminal never reaches it,
|
|
130
|
+
# so anything that ends the wait early (an abort, SIGHUP, SIGQUIT) kills
|
|
131
|
+
# the group on the way out.
|
|
132
|
+
def run_rspec(out_path, targets = [])
|
|
133
|
+
status = nil
|
|
134
|
+
@abort.deferred do
|
|
135
|
+
@child_pid = Process.spawn(baseline_env(out_path), *rspec_command(targets), chdir: @root, out: :err,
|
|
136
|
+
pgroup: true)
|
|
137
|
+
status = @events.phase(:baseline, refresh: targets.empty? ? :full : :partial, pid: @child_pid) do
|
|
138
|
+
wait_child(@child_pid)
|
|
139
|
+
end
|
|
140
|
+
# The memory sample taken as the phase ends can trip the ceiling.
|
|
141
|
+
# Stop here, before the coverage parse: that's the spike it guards.
|
|
142
|
+
raise Aborted, @abort.reason if @abort.tripped?
|
|
143
|
+
|
|
144
|
+
status.success?
|
|
145
|
+
end
|
|
146
|
+
rescue SystemCallError # `bundle` missing: `system` returned nil here
|
|
147
|
+
false
|
|
148
|
+
ensure
|
|
149
|
+
kill_group(@child_pid) if @child_pid && status.nil?
|
|
150
|
+
@child_pid = nil
|
|
151
|
+
end
|
|
152
|
+
|
|
153
|
+
def rspec_command(targets) = ["bundle", "exec", "rspec", *targets]
|
|
154
|
+
|
|
155
|
+
# The flag is checked after the wait, so a trip that lands as the child
|
|
156
|
+
# exits still stops the run here, not after the coverage parse.
|
|
157
|
+
# run_rspec kills the child's group on the way out.
|
|
158
|
+
def wait_child(pid)
|
|
159
|
+
loop do
|
|
160
|
+
_, status = Process.waitpid2(pid, Process::WNOHANG)
|
|
161
|
+
raise Aborted, @abort.reason if @abort.tripped?
|
|
162
|
+
return status if status
|
|
163
|
+
|
|
164
|
+
sleep POLL_SECONDS
|
|
165
|
+
end
|
|
166
|
+
end
|
|
167
|
+
|
|
168
|
+
def kill_group(pid)
|
|
169
|
+
Process.kill("KILL", -pid)
|
|
170
|
+
rescue Errno::ESRCH, Errno::EPERM
|
|
171
|
+
nil # already gone
|
|
172
|
+
end
|
|
173
|
+
|
|
90
174
|
def baseline_env(out_path)
|
|
91
175
|
{
|
|
92
176
|
"ACTIVE_MUTATOR" => "1",
|
|
@@ -106,36 +190,25 @@ module ActiveMutator
|
|
|
106
190
|
}
|
|
107
191
|
end
|
|
108
192
|
|
|
109
|
-
def
|
|
110
|
-
JSON.parse(File.read(@out_path)).fetch("digests", {})
|
|
111
|
-
end
|
|
112
|
-
|
|
113
|
-
# A pre-0.4.0 cache predates spec_paths and has no key; treat that as the
|
|
114
|
-
# old implicit default so existing default-config caches stay valid.
|
|
115
|
-
def stored_spec_paths(map)
|
|
116
|
-
JSON.parse(File.read(@out_path)).fetch("spec_paths", ["spec"])
|
|
117
|
-
end
|
|
118
|
-
|
|
119
|
-
def run_partial!(delta)
|
|
120
|
-
targets = delta.rerun_spec_files + delta.rerun_example_ids
|
|
193
|
+
def run_partial!(delta, cache)
|
|
121
194
|
partial_out = File.join(@cache_dir, "partial.json")
|
|
195
|
+
targets = delta.rerun_spec_files + delta.rerun_example_ids
|
|
196
|
+
part = {}
|
|
122
197
|
if targets.any?
|
|
123
|
-
|
|
124
|
-
ok = system(env, "bundle", "exec", "rspec", *targets, chdir: @root, out: :err)
|
|
198
|
+
ok = run_rspec(partial_out, targets)
|
|
125
199
|
raise BaselineFailed, "partial baseline run failed, fix the suite before mutating" unless ok
|
|
126
200
|
raise BaselineFailed, "partial baseline produced no output" unless File.exist?(partial_out)
|
|
127
201
|
|
|
128
|
-
|
|
202
|
+
part = load_payload(partial_out)
|
|
203
|
+
verify_complete!(part)
|
|
129
204
|
end
|
|
130
|
-
merge_partial!(
|
|
205
|
+
merge_partial!(cache, part, delta)
|
|
131
206
|
ensure
|
|
132
|
-
FileUtils.rm_f(partial_out)
|
|
207
|
+
FileUtils.rm_f(partial_out)
|
|
133
208
|
end
|
|
134
209
|
|
|
135
|
-
|
|
136
|
-
|
|
137
|
-
part = File.exist?(partial_out) ? JSON.parse(File.read(partial_out)) : { "records" => {}, "times" => {} }
|
|
138
|
-
|
|
210
|
+
# Edits `cache` in place; the caller stamps and writes it.
|
|
211
|
+
def merge_partial!(cache, part, delta)
|
|
139
212
|
rerun_prefixes = delta.rerun_spec_files.map { |rel| "#{rel}[" }
|
|
140
213
|
obsolete = lambda do |example_id|
|
|
141
214
|
bare = example_id.sub(%r{\A\./}, "")
|
|
@@ -151,14 +224,15 @@ module ActiveMutator
|
|
|
151
224
|
end
|
|
152
225
|
cache["records"].merge!(part.fetch("records", {}))
|
|
153
226
|
cache["times"].merge!(part.fetch("times", {}))
|
|
154
|
-
AtomicFile.write(@out_path, JSON.generate(cache))
|
|
155
227
|
end
|
|
156
228
|
|
|
157
|
-
|
|
158
|
-
|
|
229
|
+
# Writes the stamped payload once and builds the map from the hash in
|
|
230
|
+
# hand, so the file is never parsed back.
|
|
231
|
+
def stamp(data, digests)
|
|
159
232
|
data["digests"] = digests
|
|
160
233
|
data["spec_paths"] = @spec_paths
|
|
161
234
|
AtomicFile.write(@out_path, JSON.generate(data))
|
|
235
|
+
CoverageMap.new(data)
|
|
162
236
|
end
|
|
163
237
|
|
|
164
238
|
def current_digests
|
data/lib/active_mutator/cli.rb
CHANGED
|
@@ -32,7 +32,8 @@ module ActiveMutator
|
|
|
32
32
|
spec_paths: ["spec"],
|
|
33
33
|
browser_boot_seconds: 15.0, accept_survivors: false, exclude: [],
|
|
34
34
|
max_mutants: nil, debug_plan: false, fail_at: nil, adaptive_timeout: true,
|
|
35
|
-
operators: [], class_level: true, class_level_closure_cap: 10, allow_empty: false
|
|
35
|
+
operators: [], class_level: true, class_level_closure_cap: 10, allow_empty: false,
|
|
36
|
+
diagnostics: false, events_file: nil, sample_interval: 5.0, max_rss: nil
|
|
36
37
|
}
|
|
37
38
|
options.merge!(ConfigFile.load(Dir.pwd))
|
|
38
39
|
paths = OptionParser.new do |o|
|
|
@@ -69,6 +70,20 @@ module ActiveMutator
|
|
|
69
70
|
o.on("--allow-empty",
|
|
70
71
|
"Exit 0 when --since/--subject plan no mutants and the --since diff changed no code " \
|
|
71
72
|
"in a mutable source file (default: exit 1)") { options[:allow_empty] = true }
|
|
73
|
+
o.on("--diagnostics", "Print phase, mutant, and memory lines to stderr") { options[:diagnostics] = true }
|
|
74
|
+
o.on("--events FILE", "Write run events to FILE as NDJSON, one object per line") { |v| options[:events_file] = v }
|
|
75
|
+
o.on("--sample-interval S", Float,
|
|
76
|
+
"Seconds between memory samples with --diagnostics, --events, or --max-rss (default: 5)") do |v|
|
|
77
|
+
raise OptionParser::InvalidArgument, "--sample-interval must be > 0" unless v.positive?
|
|
78
|
+
options[:sample_interval] = v
|
|
79
|
+
end
|
|
80
|
+
o.on("--max-rss SIZE", "Stop the run (exit 3) when the gem's total memory reaches SIZE: 6G, 6144M, " \
|
|
81
|
+
"or MB; warns at 90%") do |v|
|
|
82
|
+
kb = MemoryCeiling.parse_kb(v)
|
|
83
|
+
raise OptionParser::InvalidArgument, "--max-rss takes a size like 6G, 6144M, or 6144 (MB)" unless kb
|
|
84
|
+
|
|
85
|
+
options[:max_rss] = kb
|
|
86
|
+
end
|
|
72
87
|
o.on("--fail-at SCORE", Float, "Exit 0 if mutation score >= SCORE even with survivors (default: any survivor fails)") do |v|
|
|
73
88
|
raise OptionParser::InvalidArgument, "--fail-at must be within 0..100" unless (0..100).cover?(v)
|
|
74
89
|
options[:fail_at] = v
|
|
@@ -7,5 +7,12 @@ module ActiveMutator
|
|
|
7
7
|
:browser_boot_seconds,
|
|
8
8
|
:accept_survivors, :exclude, :max_mutants, :debug_plan,
|
|
9
9
|
:fail_at, :adaptive_timeout, :operators,
|
|
10
|
-
:class_level, :class_level_closure_cap, :allow_empty
|
|
10
|
+
:class_level, :class_level_closure_cap, :allow_empty,
|
|
11
|
+
:diagnostics, :events_file, :sample_interval, :max_rss) do
|
|
12
|
+
# Defaults for the 0.7.0 diagnostics fields, so a Config built by hand
|
|
13
|
+
# (specs, embedding hosts) doesn't have to name them. `max_rss` is in kB.
|
|
14
|
+
def initialize(diagnostics: false, events_file: nil, sample_interval: 5.0, max_rss: nil, **fields)
|
|
15
|
+
super
|
|
16
|
+
end
|
|
17
|
+
end
|
|
11
18
|
end
|
|
@@ -26,7 +26,11 @@ module ActiveMutator
|
|
|
26
26
|
"adaptive_timeout" => :boolean,
|
|
27
27
|
"class_level" => :boolean,
|
|
28
28
|
"class_level_closure_cap" => :positive_integer,
|
|
29
|
-
"allow_empty" => :boolean
|
|
29
|
+
"allow_empty" => :boolean,
|
|
30
|
+
"diagnostics" => :boolean,
|
|
31
|
+
"events_file" => :string,
|
|
32
|
+
"sample_interval" => :positive_number,
|
|
33
|
+
"max_rss" => :size
|
|
30
34
|
}.freeze
|
|
31
35
|
|
|
32
36
|
def self.load(root)
|
|
@@ -63,6 +67,10 @@ module ActiveMutator
|
|
|
63
67
|
when :number
|
|
64
68
|
raise Error, "#{FILENAME}: #{key} must be a number" unless value.is_a?(Numeric)
|
|
65
69
|
value.to_f
|
|
70
|
+
when :positive_number
|
|
71
|
+
raise Error, "#{FILENAME}: #{key} must be a number" unless value.is_a?(Numeric)
|
|
72
|
+
raise Error, "#{FILENAME}: #{key} must be > 0" unless value.positive?
|
|
73
|
+
value.to_f
|
|
66
74
|
when :score
|
|
67
75
|
raise Error, "#{FILENAME}: #{key} must be a number" unless value.is_a?(Numeric)
|
|
68
76
|
raise Error, "#{FILENAME}: #{key} must be within 0..100" unless (0..100).cover?(value)
|
|
@@ -72,6 +80,9 @@ module ActiveMutator
|
|
|
72
80
|
raise Error, "#{FILENAME}: format must be one of #{FORMATS.join(", ")}"
|
|
73
81
|
end
|
|
74
82
|
value.tr("-", "_").to_sym
|
|
83
|
+
when :string
|
|
84
|
+
raise Error, "#{FILENAME}: #{key} must be a string" unless value.is_a?(String)
|
|
85
|
+
value
|
|
75
86
|
when :string_list
|
|
76
87
|
unless value.is_a?(Array) && value.all?(String)
|
|
77
88
|
raise Error, "#{FILENAME}: #{key} must be a list of strings"
|
|
@@ -88,6 +99,11 @@ module ActiveMutator
|
|
|
88
99
|
raise Error, "#{FILENAME}: #{key} must be true or false"
|
|
89
100
|
end
|
|
90
101
|
value
|
|
102
|
+
when :size
|
|
103
|
+
kb = MemoryCeiling.parse_kb(value)
|
|
104
|
+
raise Error, "#{FILENAME}: #{key} must be a size like 6G, 6144M, or 6144 (MB)" unless kb
|
|
105
|
+
|
|
106
|
+
kb
|
|
91
107
|
when :preload_helper
|
|
92
108
|
return :none if value == false
|
|
93
109
|
raise Error, "#{FILENAME}: preload_helper must be a path or false" unless value.is_a?(String)
|
|
@@ -8,13 +8,16 @@ module ActiveMutator
|
|
|
8
8
|
class CoverageMap
|
|
9
9
|
def self.load(path) = new(JSON.parse(File.read(path)))
|
|
10
10
|
|
|
11
|
-
attr_reader :version, :records
|
|
11
|
+
attr_reader :version, :records, :digests, :spec_paths
|
|
12
12
|
|
|
13
13
|
def initialize(data)
|
|
14
14
|
@version = data["version"]
|
|
15
15
|
@records = data.fetch("records", {})
|
|
16
16
|
@times = data.fetch("times", {})
|
|
17
17
|
@digests = data.fetch("digests", {})
|
|
18
|
+
# A pre-0.4.0 cache predates spec_paths and has no key; treat that as the
|
|
19
|
+
# old implicit default so existing default-config caches stay valid.
|
|
20
|
+
@spec_paths = data.fetch("spec_paths", ["spec"])
|
|
18
21
|
@map = build_map
|
|
19
22
|
end
|
|
20
23
|
|
|
@@ -0,0 +1,25 @@
|
|
|
1
|
+
require "json"
|
|
2
|
+
|
|
3
|
+
module ActiveMutator
|
|
4
|
+
module Diagnostics
|
|
5
|
+
# --events FILE: every event as one JSON object per line. The public,
|
|
6
|
+
# versioned contract (docs/guides/diagnostics.md): new fields may appear
|
|
7
|
+
# under v1; a rename or removal bumps VERSION. The IO is sync, so each
|
|
8
|
+
# line reaches the OS as it's written and a SIGKILLed run still leaves
|
|
9
|
+
# everything up to the kill.
|
|
10
|
+
class Ndjson
|
|
11
|
+
VERSION = 1
|
|
12
|
+
|
|
13
|
+
def initialize(io)
|
|
14
|
+
@io = io
|
|
15
|
+
@io.sync = true
|
|
16
|
+
end
|
|
17
|
+
|
|
18
|
+
def call(event)
|
|
19
|
+
line = { "v" => VERSION, "event" => event.type.to_s,
|
|
20
|
+
"t" => event.at.utc.strftime("%Y-%m-%dT%H:%M:%S.%LZ"), "elapsed" => event.elapsed.round(3) }
|
|
21
|
+
@io.write("#{JSON.generate(line.merge(event.fields.transform_keys(&:to_s)))}\n")
|
|
22
|
+
end
|
|
23
|
+
end
|
|
24
|
+
end
|
|
25
|
+
end
|
|
@@ -0,0 +1,110 @@
|
|
|
1
|
+
module ActiveMutator
|
|
2
|
+
module Diagnostics
|
|
3
|
+
# Human sizes for diagnostic lines: 812M, 1.6G. nil (no reading) is "?".
|
|
4
|
+
def self.size_kb(kb)
|
|
5
|
+
return "?" unless kb
|
|
6
|
+
return "0" if kb.zero?
|
|
7
|
+
return format("%.1fG", kb / 1_048_576.0) if kb >= 1_048_576
|
|
8
|
+
return "#{(kb / 1024.0).round}M" if kb >= 1024
|
|
9
|
+
|
|
10
|
+
"#{kb}K"
|
|
11
|
+
end
|
|
12
|
+
|
|
13
|
+
# --diagnostics: one human-readable line per event, on stderr. stdout
|
|
14
|
+
# belongs to the reporter (`--format json` must stay parseable). `only`
|
|
15
|
+
# limits it to some event types.
|
|
16
|
+
class Text
|
|
17
|
+
def initialize(root:, out: $stderr, only: nil)
|
|
18
|
+
@prefix = "#{root.chomp("/")}/"
|
|
19
|
+
@out = out
|
|
20
|
+
@only = only
|
|
21
|
+
end
|
|
22
|
+
|
|
23
|
+
def call(event)
|
|
24
|
+
return if @only && !@only.include?(event.type)
|
|
25
|
+
|
|
26
|
+
stamp = "[active_mutator #{event.at.strftime("%H:%M:%S")} +#{format("%.1f", event.elapsed)}s]"
|
|
27
|
+
@out.puts "#{stamp} #{body(event.type, event.fields)}"
|
|
28
|
+
end
|
|
29
|
+
|
|
30
|
+
private
|
|
31
|
+
|
|
32
|
+
def body(type, fields)
|
|
33
|
+
case type
|
|
34
|
+
when :phase_start, :phase_end then phase(type, fields)
|
|
35
|
+
when :mutant_start then mutant_start(fields)
|
|
36
|
+
when :mutant_end then mutant_end(fields)
|
|
37
|
+
when :memory then memory(fields)
|
|
38
|
+
when :abort then abort(fields)
|
|
39
|
+
when :memory_warning then "warn #{ceiling(fields)}"
|
|
40
|
+
when :memory_ceiling then "#{ceiling(fields)}; stopping the run"
|
|
41
|
+
else [type, *pairs(fields)].join(" ")
|
|
42
|
+
end
|
|
43
|
+
end
|
|
44
|
+
|
|
45
|
+
def phase(type, fields)
|
|
46
|
+
edge = type == :phase_start ? "start" : "end"
|
|
47
|
+
["phase", fields[:phase], edge, *pairs(fields.except(:phase))].join(" ")
|
|
48
|
+
end
|
|
49
|
+
|
|
50
|
+
def mutant_start(f)
|
|
51
|
+
"mutant start ##{f[:seq]} pid=#{f[:pid]} #{f[:lane]} #{f[:subject]} " \
|
|
52
|
+
"#{f[:file].delete_prefix(@prefix)}:#{f[:line]} #{f[:description]}"
|
|
53
|
+
end
|
|
54
|
+
|
|
55
|
+
def mutant_end(f)
|
|
56
|
+
"mutant end ##{f[:seq]} #{f[:status]} #{format("%.1f", f[:seconds])}s peak=#{Diagnostics.size_kb(f[:peak_rss_kb])}"
|
|
57
|
+
end
|
|
58
|
+
|
|
59
|
+
def abort(f)
|
|
60
|
+
running = f[:in_flight].map { |m| "##{m[:seq]} #{m[:subject]} #{m[:file].delete_prefix(@prefix)}:#{m[:line]}" }
|
|
61
|
+
"abort #{f[:reason]}; in flight: #{running.empty? ? "none" : running.join(", ")}"
|
|
62
|
+
end
|
|
63
|
+
|
|
64
|
+
# memory at 91% of --max-rss 6.0G (5.5G), or before a coverage parse:
|
|
65
|
+
# memory at 150% of --max-rss 6.0G (9.0G estimated to read a 1.7G coverage.json)
|
|
66
|
+
def ceiling(f)
|
|
67
|
+
percent = (f[:total_pss_kb] * 100.0 / f[:max_rss_kb]).round
|
|
68
|
+
total = Diagnostics.size_kb(f[:total_pss_kb])
|
|
69
|
+
if f[:coverage_bytes]
|
|
70
|
+
total += " estimated to read a #{Diagnostics.size_kb(f[:coverage_bytes] / 1024)} coverage.json"
|
|
71
|
+
end
|
|
72
|
+
"memory at #{percent}% of --max-rss #{Diagnostics.size_kb(f[:max_rss_kb])} (#{total})"
|
|
73
|
+
end
|
|
74
|
+
|
|
75
|
+
# mem parent=1.6G workers=4:3.2G baseline=2.1G total=6.9G avail=3.0G swap=0 psi=0.3 load=1.52
|
|
76
|
+
def memory(f)
|
|
77
|
+
parts = ["mem", "parent=#{size(f[:parent])}"]
|
|
78
|
+
parts << "workers=#{f[:workers].size}:#{Diagnostics.size_kb(f[:workers].sum { |w| kb(w) })}" if f[:workers].any?
|
|
79
|
+
parts << "baseline=#{size(f[:baseline])}" if f[:baseline]
|
|
80
|
+
parts << "total=#{Diagnostics.size_kb(f[:total_pss_kb])}"
|
|
81
|
+
parts.concat(system(f[:system])) if f[:system]
|
|
82
|
+
parts.join(" ")
|
|
83
|
+
end
|
|
84
|
+
|
|
85
|
+
def system(s)
|
|
86
|
+
parts = ["avail=#{Diagnostics.size_kb(s[:mem_available_kb])}"]
|
|
87
|
+
parts << "swap=#{Diagnostics.size_kb(s[:swap_total_kb] - s[:swap_free_kb])}" if s[:swap_total_kb] && s[:swap_free_kb]
|
|
88
|
+
parts << "psi=#{s[:psi_some_avg10]}" if s[:psi_some_avg10]
|
|
89
|
+
parts << "load=#{s[:load1]}" if s[:load1]
|
|
90
|
+
parts
|
|
91
|
+
end
|
|
92
|
+
|
|
93
|
+
# Not endless defs: a one-line method's line runs at
|
|
94
|
+
# load, so coverage can't tie it to the specs that call it.
|
|
95
|
+
def size(reading)
|
|
96
|
+
Diagnostics.size_kb(reading && kb(reading))
|
|
97
|
+
end
|
|
98
|
+
|
|
99
|
+
def kb(reading)
|
|
100
|
+
reading[:pss_kb] || reading[:rss_kb]
|
|
101
|
+
end
|
|
102
|
+
|
|
103
|
+
def pairs(fields)
|
|
104
|
+
fields.map do |key, value|
|
|
105
|
+
key == :bytes ? "size=#{Diagnostics.size_kb((value / 1024.0).round)}" : "#{key}=#{value}"
|
|
106
|
+
end
|
|
107
|
+
end
|
|
108
|
+
end
|
|
109
|
+
end
|
|
110
|
+
end
|