mutineer 0.11.0 → 0.11.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +38 -0
- data/README.md +39 -4
- data/lib/mutineer/cli.rb +68 -75
- data/lib/mutineer/config.rb +21 -10
- data/lib/mutineer/daemon_server.rb +21 -8
- data/lib/mutineer/external_backend.rb +132 -12
- data/lib/mutineer/isolation.rb +18 -8
- data/lib/mutineer/mutators/statement_removal.rb +12 -10
- data/lib/mutineer/rails_worker_db.rb +6 -30
- data/lib/mutineer/reporter.rb +45 -4
- data/lib/mutineer/runner.rb +49 -50
- data/lib/mutineer/version.rb +1 -1
- metadata +1 -1
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 457a9cdee5d5291635fb56dd01caf17293bc500443ca2e43e298a81c0c1f73bf
|
|
4
|
+
data.tar.gz: ffe7d126be066573d5c9c42969f994e7a9be71ecb8d301643a6deab3de5d7af3
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: 63e9df1191cdc62802dcd9dd674eb2954c416b8016017aa3bb9e689f777859f547450416e4a62f23b34f9c9171b063ff12a6639e3cda1ceb9180c48bbea673df
|
|
7
|
+
data.tar.gz: 0ace3fd9df09a67b535d55ee25f0e30d60f1dd3f03ca1027600cc216d11d9dd468f320e9b11190d595e204d6310bfe05f61ec80e6eeb33f36c123e0981ed1810
|
data/CHANGELOG.md
CHANGED
|
@@ -6,6 +6,42 @@ All notable changes to this project are documented here. The format is based on
|
|
|
6
6
|
|
|
7
7
|
## [Unreleased]
|
|
8
8
|
|
|
9
|
+
## [0.11.2] - 2026-07-24
|
|
10
|
+
|
|
11
|
+
### Fixed
|
|
12
|
+
- **`--test-command` under version managers** — scrub Mutineer's rbenv/asdf
|
|
13
|
+
version bins and bundler/gem env from the child so the suite can resolve the
|
|
14
|
+
app's Ruby via shims / `.ruby-version`. Smoke check prints a targeted hint on
|
|
15
|
+
`Bundler::RubyVersionMismatch` instead of only blaming DB/migrations. Docs
|
|
16
|
+
cover a wrapper recipe for stubborn setups (#32).
|
|
17
|
+
|
|
18
|
+
## [0.11.1] - 2026-07-23
|
|
19
|
+
|
|
20
|
+
### Fixed
|
|
21
|
+
- **`--daemon` score disclosure** — stop claiming “no coverage narrowing / lower
|
|
22
|
+
bound” on every run (narrowing already ships). Warn only when the coverage map
|
|
23
|
+
is unavailable and the run falls back to the full `--test` set (#48).
|
|
24
|
+
- **`--threshold` CI fidelity** — exit 1 when a positive threshold is set but the
|
|
25
|
+
run produced only errors/timeouts/uncapturable (nil score with a broken harness)
|
|
26
|
+
instead of silently green; pure no_coverage / all-ignored still skips the gate
|
|
27
|
+
(#49). Non-numeric `--threshold` / YAML `threshold` is a usage error (exit 2),
|
|
28
|
+
not a silent gate-off (#51).
|
|
29
|
+
- **`--daemon` contracts** — reject `--framework rspec` (exit 2); force
|
|
30
|
+
`--strategy reload` with a clear warning when redefine was requested (daemon
|
|
31
|
+
whole-file only) (#50). **In-process `--rails` always serial** unless
|
|
32
|
+
`--daemon` (only daemon has per-worker DB isolation) (#55).
|
|
33
|
+
- **Isolation timeout** — kill the child process group and honor a clean exit
|
|
34
|
+
that races the deadline (not always timeout) (#52). **External backend** —
|
|
35
|
+
signal death is `error`, not `killed` (#53).
|
|
36
|
+
- **`--fail-fast` is serial** on the in-process path (matches daemon) so the
|
|
37
|
+
survivor set is deterministic under any `--jobs` (#54).
|
|
38
|
+
- **Dry-run uses `collect_jobs`** so candidates match a real run (#59).
|
|
39
|
+
- **Daemon schema load once per worker** (not every mutant fork) (#56).
|
|
40
|
+
- **Nested `statement_removal`** visits inner statement lists (#62).
|
|
41
|
+
- Dead seams: remove unused `RailsWorkerDb.provision`, simplify parallel daemon
|
|
42
|
+
fail-fast branch, drop unused `validate_daemon!` arg (#60).
|
|
43
|
+
- Archive historical implementation spec under `docs/archive/` (#61).
|
|
44
|
+
|
|
9
45
|
## [0.11.0] - 2026-07-02
|
|
10
46
|
|
|
11
47
|
### Added
|
|
@@ -217,6 +253,8 @@ Rails hardening + CI batch (issues #8–#13), all verified Rails-free.
|
|
|
217
253
|
- `.mutineer.yml` configuration (CLI > config > default precedence).
|
|
218
254
|
- Byte-correct source handling for multibyte (UTF-8) sources.
|
|
219
255
|
|
|
256
|
+
[0.11.2]: https://github.com/davidteren/mutineer/releases/tag/v0.11.2
|
|
257
|
+
[0.11.1]: https://github.com/davidteren/mutineer/releases/tag/v0.11.1
|
|
220
258
|
[0.11.0]: https://github.com/davidteren/mutineer/releases/tag/v0.11.0
|
|
221
259
|
[0.10.0]: https://github.com/davidteren/mutineer/releases/tag/v0.10.0
|
|
222
260
|
[0.9.1]: https://github.com/davidteren/mutineer/releases/tag/v0.9.1
|
data/README.md
CHANGED
|
@@ -151,11 +151,46 @@ RAILS_ENV=test mutineer run app/models/order.rb \
|
|
|
151
151
|
|
|
152
152
|
- **`%{files}`** is required; it expands to the `--test` paths as separate
|
|
153
153
|
arguments (a path with a space stays one argument — there is no shell).
|
|
154
|
-
- **Environment
|
|
155
|
-
command
|
|
156
|
-
|
|
154
|
+
- **Environment:** vars like `RAILS_ENV` / `DATABASE_URL` set on the Mutineer
|
|
155
|
+
command are inherited. Mutineer **unsets** `BUNDLE_*`, `GEM_*`, `RUBY*`,
|
|
156
|
+
`RBENV_VERSION`, `ASDF_RUBY_VERSION`, and `RBENV_DIR` in the child (so
|
|
157
|
+
Mutineer's own Ruby cannot pin the suite), and drops version-manager
|
|
158
|
+
**version bins** (e.g. `~/.rbenv/versions/3.4.x/bin`) from `PATH`, then
|
|
159
|
+
prepends rbenv/asdf shims when a pin was scrubbed. Do not rely on
|
|
160
|
+
`RBENV_VERSION=…` on the Mutineer command for the suite; use `.ruby-version`
|
|
161
|
+
or a wrapper. Don't put `KEY=val` prefixes *inside* `--test-command` (no shell;
|
|
162
|
+
that would be treated as the program name).
|
|
163
|
+
|
|
164
|
+
#### Under a version manager (rbenv / asdf / chruby)
|
|
165
|
+
|
|
166
|
+
Automatic scrub targets **rbenv** and **asdf** (shims + version bins). **chruby**
|
|
167
|
+
has no shims: Mutineer still strips `…/rubies/…/bin` so it cannot leave Mutineer's
|
|
168
|
+
Ruby pinned, but you need a wrapper that sources chruby and selects the app
|
|
169
|
+
version. If the smoke check still reports a **Ruby version mismatch**, wrap the
|
|
170
|
+
suite. Example rbenv wrapper (`bin/mutineer-test` in the app):
|
|
157
171
|
|
|
158
|
-
|
|
172
|
+
```sh
|
|
173
|
+
#!/usr/bin/env bash
|
|
174
|
+
set -euo pipefail
|
|
175
|
+
cd "$(dirname "$0")/.."
|
|
176
|
+
unset GEM_HOME GEM_PATH RUBYLIB RUBYOPT BUNDLE_GEMFILE BUNDLE_BIN_PATH BUNDLER_VERSION
|
|
177
|
+
export RBENV_VERSION="$(cat .ruby-version 2>/dev/null || true)"
|
|
178
|
+
export RAILS_ENV="${RAILS_ENV:-test}"
|
|
179
|
+
export PATH="${HOME}/.rbenv/shims:${PATH}"
|
|
180
|
+
exec bundle exec rails test "$@"
|
|
181
|
+
```
|
|
182
|
+
|
|
183
|
+
```sh
|
|
184
|
+
# Mutineer on 3.4+; suite on the app's Ruby via the wrapper
|
|
185
|
+
mutineer run app/models/order.rb \
|
|
186
|
+
--test test/models/order_test.rb \
|
|
187
|
+
--test-command "bin/mutineer-test %{files}"
|
|
188
|
+
```
|
|
189
|
+
|
|
190
|
+
Mutineer also surfaces a targeted smoke-check message when Bundler prints
|
|
191
|
+
`RubyVersionMismatch`, instead of only blaming DB/migrations.
|
|
192
|
+
|
|
193
|
+
Tradeoffs — this path is correct but not free:
|
|
159
194
|
|
|
160
195
|
- **Slower:** your app re-boots for every mutant (no shared boot yet).
|
|
161
196
|
- **No coverage narrowing:** every mutant runs the *full* `--test` set, so the
|
data/lib/mutineer/cli.rb
CHANGED
|
@@ -49,8 +49,8 @@ module Mutineer
|
|
|
49
49
|
--rails Sugar for --boot config/environment --strategy redefine
|
|
50
50
|
--test-command CMD Run the target suite in the app's own runtime as a
|
|
51
51
|
subprocess (for apps on Ruby < 3.4). CMD must contain
|
|
52
|
-
%{files}
|
|
53
|
-
|
|
52
|
+
%{files}. Scrubs Mutineer Ruby PATH pins; set RAILS_ENV
|
|
53
|
+
on the mutineer command (not as KEY=val inside CMD)
|
|
54
54
|
--daemon Boot the app ONCE in a persistent daemon and fork per
|
|
55
55
|
mutant, with per-worker DB isolation so --jobs N is safe
|
|
56
56
|
under Rails (needs --rails/--boot; not with --test-command)
|
|
@@ -93,12 +93,20 @@ module Mutineer
|
|
|
93
93
|
end
|
|
94
94
|
o.on("--list-operators") { show_operators = true }
|
|
95
95
|
o.on("--dry-run") { opts[:dry_run] = true }
|
|
96
|
-
o.on("--fail-fast") { opts[:fail_fast] = true }
|
|
96
|
+
o.on("--fail-fast") { opts[:fail_fast] = true; explicit << :fail_fast }
|
|
97
97
|
o.on("--only NAME") { |v| opts[:only] = v; explicit << :only }
|
|
98
98
|
o.on("--since REF") { |v| opts[:since] = v; explicit << :since }
|
|
99
99
|
o.on("--test FILE") { |v| (opts[:tests] ||= []) << v }
|
|
100
100
|
o.on("--operators LIST") { |v| opts[:operators] = v.split(",").map(&:strip); explicit << :operators }
|
|
101
|
-
o.on("--threshold FLOAT")
|
|
101
|
+
o.on("--threshold FLOAT") do |v|
|
|
102
|
+
f = Float(v, exception: false)
|
|
103
|
+
if f.nil?
|
|
104
|
+
warn "mutineer: --threshold requires a number between 0 and 100 (got: #{v.inspect})"
|
|
105
|
+
exit 2
|
|
106
|
+
end
|
|
107
|
+
opts[:threshold] = f
|
|
108
|
+
explicit << :threshold
|
|
109
|
+
end
|
|
102
110
|
o.on("--jobs N") { |v| opts[:jobs] = v; explicit << :jobs }
|
|
103
111
|
o.on("--strategy STRAT") { |v| opts[:strategy] = v; explicit << :strategy }
|
|
104
112
|
o.on("--framework NAME") { |v| opts[:framework] = v; explicit << :framework }
|
|
@@ -248,21 +256,21 @@ module Mutineer
|
|
|
248
256
|
end
|
|
249
257
|
|
|
250
258
|
validate_test_command!(config) if config.test_command
|
|
251
|
-
validate_daemon!(config, explicit) if config.daemon
|
|
252
259
|
|
|
253
260
|
validate_since!(config) if config.since
|
|
254
261
|
preflight_output!(config.output) if config.output
|
|
255
262
|
preflight_baseline!(config.baseline) if config.baseline
|
|
256
263
|
|
|
257
|
-
#
|
|
258
|
-
#
|
|
259
|
-
# populated config.tests. Runs after every flag/usage check above so a
|
|
260
|
-
# mistyped flag still reports the flag; skipped under --dry-run (no tests
|
|
261
|
-
# needed). validate_paths! then sees the inferred (real) tests.
|
|
264
|
+
# When --test is omitted, infer each source's test by convention. Autopair
|
|
265
|
+
# also re-detects framework from inferred tests when --framework was not set.
|
|
262
266
|
autopair!(config, explicit) unless config.dry_run
|
|
263
267
|
|
|
264
|
-
#
|
|
265
|
-
#
|
|
268
|
+
# Daemon validation runs AFTER autopair so auto-inferred *_spec.rb tests
|
|
269
|
+
# cannot bypass the RSpec rejection (framework would still be minitest if we
|
|
270
|
+
# validated before discovery).
|
|
271
|
+
validate_daemon!(config) if config.daemon
|
|
272
|
+
|
|
273
|
+
# Boot mode needs at least one --test file (nothing to select from otherwise).
|
|
266
274
|
if config.boot && config.tests.empty?
|
|
267
275
|
warn "mutineer: --boot/--rails requires at least one --test file"
|
|
268
276
|
exit 2
|
|
@@ -307,28 +315,34 @@ module Mutineer
|
|
|
307
315
|
config.jobs = 1
|
|
308
316
|
end
|
|
309
317
|
|
|
310
|
-
#
|
|
311
|
-
#
|
|
312
|
-
#
|
|
313
|
-
#
|
|
314
|
-
# Config.resolve already keeps --jobs serial unless the user explicitly asks for
|
|
315
|
-
# parallelism, and each worker gets its own SQLite database on demand — so there is
|
|
316
|
-
# no "missing worker DB" precondition to check here for SQLite. Postgres worker-DB
|
|
317
|
-
# provisioning + its missing-DB error (KTD-9) arrives with the Postgres adapter (U10).
|
|
318
|
+
# --daemon selects the persistent-daemon backend (boot once, fork per mutant,
|
|
319
|
+
# per-worker DB isolation on SQLite). Usage errors exit 2: cannot combine with
|
|
320
|
+
# --test-command; requires --rails or --boot; minitest only; reload strategy only
|
|
321
|
+
# (redefine needs a shared VM surgical path the daemon does not ship).
|
|
318
322
|
#
|
|
319
323
|
# @api private
|
|
320
324
|
# @param config [Mutineer::Config] run configuration.
|
|
321
|
-
# @param explicit [Set<Symbol>] explicit CLI fields.
|
|
322
325
|
# @return [void]
|
|
323
|
-
def self.validate_daemon!(config
|
|
326
|
+
def self.validate_daemon!(config)
|
|
324
327
|
if config.test_command
|
|
325
328
|
warn "mutineer: choose one backend — --daemon and --test-command cannot be combined"
|
|
326
329
|
exit 2
|
|
327
330
|
end
|
|
328
|
-
|
|
331
|
+
unless config.rails || config.boot
|
|
332
|
+
warn "mutineer: --daemon needs an app to boot; add --rails (or --boot FILE)"
|
|
333
|
+
exit 2
|
|
334
|
+
end
|
|
335
|
+
if config.framework == "rspec"
|
|
336
|
+
warn "mutineer: --daemon supports only --framework minitest " \
|
|
337
|
+
"(rspec is not implemented on the daemon path yet)"
|
|
338
|
+
exit 2
|
|
339
|
+
end
|
|
340
|
+
return if config.strategy == "reload"
|
|
329
341
|
|
|
330
|
-
|
|
331
|
-
|
|
342
|
+
# --rails defaults strategy to redefine; daemon always whole-file loads.
|
|
343
|
+
warn "[mutineer] --daemon uses --strategy reload " \
|
|
344
|
+
"(redefine is not supported on the daemon path); forcing reload."
|
|
345
|
+
config.strategy = "reload"
|
|
332
346
|
end
|
|
333
347
|
|
|
334
348
|
# --since needs a real git repo and a resolvable ref; either failure is a
|
|
@@ -458,26 +472,18 @@ module Mutineer
|
|
|
458
472
|
reporter.report(out: $stdout, err: $stderr, threshold: config.threshold,
|
|
459
473
|
format: config.format, output: config.output, baseline: delta)
|
|
460
474
|
|
|
461
|
-
#
|
|
462
|
-
#
|
|
463
|
-
#
|
|
464
|
-
#
|
|
475
|
+
# Warn (stderr, so it never pollutes json/html) that an external run's score
|
|
476
|
+
# is not comparable to an in-process run: no coverage narrowing (uncovered
|
|
477
|
+
# mutants count as survivors), and an infra failure is scored as a kill
|
|
478
|
+
# (upper bound). Daemon coverage fallback warnings are emitted from the runner
|
|
479
|
+
# only when the map is unavailable — not on every --daemon run.
|
|
465
480
|
if config.test_command
|
|
466
481
|
warn "[mutineer] --test-command score is an upper bound, not comparable to an " \
|
|
467
482
|
"in-process run: no coverage narrowing (uncovered mutants count as survivors) " \
|
|
468
483
|
"and an infra failure is scored as a kill."
|
|
469
484
|
end
|
|
470
485
|
|
|
471
|
-
#
|
|
472
|
-
# coverage narrowing yet (Phase 2c), so this score is a lower bound (uncovered
|
|
473
|
-
# mutants count as survivors) and not comparable to an in-process run.
|
|
474
|
-
if config.daemon
|
|
475
|
-
warn "[mutineer] --daemon score is a lower bound, not comparable to an " \
|
|
476
|
-
"in-process run: no coverage narrowing yet (uncovered mutants count as " \
|
|
477
|
-
"survivors)."
|
|
478
|
-
end
|
|
479
|
-
|
|
480
|
-
# #14: nudge toward the opt-in tier-2 operators (human report only — never
|
|
486
|
+
# Nudge toward the opt-in tier-2 operators (human report only — never
|
|
481
487
|
# pollute JSON output).
|
|
482
488
|
if !%w[json html].include?(config.format) && (hint = tier2_hint(config.operators))
|
|
483
489
|
puts hint
|
|
@@ -504,50 +510,37 @@ module Mutineer
|
|
|
504
510
|
"enable with --operators <list>."
|
|
505
511
|
end
|
|
506
512
|
|
|
507
|
-
# Runs dry-run mode.
|
|
513
|
+
# Runs dry-run mode. Reuses Runner.collect_jobs (+ filter_since) so the
|
|
514
|
+
# candidate list cannot drift from a real run's job selection.
|
|
508
515
|
#
|
|
509
516
|
# @param config [Mutineer::Config] run configuration.
|
|
510
517
|
# @return [void]
|
|
511
518
|
def self.dry_run(config)
|
|
512
519
|
operator_classes = MutatorRegistry.resolve(config.operators || MutatorRegistry::DEFAULT_NAMES)
|
|
513
|
-
|
|
520
|
+
jobs, ignored_results, source_map = Runner.collect_jobs(config, operator_classes)
|
|
521
|
+
# Narrow jobs and ignored the same way so the summary matches the printed list.
|
|
522
|
+
if config.since
|
|
523
|
+
jobs = Runner.filter_since(jobs, source_map, config)
|
|
524
|
+
ignored_jobs = ignored_results.map { |r| [r.subject, r.mutation, r.id] }
|
|
525
|
+
ignored = Runner.filter_since(ignored_jobs, source_map, config).size
|
|
526
|
+
else
|
|
527
|
+
ignored = ignored_results.size
|
|
528
|
+
end
|
|
529
|
+
|
|
514
530
|
per_operator = Hash.new(0)
|
|
515
531
|
skipped = 0
|
|
516
|
-
|
|
517
|
-
|
|
518
|
-
|
|
519
|
-
|
|
520
|
-
|
|
521
|
-
changed = if config.since
|
|
522
|
-
ChangedLines.for(ref: config.since, files: config.sources,
|
|
523
|
-
project_root: config.project_root)
|
|
524
|
-
end
|
|
525
|
-
|
|
526
|
-
Project.discover(config.sources, only: config.only).each do |subject|
|
|
527
|
-
source = (sources[subject.file] ||= Parser.parse_file(subject.file).source.source)
|
|
528
|
-
# #22: honor suppression so the preview matches what a real run mutates.
|
|
529
|
-
# Mirror execute's per-subject shape (ids need the full mutation list).
|
|
530
|
-
disabled = Runner.suppress_map(source)
|
|
531
|
-
mutations = operator_classes.flat_map { |klass| klass.new.mutations_for(subject, source) }
|
|
532
|
-
ids = MutantId.for_subject(subject, source, mutations)
|
|
533
|
-
mutations.each_with_index do |mutation, i|
|
|
534
|
-
unless mutation.valid?(source)
|
|
535
|
-
skipped += 1
|
|
536
|
-
next
|
|
537
|
-
end
|
|
538
|
-
line = source.byteslice(0, mutation.start_offset).count("\n") + 1
|
|
539
|
-
next if changed && !changed[File.expand_path(subject.file, config.project_root)]&.include?(line)
|
|
540
|
-
|
|
541
|
-
if Runner.suppressed?(mutation.operator, line, ids[i], disabled, ignore_set)
|
|
542
|
-
ignored += 1
|
|
543
|
-
next
|
|
544
|
-
end
|
|
545
|
-
|
|
546
|
-
per_operator[mutation.operator] += 1
|
|
547
|
-
original = source.byteslice(mutation.start_offset...mutation.end_offset)
|
|
548
|
-
puts "[#{mutation.operator}] #{subject.qualified_name} " \
|
|
549
|
-
"#{subject.file}:#{line} `#{original}` -> `#{mutation.replacement}`"
|
|
532
|
+
jobs.each do |subject, mutation, _id|
|
|
533
|
+
source = source_map[subject.file]
|
|
534
|
+
unless mutation.valid?(source)
|
|
535
|
+
skipped += 1
|
|
536
|
+
next
|
|
550
537
|
end
|
|
538
|
+
|
|
539
|
+
line = source.byteslice(0, mutation.start_offset).count("\n") + 1
|
|
540
|
+
per_operator[mutation.operator] += 1
|
|
541
|
+
original = source.byteslice(mutation.start_offset...mutation.end_offset)
|
|
542
|
+
puts "[#{mutation.operator}] #{subject.qualified_name} " \
|
|
543
|
+
"#{subject.file}:#{line} `#{original}` -> `#{mutation.replacement}`"
|
|
551
544
|
end
|
|
552
545
|
|
|
553
546
|
total = per_operator.values.sum
|
data/lib/mutineer/config.rb
CHANGED
|
@@ -112,17 +112,21 @@ module Mutineer
|
|
|
112
112
|
file_hash.each { |k, v| merged[k] = v unless explicit.include?(k) }
|
|
113
113
|
config = new(**merged)
|
|
114
114
|
|
|
115
|
-
# --rails sugar: boot config/environment
|
|
116
|
-
#
|
|
117
|
-
#
|
|
115
|
+
# --rails sugar: boot config/environment. Prefer redefine only for the
|
|
116
|
+
# in-process path (daemon is whole-file reload only). In-process --rails
|
|
117
|
+
# shares one test database — force serial unless --daemon.
|
|
118
118
|
if config.rails
|
|
119
119
|
config.boot ||= "config/environment"
|
|
120
|
-
config.
|
|
121
|
-
|
|
122
|
-
|
|
123
|
-
|
|
124
|
-
|
|
125
|
-
|
|
120
|
+
unless config.daemon || explicit.include?(:strategy)
|
|
121
|
+
config.strategy = "redefine"
|
|
122
|
+
end
|
|
123
|
+
unless config.daemon
|
|
124
|
+
if config.jobs.to_i > 1
|
|
125
|
+
warn "[mutineer] --rails without --daemon runs serially (shared test DB); " \
|
|
126
|
+
"forcing --jobs 1. Use --daemon for safe --jobs N."
|
|
127
|
+
end
|
|
128
|
+
config.jobs = 1
|
|
129
|
+
end
|
|
126
130
|
end
|
|
127
131
|
|
|
128
132
|
# Auto-detect the framework only when neither CLI nor config file set it
|
|
@@ -162,13 +166,20 @@ module Mutineer
|
|
|
162
166
|
case known_key
|
|
163
167
|
when "operators" then filter_operators(Array(value).map(&:to_s), file_name)
|
|
164
168
|
when "jobs" then value.to_i
|
|
165
|
-
when "threshold"
|
|
169
|
+
when "threshold"
|
|
170
|
+
f = Float(value, exception: false)
|
|
171
|
+
if f.nil?
|
|
172
|
+
raise ConfigError, "#{file_name}: threshold must be a number between 0 and 100 " \
|
|
173
|
+
"(got: #{value.inspect})"
|
|
174
|
+
end
|
|
175
|
+
f
|
|
166
176
|
when "require" then Array(value).map(&:to_s)
|
|
167
177
|
when "boot" then value.to_s
|
|
168
178
|
when "framework" then value.to_s
|
|
169
179
|
when "rails" then value == true || value.to_s == "true"
|
|
170
180
|
when "daemon" then value == true || value.to_s == "true"
|
|
171
181
|
when "verbose" then value == true || value.to_s == "true"
|
|
182
|
+
when "fail_fast" then value == true || value.to_s == "true"
|
|
172
183
|
when "ignore" then Array(value).map(&:to_s)
|
|
173
184
|
when "baseline" then value.to_s
|
|
174
185
|
when "test_command" then value.to_s
|
|
@@ -127,6 +127,8 @@ module Mutineer
|
|
|
127
127
|
@worker_db = RailsWorkerDb.available? ? RailsWorkerDb : nil
|
|
128
128
|
schema = cfg["schema"] && File.expand_path(cfg["schema"])
|
|
129
129
|
@schema_path = schema if schema && File.exist?(schema)
|
|
130
|
+
# Schema is loaded once per worker slot on first use (not every mutant fork).
|
|
131
|
+
@schema_ready = {}
|
|
130
132
|
rescue LoadError => e
|
|
131
133
|
@errio.puts("[daemon] worker-DB routing unavailable: #{e.message}")
|
|
132
134
|
@worker_db = nil
|
|
@@ -165,18 +167,18 @@ module Mutineer
|
|
|
165
167
|
def run_mutant(req)
|
|
166
168
|
timeout = req.fetch("timeout", 30)
|
|
167
169
|
worker = req.fetch("worker", 0)
|
|
170
|
+
# Load schema until the first killed/survived fork for this worker slot.
|
|
171
|
+
schema_for_fork = (@worker_db && @schema_path && !@schema_ready[worker]) ? @schema_path : nil
|
|
168
172
|
pid = fork do
|
|
169
|
-
# New process group so a per-fork timeout can SIGKILL the whole subtree
|
|
170
|
-
#
|
|
171
|
-
# test-framework output can never corrupt the IPC pipe (KTD-6).
|
|
173
|
+
# New process group so a per-fork timeout can SIGKILL the whole subtree,
|
|
174
|
+
# and silence the child's stdout so test output never corrupts the IPC pipe.
|
|
172
175
|
Process.setpgid(0, 0) rescue nil # rubocop:disable Style/RescueModifier
|
|
173
176
|
$stdout.reopen(File::NULL, "w")
|
|
174
177
|
code =
|
|
175
178
|
begin
|
|
176
|
-
# Route THIS fork at its own worker database before any test loads
|
|
177
|
-
#
|
|
178
|
-
|
|
179
|
-
@worker_db&.after_fork(worker, @schema_path)
|
|
179
|
+
# Route THIS fork at its own worker database before any test loads.
|
|
180
|
+
# A routing failure raises here and is scored `error`, never a false verdict.
|
|
181
|
+
@worker_db&.after_fork(worker, schema_for_fork)
|
|
180
182
|
apply_payload(req["payload"])
|
|
181
183
|
run_tests(Array(req["tests"]))
|
|
182
184
|
rescue Exception => e # rubocop:disable Lint/RescueException
|
|
@@ -186,6 +188,10 @@ module Mutineer
|
|
|
186
188
|
exit!(code)
|
|
187
189
|
end
|
|
188
190
|
verdict = wait_verdict(pid, timeout)
|
|
191
|
+
# Mark ready only when the child finished cleanly after schema load
|
|
192
|
+
# (killed/survived). Timeout can interrupt mid-load_schema; error is a
|
|
193
|
+
# routing failure — both leave the slot unready so the next fork reloads.
|
|
194
|
+
@schema_ready[worker] = true if schema_for_fork && %w[killed survived].include?(verdict)
|
|
189
195
|
# A SIGKILLed timeout child skipped its Tempfile unlink — sweep the orphan so
|
|
190
196
|
# it can't outlive the run or trip Zeitwerk on a later fork.
|
|
191
197
|
sweep_temps if verdict == "timeout"
|
|
@@ -222,7 +228,14 @@ module Mutineer
|
|
|
222
228
|
rescue Errno::ESRCH, Errno::EPERM
|
|
223
229
|
Process.kill(:KILL, pid) rescue nil # rubocop:disable Style/RescueModifier
|
|
224
230
|
end
|
|
225
|
-
|
|
231
|
+
begin
|
|
232
|
+
_reaped, status = Process.waitpid2(pid)
|
|
233
|
+
if status && status.exited? && !status.signaled?
|
|
234
|
+
return { 0 => "survived", 1 => "killed" }.fetch(status.exitstatus, "error")
|
|
235
|
+
end
|
|
236
|
+
rescue Errno::ECHILD
|
|
237
|
+
# already reaped
|
|
238
|
+
end
|
|
226
239
|
return "timeout"
|
|
227
240
|
end
|
|
228
241
|
sleep POLL
|
|
@@ -10,17 +10,21 @@ module Mutineer
|
|
|
10
10
|
# tests. The CLI maps this to a runtime error (exit 1), not a usage error.
|
|
11
11
|
class SmokeCheckError < StandardError; end
|
|
12
12
|
|
|
13
|
-
#
|
|
14
|
-
#
|
|
15
|
-
#
|
|
13
|
+
# External execution backend. Runs the user's `--test-command` as a subprocess
|
|
14
|
+
# in the app's own runtime (whatever Ruby its bundle resolves to), so mutineer
|
|
15
|
+
# (Ruby ≥ 3.4) can mutation-test apps pinned to an older Ruby.
|
|
16
16
|
#
|
|
17
17
|
# This is deliberately NOT a `TestRunners` framework adapter: those return an
|
|
18
18
|
# Integer 0/1 from inside a fork and are dispatched by framework name. This is a
|
|
19
19
|
# whole backend — it spawns a process, enforces a wall-clock timeout, and maps
|
|
20
|
-
# the exit status to a Result. The mapping is the
|
|
20
|
+
# the exit status to a Result. The mapping is the same direction as in-process
|
|
21
21
|
# (suite passes => survived, suite fails => killed) but coarser: it cannot tell an
|
|
22
22
|
# infrastructure error from a genuine kill, so the smoke check (below) guards the
|
|
23
|
-
# persistent case and the score is disclosed as an upper bound
|
|
23
|
+
# persistent case and the score is disclosed as an upper bound.
|
|
24
|
+
#
|
|
25
|
+
# Child environment: bundler/gem env injected by Mutineer's Ruby is stripped, and
|
|
26
|
+
# version-manager concrete version bins (e.g. `~/.rbenv/versions/3.4.9/bin`) are
|
|
27
|
+
# removed from PATH so shims / `.ruby-version` can select the app's Ruby (#32).
|
|
24
28
|
module ExternalBackend
|
|
25
29
|
# Generous ceiling for the one-off smoke/calibration run (a cold app boot plus
|
|
26
30
|
# the full suite). The per-mutant timeout is derived from how long this took.
|
|
@@ -29,6 +33,24 @@ module Mutineer
|
|
|
29
33
|
# this backend waits on an external process TREE, not an in-process fork.
|
|
30
34
|
POLL = 0.02
|
|
31
35
|
|
|
36
|
+
# PATH entries that pin a concrete Ruby under a version manager (ahead of
|
|
37
|
+
# shims). Optional trailing slash; normal bins are left alone.
|
|
38
|
+
VERSION_BIN_PATH = %r{
|
|
39
|
+
(?:
|
|
40
|
+
/\.rbenv/versions/[^/]+/bin
|
|
41
|
+
|/\.asdf/installs/ruby/[^/]+/bin
|
|
42
|
+
|/\.rubies/[^/]+/bin
|
|
43
|
+
|/rubies/[^/]+/bin
|
|
44
|
+
)/?\z
|
|
45
|
+
}x
|
|
46
|
+
|
|
47
|
+
# Env keys Process.spawn must unset (nil value) so Mutineer's Ruby/bundler
|
|
48
|
+
# context does not leak. Spawn merges env onto the parent; omitting a key
|
|
49
|
+
# does NOT clear it.
|
|
50
|
+
CLEAR_ENV_KEYS = %w[
|
|
51
|
+
BUNDLER_VERSION RBENV_VERSION ASDF_RUBY_VERSION RBENV_DIR
|
|
52
|
+
].freeze
|
|
53
|
+
|
|
32
54
|
# Turn a command template into an argv array (no shell → no eval, no
|
|
33
55
|
# injection). The `%{files}` token expands IN PLACE to N separate argv
|
|
34
56
|
# elements — one per path, unescaped — so a path containing a space stays a
|
|
@@ -41,10 +63,63 @@ module Mutineer
|
|
|
41
63
|
Shellwords.split(command).flat_map { |tok| tok == "%{files}" ? files : [tok] }
|
|
42
64
|
end
|
|
43
65
|
|
|
66
|
+
# Environment delta for the test-command child. Process.spawn merges this
|
|
67
|
+
# onto the parent env: set a key to +nil+ to unset it. Scrubs Mutineer/bundler
|
|
68
|
+
# injection and version-manager PATH pins so shims / `.ruby-version` can win.
|
|
69
|
+
# Keeps other vars (e.g. `RAILS_ENV`) so setting them on the Mutineer command
|
|
70
|
+
# still reaches the suite.
|
|
71
|
+
#
|
|
72
|
+
# @api private
|
|
73
|
+
# @return [Hash{String => String, nil}] env for Process.spawn.
|
|
74
|
+
def self.child_env
|
|
75
|
+
env = {}
|
|
76
|
+
cleared_pin = false
|
|
77
|
+
ENV.each_key do |k|
|
|
78
|
+
next unless clear_env_key?(k)
|
|
79
|
+
|
|
80
|
+
env[k] = nil
|
|
81
|
+
cleared_pin = true
|
|
82
|
+
end
|
|
83
|
+
path = ENV["PATH"].to_s.split(File::PATH_SEPARATOR)
|
|
84
|
+
kept = path.reject { |p| p.match?(VERSION_BIN_PATH) }
|
|
85
|
+
scrubbed_bin = kept.size != path.size
|
|
86
|
+
# Only reorder PATH when we scrubbed a pin; otherwise leave the user's PATH alone.
|
|
87
|
+
path = if cleared_pin || scrubbed_bin
|
|
88
|
+
(version_manager_shim_dirs + kept).uniq
|
|
89
|
+
else
|
|
90
|
+
kept
|
|
91
|
+
end
|
|
92
|
+
env["PATH"] = path.join(File::PATH_SEPARATOR)
|
|
93
|
+
env
|
|
94
|
+
end
|
|
95
|
+
|
|
96
|
+
# True when the key must be unset in the child (bundler/gem/version-manager).
|
|
97
|
+
#
|
|
98
|
+
# @api private
|
|
99
|
+
# @param key [String] environment variable name.
|
|
100
|
+
# @return [Boolean]
|
|
101
|
+
def self.clear_env_key?(key)
|
|
102
|
+
key.start_with?("BUNDLE_", "RUBY", "GEM_") || CLEAR_ENV_KEYS.include?(key)
|
|
103
|
+
end
|
|
104
|
+
|
|
105
|
+
# Existing shim directories for rbenv / asdf (chruby uses PATH without shims).
|
|
106
|
+
#
|
|
107
|
+
# @api private
|
|
108
|
+
# @return [Array<String>] absolute shim paths that exist.
|
|
109
|
+
def self.version_manager_shim_dirs
|
|
110
|
+
home = ENV["HOME"]
|
|
111
|
+
return [] if home.nil? || home.empty?
|
|
112
|
+
|
|
113
|
+
[
|
|
114
|
+
File.join(home, ".rbenv", "shims"),
|
|
115
|
+
File.join(home, ".asdf", "shims")
|
|
116
|
+
].select { |d| File.directory?(d) }
|
|
117
|
+
end
|
|
118
|
+
|
|
44
119
|
# Runs the command for ONE mutant against whatever is currently on disk (the
|
|
45
120
|
# caller has already swapped the mutant in via FileSwap). Maps the outcome to a
|
|
46
|
-
# Result.
|
|
47
|
-
#
|
|
121
|
+
# Result. Uses {child_env} so version-manager PATH pins from Mutineer's Ruby do
|
|
122
|
+
# not force the suite onto the wrong interpreter.
|
|
48
123
|
#
|
|
49
124
|
# @param command [String] the --test-command template.
|
|
50
125
|
# @param files [Array<String>] test file paths.
|
|
@@ -62,6 +137,12 @@ module Mutineer
|
|
|
62
137
|
Result.timeout
|
|
63
138
|
else # :exited
|
|
64
139
|
return Result.survived if code&.zero?
|
|
140
|
+
# Signal death (nil exitstatus) is infrastructure, not a suite assertion
|
|
141
|
+
# failure — match Isolation/daemon (error), do not inflate kill rate.
|
|
142
|
+
if code.nil?
|
|
143
|
+
warn output if verbose && !output.empty?
|
|
144
|
+
return Result.error("test-command terminated by signal")
|
|
145
|
+
end
|
|
65
146
|
|
|
66
147
|
warn output if verbose && !output.empty?
|
|
67
148
|
Result.killed
|
|
@@ -87,10 +168,42 @@ module Mutineer
|
|
|
87
168
|
else "exited #{code}"
|
|
88
169
|
end
|
|
89
170
|
detail = output.empty? ? "" : "\n--- last output ---\n#{tail(output)}"
|
|
171
|
+
hint = ruby_version_mismatch_hint(output)
|
|
90
172
|
raise SmokeCheckError,
|
|
91
|
-
"the test command #{reason} against the UNMUTATED source —
|
|
92
|
-
"
|
|
93
|
-
|
|
173
|
+
"the test command #{reason} against the UNMUTATED source — " \
|
|
174
|
+
"#{hint || generic_env_hint}.#{detail}"
|
|
175
|
+
end
|
|
176
|
+
|
|
177
|
+
# Generic smoke-failure framing when no Ruby-version mismatch is detected.
|
|
178
|
+
#
|
|
179
|
+
# @api private
|
|
180
|
+
# @return [String]
|
|
181
|
+
def self.generic_env_hint
|
|
182
|
+
"the unmutated suite is not green (failing tests, or a broken environment: " \
|
|
183
|
+
"DB, RAILS_ENV, migrations) — fix that before scoring"
|
|
184
|
+
end
|
|
185
|
+
|
|
186
|
+
# Targeted hint when Bundler reports a RubyVersionMismatch (common under
|
|
187
|
+
# rbenv/asdf/chruby when Mutineer's version bin sits ahead of shims on PATH).
|
|
188
|
+
#
|
|
189
|
+
# @api private
|
|
190
|
+
# @param output [String] captured child stdout+stderr.
|
|
191
|
+
# @return [String, nil] hint text, or nil when not a version mismatch.
|
|
192
|
+
def self.ruby_version_mismatch_hint(output)
|
|
193
|
+
return nil if output.nil? || output.empty?
|
|
194
|
+
return nil unless output.match?(/RubyVersionMismatch|Your Ruby version is .* but your Gemfile specified/i)
|
|
195
|
+
|
|
196
|
+
ran = output[/Your Ruby version is ([0-9.]+)/i, 1]
|
|
197
|
+
want = output[/Gemfile specified ([0-9.]+)/i, 1]
|
|
198
|
+
parts = +"Detected a Ruby version mismatch"
|
|
199
|
+
parts << ": the test command ran under #{ran}" if ran
|
|
200
|
+
parts << " but the app expects #{want}" if want
|
|
201
|
+
parts << ". Mutineer already scrubbed version-manager bins and RBENV_VERSION/" \
|
|
202
|
+
"ASDF_RUBY_VERSION from the child env; the suite still picked the wrong " \
|
|
203
|
+
"Ruby. Use a wrapper that re-selects the app Ruby and ensure " \
|
|
204
|
+
".ruby-version / .tool-versions is present " \
|
|
205
|
+
"(see README: Apps on Ruby < 3.4 → Under a version manager)"
|
|
206
|
+
parts
|
|
94
207
|
end
|
|
95
208
|
|
|
96
209
|
# Spawns the command to a captured combined-output tempfile, enforces a
|
|
@@ -116,7 +229,8 @@ module Mutineer
|
|
|
116
229
|
# explicit [program, argv0] form guarantees the no-shell exec path even for a
|
|
117
230
|
# degenerate single-element argv (Process.spawn(*argv) would route a lone
|
|
118
231
|
# metachar-bearing string through /bin/sh, breaking the argv-only invariant).
|
|
119
|
-
pid = Process.spawn([argv.first, argv.first], *argv[1..],
|
|
232
|
+
pid = Process.spawn(child_env, [argv.first, argv.first], *argv[1..],
|
|
233
|
+
out: out, err: %i[child out], pgroup: true)
|
|
120
234
|
kind, code = wait_with_timeout(pid, timeout)
|
|
121
235
|
elapsed = Process.clock_gettime(Process::CLOCK_MONOTONIC) - start
|
|
122
236
|
out.rewind
|
|
@@ -151,7 +265,13 @@ module Mutineer
|
|
|
151
265
|
rescue Errno::ESRCH, Errno::EPERM
|
|
152
266
|
Process.kill(:KILL, pid) rescue nil # rubocop:disable Style/RescueModifier
|
|
153
267
|
end
|
|
154
|
-
|
|
268
|
+
begin
|
|
269
|
+
_reaped, status = Process.waitpid2(pid)
|
|
270
|
+
# Finished cleanly between poll and kill — honor real exit code.
|
|
271
|
+
return [:exited, status.exitstatus] if status && status.exited? && !status.signaled?
|
|
272
|
+
rescue Errno::ECHILD
|
|
273
|
+
# already reaped
|
|
274
|
+
end
|
|
155
275
|
return [:timeout, nil]
|
|
156
276
|
end
|
|
157
277
|
sleep POLL
|
data/lib/mutineer/isolation.rb
CHANGED
|
@@ -31,6 +31,9 @@ module Mutineer
|
|
|
31
31
|
# @return [Mutineer::Result] result from the child process.
|
|
32
32
|
def self.run(timeout: DEFAULT_TIMEOUT)
|
|
33
33
|
pid = fork do
|
|
34
|
+
# Own process group so a timeout kill can reap grandchildren (match
|
|
35
|
+
# daemon/external backends). Best-effort: if setpgid fails, kill the pid.
|
|
36
|
+
Process.setpgid(0, 0) rescue nil # rubocop:disable Style/RescueModifier
|
|
34
37
|
code = 0
|
|
35
38
|
begin
|
|
36
39
|
result = yield
|
|
@@ -48,20 +51,27 @@ module Mutineer
|
|
|
48
51
|
exit!(code)
|
|
49
52
|
end
|
|
50
53
|
|
|
51
|
-
# Single-threaded deadline poll
|
|
52
|
-
#
|
|
53
|
-
# shows the child is still alive past the deadline
|
|
54
|
-
# never hit a reaped/recycled pid. Timeout is a parent-side fact
|
|
55
|
-
# (deadline reached), not status.signaled? (which is true for ANY signal
|
|
56
|
-
# death, e.g. SIGSEGV).
|
|
54
|
+
# Single-threaded deadline poll: we are the ONLY caller of waitpid on this
|
|
55
|
+
# pid, so we never reap-then-kill. SIGKILL the process group only after
|
|
56
|
+
# WNOHANG shows the child is still alive past the deadline.
|
|
57
57
|
deadline = Process.clock_gettime(Process::CLOCK_MONOTONIC) + timeout
|
|
58
58
|
loop do
|
|
59
59
|
reaped, status = Process.waitpid2(pid, Process::WNOHANG)
|
|
60
60
|
return decode(status) if reaped
|
|
61
61
|
|
|
62
62
|
if Process.clock_gettime(Process::CLOCK_MONOTONIC) >= deadline
|
|
63
|
-
|
|
64
|
-
|
|
63
|
+
begin
|
|
64
|
+
Process.kill(:KILL, -pid)
|
|
65
|
+
rescue Errno::ESRCH, Errno::EPERM
|
|
66
|
+
Process.kill(:KILL, pid) rescue nil # rubocop:disable Style/RescueModifier
|
|
67
|
+
end
|
|
68
|
+
begin
|
|
69
|
+
_reaped, status = Process.waitpid2(pid)
|
|
70
|
+
# Child may have finished cleanly between WNOHANG and kill; honor it.
|
|
71
|
+
return decode(status) if status && status.exited? && !status.signaled?
|
|
72
|
+
rescue Errno::ECHILD
|
|
73
|
+
# already reaped
|
|
74
|
+
end
|
|
65
75
|
return Result.timeout
|
|
66
76
|
end
|
|
67
77
|
sleep 0.005
|
|
@@ -14,17 +14,19 @@ module Mutineer
|
|
|
14
14
|
# @return [void]
|
|
15
15
|
def visit_statements_node(node)
|
|
16
16
|
stmts = node.body
|
|
17
|
-
|
|
18
|
-
|
|
19
|
-
|
|
20
|
-
|
|
21
|
-
|
|
22
|
-
|
|
23
|
-
|
|
24
|
-
|
|
25
|
-
|
|
26
|
-
|
|
17
|
+
if stmts.length >= 2
|
|
18
|
+
stmts[0...-1].each do |stmt|
|
|
19
|
+
loc = stmt.location
|
|
20
|
+
@mutations << Mutation.new(
|
|
21
|
+
start_offset: loc.start_offset,
|
|
22
|
+
end_offset: loc.end_offset,
|
|
23
|
+
replacement: "nil",
|
|
24
|
+
operator: :statement_removal
|
|
25
|
+
)
|
|
26
|
+
end
|
|
27
27
|
end
|
|
28
|
+
# Nested statement lists (if/begin/while bodies) are separate StatementsNodes.
|
|
29
|
+
super
|
|
28
30
|
end
|
|
29
31
|
end
|
|
30
32
|
end
|
|
@@ -17,10 +17,9 @@ module Mutineer
|
|
|
17
17
|
# fixtures then repopulate that isolated database per test.
|
|
18
18
|
#
|
|
19
19
|
# Scope: this pass ships the **SQLite** adapter (per-worker file, hermetic,
|
|
20
|
-
# spike-proven). Postgres per-worker
|
|
21
|
-
#
|
|
22
|
-
#
|
|
23
|
-
# rather than silently mis-routing.
|
|
20
|
+
# spike-proven). Postgres per-worker DBs (`CREATE DATABASE <db>-<worker>`) are not
|
|
21
|
+
# implemented yet; a non-SQLite config raises a clear NotImplementedError rather
|
|
22
|
+
# than silently mis-routing.
|
|
24
23
|
#
|
|
25
24
|
# Honest limit (KTD-5): routing failures surface as `error` via {verify_connection!}.
|
|
26
25
|
# Re-raising an AR error that fires *inside a test body* past Minitest (so an in-test
|
|
@@ -63,7 +62,7 @@ module Mutineer
|
|
|
63
62
|
# What's gated is runtime PROVISIONING: SQLite files are created on connect,
|
|
64
63
|
# but Postgres needs an explicit `CREATE DATABASE` per worker (U10). Until that
|
|
65
64
|
# lands, refuse non-SQLite loudly rather than route to a database that doesn't
|
|
66
|
-
# exist
|
|
65
|
+
# exist until Postgres worker creation is implemented.
|
|
67
66
|
unless adapter.start_with?("sqlite")
|
|
68
67
|
raise NotImplementedError,
|
|
69
68
|
"worker-DB isolation currently provisions SQLite only (got adapter #{adapter.inspect}); " \
|
|
@@ -112,31 +111,8 @@ module Mutineer
|
|
|
112
111
|
verify_connection!
|
|
113
112
|
end
|
|
114
113
|
|
|
115
|
-
#
|
|
116
|
-
#
|
|
117
|
-
# provisioning entry point (V2 — never silent auto-create mid-audit) and the seam
|
|
118
|
-
# U10 extends for Postgres (`CREATE DATABASE` per worker). For SQLite the files are
|
|
119
|
-
# created on connect and the schema load makes them ready; idempotent to re-run.
|
|
120
|
-
#
|
|
121
|
-
# @param worker_count [Integer] number of worker databases to provision.
|
|
122
|
-
# @param schema_path [String, nil] absolute path to `db/schema.rb`, or nil to skip.
|
|
123
|
-
# @return [void]
|
|
124
|
-
def self.provision(worker_count, schema_path)
|
|
125
|
-
return unless available?
|
|
126
|
-
|
|
127
|
-
original = ActiveRecord::Base.connection_db_config
|
|
128
|
-
(0...worker_count).each do |worker|
|
|
129
|
-
ActiveRecord::Base.establish_connection(worker_db_config(worker))
|
|
130
|
-
load_schema(schema_path) if schema_path
|
|
131
|
-
end
|
|
132
|
-
ensure
|
|
133
|
-
ActiveRecord::Base.establish_connection(original) if original
|
|
134
|
-
end
|
|
135
|
-
|
|
136
|
-
# Load a Rails `schema.rb` into the current connection with its output silenced,
|
|
137
|
-
# so a parent-side {provision} call can never spill schema chatter onto the daemon
|
|
138
|
-
# IPC pipe. (In a fork the child's stdout is already `File::NULL`; this guards the
|
|
139
|
-
# parent path too.)
|
|
114
|
+
# Load a Rails `schema.rb` into the current connection with output silenced
|
|
115
|
+
# (fork child stdout is already File::NULL; this is belt-and-braces).
|
|
140
116
|
#
|
|
141
117
|
# @param schema_path [String] absolute path to `db/schema.rb`.
|
|
142
118
|
# @return [void]
|
data/lib/mutineer/reporter.rb
CHANGED
|
@@ -69,12 +69,21 @@ module Mutineer
|
|
|
69
69
|
verdict(out, threshold) if threshold && threshold.positive?
|
|
70
70
|
end
|
|
71
71
|
|
|
72
|
-
# 0 pass / 1 below threshold. Usage errors (exit 2)
|
|
72
|
+
# 0 pass / 1 below threshold or untestable-with-errors. Usage errors (exit 2)
|
|
73
|
+
# are the CLI's job. When the score is nil (nothing killed or survived), pure
|
|
74
|
+
# no_coverage / all-ignored / empty still skip the gate; if any mutant was
|
|
75
|
+
# errored, timed out, or uncapturable, fail the gate so a broken harness
|
|
76
|
+
# cannot green CI under --threshold.
|
|
73
77
|
def exit_code(threshold:)
|
|
74
78
|
return 0 if threshold.nil? || threshold <= 0
|
|
75
79
|
|
|
76
80
|
score = @agg.mutation_score
|
|
77
|
-
|
|
81
|
+
if score.nil?
|
|
82
|
+
broken = @agg.errored_count + @agg.timeout_count + @agg.uncapturable_count
|
|
83
|
+
return 1 if @agg.total.positive? && broken.positive?
|
|
84
|
+
|
|
85
|
+
return 0 # pure no_coverage / ignored / empty — gate skipped
|
|
86
|
+
end
|
|
78
87
|
|
|
79
88
|
score >= threshold ? 0 : 1
|
|
80
89
|
end
|
|
@@ -380,12 +389,38 @@ module Mutineer
|
|
|
380
389
|
"#{@agg.ignored_count} ignored excluded"
|
|
381
390
|
if score.nil?
|
|
382
391
|
out.puts "Mutation score: N/A (no covered mutants)"
|
|
383
|
-
|
|
392
|
+
if broken_nil_score?
|
|
393
|
+
err.puts "[mutineer] no covered mutations (#{broken_nil_score_detail}); " \
|
|
394
|
+
"threshold gate fails under a positive --threshold (broken harness)."
|
|
395
|
+
else
|
|
396
|
+
err.puts "[mutineer] no covered mutations; mutation score is N/A and the threshold check is skipped."
|
|
397
|
+
end
|
|
384
398
|
else
|
|
385
399
|
out.puts "Mutation score: #{score}% (killed / (killed + survived); #{excluded})"
|
|
386
400
|
end
|
|
387
401
|
end
|
|
388
402
|
|
|
403
|
+
# True when nil score is due to errors/timeouts/uncapturable (gate must fail).
|
|
404
|
+
#
|
|
405
|
+
# @api private
|
|
406
|
+
# @return [Boolean]
|
|
407
|
+
def broken_nil_score?
|
|
408
|
+
@agg.total.positive? &&
|
|
409
|
+
(@agg.errored_count + @agg.timeout_count + @agg.uncapturable_count).positive?
|
|
410
|
+
end
|
|
411
|
+
|
|
412
|
+
# Human-readable counts for a broken nil-score run.
|
|
413
|
+
#
|
|
414
|
+
# @api private
|
|
415
|
+
# @return [String]
|
|
416
|
+
def broken_nil_score_detail
|
|
417
|
+
parts = []
|
|
418
|
+
parts << "#{@agg.errored_count} errored" if @agg.errored_count.positive?
|
|
419
|
+
parts << "#{@agg.timeout_count} timeout" if @agg.timeout_count.positive?
|
|
420
|
+
parts << "#{@agg.uncapturable_count} uncapturable" if @agg.uncapturable_count.positive?
|
|
421
|
+
parts.join(", ")
|
|
422
|
+
end
|
|
423
|
+
|
|
389
424
|
# #11: one line per source after the global summary, so a multi-source run
|
|
390
425
|
# shows which file is weak. Omitted for a single-source run — the global
|
|
391
426
|
# summary already says everything (ponytail: no redundant one-line block).
|
|
@@ -470,7 +505,13 @@ module Mutineer
|
|
|
470
505
|
# @return [void]
|
|
471
506
|
def verdict(out, threshold)
|
|
472
507
|
score = @agg.mutation_score
|
|
473
|
-
|
|
508
|
+
if score.nil?
|
|
509
|
+
if broken_nil_score?
|
|
510
|
+
out.puts "FAILED: no covered mutants (#{broken_nil_score_detail}); " \
|
|
511
|
+
"threshold #{threshold}% cannot pass with a broken harness"
|
|
512
|
+
end
|
|
513
|
+
return
|
|
514
|
+
end
|
|
474
515
|
|
|
475
516
|
if score >= threshold
|
|
476
517
|
out.puts "PASSED: #{score}% >= threshold #{threshold}%"
|
data/lib/mutineer/runner.rb
CHANGED
|
@@ -112,13 +112,15 @@ module Mutineer
|
|
|
112
112
|
sweep_orphans(dirs)
|
|
113
113
|
|
|
114
114
|
strategy = config.strategy
|
|
115
|
+
# Fail-fast must be serial: a parallel stop_when fires on the first survivor
|
|
116
|
+
# by wall-clock, not input order, so the survivor set would diverge from
|
|
117
|
+
# --jobs 1 (daemon already forces serial for the same reason).
|
|
118
|
+
jobs_n = config.fail_fast ? 1 : config.jobs
|
|
115
119
|
results =
|
|
116
120
|
begin
|
|
117
121
|
framework = config.framework
|
|
118
|
-
# #21: --fail-fast stops scheduling new mutants after the first survivor;
|
|
119
|
-
# in-flight workers drain, unscheduled jobs stay nil (dropped below).
|
|
120
122
|
stop_when = config.fail_fast ? ->(r) { r.survived? } : nil
|
|
121
|
-
bare = WorkerPool.new(
|
|
123
|
+
bare = WorkerPool.new(jobs_n).run(jobs, stop_when: stop_when) do |subject, mutation|
|
|
122
124
|
run(mutation, source_file: subject.file, coverage_map: coverage_map,
|
|
123
125
|
subject: subject, strategy: strategy, rails: config.rails, framework: framework)
|
|
124
126
|
end
|
|
@@ -229,13 +231,12 @@ module Mutineer
|
|
|
229
231
|
end
|
|
230
232
|
end
|
|
231
233
|
|
|
232
|
-
#
|
|
233
|
-
#
|
|
234
|
-
#
|
|
235
|
-
#
|
|
236
|
-
#
|
|
237
|
-
#
|
|
238
|
-
# runs the full `--test` set.
|
|
234
|
+
# Daemon backend: boot the app once in a persistent subprocess and fork per
|
|
235
|
+
# mutant. Tool-side we build the ready-to-`load` payload (whole-file reload by
|
|
236
|
+
# default) and ship it; the daemon needs no Prism/mutineer. Coverage is built
|
|
237
|
+
# once via a short-lived daemon so each mutant runs only its covering tests.
|
|
238
|
+
# When jobs > 1, each worker uses its own database (SQLite). Fail-fast forces
|
|
239
|
+
# serial so the survivor set matches jobs 1.
|
|
239
240
|
#
|
|
240
241
|
# @return [Array(Mutineer::AggregateResult, Hash<String,String>)] aggregate and source map.
|
|
241
242
|
def self.execute_daemon(config, operator_classes)
|
|
@@ -243,10 +244,9 @@ module Mutineer
|
|
|
243
244
|
jobs = filter_since(jobs, source_map, config) if config.since
|
|
244
245
|
abs_tests = config.tests.map { |t| File.expand_path(t, config.project_root) }
|
|
245
246
|
|
|
246
|
-
#
|
|
247
|
-
#
|
|
248
|
-
#
|
|
249
|
-
# --test set (no narrowing) rather than mis-scoring everything as no_coverage.
|
|
247
|
+
# Build the coverage map once (app-side). nil when the build fails — runners
|
|
248
|
+
# fall back to the full --test set (and emit a stderr warning) rather than
|
|
249
|
+
# mis-scoring everything as no_coverage.
|
|
250
250
|
coverage_map = daemon_coverage_map(config, abs_tests)
|
|
251
251
|
|
|
252
252
|
# #26/U6: worker count = resolved --jobs, capped at the job count (no idle
|
|
@@ -270,24 +270,16 @@ module Mutineer
|
|
|
270
270
|
[AggregateResult.new(results + ignored_results), source_map]
|
|
271
271
|
end
|
|
272
272
|
|
|
273
|
-
#
|
|
274
|
-
#
|
|
273
|
+
# Build the coverage map via a short-lived daemon (boots the app once, captures
|
|
274
|
+
# per-test coverage app-side, ships the map back). Returns a query-only
|
|
275
275
|
# CoverageMap, or nil when the build fails / returns empty — callers then run the
|
|
276
|
-
# full --test set
|
|
277
|
-
#
|
|
278
|
-
#
|
|
276
|
+
# full --test set. Coverage-build IPC has no wall-clock (same limitation as
|
|
277
|
+
# in-process build_via_fork). A normal nonempty map scores like in-process;
|
|
278
|
+
# nil falls back to the full suite (more testing, not comparable).
|
|
279
279
|
#
|
|
280
280
|
# @param config [Mutineer::Config] the run config.
|
|
281
281
|
# @param abs_tests [Array<String>] absolute --test paths.
|
|
282
282
|
# @return [Mutineer::CoverageMap, nil]
|
|
283
|
-
#
|
|
284
|
-
# ponytail: the coverage-build IPC read is unbounded (no wall-clock) — a --test
|
|
285
|
-
# file that infinite-loops during capture wedges this call, mirroring the existing
|
|
286
|
-
# in-process build_via_fork limitation. A capture timeout is a follow-up. When the
|
|
287
|
-
# map comes back empty (all captures failed), this returns nil and the run falls
|
|
288
|
-
# back to the FULL --test set per mutant — so on a total capture failure the daemon
|
|
289
|
-
# score is an upper bound (more testing), diverging from the in-process no_coverage
|
|
290
|
-
# scoring for that degenerate case only; a normal (nonempty) map scores identically.
|
|
291
283
|
def self.daemon_coverage_map(config, abs_tests)
|
|
292
284
|
client = DaemonClient.new(boot: daemon_boot_config(config, abs_tests, coverage: true),
|
|
293
285
|
app_root: config.project_root).start
|
|
@@ -296,14 +288,30 @@ module Mutineer
|
|
|
296
288
|
ensure
|
|
297
289
|
client.quit
|
|
298
290
|
end
|
|
299
|
-
|
|
291
|
+
unless data && !(data["map"] || {}).empty?
|
|
292
|
+
reason = data.is_a?(Hash) && data["error"] ? data["error"] : "empty map"
|
|
293
|
+
warn_daemon_coverage_fallback(reason)
|
|
294
|
+
return nil
|
|
295
|
+
end
|
|
300
296
|
|
|
301
297
|
CoverageMap.from_data(map: data["map"], failed_test_files: data["failed_test_files"] || [],
|
|
302
298
|
project_root: config.project_root)
|
|
303
|
-
rescue DaemonBootError
|
|
299
|
+
rescue DaemonBootError => e
|
|
300
|
+
warn_daemon_coverage_fallback("#{e.class}: #{e.message}")
|
|
304
301
|
nil
|
|
305
302
|
end
|
|
306
303
|
|
|
304
|
+
# Stderr note when daemon coverage is unavailable (full --test set per mutant).
|
|
305
|
+
#
|
|
306
|
+
# @api private
|
|
307
|
+
# @param reason [String] short cause (boot error message, empty map, …).
|
|
308
|
+
# @return [void]
|
|
309
|
+
def self.warn_daemon_coverage_fallback(reason = "unknown")
|
|
310
|
+
warn "[mutineer] daemon coverage map unavailable (#{reason}); running every " \
|
|
311
|
+
"mutant against the full --test set (score not comparable to an in-process run)."
|
|
312
|
+
end
|
|
313
|
+
private_class_method :warn_daemon_coverage_fallback
|
|
314
|
+
|
|
307
315
|
# Serial daemon path: one daemon (worker 0), one mutant at a time. Honors
|
|
308
316
|
# --fail-fast (#21: stop at the first survivor).
|
|
309
317
|
#
|
|
@@ -324,24 +332,16 @@ module Mutineer
|
|
|
324
332
|
results
|
|
325
333
|
end
|
|
326
334
|
|
|
327
|
-
# Parallel daemon path
|
|
328
|
-
#
|
|
329
|
-
#
|
|
330
|
-
#
|
|
331
|
-
# placed by input index and compacted, so the verdict SET is identical to serial
|
|
332
|
-
# regardless of finish order (R7). `execute_daemon` never calls this with
|
|
333
|
-
# `config.fail_fast` set (forced to the serial path instead) — a wall-clock stop
|
|
334
|
-
# here, unlike serial's input-index break, would make the survivor set and score
|
|
335
|
-
# non-deterministic. The stop-flag machinery below stays as a defensive no-op for
|
|
336
|
-
# any direct caller that bypasses that guard.
|
|
335
|
+
# Parallel daemon path: N daemon handles, each pinned to its own worker slot
|
|
336
|
+
# (own DB). A shared queue of job indices feeds N tool-side threads; results
|
|
337
|
+
# are placed by input index so the verdict set matches serial. Callers must
|
|
338
|
+
# not pass fail_fast here (execute_daemon forces serial for fail-fast).
|
|
337
339
|
#
|
|
338
340
|
# @return [Array<Mutineer::Result>] completed results in input order.
|
|
339
341
|
def self.run_daemon_parallel(jobs, worker_count, config, abs_tests, coverage_map, source_map)
|
|
340
|
-
results
|
|
341
|
-
queue
|
|
342
|
+
results = Array.new(jobs.size)
|
|
343
|
+
queue = Queue.new
|
|
342
344
|
jobs.each_index { |i| queue << i }
|
|
343
|
-
stop = false
|
|
344
|
-
stop_mutex = Mutex.new
|
|
345
345
|
|
|
346
346
|
clients = Array.new(worker_count) do
|
|
347
347
|
DaemonClient.new(boot: daemon_boot_config(config, abs_tests),
|
|
@@ -350,15 +350,13 @@ module Mutineer
|
|
|
350
350
|
|
|
351
351
|
clients.each_with_index.map do |client, worker|
|
|
352
352
|
Thread.new do
|
|
353
|
-
|
|
353
|
+
loop do
|
|
354
354
|
i = begin
|
|
355
|
-
queue.pop(true)
|
|
355
|
+
queue.pop(true)
|
|
356
356
|
rescue ThreadError
|
|
357
357
|
break
|
|
358
358
|
end
|
|
359
|
-
|
|
360
|
-
results[i] = r
|
|
361
|
-
stop_mutex.synchronize { stop = true } if config.fail_fast && r.survived?
|
|
359
|
+
results[i] = daemon_job_result(jobs[i], i, client, worker, config, coverage_map, abs_tests, source_map)
|
|
362
360
|
end
|
|
363
361
|
ensure
|
|
364
362
|
client.quit
|
|
@@ -433,8 +431,9 @@ module Mutineer
|
|
|
433
431
|
[:run, chosen.map { |t| File.expand_path(t, coverage_map.project_root) }]
|
|
434
432
|
end
|
|
435
433
|
|
|
436
|
-
# Default per-mutant timeout on the daemon path.
|
|
437
|
-
#
|
|
434
|
+
# Default per-mutant timeout on the daemon path (seconds). Coverage narrowing
|
|
435
|
+
# usually keeps each job short; this still covers a slow suite or full-suite
|
|
436
|
+
# fallback when the coverage map is unavailable.
|
|
438
437
|
DAEMON_TIMEOUT = 60
|
|
439
438
|
|
|
440
439
|
# The boot config the daemon needs to boot the app once: where to boot, the test
|
data/lib/mutineer/version.rb
CHANGED