mutineer 0.10.0 → 0.11.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +19 -0
- data/README.md +30 -1
- data/lib/mutineer/cli.rb +40 -0
- data/lib/mutineer/config.rb +4 -4
- data/lib/mutineer/coverage_map.rb +27 -7
- data/lib/mutineer/daemon_client.rb +17 -2
- data/lib/mutineer/daemon_server.rb +76 -5
- data/lib/mutineer/rails_worker_db.rb +161 -0
- data/lib/mutineer/runner.rb +210 -41
- data/lib/mutineer/version.rb +1 -1
- metadata +2 -1
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 2dd1784f527b49eaed8d13a3e03c59c8a30082a44cf625f3a6ce1f252ffb3a60
|
|
4
|
+
data.tar.gz: 164d53165ea6f85c31158783ea43f8f2c88c3df058240d748df01f960da23897
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: a8bcaae29c133ca1ac0f9da31dc143b7b5deeef347104e209386cf6029252e77586d1240440e97bcec082e9e11a355087ffda91e7ca5495de85ac0f6bf1c2155
|
|
7
|
+
data.tar.gz: 06f108bfbb0f4b32318ca120e086ce88516b036ebc622e37b81963f4f93b7a87d5cdb6403b95cd69aa7a196e910718ba52b632d376a10f2763bf05f9f94a8fc8
|
data/CHANGELOG.md
CHANGED
|
@@ -6,6 +6,23 @@ All notable changes to this project are documented here. The format is based on
|
|
|
6
6
|
|
|
7
7
|
## [Unreleased]
|
|
8
8
|
|
|
9
|
+
## [0.11.0] - 2026-07-02
|
|
10
|
+
|
|
11
|
+
### Added
|
|
12
|
+
- **`--daemon` backend — fast, parallel-safe Rails mutation testing** (#26/#27
|
|
13
|
+
Phase 2). Boots the app **once** in a persistent daemon and forks per mutant
|
|
14
|
+
(restoring shared-boot speed), and gives **each parallel worker its own
|
|
15
|
+
database** so `--jobs N` is safe under Rails for the first time — parallel
|
|
16
|
+
verdicts are proven identical to serial (no fixture cross-talk). Coverage
|
|
17
|
+
narrowing is restored on this path (each mutant runs only its covering tests;
|
|
18
|
+
a mutant on an uncovered line is `no_coverage`), so the daemon score is
|
|
19
|
+
comparable to the in-process `--rails` score. Opt in with `--rails --daemon`
|
|
20
|
+
(also `daemon: true` in `.mutineer.yml`); `--daemon` can't be combined with
|
|
21
|
+
`--test-command`. **SQLite** today (hermetic, CI-proven); **Postgres**
|
|
22
|
+
per-worker provisioning is in progress (#34/#35). The gem core stays Prism +
|
|
23
|
+
stdlib, zero runtime dependencies — worker-DB routing uses the app's own
|
|
24
|
+
ActiveRecord.
|
|
25
|
+
|
|
9
26
|
## [0.10.0] - 2026-07-02
|
|
10
27
|
|
|
11
28
|
### Added
|
|
@@ -200,6 +217,8 @@ Rails hardening + CI batch (issues #8–#13), all verified Rails-free.
|
|
|
200
217
|
- `.mutineer.yml` configuration (CLI > config > default precedence).
|
|
201
218
|
- Byte-correct source handling for multibyte (UTF-8) sources.
|
|
202
219
|
|
|
220
|
+
[0.11.0]: https://github.com/davidteren/mutineer/releases/tag/v0.11.0
|
|
221
|
+
[0.10.0]: https://github.com/davidteren/mutineer/releases/tag/v0.10.0
|
|
203
222
|
[0.9.1]: https://github.com/davidteren/mutineer/releases/tag/v0.9.1
|
|
204
223
|
[0.9.0]: https://github.com/davidteren/mutineer/releases/tag/v0.9.0
|
|
205
224
|
[0.8.0]: https://github.com/davidteren/mutineer/releases/tag/v0.8.0
|
data/README.md
CHANGED
|
@@ -57,6 +57,7 @@ mutineer run lib/calculator.rb --test test/calculator_test.rb --threshold 90
|
|
|
57
57
|
| `--verbose` | Surface the real error when a fork capture fails (alias `--debug`) |
|
|
58
58
|
| `--strategy NAME` | Mutation application: `reload` whole-file (default) or `redefine` surgical (`7a`/`7b` accepted as deprecated aliases) |
|
|
59
59
|
| `--test-command CMD` | Run the suite as a subprocess in the app's own runtime (for apps on Ruby < 3.4); `CMD` must contain `%{files}`. See [Apps on Ruby < 3.4](#apps-on-ruby--34) |
|
|
60
|
+
| `--daemon` | Boot the app once in a persistent daemon and fork per mutant, with per-worker DB isolation so `--jobs N` is safe under Rails (needs `--rails`/`--boot`; not with `--test-command`). See [the daemon backend](#faster-parallel-safe-rails-the---daemon-backend) |
|
|
60
61
|
| `--format human\|json\|html` | Report format (default: human; `html` is a self-contained file) |
|
|
61
62
|
| `--output FILE` | Write the report to FILE instead of stdout |
|
|
62
63
|
| `--dry-run` | List candidate mutations without executing (honors suppression) |
|
|
@@ -104,6 +105,33 @@ Add Mutineer to your Gemfile's test group:
|
|
|
104
105
|
gem "mutineer", group: :test, require: false
|
|
105
106
|
```
|
|
106
107
|
|
|
108
|
+
### Faster, parallel-safe Rails (the `--daemon` backend)
|
|
109
|
+
|
|
110
|
+
`--rails` boots your app once but runs mutants **serially** — parallel `--jobs`
|
|
111
|
+
under Rails is unsafe, because every worker shares one test database and clobbers
|
|
112
|
+
the others' fixtures. `--daemon` fixes both: it boots the app once in a persistent
|
|
113
|
+
helper and forks per mutant, and gives **each parallel worker its own database**,
|
|
114
|
+
so `--jobs N` is safe and its verdicts are proven identical to a serial run.
|
|
115
|
+
|
|
116
|
+
```sh
|
|
117
|
+
RAILS_ENV=test bundle exec mutineer run \
|
|
118
|
+
app/models/order.rb --test test/models/order_test.rb \
|
|
119
|
+
--rails --daemon --jobs 4
|
|
120
|
+
```
|
|
121
|
+
|
|
122
|
+
- **One boot, forked per mutant** — restores the shared-boot speed.
|
|
123
|
+
- **Coverage-guided** — each mutant runs only its covering tests (like `--rails`);
|
|
124
|
+
a mutant on an uncovered line is `no_coverage`, so the score stays comparable to
|
|
125
|
+
the in-process `--rails` score.
|
|
126
|
+
- **Safe `--jobs N`** — each worker routes to its own copy of the test database, so
|
|
127
|
+
parallel verdicts equal serial (no fixture cross-talk).
|
|
128
|
+
- **One backend at a time** — `--daemon` can't be combined with `--test-command`
|
|
129
|
+
(choose one), and it needs an app to boot (`--rails` or `--boot`).
|
|
130
|
+
|
|
131
|
+
Status: **SQLite** today (hermetic, CI-proven). **Postgres** per-worker
|
|
132
|
+
provisioning is in progress (#34/#35); until it lands, use `--daemon` with a
|
|
133
|
+
SQLite test database, or drop `--jobs` to run serially on other adapters.
|
|
134
|
+
|
|
107
135
|
### Apps on Ruby < 3.4
|
|
108
136
|
|
|
109
137
|
Mutineer's own process needs Ruby ≥ 3.4 (it parses with stdlib Prism), and the
|
|
@@ -136,7 +164,8 @@ Tradeoffs (Phase 1) — this path is correct but not free:
|
|
|
136
164
|
is scored as a kill. Mutineer prints this caveat on every run and aborts up
|
|
137
165
|
front (a "smoke check") if your unmutated suite isn't green.
|
|
138
166
|
- **Reload strategy only** (`--strategy redefine` is rejected on this path) and
|
|
139
|
-
**serial** (`--jobs` is forced to 1
|
|
167
|
+
**serial** (`--jobs` is forced to 1). For apps on Ruby ≥ 3.4, `--daemon` gives
|
|
168
|
+
safe parallelism instead (see [the daemon backend](#faster-parallel-safe-rails-the---daemon-backend)).
|
|
140
169
|
|
|
141
170
|
## Suppressing equivalent mutants
|
|
142
171
|
|
data/lib/mutineer/cli.rb
CHANGED
|
@@ -51,6 +51,9 @@ module Mutineer
|
|
|
51
51
|
subprocess (for apps on Ruby < 3.4). CMD must contain
|
|
52
52
|
%{files} (expands to the --test paths). Env is inherited,
|
|
53
53
|
e.g. RAILS_ENV=test mutineer run ... --test-command "..."
|
|
54
|
+
--daemon Boot the app ONCE in a persistent daemon and fork per
|
|
55
|
+
mutant, with per-worker DB isolation so --jobs N is safe
|
|
56
|
+
under Rails (needs --rails/--boot; not with --test-command)
|
|
54
57
|
--format human|json|html Report format (default: human)
|
|
55
58
|
--output FILE Write the report to FILE instead of stdout
|
|
56
59
|
--dry-run List mutations without executing
|
|
@@ -112,6 +115,9 @@ module Mutineer
|
|
|
112
115
|
# #27: run the target suite as a subprocess in the app's OWN runtime so
|
|
113
116
|
# mutineer (Ruby >= 3.4) can mutation-test apps pinned to an older Ruby.
|
|
114
117
|
o.on("--test-command CMD") { |v| opts[:test_command] = v; explicit << :test_command }
|
|
118
|
+
# #26/#27 Phase 2: boot the app ONCE in a persistent daemon and fork per
|
|
119
|
+
# mutant, with per-worker DB isolation so --jobs N is safe under Rails.
|
|
120
|
+
o.on("--daemon") { opts[:daemon] = true; explicit << :daemon }
|
|
115
121
|
end
|
|
116
122
|
|
|
117
123
|
begin
|
|
@@ -242,6 +248,7 @@ module Mutineer
|
|
|
242
248
|
end
|
|
243
249
|
|
|
244
250
|
validate_test_command!(config) if config.test_command
|
|
251
|
+
validate_daemon!(config, explicit) if config.daemon
|
|
245
252
|
|
|
246
253
|
validate_since!(config) if config.since
|
|
247
254
|
preflight_output!(config.output) if config.output
|
|
@@ -300,6 +307,30 @@ module Mutineer
|
|
|
300
307
|
config.jobs = 1
|
|
301
308
|
end
|
|
302
309
|
|
|
310
|
+
# #26/#27 Phase 2 (U8): --daemon selects the persistent-daemon backend (boot once,
|
|
311
|
+
# fork per mutant, per-worker DB isolation). Two usage errors, both exit 2 (KTD-10):
|
|
312
|
+
# it cannot be combined with --test-command (choose ONE backend — never silently
|
|
313
|
+
# pick one), and it requires an app to boot (--rails or --boot). Under Rails,
|
|
314
|
+
# Config.resolve already keeps --jobs serial unless the user explicitly asks for
|
|
315
|
+
# parallelism, and each worker gets its own SQLite database on demand — so there is
|
|
316
|
+
# no "missing worker DB" precondition to check here for SQLite. Postgres worker-DB
|
|
317
|
+
# provisioning + its missing-DB error (KTD-9) arrives with the Postgres adapter (U10).
|
|
318
|
+
#
|
|
319
|
+
# @api private
|
|
320
|
+
# @param config [Mutineer::Config] run configuration.
|
|
321
|
+
# @param explicit [Set<Symbol>] explicit CLI fields.
|
|
322
|
+
# @return [void]
|
|
323
|
+
def self.validate_daemon!(config, _explicit = Set.new)
|
|
324
|
+
if config.test_command
|
|
325
|
+
warn "mutineer: choose one backend — --daemon and --test-command cannot be combined"
|
|
326
|
+
exit 2
|
|
327
|
+
end
|
|
328
|
+
return if config.rails || config.boot
|
|
329
|
+
|
|
330
|
+
warn "mutineer: --daemon needs an app to boot; add --rails (or --boot FILE)"
|
|
331
|
+
exit 2
|
|
332
|
+
end
|
|
333
|
+
|
|
303
334
|
# --since needs a real git repo and a resolvable ref; either failure is a
|
|
304
335
|
# usage error (exit 2) so CI sees "bad invocation," not "tests too weak."
|
|
305
336
|
# Validates the --since ref.
|
|
@@ -437,6 +468,15 @@ module Mutineer
|
|
|
437
468
|
"and an infra failure is scored as a kill."
|
|
438
469
|
end
|
|
439
470
|
|
|
471
|
+
# #26/U8: same disclosure as --test-command above — the daemon backend has no
|
|
472
|
+
# coverage narrowing yet (Phase 2c), so this score is a lower bound (uncovered
|
|
473
|
+
# mutants count as survivors) and not comparable to an in-process run.
|
|
474
|
+
if config.daemon
|
|
475
|
+
warn "[mutineer] --daemon score is a lower bound, not comparable to an " \
|
|
476
|
+
"in-process run: no coverage narrowing yet (uncovered mutants count as " \
|
|
477
|
+
"survivors)."
|
|
478
|
+
end
|
|
479
|
+
|
|
440
480
|
# #14: nudge toward the opt-in tier-2 operators (human report only — never
|
|
441
481
|
# pollute JSON output).
|
|
442
482
|
if !%w[json html].include?(config.format) && (hint = tier2_hint(config.operators))
|
data/lib/mutineer/config.rb
CHANGED
|
@@ -25,9 +25,8 @@ module Mutineer
|
|
|
25
25
|
:cache_dir, :project_root, :load_paths,
|
|
26
26
|
:jobs, :format, :output, :strategy, :require_paths,
|
|
27
27
|
:boot, :rails, :since, :framework, :verbose, :ignore,
|
|
28
|
-
# :daemon
|
|
29
|
-
#
|
|
30
|
-
# coerce + KNOWN_KEYS). For now they're set programmatically (tests/Runner).
|
|
28
|
+
# :daemon is user-facing as of U8 (--daemon flag + KNOWN_KEYS + boolean coerce).
|
|
29
|
+
# :daemon_timeout stays programmatic (set by tests/Runner; no flag yet).
|
|
31
30
|
:baseline, :baseline_epsilon, :fail_fast, :test_command,
|
|
32
31
|
:daemon, :daemon_timeout,
|
|
33
32
|
keyword_init: true
|
|
@@ -35,7 +34,7 @@ module Mutineer
|
|
|
35
34
|
# Config file name.
|
|
36
35
|
CONFIG_FILE = ".mutineer.yml"
|
|
37
36
|
# Keys accepted in .mutineer.yml (R7). `require` maps to the :require_paths field.
|
|
38
|
-
KNOWN_KEYS = %w[operators jobs threshold only require boot rails since framework verbose ignore baseline fail_fast test_command].freeze
|
|
37
|
+
KNOWN_KEYS = %w[operators jobs threshold only require boot rails since framework verbose ignore baseline fail_fast test_command daemon].freeze
|
|
39
38
|
|
|
40
39
|
def initialize(**kwargs)
|
|
41
40
|
super
|
|
@@ -168,6 +167,7 @@ module Mutineer
|
|
|
168
167
|
when "boot" then value.to_s
|
|
169
168
|
when "framework" then value.to_s
|
|
170
169
|
when "rails" then value == true || value.to_s == "true"
|
|
170
|
+
when "daemon" then value == true || value.to_s == "true"
|
|
171
171
|
when "verbose" then value == true || value.to_s == "true"
|
|
172
172
|
when "ignore" then Array(value).map(&:to_s)
|
|
173
173
|
when "baseline" then value.to_s
|
|
@@ -21,7 +21,24 @@ module Mutineer
|
|
|
21
21
|
class CoverageMap
|
|
22
22
|
DEFAULT_CAPTURE_TIMEOUT = 120 # seconds, per coverage subprocess (R3)
|
|
23
23
|
|
|
24
|
-
attr_reader :project_root, :failed_test_files, :phase_a_ran
|
|
24
|
+
attr_reader :project_root, :failed_test_files, :phase_a_ran, :map
|
|
25
|
+
|
|
26
|
+
# Build a QUERY-ONLY map from data captured elsewhere (the daemon builds the map
|
|
27
|
+
# app-side and ships `map` + `failed_test_files` over IPC; the tool reconstructs it
|
|
28
|
+
# here for per-mutant selection). Skips the capture machinery entirely — only the
|
|
29
|
+
# three fields #tests_for / #method_uncapturable? read are set (U7).
|
|
30
|
+
#
|
|
31
|
+
# @param map [Hash] the "file:line" => [test_files] map.
|
|
32
|
+
# @param failed_test_files [Array<String>] test files whose capture failed.
|
|
33
|
+
# @param project_root [String] project root (for path relativization).
|
|
34
|
+
# @return [Mutineer::CoverageMap] a query-only map.
|
|
35
|
+
def self.from_data(map:, failed_test_files:, project_root:)
|
|
36
|
+
instance = allocate
|
|
37
|
+
instance.instance_variable_set(:@map, map || {})
|
|
38
|
+
instance.instance_variable_set(:@failed_test_files, failed_test_files || [])
|
|
39
|
+
instance.instance_variable_set(:@project_root, project_root)
|
|
40
|
+
instance
|
|
41
|
+
end
|
|
25
42
|
|
|
26
43
|
def initialize(source_paths:, test_paths:, cache_dir: ".mutineer",
|
|
27
44
|
load_paths: ["lib"], project_root: Dir.pwd,
|
|
@@ -54,9 +71,9 @@ module Mutineer
|
|
|
54
71
|
# the booted parent instead. Inverts into the same map #tests_for reads, and
|
|
55
72
|
# reuses the digest cache (the digest mixes in the boot file so a boot cache
|
|
56
73
|
# never collides with a standalone one).
|
|
57
|
-
def build_via_fork(
|
|
74
|
+
def build_via_fork(after_fork: nil)
|
|
58
75
|
warn_external_sources
|
|
59
|
-
cached_or { run_phase_a_via_fork(
|
|
76
|
+
cached_or { run_phase_a_via_fork(after_fork: after_fork) }
|
|
60
77
|
end
|
|
61
78
|
|
|
62
79
|
# Phase B lookup: the test files that cover `file:line`, or [] when none do.
|
|
@@ -159,7 +176,7 @@ module Mutineer
|
|
|
159
176
|
# per-source coverage counts. record() inverts them exactly as the subprocess
|
|
160
177
|
# path does. ponytail: serial fork (one test at a time) — boot apps fork
|
|
161
178
|
# cheaply via COW and per-test isolation matters more than throughput here.
|
|
162
|
-
def run_phase_a_via_fork(
|
|
179
|
+
def run_phase_a_via_fork(after_fork:)
|
|
163
180
|
@phase_a_ran = true
|
|
164
181
|
@map = {}
|
|
165
182
|
@failed_test_files = []
|
|
@@ -169,7 +186,7 @@ module Mutineer
|
|
|
169
186
|
# Tri-state payload (KTD-1): Hash = coverage, String = error diagnostic
|
|
170
187
|
# from the child, nil = pipe gone / empty.
|
|
171
188
|
# ponytail/#9: this String diagnostic is what #9 turns into an :uncapturable status.
|
|
172
|
-
case (coverage = fork_capture(absolute(test_path), abs_sources,
|
|
189
|
+
case (coverage = fork_capture(absolute(test_path), abs_sources, after_fork))
|
|
173
190
|
when Hash then record(coverage, test_path)
|
|
174
191
|
when String
|
|
175
192
|
fail_test(test_path, @verbose ? "fork capture failed: #{coverage}" :
|
|
@@ -182,7 +199,7 @@ module Mutineer
|
|
|
182
199
|
# Fork the booted parent, run one test under the inherited Coverage, and
|
|
183
200
|
# return its per-source counts hash (or nil on failure). Reuses the same
|
|
184
201
|
# fork + Marshal-over-pipe + hard-exit! discipline as WorkerPool/Isolation.
|
|
185
|
-
def fork_capture(abs_test, abs_sources,
|
|
202
|
+
def fork_capture(abs_test, abs_sources, after_fork)
|
|
186
203
|
rd, wr = IO.pipe
|
|
187
204
|
# #19: Marshal output is binary — an un-binmoded pipe can raise
|
|
188
205
|
# Encoding::UndefinedConversionError on write, which the child's rescue then
|
|
@@ -193,7 +210,10 @@ module Mutineer
|
|
|
193
210
|
rd.close
|
|
194
211
|
payload =
|
|
195
212
|
begin
|
|
196
|
-
|
|
213
|
+
# Fork-safety hook: the in-process path reconnects AR; the daemon routes
|
|
214
|
+
# to its worker DB. Nil (non-Rails) = no-op. Injected so this file needs
|
|
215
|
+
# neither Runner (Prism) nor Rails.
|
|
216
|
+
after_fork&.call
|
|
197
217
|
Coverage.result(clear: true, stop: false) # discard pre-test delta
|
|
198
218
|
TestRunners.for(@framework).run([abs_test])
|
|
199
219
|
# lines:true yields {file => {lines: [...]}}; reduce to the counts
|
|
@@ -55,15 +55,17 @@ module Mutineer
|
|
|
55
55
|
# @param payload [Hash] {"code" => mutated ruby, "source_file" => path}.
|
|
56
56
|
# @param tests [Array<String>] covering test file paths.
|
|
57
57
|
# @param timeout [Numeric] per-mutant wall-clock timeout (seconds).
|
|
58
|
+
# @param worker [Integer] worker slot; the daemon routes the fork to
|
|
59
|
+
# `<db>-<worker>` (#26 isolation). Defaults to 0 (serial in U5).
|
|
58
60
|
# @return [String] one of survived/killed/error/timeout.
|
|
59
|
-
def request(id:, payload:, tests:, timeout:)
|
|
61
|
+
def request(id:, payload:, tests:, timeout:, worker: 0)
|
|
60
62
|
# A crash can surface on the WRITE (daemon died idle between requests →
|
|
61
63
|
# Errno::EPIPE) as well as the read (EOF), so guard both: either way, respawn
|
|
62
64
|
# for future mutants and score THIS one error (re-running a crash-causing
|
|
63
65
|
# mutant could loop). Never let a dead pipe abort the whole run.
|
|
64
66
|
reply =
|
|
65
67
|
begin
|
|
66
|
-
send_line("id" => id, "payload" => payload, "tests" => tests, "timeout" => timeout)
|
|
68
|
+
send_line("id" => id, "worker" => worker, "payload" => payload, "tests" => tests, "timeout" => timeout)
|
|
67
69
|
read_line
|
|
68
70
|
rescue Errno::EPIPE, IOError
|
|
69
71
|
nil
|
|
@@ -74,6 +76,19 @@ module Mutineer
|
|
|
74
76
|
"error"
|
|
75
77
|
end
|
|
76
78
|
|
|
79
|
+
# #26/U7: ask the daemon to build the coverage map app-side and return it.
|
|
80
|
+
# One-shot control message (no id). Returns `{"map"=>..., "failed_test_files"=>...}`
|
|
81
|
+
# (possibly with an `"error"`), or nil if the daemon vanished — the caller then
|
|
82
|
+
# falls back to running the full test set (no narrowing) rather than mis-scoring.
|
|
83
|
+
#
|
|
84
|
+
# @return [Hash, nil] the coverage payload, or nil on a dead pipe.
|
|
85
|
+
def coverage
|
|
86
|
+
send_line("cmd" => "coverage")
|
|
87
|
+
read_line
|
|
88
|
+
rescue Errno::EPIPE, IOError
|
|
89
|
+
nil
|
|
90
|
+
end
|
|
91
|
+
|
|
77
92
|
# Graceful shutdown; leaves no orphaned daemon/child.
|
|
78
93
|
#
|
|
79
94
|
# @return [void]
|
|
@@ -20,17 +20,23 @@ module Mutineer
|
|
|
20
20
|
#
|
|
21
21
|
# Protocol (one JSON object per line, both directions):
|
|
22
22
|
# boot in : {"cmd":"boot","project_root":"...","boot":"config/environment",
|
|
23
|
-
# "load_paths":["test"],"framework":"minitest","rails":true}
|
|
23
|
+
# "load_paths":["test"],"framework":"minitest","rails":true,"schema":"db/schema.rb"}
|
|
24
24
|
# ready out: {"ready":true,"ruby":"3.3.6"} (or {"ready":false,"error":"..."} then exit)
|
|
25
|
-
# run in : {"id":N,"payload":{"code":"<ruby>","source_file":"app/models/order.rb"},
|
|
25
|
+
# run in : {"id":N,"worker":I,"payload":{"code":"<ruby>","source_file":"app/models/order.rb"},
|
|
26
26
|
# "tests":["test/models/order_test.rb"],"timeout":30}
|
|
27
27
|
# verdict : {"id":N,"verdict":"survived"|"killed"|"error"|"timeout"}
|
|
28
28
|
# quit in : {"cmd":"quit"}
|
|
29
29
|
#
|
|
30
|
+
# Worker isolation (#26/U5): when the app is Rails, each fork is routed to its own
|
|
31
|
+
# database `<db>-<worker>` via {RailsWorkerDb} BEFORE any test loads, so concurrent
|
|
32
|
+
# workers can't clobber each other's transactional fixtures. `worker` defaults to 0
|
|
33
|
+
# (serial). SQLite this pass; Postgres provisioning is U10.
|
|
34
|
+
#
|
|
30
35
|
# Verdict mapping (KTD-5, Phase-2a honest limit): child exit 0=survived (suite
|
|
31
|
-
# passed), 1=killed (suite failed), 2=error (child raised AROUND the test — load
|
|
32
|
-
# boot failure); parent-detected timeout. Tagging an in-TEST DB
|
|
33
|
-
# (
|
|
36
|
+
# passed), 1=killed (suite failed), 2=error (child raised AROUND the test — load,
|
|
37
|
+
# boot, or worker-DB routing failure); parent-detected timeout. Tagging an in-TEST DB
|
|
38
|
+
# error (one fired inside a test body, vs at routing time) as `error` rather than
|
|
39
|
+
# `killed` is a U6 concern — only observable under the concurrent gate.
|
|
34
40
|
module DaemonServer
|
|
35
41
|
# Poll interval (seconds) for the per-fork deadline wait loop.
|
|
36
42
|
POLL = 0.02
|
|
@@ -67,6 +73,14 @@ module Mutineer
|
|
|
67
73
|
end
|
|
68
74
|
break if req["cmd"] == "quit"
|
|
69
75
|
|
|
76
|
+
# #26/U7: build the coverage map app-side and ship it to the tool, which
|
|
77
|
+
# then selects covering tests per mutant. One-shot control message.
|
|
78
|
+
if req["cmd"] == "coverage"
|
|
79
|
+
output.puts(JSON.generate(build_coverage_map))
|
|
80
|
+
output.flush
|
|
81
|
+
next
|
|
82
|
+
end
|
|
83
|
+
|
|
70
84
|
output.puts(JSON.generate(run_mutant(req)))
|
|
71
85
|
output.flush
|
|
72
86
|
end
|
|
@@ -77,16 +91,24 @@ module Mutineer
|
|
|
77
91
|
# BOOT ONCE. chdir + require the app's boot file so the whole app is loaded and
|
|
78
92
|
# inherited by every fork. Never requires mutineer.
|
|
79
93
|
def boot!(cfg)
|
|
94
|
+
@cfg = cfg
|
|
80
95
|
@framework = cfg.fetch("framework", "minitest")
|
|
81
96
|
@source_dirs = Array(cfg["source_dirs"]).map { |d| File.expand_path(d) }
|
|
82
97
|
Dir.chdir(cfg["project_root"]) if cfg["project_root"]
|
|
83
98
|
ENV["RAILS_ENV"] ||= "test" if cfg["rails"]
|
|
84
99
|
Array(cfg["load_paths"]).each { |d| $LOAD_PATH.unshift(File.expand_path(d)) }
|
|
100
|
+
# #26/U7: start Coverage BEFORE the app loads, so booted source lines are
|
|
101
|
+
# instrumented — the map build (build_via_fork) forks this booted parent.
|
|
102
|
+
if cfg["coverage"]
|
|
103
|
+
require "coverage"
|
|
104
|
+
Coverage.start(lines: true)
|
|
105
|
+
end
|
|
85
106
|
# Clear any mutant tempfile a prior SIGKILLed timeout child orphaned in a
|
|
86
107
|
# source dir BEFORE the app boots — Zeitwerk would otherwise choke on the
|
|
87
108
|
# tempfile's non-constant name during autoload setup.
|
|
88
109
|
sweep_temps
|
|
89
110
|
require File.expand_path(cfg["boot"]) if cfg["boot"]
|
|
111
|
+
setup_worker_db(cfg) if cfg["rails"]
|
|
90
112
|
rescue Exception => e # rubocop:disable Lint/RescueException
|
|
91
113
|
# Boot failed (bad boot path, app error) — tell the client and exit so it can
|
|
92
114
|
# surface a clean error rather than hang on the handshake.
|
|
@@ -95,9 +117,54 @@ module Mutineer
|
|
|
95
117
|
exit!(1)
|
|
96
118
|
end
|
|
97
119
|
|
|
120
|
+
# Load the per-worker DB adapter app-side (sibling gem file, by relative path so
|
|
121
|
+
# it bypasses the app bundle — like this daemon itself). No-op unless the app has
|
|
122
|
+
# ActiveRecord. Records the adapter + schema path so each fork can route to its
|
|
123
|
+
# own database (#26 isolation, U5). SQLite-only this pass; a non-SQLite config
|
|
124
|
+
# raises in the fork and reads as `error`, never a mis-routed verdict.
|
|
125
|
+
def setup_worker_db(cfg)
|
|
126
|
+
require_relative "rails_worker_db"
|
|
127
|
+
@worker_db = RailsWorkerDb.available? ? RailsWorkerDb : nil
|
|
128
|
+
schema = cfg["schema"] && File.expand_path(cfg["schema"])
|
|
129
|
+
@schema_path = schema if schema && File.exist?(schema)
|
|
130
|
+
rescue LoadError => e
|
|
131
|
+
@errio.puts("[daemon] worker-DB routing unavailable: #{e.message}")
|
|
132
|
+
@worker_db = nil
|
|
133
|
+
end
|
|
134
|
+
|
|
135
|
+
# #26/U7: build the coverage map app-side (Coverage was started at boot) and
|
|
136
|
+
# return it as `{map, failed_test_files}` for the tool to select covering tests.
|
|
137
|
+
# Capture forks route to worker 0's DB (isolated, serial). On any failure return
|
|
138
|
+
# an empty map + an error string — the tool then falls back to the full test set
|
|
139
|
+
# rather than mis-scoring everything as no_coverage.
|
|
140
|
+
def build_coverage_map
|
|
141
|
+
require_relative "coverage_map"
|
|
142
|
+
root = @cfg["project_root"] || Dir.pwd
|
|
143
|
+
cmap = CoverageMap.new(
|
|
144
|
+
source_paths: Array(@cfg["sources"]), test_paths: Array(@cfg["tests"]),
|
|
145
|
+
load_paths: Array(@cfg["load_paths"]), project_root: root,
|
|
146
|
+
boot_path: @cfg["boot"], framework: @framework, cache_dir: File.join(root, ".mutineer")
|
|
147
|
+
).build_via_fork(after_fork: coverage_after_fork)
|
|
148
|
+
{ "map" => cmap.map, "failed_test_files" => cmap.failed_test_files }
|
|
149
|
+
rescue Exception => e # rubocop:disable Lint/RescueException
|
|
150
|
+
@errio.puts("[daemon] coverage build failed: #{e.class}: #{e.message}")
|
|
151
|
+
{ "map" => {}, "failed_test_files" => [], "error" => "#{e.class}: #{e.message}" }
|
|
152
|
+
end
|
|
153
|
+
|
|
154
|
+
# Fork-safety hook for coverage capture: route each capture fork to worker 0's
|
|
155
|
+
# isolated DB (captures run serially, so one worker is enough). Nil when the app
|
|
156
|
+
# has no worker-DB adapter (non-Rails) — capture then runs as before.
|
|
157
|
+
def coverage_after_fork
|
|
158
|
+
return nil unless @worker_db
|
|
159
|
+
|
|
160
|
+
schema = @schema_path
|
|
161
|
+
-> { @worker_db.after_fork(0, schema) }
|
|
162
|
+
end
|
|
163
|
+
|
|
98
164
|
# Fork a child to run one mutant in isolation; decode its exit into a verdict.
|
|
99
165
|
def run_mutant(req)
|
|
100
166
|
timeout = req.fetch("timeout", 30)
|
|
167
|
+
worker = req.fetch("worker", 0)
|
|
101
168
|
pid = fork do
|
|
102
169
|
# New process group so a per-fork timeout can SIGKILL the whole subtree
|
|
103
170
|
# (carries the Phase-1 pgroup discipline), and silence the child's stdout so
|
|
@@ -106,6 +173,10 @@ module Mutineer
|
|
|
106
173
|
$stdout.reopen(File::NULL, "w")
|
|
107
174
|
code =
|
|
108
175
|
begin
|
|
176
|
+
# Route THIS fork at its own worker database before any test loads, so
|
|
177
|
+
# concurrent workers can't clobber each other's fixtures (#26). A routing
|
|
178
|
+
# failure raises here and is scored `error` below, never a false verdict.
|
|
179
|
+
@worker_db&.after_fork(worker, @schema_path)
|
|
109
180
|
apply_payload(req["payload"])
|
|
110
181
|
run_tests(Array(req["tests"]))
|
|
111
182
|
rescue Exception => e # rubocop:disable Lint/RescueException
|
|
@@ -0,0 +1,161 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
module Mutineer
|
|
4
|
+
# #26/#27 Phase 2b (U5) — per-worker database isolation for the daemon path.
|
|
5
|
+
#
|
|
6
|
+
# Loaded APP-SIDE by {DaemonServer} (a sibling gem file, pulled in by absolute path
|
|
7
|
+
# so it bypasses the app bundle — the same trick {DaemonClient} uses to run
|
|
8
|
+
# `daemon_server.rb` under a bundle that has no mutineer). It uses the app's OWN
|
|
9
|
+
# already-booted ActiveRecord and NEVER `require "active_record"` (R10/KTD-8): every
|
|
10
|
+
# method that touches AR first confirms {available?}, so the daemon core stays
|
|
11
|
+
# framework-agnostic and the gem keeps its zero-runtime-dependency promise.
|
|
12
|
+
#
|
|
13
|
+
# Isolation model (KTD-7): each parallel worker gets its OWN database so concurrent
|
|
14
|
+
# forks can't clobber each other's transactional fixtures (the measured #26
|
|
15
|
+
# corruption). {after_fork} runs inside a freshly-forked child and points that
|
|
16
|
+
# child's connection at the worker's database BEFORE any test loads; transactional
|
|
17
|
+
# fixtures then repopulate that isolated database per test.
|
|
18
|
+
#
|
|
19
|
+
# Scope: this pass ships the **SQLite** adapter (per-worker file, hermetic,
|
|
20
|
+
# spike-proven). Postgres per-worker provisioning (`CREATE DATABASE <db>-<worker>` —
|
|
21
|
+
# the measured corruption case) extends {worker_db_config}/{provision} at the marked
|
|
22
|
+
# seam in U10; until then a non-SQLite config raises a clear NotImplementedError
|
|
23
|
+
# rather than silently mis-routing.
|
|
24
|
+
#
|
|
25
|
+
# Honest limit (KTD-5): routing failures surface as `error` via {verify_connection!}.
|
|
26
|
+
# Re-raising an AR error that fires *inside a test body* past Minitest (so an in-test
|
|
27
|
+
# DB failure is `error`, not `killed`) is only observable under concurrent load and
|
|
28
|
+
# is deferred to U6 with the parallel gate — noted, not silently skipped.
|
|
29
|
+
module RailsWorkerDb
|
|
30
|
+
# True when the app has ActiveRecord loaded — the only condition under which any
|
|
31
|
+
# other method here may touch AR. Never triggers an autoload/require of AR itself.
|
|
32
|
+
#
|
|
33
|
+
# @return [Boolean]
|
|
34
|
+
def self.available?
|
|
35
|
+
defined?(ActiveRecord::Base) ? true : false
|
|
36
|
+
end
|
|
37
|
+
|
|
38
|
+
# Derive a per-worker database path from a base path by inserting `-<worker>`
|
|
39
|
+
# before the extension. Pure string transform (no AR) so it is unit-testable in
|
|
40
|
+
# the zero-dep suite. `storage/test.sqlite3`, worker 1 -> `storage/test-1.sqlite3`.
|
|
41
|
+
#
|
|
42
|
+
# @param database [String] the base database path.
|
|
43
|
+
# @param worker [Integer] the worker slot index (0..N-1).
|
|
44
|
+
# @return [String] the per-worker database path.
|
|
45
|
+
def self.worker_database_path(database, worker)
|
|
46
|
+
ext = File.extname(database)
|
|
47
|
+
"#{database.delete_suffix(ext)}-#{worker}#{ext}"
|
|
48
|
+
end
|
|
49
|
+
|
|
50
|
+
# Build the AR connection config for one worker by copying the app's current
|
|
51
|
+
# (default test) config and swapping in the per-worker database path. SQLite only
|
|
52
|
+
# this pass — a non-SQLite adapter raises so the SQLite-first scope fails loud
|
|
53
|
+
# instead of mis-routing (Postgres is U10).
|
|
54
|
+
#
|
|
55
|
+
# @param worker [Integer] the worker slot index.
|
|
56
|
+
# @return [Hash] a symbol-keyed AR configuration hash for the worker database.
|
|
57
|
+
# @raise [NotImplementedError] when the app's database is non-SQLite or in-memory.
|
|
58
|
+
def self.worker_db_config(worker)
|
|
59
|
+
hash = ActiveRecord::Base.connection_db_config.configuration_hash
|
|
60
|
+
adapter = hash[:adapter].to_s
|
|
61
|
+
# U10 seam: the config-SHAPING below (per_worker_config) is already
|
|
62
|
+
# adapter-general — it derives correct SQLite *and* Postgres worker-DB names.
|
|
63
|
+
# What's gated is runtime PROVISIONING: SQLite files are created on connect,
|
|
64
|
+
# but Postgres needs an explicit `CREATE DATABASE` per worker (U10). Until that
|
|
65
|
+
# lands, refuse non-SQLite loudly rather than route to a database that doesn't
|
|
66
|
+
# exist. To finish U10: implement provision() for PG and drop this guard.
|
|
67
|
+
unless adapter.start_with?("sqlite")
|
|
68
|
+
raise NotImplementedError,
|
|
69
|
+
"worker-DB isolation currently provisions SQLite only (got adapter #{adapter.inspect}); " \
|
|
70
|
+
"Postgres per-worker provisioning is U10 (#26/#35) — use a SQLite test DB, or drop --jobs."
|
|
71
|
+
end
|
|
72
|
+
|
|
73
|
+
per_worker_config(hash, worker)
|
|
74
|
+
end
|
|
75
|
+
|
|
76
|
+
# Pure config-shaping (no AR): given a connection config hash, return the
|
|
77
|
+
# per-worker variant with its database swapped to the worker's own name.
|
|
78
|
+
# Adapter-general — SQLite (`storage/test.sqlite3` → `storage/test-<w>.sqlite3`)
|
|
79
|
+
# and Postgres (`myapp_test` → `myapp_test-<w>`, Rails `parallelize` naming) both
|
|
80
|
+
# fall out of {worker_database_path}. Extracted + unit-tested so the Postgres
|
|
81
|
+
# SHAPE is proven ready for U10 without a live database.
|
|
82
|
+
#
|
|
83
|
+
# @param config_hash [Hash] a connection config hash (symbol or string keys).
|
|
84
|
+
# @param worker [Integer] the worker slot index.
|
|
85
|
+
# @return [Hash] the per-worker config (symbol keys), database swapped.
|
|
86
|
+
# @raise [NotImplementedError] for an in-memory or empty database (no per-worker split).
|
|
87
|
+
def self.per_worker_config(config_hash, worker)
|
|
88
|
+
hash = config_hash.transform_keys(&:to_sym)
|
|
89
|
+
database = hash[:database].to_s
|
|
90
|
+
if database.empty? || database == ":memory:"
|
|
91
|
+
raise NotImplementedError,
|
|
92
|
+
"worker-DB isolation needs a file/name-backed database (got #{database.inspect})."
|
|
93
|
+
end
|
|
94
|
+
|
|
95
|
+
hash.merge(database: worker_database_path(database, worker))
|
|
96
|
+
end
|
|
97
|
+
|
|
98
|
+
# Child-side (after fork): route this process's ActiveRecord at the worker's own
|
|
99
|
+
# database and confirm it is reachable, so a routing failure reads as `error`
|
|
100
|
+
# (via the daemon's child rescue) rather than a false verdict. Loads the schema
|
|
101
|
+
# into the worker database when a schema path is given (idempotent — schema.rb
|
|
102
|
+
# runs with `force: true`), covering a fresh worker file.
|
|
103
|
+
#
|
|
104
|
+
# @param worker [Integer] the worker slot index.
|
|
105
|
+
# @param schema_path [String, nil] absolute path to `db/schema.rb`, or nil to skip.
|
|
106
|
+
# @return [void]
|
|
107
|
+
def self.after_fork(worker, schema_path = nil)
|
|
108
|
+
return unless available?
|
|
109
|
+
|
|
110
|
+
ActiveRecord::Base.establish_connection(worker_db_config(worker))
|
|
111
|
+
load_schema(schema_path) if schema_path
|
|
112
|
+
verify_connection!
|
|
113
|
+
end
|
|
114
|
+
|
|
115
|
+
# Explicit, parent-side provisioning: create + schema-load every worker database
|
|
116
|
+
# up front, then restore the app's default connection. This is the EXPLICIT
|
|
117
|
+
# provisioning entry point (V2 — never silent auto-create mid-audit) and the seam
|
|
118
|
+
# U10 extends for Postgres (`CREATE DATABASE` per worker). For SQLite the files are
|
|
119
|
+
# created on connect and the schema load makes them ready; idempotent to re-run.
|
|
120
|
+
#
|
|
121
|
+
# @param worker_count [Integer] number of worker databases to provision.
|
|
122
|
+
# @param schema_path [String, nil] absolute path to `db/schema.rb`, or nil to skip.
|
|
123
|
+
# @return [void]
|
|
124
|
+
def self.provision(worker_count, schema_path)
|
|
125
|
+
return unless available?
|
|
126
|
+
|
|
127
|
+
original = ActiveRecord::Base.connection_db_config
|
|
128
|
+
(0...worker_count).each do |worker|
|
|
129
|
+
ActiveRecord::Base.establish_connection(worker_db_config(worker))
|
|
130
|
+
load_schema(schema_path) if schema_path
|
|
131
|
+
end
|
|
132
|
+
ensure
|
|
133
|
+
ActiveRecord::Base.establish_connection(original) if original
|
|
134
|
+
end
|
|
135
|
+
|
|
136
|
+
# Load a Rails `schema.rb` into the current connection with its output silenced,
|
|
137
|
+
# so a parent-side {provision} call can never spill schema chatter onto the daemon
|
|
138
|
+
# IPC pipe. (In a fork the child's stdout is already `File::NULL`; this guards the
|
|
139
|
+
# parent path too.)
|
|
140
|
+
#
|
|
141
|
+
# @param schema_path [String] absolute path to `db/schema.rb`.
|
|
142
|
+
# @return [void]
|
|
143
|
+
def self.load_schema(schema_path)
|
|
144
|
+
ActiveRecord::Migration.verbose = false if defined?(ActiveRecord::Migration)
|
|
145
|
+
original = $stdout
|
|
146
|
+
$stdout = File.open(File::NULL, "w")
|
|
147
|
+
load schema_path
|
|
148
|
+
ensure
|
|
149
|
+
$stdout.close unless $stdout.equal?(original)
|
|
150
|
+
$stdout = original
|
|
151
|
+
end
|
|
152
|
+
|
|
153
|
+
# Force a round-trip to the freshly-routed connection so a broken route fails HERE
|
|
154
|
+
# (→ `error`) instead of later masquerading as a test failure (→ false `killed`).
|
|
155
|
+
#
|
|
156
|
+
# @return [void]
|
|
157
|
+
def self.verify_connection!
|
|
158
|
+
ActiveRecord::Base.connection.execute("SELECT 1")
|
|
159
|
+
end
|
|
160
|
+
end
|
|
161
|
+
end
|
data/lib/mutineer/runner.rb
CHANGED
|
@@ -90,7 +90,7 @@ module Mutineer
|
|
|
90
90
|
load_paths: config.load_paths, framework: config.framework,
|
|
91
91
|
boot_path: File.expand_path(config.boot, config.project_root),
|
|
92
92
|
verbose: config.verbose
|
|
93
|
-
).build_via_fork(
|
|
93
|
+
).build_via_fork(after_fork: (config.rails ? -> { reconnect_active_record } : nil))
|
|
94
94
|
else
|
|
95
95
|
coverage_map = CoverageMap.new(
|
|
96
96
|
source_paths: config.sources, test_paths: config.tests,
|
|
@@ -241,37 +241,196 @@ module Mutineer
|
|
|
241
241
|
def self.execute_daemon(config, operator_classes)
|
|
242
242
|
jobs, ignored_results, source_map = collect_jobs(config, operator_classes)
|
|
243
243
|
jobs = filter_since(jobs, source_map, config) if config.since
|
|
244
|
-
|
|
245
244
|
abs_tests = config.tests.map { |t| File.expand_path(t, config.project_root) }
|
|
246
|
-
|
|
245
|
+
|
|
246
|
+
# #26/U7: build the coverage map once (app-side, via a short-lived daemon) so
|
|
247
|
+
# each mutant runs ONLY its covering tests and a mutant on an uncovered line is
|
|
248
|
+
# no_coverage. nil when the build fails — the runners fall back to the full
|
|
249
|
+
# --test set (no narrowing) rather than mis-scoring everything as no_coverage.
|
|
250
|
+
coverage_map = daemon_coverage_map(config, abs_tests)
|
|
251
|
+
|
|
252
|
+
# #26/U6: worker count = resolved --jobs, capped at the job count (no idle
|
|
253
|
+
# daemons). >1 → N concurrent daemon handles, each on its OWN worker DB (V6:
|
|
254
|
+
# N-handles, the spike-proven shape). 1 → the serial single-daemon path.
|
|
255
|
+
# --fail-fast forces serial: parallel's stop flag fires on the first survivor
|
|
256
|
+
# by WALL-CLOCK, not input index, so the verdict set would diverge from serial
|
|
257
|
+
# (a different, non-deterministic survivor set/score) — the "identical to
|
|
258
|
+
# --jobs 1" guarantee below only holds when fail-fast can't race.
|
|
259
|
+
worker_count = [config.jobs || 1, 1].max
|
|
260
|
+
worker_count = 1 if config.fail_fast
|
|
261
|
+
worker_count = [worker_count, jobs.size].min if jobs.size.positive?
|
|
262
|
+
|
|
263
|
+
results =
|
|
264
|
+
if worker_count > 1
|
|
265
|
+
run_daemon_parallel(jobs, worker_count, config, abs_tests, coverage_map, source_map)
|
|
266
|
+
else
|
|
267
|
+
run_daemon_serial(jobs, config, abs_tests, coverage_map, source_map)
|
|
268
|
+
end
|
|
269
|
+
|
|
270
|
+
[AggregateResult.new(results + ignored_results), source_map]
|
|
271
|
+
end
|
|
272
|
+
|
|
273
|
+
# #26/U7: build the coverage map via a short-lived daemon (boots the app once,
|
|
274
|
+
# captures per-test coverage app-side, ships the map back). Returns a query-only
|
|
275
|
+
# CoverageMap, or nil when the build fails / returns empty — callers then run the
|
|
276
|
+
# full --test set (no narrowing) rather than mis-scoring. The dedicated daemon
|
|
277
|
+
# costs one extra boot; correctness over that boot (ponytail — the map is reused
|
|
278
|
+
# for every mutant).
|
|
279
|
+
#
|
|
280
|
+
# @param config [Mutineer::Config] the run config.
|
|
281
|
+
# @param abs_tests [Array<String>] absolute --test paths.
|
|
282
|
+
# @return [Mutineer::CoverageMap, nil]
|
|
283
|
+
#
|
|
284
|
+
# ponytail: the coverage-build IPC read is unbounded (no wall-clock) — a --test
|
|
285
|
+
# file that infinite-loops during capture wedges this call, mirroring the existing
|
|
286
|
+
# in-process build_via_fork limitation. A capture timeout is a follow-up. When the
|
|
287
|
+
# map comes back empty (all captures failed), this returns nil and the run falls
|
|
288
|
+
# back to the FULL --test set per mutant — so on a total capture failure the daemon
|
|
289
|
+
# score is an upper bound (more testing), diverging from the in-process no_coverage
|
|
290
|
+
# scoring for that degenerate case only; a normal (nonempty) map scores identically.
|
|
291
|
+
def self.daemon_coverage_map(config, abs_tests)
|
|
292
|
+
client = DaemonClient.new(boot: daemon_boot_config(config, abs_tests, coverage: true),
|
|
247
293
|
app_root: config.project_root).start
|
|
294
|
+
data = begin
|
|
295
|
+
client.coverage
|
|
296
|
+
ensure
|
|
297
|
+
client.quit
|
|
298
|
+
end
|
|
299
|
+
return nil unless data && !(data["map"] || {}).empty?
|
|
300
|
+
|
|
301
|
+
CoverageMap.from_data(map: data["map"], failed_test_files: data["failed_test_files"] || [],
|
|
302
|
+
project_root: config.project_root)
|
|
303
|
+
rescue DaemonBootError
|
|
304
|
+
nil
|
|
305
|
+
end
|
|
248
306
|
|
|
307
|
+
# Serial daemon path: one daemon (worker 0), one mutant at a time. Honors
|
|
308
|
+
# --fail-fast (#21: stop at the first survivor).
|
|
309
|
+
#
|
|
310
|
+
# @return [Array<Mutineer::Result>] results in input order.
|
|
311
|
+
def self.run_daemon_serial(jobs, config, abs_tests, coverage_map, source_map)
|
|
312
|
+
client = DaemonClient.new(boot: daemon_boot_config(config, abs_tests),
|
|
313
|
+
app_root: config.project_root).start
|
|
249
314
|
results = []
|
|
250
315
|
begin
|
|
251
|
-
jobs.each_with_index do |
|
|
252
|
-
|
|
253
|
-
|
|
254
|
-
|
|
255
|
-
# that would fail to load and read as a false `killed`.
|
|
256
|
-
r =
|
|
257
|
-
if Parser.parse_string(mutated).errors.any?
|
|
258
|
-
Result.skipped
|
|
259
|
-
else
|
|
260
|
-
verdict = client.request(
|
|
261
|
-
id: i, timeout: config.daemon_timeout || DAEMON_TIMEOUT,
|
|
262
|
-
payload: { "code" => mutated, "source_file" => File.expand_path(subject.file, config.project_root) },
|
|
263
|
-
tests: abs_tests
|
|
264
|
-
)
|
|
265
|
-
daemon_result(verdict)
|
|
266
|
-
end
|
|
267
|
-
results << r.with(subject: subject, mutation: mutation, id: id)
|
|
268
|
-
break if config.fail_fast && r.survived? # #21: stop at the first survivor
|
|
316
|
+
jobs.each_with_index do |job, i|
|
|
317
|
+
r = daemon_job_result(job, i, client, 0, config, coverage_map, abs_tests, source_map)
|
|
318
|
+
results << r
|
|
319
|
+
break if config.fail_fast && r.survived?
|
|
269
320
|
end
|
|
270
321
|
ensure
|
|
271
322
|
client.quit
|
|
272
323
|
end
|
|
324
|
+
results
|
|
325
|
+
end
|
|
273
326
|
|
|
274
|
-
|
|
327
|
+
# Parallel daemon path (#26/U6): N daemon handles, each pinned to its own worker
|
|
328
|
+
# slot (→ its own DB, so concurrent workers can't clobber each other's fixtures).
|
|
329
|
+
# A shared queue of job indices feeds N tool-side threads; each thread blocks on
|
|
330
|
+
# IPC (GVL released), so the N daemons run genuinely concurrently. Results are
|
|
331
|
+
# placed by input index and compacted, so the verdict SET is identical to serial
|
|
332
|
+
# regardless of finish order (R7). `execute_daemon` never calls this with
|
|
333
|
+
# `config.fail_fast` set (forced to the serial path instead) — a wall-clock stop
|
|
334
|
+
# here, unlike serial's input-index break, would make the survivor set and score
|
|
335
|
+
# non-deterministic. The stop-flag machinery below stays as a defensive no-op for
|
|
336
|
+
# any direct caller that bypasses that guard.
|
|
337
|
+
#
|
|
338
|
+
# @return [Array<Mutineer::Result>] completed results in input order.
|
|
339
|
+
def self.run_daemon_parallel(jobs, worker_count, config, abs_tests, coverage_map, source_map)
|
|
340
|
+
results = Array.new(jobs.size)
|
|
341
|
+
queue = Queue.new
|
|
342
|
+
jobs.each_index { |i| queue << i }
|
|
343
|
+
stop = false
|
|
344
|
+
stop_mutex = Mutex.new
|
|
345
|
+
|
|
346
|
+
clients = Array.new(worker_count) do
|
|
347
|
+
DaemonClient.new(boot: daemon_boot_config(config, abs_tests),
|
|
348
|
+
app_root: config.project_root).start
|
|
349
|
+
end
|
|
350
|
+
|
|
351
|
+
clients.each_with_index.map do |client, worker|
|
|
352
|
+
Thread.new do
|
|
353
|
+
until stop_mutex.synchronize { stop }
|
|
354
|
+
i = begin
|
|
355
|
+
queue.pop(true) # non-blocking; ThreadError when drained
|
|
356
|
+
rescue ThreadError
|
|
357
|
+
break
|
|
358
|
+
end
|
|
359
|
+
r = daemon_job_result(jobs[i], i, client, worker, config, coverage_map, abs_tests, source_map)
|
|
360
|
+
results[i] = r
|
|
361
|
+
stop_mutex.synchronize { stop = true } if config.fail_fast && r.survived?
|
|
362
|
+
end
|
|
363
|
+
ensure
|
|
364
|
+
client.quit
|
|
365
|
+
end
|
|
366
|
+
end.each(&:join)
|
|
367
|
+
|
|
368
|
+
results.compact
|
|
369
|
+
end
|
|
370
|
+
|
|
371
|
+
# Build the payload for one job, run it on the given daemon/worker, and attach
|
|
372
|
+
# the subject/mutation/id — the shared body of both daemon paths.
|
|
373
|
+
#
|
|
374
|
+
# @param job [Array(Mutineer::Subject, Mutineer::Mutation, String)] the work item.
|
|
375
|
+
# @param req_id [Integer] request id (echoed back for IPC ordering safety).
|
|
376
|
+
# @param client [Mutineer::DaemonClient] the daemon handle to run on.
|
|
377
|
+
# @param worker [Integer] the worker slot (→ worker DB) this daemon routes to.
|
|
378
|
+
# @return [Mutineer::Result] the decorated result.
|
|
379
|
+
def self.daemon_job_result(job, req_id, client, worker, config, coverage_map, abs_tests, source_map)
|
|
380
|
+
subject, mutation, id = job
|
|
381
|
+
source = source_map[subject.file]
|
|
382
|
+
mutated = mutation.apply(source)
|
|
383
|
+
# KTD-8 (carried): skip an invalid mutant tool-side — never ship a payload that
|
|
384
|
+
# would fail to load and read as a false `killed`.
|
|
385
|
+
# #26/U7: narrow to covering tests (shared with the in-process path via
|
|
386
|
+
# coverage_selection, so scores match). :verdict = no_coverage/uncapturable, no
|
|
387
|
+
# fork. No map (build failed) → run the full --test set (fallback, not narrowed).
|
|
388
|
+
sel = coverage_map && coverage_selection(subject.file, mutation, subject, source, coverage_map)
|
|
389
|
+
r =
|
|
390
|
+
if Parser.parse_string(mutated).errors.any?
|
|
391
|
+
Result.skipped
|
|
392
|
+
elsif sel && sel[0] == :verdict
|
|
393
|
+
sel[1]
|
|
394
|
+
else
|
|
395
|
+
verdict = client.request(
|
|
396
|
+
id: req_id, worker: worker, timeout: config.daemon_timeout || DAEMON_TIMEOUT,
|
|
397
|
+
payload: { "code" => mutated, "source_file" => File.expand_path(subject.file, config.project_root) },
|
|
398
|
+
tests: sel ? sel[1] : abs_tests
|
|
399
|
+
)
|
|
400
|
+
daemon_result(verdict)
|
|
401
|
+
end
|
|
402
|
+
r.with(subject: subject, mutation: mutation, id: id)
|
|
403
|
+
end
|
|
404
|
+
|
|
405
|
+
# Coverage-based test selection, shared by the in-process ({run}) and daemon
|
|
406
|
+
# paths so both narrow identically (score parity, U7/V5). Returns
|
|
407
|
+
# `[:run, abs_test_paths]` when some test covers the mutant's line, or
|
|
408
|
+
# `[:verdict, Result]` (no_coverage / uncapturable) when none do.
|
|
409
|
+
#
|
|
410
|
+
# #9/#25: an empty selection is `:uncapturable` (not `:no_coverage`) when the
|
|
411
|
+
# mutant's enclosing method body got coverage from no *successful* capture but a
|
|
412
|
+
# sibling test failed to capture — the coverage was lost, not absent. Both are
|
|
413
|
+
# excluded from the score denominator, so this distinction is reporting-only and
|
|
414
|
+
# never changes the daemon-vs-in-process score.
|
|
415
|
+
#
|
|
416
|
+
# @param source_file [String] the mutated source file path.
|
|
417
|
+
# @param mutation [Mutineer::Mutation] the mutation (for its line offset).
|
|
418
|
+
# @param subject [Mutineer::Subject, nil] the subject (for its method body range).
|
|
419
|
+
# @param source [String] the original source text.
|
|
420
|
+
# @param coverage_map [Mutineer::CoverageMap] the built/loaded coverage map.
|
|
421
|
+
# @return [Array(Symbol, Object)] `[:run, Array<String>]` or `[:verdict, Result]`.
|
|
422
|
+
def self.coverage_selection(source_file, mutation, subject, source, coverage_map)
|
|
423
|
+
line = source.byteslice(0, mutation.start_offset).count("\n") + 1
|
|
424
|
+
chosen = coverage_map.tests_for(source_file, line)
|
|
425
|
+
if chosen.empty?
|
|
426
|
+
# Method BODY range, not the whole def: the def/end lines are "covered" at
|
|
427
|
+
# class-load even when the body never runs (body_loc is the statements' span).
|
|
428
|
+
loc = subject&.body_loc
|
|
429
|
+
range = loc ? (loc.start_line..loc.end_line) : (line..line)
|
|
430
|
+
return [:verdict, coverage_map.method_uncapturable?(source_file, range) ? Result.uncapturable : Result.no_coverage]
|
|
431
|
+
end
|
|
432
|
+
|
|
433
|
+
[:run, chosen.map { |t| File.expand_path(t, coverage_map.project_root) }]
|
|
275
434
|
end
|
|
276
435
|
|
|
277
436
|
# Default per-mutant timeout on the daemon path. Generous because 2a runs the full
|
|
@@ -281,17 +440,39 @@ module Mutineer
|
|
|
281
440
|
# The boot config the daemon needs to boot the app once: where to boot, the test
|
|
282
441
|
# load roots (so `require "test_helper"` resolves in every fork), framework, and
|
|
283
442
|
# whether this is Rails.
|
|
284
|
-
def self.daemon_boot_config(config, abs_tests)
|
|
443
|
+
def self.daemon_boot_config(config, abs_tests, coverage: false)
|
|
285
444
|
{
|
|
286
445
|
project_root: config.project_root,
|
|
287
446
|
boot: File.expand_path(config.boot || "config/environment", config.project_root),
|
|
288
447
|
load_paths: test_load_roots(abs_tests),
|
|
289
448
|
source_dirs: source_dirs(config), # so the daemon can sweep orphan mutant temps
|
|
290
449
|
framework: config.framework,
|
|
291
|
-
rails: config.rails
|
|
450
|
+
rails: config.rails,
|
|
451
|
+
# #26/U5: schema for per-worker DB isolation. Sent when present; the daemon
|
|
452
|
+
# skips worker-DB schema loading if the path is absent (e.g. structure.sql apps).
|
|
453
|
+
schema: daemon_schema_path(config),
|
|
454
|
+
# #26/U7: coverage narrowing. Only the short-lived map-building daemon starts
|
|
455
|
+
# Coverage (before boot); worker daemons boot with it OFF (no wasted
|
|
456
|
+
# instrumentation/memory across every mutant fork). `sources`/`tests` are the
|
|
457
|
+
# map-build inputs.
|
|
458
|
+
coverage: coverage,
|
|
459
|
+
sources: config.sources.map { |s| File.expand_path(s, config.project_root) },
|
|
460
|
+
tests: abs_tests
|
|
292
461
|
}
|
|
293
462
|
end
|
|
294
463
|
|
|
464
|
+
# Absolute path to the app's `db/schema.rb` if it exists, else nil. Used by the
|
|
465
|
+
# daemon to schema-load each fork's isolated worker database (#26/U5). Only
|
|
466
|
+
# `schema.rb` is supported this pass; `structure.sql` apps get nil and fall back to
|
|
467
|
+
# whatever the worker DB already holds (Postgres provisioning is U10).
|
|
468
|
+
#
|
|
469
|
+
# @param config [Mutineer::Config] the run config.
|
|
470
|
+
# @return [String, nil] absolute schema path or nil.
|
|
471
|
+
def self.daemon_schema_path(config)
|
|
472
|
+
path = File.expand_path("db/schema.rb", config.project_root)
|
|
473
|
+
File.exist?(path) ? path : nil
|
|
474
|
+
end
|
|
475
|
+
|
|
295
476
|
# Map a daemon verdict string to a Result. The daemon reports the four run-time
|
|
296
477
|
# states it can decide (KTD-5); pre-fork states (skipped/no_coverage/…) are
|
|
297
478
|
# resolved tool-side before a request is ever sent.
|
|
@@ -426,24 +607,12 @@ module Mutineer
|
|
|
426
607
|
|
|
427
608
|
# Coverage selection (both standalone and boot mode): a mutation on a line
|
|
428
609
|
# no test exercises is :no_coverage (no fork); otherwise exactly the
|
|
429
|
-
# covering test files run in the child.
|
|
430
|
-
|
|
431
|
-
|
|
432
|
-
|
|
433
|
-
# errored during capture (coverage lost) — the latter is :uncapturable.
|
|
434
|
-
# #25: taint per-METHOD (the mutant's enclosing def range), not whole-file,
|
|
435
|
-
# so a covered method's uncovered line stays :no_coverage while a method
|
|
436
|
-
# reachable only by a failed capture is :uncapturable.
|
|
437
|
-
if chosen.empty?
|
|
438
|
-
# Use the method BODY range, not the whole def: the `def`/`end` lines are
|
|
439
|
-
# "covered" at class-load even when the body never runs, which would mask
|
|
440
|
-
# an uncovered method. body_loc is the body statements' span.
|
|
441
|
-
loc = subject&.body_loc
|
|
442
|
-
range = loc ? (loc.start_line..loc.end_line) : (line..line)
|
|
443
|
-
return coverage_map.method_uncapturable?(source_file, range) ? Result.uncapturable : Result.no_coverage
|
|
444
|
-
end
|
|
610
|
+
# covering test files run in the child. Shared with the daemon path so both
|
|
611
|
+
# narrow identically (score parity, U7/V5).
|
|
612
|
+
kind, payload = coverage_selection(source_file, mutation, subject, source, coverage_map)
|
|
613
|
+
return payload if kind == :verdict
|
|
445
614
|
|
|
446
|
-
abs_tests =
|
|
615
|
+
abs_tests = payload
|
|
447
616
|
|
|
448
617
|
Isolation.run(timeout: timeout) do
|
|
449
618
|
# Forking inherits the parent's live DB connection; sharing one socket
|
data/lib/mutineer/version.rb
CHANGED
metadata
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
|
2
2
|
name: mutineer
|
|
3
3
|
version: !ruby/object:Gem::Version
|
|
4
|
-
version: 0.
|
|
4
|
+
version: 0.11.0
|
|
5
5
|
platform: ruby
|
|
6
6
|
authors:
|
|
7
7
|
- David Teren
|
|
@@ -95,6 +95,7 @@ files:
|
|
|
95
95
|
- lib/mutineer/pairing.rb
|
|
96
96
|
- lib/mutineer/parser.rb
|
|
97
97
|
- lib/mutineer/project.rb
|
|
98
|
+
- lib/mutineer/rails_worker_db.rb
|
|
98
99
|
- lib/mutineer/reporter.rb
|
|
99
100
|
- lib/mutineer/result.rb
|
|
100
101
|
- lib/mutineer/runner.rb
|