mutineer 0.9.1 → 0.11.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +35 -0
- data/README.md +67 -1
- data/lib/mutineer/cli.rb +100 -0
- data/lib/mutineer/config.rb +8 -2
- data/lib/mutineer/coverage_map.rb +27 -7
- data/lib/mutineer/daemon_client.rb +187 -0
- data/lib/mutineer/daemon_server.rb +261 -0
- data/lib/mutineer/external_backend.rb +168 -0
- data/lib/mutineer/file_swap.rb +95 -0
- data/lib/mutineer/rails_worker_db.rb +161 -0
- data/lib/mutineer/runner.rb +384 -45
- data/lib/mutineer/version.rb +1 -1
- metadata +6 -1
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 2dd1784f527b49eaed8d13a3e03c59c8a30082a44cf625f3a6ce1f252ffb3a60
|
|
4
|
+
data.tar.gz: 164d53165ea6f85c31158783ea43f8f2c88c3df058240d748df01f960da23897
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: a8bcaae29c133ca1ac0f9da31dc143b7b5deeef347104e209386cf6029252e77586d1240440e97bcec082e9e11a355087ffda91e7ca5495de85ac0f6bf1c2155
|
|
7
|
+
data.tar.gz: 06f108bfbb0f4b32318ca120e086ce88516b036ebc622e37b81963f4f93b7a87d5cdb6403b95cd69aa7a196e910718ba52b632d376a10f2763bf05f9f94a8fc8
|
data/CHANGELOG.md
CHANGED
|
@@ -4,6 +4,39 @@ All notable changes to this project are documented here. The format is based on
|
|
|
4
4
|
[Keep a Changelog](https://keepachangelog.com/), and this project adheres to
|
|
5
5
|
[Semantic Versioning](https://semver.org/).
|
|
6
6
|
|
|
7
|
+
## [Unreleased]
|
|
8
|
+
|
|
9
|
+
## [0.11.0] - 2026-07-02
|
|
10
|
+
|
|
11
|
+
### Added
|
|
12
|
+
- **`--daemon` backend — fast, parallel-safe Rails mutation testing** (#26/#27
|
|
13
|
+
Phase 2). Boots the app **once** in a persistent daemon and forks per mutant
|
|
14
|
+
(restoring shared-boot speed), and gives **each parallel worker its own
|
|
15
|
+
database** so `--jobs N` is safe under Rails for the first time — parallel
|
|
16
|
+
verdicts are proven identical to serial (no fixture cross-talk). Coverage
|
|
17
|
+
narrowing is restored on this path (each mutant runs only its covering tests;
|
|
18
|
+
a mutant on an uncovered line is `no_coverage`), so the daemon score is
|
|
19
|
+
comparable to the in-process `--rails` score. Opt in with `--rails --daemon`
|
|
20
|
+
(also `daemon: true` in `.mutineer.yml`); `--daemon` can't be combined with
|
|
21
|
+
`--test-command`. **SQLite** today (hermetic, CI-proven); **Postgres**
|
|
22
|
+
per-worker provisioning is in progress (#34/#35). The gem core stays Prism +
|
|
23
|
+
stdlib, zero runtime dependencies — worker-DB routing uses the app's own
|
|
24
|
+
ActiveRecord.
|
|
25
|
+
|
|
26
|
+
## [0.10.0] - 2026-07-02
|
|
27
|
+
|
|
28
|
+
### Added
|
|
29
|
+
- **`--test-command` external backend** (#27) — mutation-test apps pinned to Ruby
|
|
30
|
+
< 3.4. Mutineer stays on ≥ 3.4 but runs your suite as a subprocess in the app's
|
|
31
|
+
own runtime via `--test-command "bundle exec rails test %{files}"` (`%{files}`
|
|
32
|
+
expands to the `--test` paths; env is inherited). The mutant is applied on disk
|
|
33
|
+
with crash-safe backup/restore (self-heals a hard-killed run on next startup); a
|
|
34
|
+
smoke check aborts before scoring if the unmutated suite isn't green. This path
|
|
35
|
+
is reload-only, serial (`--jobs` forced to 1), and does no coverage narrowing —
|
|
36
|
+
so its score is an upper bound, not comparable to an in-process `--rails` score
|
|
37
|
+
(Mutineer prints this caveat). Also settable as `test_command:` in `.mutineer.yml`.
|
|
38
|
+
Safe parallelism for this path is tracked in #26.
|
|
39
|
+
|
|
7
40
|
## [0.9.1] - 2026-07-01
|
|
8
41
|
|
|
9
42
|
### Fixed
|
|
@@ -184,6 +217,8 @@ Rails hardening + CI batch (issues #8–#13), all verified Rails-free.
|
|
|
184
217
|
- `.mutineer.yml` configuration (CLI > config > default precedence).
|
|
185
218
|
- Byte-correct source handling for multibyte (UTF-8) sources.
|
|
186
219
|
|
|
220
|
+
[0.11.0]: https://github.com/davidteren/mutineer/releases/tag/v0.11.0
|
|
221
|
+
[0.10.0]: https://github.com/davidteren/mutineer/releases/tag/v0.10.0
|
|
187
222
|
[0.9.1]: https://github.com/davidteren/mutineer/releases/tag/v0.9.1
|
|
188
223
|
[0.9.0]: https://github.com/davidteren/mutineer/releases/tag/v0.9.0
|
|
189
224
|
[0.8.0]: https://github.com/davidteren/mutineer/releases/tag/v0.8.0
|
data/README.md
CHANGED
|
@@ -56,6 +56,8 @@ mutineer run lib/calculator.rb --test test/calculator_test.rb --threshold 90
|
|
|
56
56
|
| `--jobs N` | Parallel worker count (default: processor count; `1` under `--rails`) |
|
|
57
57
|
| `--verbose` | Surface the real error when a fork capture fails (alias `--debug`) |
|
|
58
58
|
| `--strategy NAME` | Mutation application: `reload` whole-file (default) or `redefine` surgical (`7a`/`7b` accepted as deprecated aliases) |
|
|
59
|
+
| `--test-command CMD` | Run the suite as a subprocess in the app's own runtime (for apps on Ruby < 3.4); `CMD` must contain `%{files}`. See [Apps on Ruby < 3.4](#apps-on-ruby--34) |
|
|
60
|
+
| `--daemon` | Boot the app once in a persistent daemon and fork per mutant, with per-worker DB isolation so `--jobs N` is safe under Rails (needs `--rails`/`--boot`; not with `--test-command`). See [the daemon backend](#faster-parallel-safe-rails-the---daemon-backend) |
|
|
59
61
|
| `--format human\|json\|html` | Report format (default: human; `html` is a self-contained file) |
|
|
60
62
|
| `--output FILE` | Write the report to FILE instead of stdout |
|
|
61
63
|
| `--dry-run` | List candidate mutations without executing (honors suppression) |
|
|
@@ -103,6 +105,68 @@ Add Mutineer to your Gemfile's test group:
|
|
|
103
105
|
gem "mutineer", group: :test, require: false
|
|
104
106
|
```
|
|
105
107
|
|
|
108
|
+
### Faster, parallel-safe Rails (the `--daemon` backend)
|
|
109
|
+
|
|
110
|
+
`--rails` boots your app once but runs mutants **serially** — parallel `--jobs`
|
|
111
|
+
under Rails is unsafe, because every worker shares one test database and clobbers
|
|
112
|
+
the others' fixtures. `--daemon` fixes both: it boots the app once in a persistent
|
|
113
|
+
helper and forks per mutant, and gives **each parallel worker its own database**,
|
|
114
|
+
so `--jobs N` is safe and its verdicts are proven identical to a serial run.
|
|
115
|
+
|
|
116
|
+
```sh
|
|
117
|
+
RAILS_ENV=test bundle exec mutineer run \
|
|
118
|
+
app/models/order.rb --test test/models/order_test.rb \
|
|
119
|
+
--rails --daemon --jobs 4
|
|
120
|
+
```
|
|
121
|
+
|
|
122
|
+
- **One boot, forked per mutant** — restores the shared-boot speed.
|
|
123
|
+
- **Coverage-guided** — each mutant runs only its covering tests (like `--rails`);
|
|
124
|
+
a mutant on an uncovered line is `no_coverage`, so the score stays comparable to
|
|
125
|
+
the in-process `--rails` score.
|
|
126
|
+
- **Safe `--jobs N`** — each worker routes to its own copy of the test database, so
|
|
127
|
+
parallel verdicts equal serial (no fixture cross-talk).
|
|
128
|
+
- **One backend at a time** — `--daemon` can't be combined with `--test-command`
|
|
129
|
+
(choose one), and it needs an app to boot (`--rails` or `--boot`).
|
|
130
|
+
|
|
131
|
+
Status: **SQLite** today (hermetic, CI-proven). **Postgres** per-worker
|
|
132
|
+
provisioning is in progress (#34/#35); until it lands, use `--daemon` with a
|
|
133
|
+
SQLite test database, or drop `--jobs` to run serially on other adapters.
|
|
134
|
+
|
|
135
|
+
### Apps on Ruby < 3.4
|
|
136
|
+
|
|
137
|
+
Mutineer's own process needs Ruby ≥ 3.4 (it parses with stdlib Prism), and the
|
|
138
|
+
`--rails` path above boots your app *inside Mutineer's process* — so it can't run
|
|
139
|
+
against an app pinned to an older Ruby (`ruby "3.1.6"` in the Gemfile), where the
|
|
140
|
+
bundle rejects 3.4.
|
|
141
|
+
|
|
142
|
+
`--test-command` decouples the two: Mutineer stays on ≥ 3.4, but your suite runs
|
|
143
|
+
as a **subprocess in your app's own runtime** (whatever Ruby its bundle resolves
|
|
144
|
+
to). Run Mutineer with a 3.4+ Ruby and hand it the command that runs your tests:
|
|
145
|
+
|
|
146
|
+
```sh
|
|
147
|
+
RAILS_ENV=test mutineer run app/models/order.rb \
|
|
148
|
+
--test test/models/order_test.rb \
|
|
149
|
+
--test-command "bundle exec rails test %{files}"
|
|
150
|
+
```
|
|
151
|
+
|
|
152
|
+
- **`%{files}`** is required; it expands to the `--test` paths as separate
|
|
153
|
+
arguments (a path with a space stays one argument — there is no shell).
|
|
154
|
+
- **Environment** is inherited by the subprocess, so set it on the Mutineer
|
|
155
|
+
command (e.g. the leading `RAILS_ENV=test` above). Don't put `KEY=val` inside
|
|
156
|
+
`--test-command`.
|
|
157
|
+
|
|
158
|
+
Tradeoffs (Phase 1) — this path is correct but not free:
|
|
159
|
+
|
|
160
|
+
- **Slower:** your app re-boots for every mutant (no shared boot yet).
|
|
161
|
+
- **No coverage narrowing:** every mutant runs the *full* `--test` set, so the
|
|
162
|
+
score is an **upper bound and not comparable to an in-process (`--rails`)
|
|
163
|
+
score** — uncovered mutants count as survivors, and an infrastructure failure
|
|
164
|
+
is scored as a kill. Mutineer prints this caveat on every run and aborts up
|
|
165
|
+
front (a "smoke check") if your unmutated suite isn't green.
|
|
166
|
+
- **Reload strategy only** (`--strategy redefine` is rejected on this path) and
|
|
167
|
+
**serial** (`--jobs` is forced to 1). For apps on Ruby ≥ 3.4, `--daemon` gives
|
|
168
|
+
safe parallelism instead (see [the daemon backend](#faster-parallel-safe-rails-the---daemon-backend)).
|
|
169
|
+
|
|
106
170
|
## Suppressing equivalent mutants
|
|
107
171
|
|
|
108
172
|
Some mutants are equivalent (behaviour-identical) and survive forever — keeping a
|
|
@@ -166,7 +230,9 @@ walking up). CLI flags override config; config overrides defaults.
|
|
|
166
230
|
|
|
167
231
|
Sources are positional CLI arguments and test files come from `--test`; the
|
|
168
232
|
config file accepts these keys: `operators`, `threshold`, `jobs`, `only`,
|
|
169
|
-
`require` (extra files to load before mutating),
|
|
233
|
+
`require` (extra files to load before mutating), `boot`/`rails`, and
|
|
234
|
+
`test_command` (the external-runtime suite command — see
|
|
235
|
+
[Apps on Ruby < 3.4](#apps-on-ruby--34)).
|
|
170
236
|
|
|
171
237
|
```yaml
|
|
172
238
|
# .mutineer.yml
|
data/lib/mutineer/cli.rb
CHANGED
|
@@ -47,6 +47,13 @@ module Mutineer
|
|
|
47
47
|
--boot FILE Require FILE once in the parent to boot the app env, then
|
|
48
48
|
fork per mutant (Rails apps; requires --test)
|
|
49
49
|
--rails Sugar for --boot config/environment --strategy redefine
|
|
50
|
+
--test-command CMD Run the target suite in the app's own runtime as a
|
|
51
|
+
subprocess (for apps on Ruby < 3.4). CMD must contain
|
|
52
|
+
%{files} (expands to the --test paths). Env is inherited,
|
|
53
|
+
e.g. RAILS_ENV=test mutineer run ... --test-command "..."
|
|
54
|
+
--daemon Boot the app ONCE in a persistent daemon and fork per
|
|
55
|
+
mutant, with per-worker DB isolation so --jobs N is safe
|
|
56
|
+
under Rails (needs --rails/--boot; not with --test-command)
|
|
50
57
|
--format human|json|html Report format (default: human)
|
|
51
58
|
--output FILE Write the report to FILE instead of stdout
|
|
52
59
|
--dry-run List mutations without executing
|
|
@@ -105,6 +112,12 @@ module Mutineer
|
|
|
105
112
|
# typed (CLI wins over the file). --baseline-epsilon is CLI-only.
|
|
106
113
|
o.on("--baseline FILE") { |v| opts[:baseline] = v; explicit << :baseline }
|
|
107
114
|
o.on("--baseline-epsilon FLOAT") { |v| opts[:baseline_epsilon] = v.to_f }
|
|
115
|
+
# #27: run the target suite as a subprocess in the app's OWN runtime so
|
|
116
|
+
# mutineer (Ruby >= 3.4) can mutation-test apps pinned to an older Ruby.
|
|
117
|
+
o.on("--test-command CMD") { |v| opts[:test_command] = v; explicit << :test_command }
|
|
118
|
+
# #26/#27 Phase 2: boot the app ONCE in a persistent daemon and fork per
|
|
119
|
+
# mutant, with per-worker DB isolation so --jobs N is safe under Rails.
|
|
120
|
+
o.on("--daemon") { opts[:daemon] = true; explicit << :daemon }
|
|
108
121
|
end
|
|
109
122
|
|
|
110
123
|
begin
|
|
@@ -188,6 +201,11 @@ module Mutineer
|
|
|
188
201
|
rescue Mutineer::ParseError => e
|
|
189
202
|
warn "mutineer: error reading: #{e.message}"
|
|
190
203
|
exit 1
|
|
204
|
+
rescue Mutineer::SmokeCheckError => e
|
|
205
|
+
# #27: the unmutated suite isn't green under --test-command — a broken
|
|
206
|
+
# environment, not weak tests. Runtime error (exit 1), not usage (exit 2).
|
|
207
|
+
warn "mutineer: #{e.message}"
|
|
208
|
+
exit 1
|
|
191
209
|
end
|
|
192
210
|
|
|
193
211
|
# Flag validation: every flag/usage failure exits 2 (C7), consistent with the
|
|
@@ -229,6 +247,9 @@ module Mutineer
|
|
|
229
247
|
exit 2
|
|
230
248
|
end
|
|
231
249
|
|
|
250
|
+
validate_test_command!(config) if config.test_command
|
|
251
|
+
validate_daemon!(config, explicit) if config.daemon
|
|
252
|
+
|
|
232
253
|
validate_since!(config) if config.since
|
|
233
254
|
preflight_output!(config.output) if config.output
|
|
234
255
|
preflight_baseline!(config.baseline) if config.baseline
|
|
@@ -250,6 +271,66 @@ module Mutineer
|
|
|
250
271
|
validate_paths!(config)
|
|
251
272
|
end
|
|
252
273
|
|
|
274
|
+
# #27: --test-command runs the target suite in the app's own runtime. Validate
|
|
275
|
+
# its shape up front (usage errors → exit 2) and force serial execution: each
|
|
276
|
+
# subprocess boots the app and opens its own fixture transaction against the
|
|
277
|
+
# same DB, so --jobs > 1 would corrupt results (the #12 fixture-contention
|
|
278
|
+
# hazard). Unlike --rails, this path has NO per-worker DB isolation to opt into,
|
|
279
|
+
# so an explicit --jobs N is forced to 1 rather than honored (KTD-5).
|
|
280
|
+
# Validates the --test-command configuration.
|
|
281
|
+
#
|
|
282
|
+
# @api private
|
|
283
|
+
# @param config [Mutineer::Config] run configuration.
|
|
284
|
+
# @return [void]
|
|
285
|
+
def self.validate_test_command!(config)
|
|
286
|
+
if config.test_command.strip.empty?
|
|
287
|
+
warn "mutineer: --test-command must not be empty"
|
|
288
|
+
exit 2
|
|
289
|
+
end
|
|
290
|
+
unless config.test_command.include?("%{files}")
|
|
291
|
+
warn "mutineer: --test-command must contain %{files} (where the --test paths are substituted)"
|
|
292
|
+
exit 2
|
|
293
|
+
end
|
|
294
|
+
if config.boot
|
|
295
|
+
warn "mutineer: --test-command cannot be combined with --boot/--rails " \
|
|
296
|
+
"(the external subprocess boots the app itself)"
|
|
297
|
+
exit 2
|
|
298
|
+
end
|
|
299
|
+
if config.strategy == "redefine"
|
|
300
|
+
warn "mutineer: --test-command supports only --strategy reload " \
|
|
301
|
+
"(surgical redefine needs a shared VM; the subprocess has its own)"
|
|
302
|
+
exit 2
|
|
303
|
+
end
|
|
304
|
+
return unless config.jobs > 1
|
|
305
|
+
|
|
306
|
+
warn "[mutineer] --test-command runs serially (no per-worker DB isolation yet); forcing --jobs 1."
|
|
307
|
+
config.jobs = 1
|
|
308
|
+
end
|
|
309
|
+
|
|
310
|
+
# #26/#27 Phase 2 (U8): --daemon selects the persistent-daemon backend (boot once,
|
|
311
|
+
# fork per mutant, per-worker DB isolation). Two usage errors, both exit 2 (KTD-10):
|
|
312
|
+
# it cannot be combined with --test-command (choose ONE backend — never silently
|
|
313
|
+
# pick one), and it requires an app to boot (--rails or --boot). Under Rails,
|
|
314
|
+
# Config.resolve already keeps --jobs serial unless the user explicitly asks for
|
|
315
|
+
# parallelism, and each worker gets its own SQLite database on demand — so there is
|
|
316
|
+
# no "missing worker DB" precondition to check here for SQLite. Postgres worker-DB
|
|
317
|
+
# provisioning + its missing-DB error (KTD-9) arrives with the Postgres adapter (U10).
|
|
318
|
+
#
|
|
319
|
+
# @api private
|
|
320
|
+
# @param config [Mutineer::Config] run configuration.
|
|
321
|
+
# @param explicit [Set<Symbol>] explicit CLI fields.
|
|
322
|
+
# @return [void]
|
|
323
|
+
def self.validate_daemon!(config, _explicit = Set.new)
|
|
324
|
+
if config.test_command
|
|
325
|
+
warn "mutineer: choose one backend — --daemon and --test-command cannot be combined"
|
|
326
|
+
exit 2
|
|
327
|
+
end
|
|
328
|
+
return if config.rails || config.boot
|
|
329
|
+
|
|
330
|
+
warn "mutineer: --daemon needs an app to boot; add --rails (or --boot FILE)"
|
|
331
|
+
exit 2
|
|
332
|
+
end
|
|
333
|
+
|
|
253
334
|
# --since needs a real git repo and a resolvable ref; either failure is a
|
|
254
335
|
# usage error (exit 2) so CI sees "bad invocation," not "tests too weak."
|
|
255
336
|
# Validates the --since ref.
|
|
@@ -377,6 +458,25 @@ module Mutineer
|
|
|
377
458
|
reporter.report(out: $stdout, err: $stderr, threshold: config.threshold,
|
|
378
459
|
format: config.format, output: config.output, baseline: delta)
|
|
379
460
|
|
|
461
|
+
# #27/KTD-6: warn (stderr, so it never pollutes json/html) that an external
|
|
462
|
+
# run's score is not comparable to an in-process run — it has no coverage
|
|
463
|
+
# narrowing, so uncovered mutants count as survivors, and an infra failure is
|
|
464
|
+
# scored as a kill (upper bound).
|
|
465
|
+
if config.test_command
|
|
466
|
+
warn "[mutineer] --test-command score is an upper bound, not comparable to an " \
|
|
467
|
+
"in-process run: no coverage narrowing (uncovered mutants count as survivors) " \
|
|
468
|
+
"and an infra failure is scored as a kill."
|
|
469
|
+
end
|
|
470
|
+
|
|
471
|
+
# #26/U8: same disclosure as --test-command above — the daemon backend has no
|
|
472
|
+
# coverage narrowing yet (Phase 2c), so this score is a lower bound (uncovered
|
|
473
|
+
# mutants count as survivors) and not comparable to an in-process run.
|
|
474
|
+
if config.daemon
|
|
475
|
+
warn "[mutineer] --daemon score is a lower bound, not comparable to an " \
|
|
476
|
+
"in-process run: no coverage narrowing yet (uncovered mutants count as " \
|
|
477
|
+
"survivors)."
|
|
478
|
+
end
|
|
479
|
+
|
|
380
480
|
# #14: nudge toward the opt-in tier-2 operators (human report only — never
|
|
381
481
|
# pollute JSON output).
|
|
382
482
|
if !%w[json html].include?(config.format) && (hint = tier2_hint(config.operators))
|
data/lib/mutineer/config.rb
CHANGED
|
@@ -25,13 +25,16 @@ module Mutineer
|
|
|
25
25
|
:cache_dir, :project_root, :load_paths,
|
|
26
26
|
:jobs, :format, :output, :strategy, :require_paths,
|
|
27
27
|
:boot, :rails, :since, :framework, :verbose, :ignore,
|
|
28
|
-
|
|
28
|
+
# :daemon is user-facing as of U8 (--daemon flag + KNOWN_KEYS + boolean coerce).
|
|
29
|
+
# :daemon_timeout stays programmatic (set by tests/Runner; no flag yet).
|
|
30
|
+
:baseline, :baseline_epsilon, :fail_fast, :test_command,
|
|
31
|
+
:daemon, :daemon_timeout,
|
|
29
32
|
keyword_init: true
|
|
30
33
|
) do
|
|
31
34
|
# Config file name.
|
|
32
35
|
CONFIG_FILE = ".mutineer.yml"
|
|
33
36
|
# Keys accepted in .mutineer.yml (R7). `require` maps to the :require_paths field.
|
|
34
|
-
KNOWN_KEYS = %w[operators jobs threshold only require boot rails since framework verbose ignore baseline fail_fast].freeze
|
|
37
|
+
KNOWN_KEYS = %w[operators jobs threshold only require boot rails since framework verbose ignore baseline fail_fast test_command daemon].freeze
|
|
35
38
|
|
|
36
39
|
def initialize(**kwargs)
|
|
37
40
|
super
|
|
@@ -51,6 +54,7 @@ module Mutineer
|
|
|
51
54
|
self.ignore ||= []
|
|
52
55
|
self.baseline_epsilon ||= 0.0
|
|
53
56
|
self.fail_fast = false if fail_fast.nil?
|
|
57
|
+
self.daemon = false if daemon.nil?
|
|
54
58
|
end
|
|
55
59
|
|
|
56
60
|
# Walk from `start` toward `home`, returning the first .mutineer.yml path found
|
|
@@ -163,9 +167,11 @@ module Mutineer
|
|
|
163
167
|
when "boot" then value.to_s
|
|
164
168
|
when "framework" then value.to_s
|
|
165
169
|
when "rails" then value == true || value.to_s == "true"
|
|
170
|
+
when "daemon" then value == true || value.to_s == "true"
|
|
166
171
|
when "verbose" then value == true || value.to_s == "true"
|
|
167
172
|
when "ignore" then Array(value).map(&:to_s)
|
|
168
173
|
when "baseline" then value.to_s
|
|
174
|
+
when "test_command" then value.to_s
|
|
169
175
|
else value
|
|
170
176
|
end
|
|
171
177
|
end
|
|
@@ -21,7 +21,24 @@ module Mutineer
|
|
|
21
21
|
class CoverageMap
|
|
22
22
|
DEFAULT_CAPTURE_TIMEOUT = 120 # seconds, per coverage subprocess (R3)
|
|
23
23
|
|
|
24
|
-
attr_reader :project_root, :failed_test_files, :phase_a_ran
|
|
24
|
+
attr_reader :project_root, :failed_test_files, :phase_a_ran, :map
|
|
25
|
+
|
|
26
|
+
# Build a QUERY-ONLY map from data captured elsewhere (the daemon builds the map
|
|
27
|
+
# app-side and ships `map` + `failed_test_files` over IPC; the tool reconstructs it
|
|
28
|
+
# here for per-mutant selection). Skips the capture machinery entirely — only the
|
|
29
|
+
# three fields #tests_for / #method_uncapturable? read are set (U7).
|
|
30
|
+
#
|
|
31
|
+
# @param map [Hash] the "file:line" => [test_files] map.
|
|
32
|
+
# @param failed_test_files [Array<String>] test files whose capture failed.
|
|
33
|
+
# @param project_root [String] project root (for path relativization).
|
|
34
|
+
# @return [Mutineer::CoverageMap] a query-only map.
|
|
35
|
+
def self.from_data(map:, failed_test_files:, project_root:)
|
|
36
|
+
instance = allocate
|
|
37
|
+
instance.instance_variable_set(:@map, map || {})
|
|
38
|
+
instance.instance_variable_set(:@failed_test_files, failed_test_files || [])
|
|
39
|
+
instance.instance_variable_set(:@project_root, project_root)
|
|
40
|
+
instance
|
|
41
|
+
end
|
|
25
42
|
|
|
26
43
|
def initialize(source_paths:, test_paths:, cache_dir: ".mutineer",
|
|
27
44
|
load_paths: ["lib"], project_root: Dir.pwd,
|
|
@@ -54,9 +71,9 @@ module Mutineer
|
|
|
54
71
|
# the booted parent instead. Inverts into the same map #tests_for reads, and
|
|
55
72
|
# reuses the digest cache (the digest mixes in the boot file so a boot cache
|
|
56
73
|
# never collides with a standalone one).
|
|
57
|
-
def build_via_fork(
|
|
74
|
+
def build_via_fork(after_fork: nil)
|
|
58
75
|
warn_external_sources
|
|
59
|
-
cached_or { run_phase_a_via_fork(
|
|
76
|
+
cached_or { run_phase_a_via_fork(after_fork: after_fork) }
|
|
60
77
|
end
|
|
61
78
|
|
|
62
79
|
# Phase B lookup: the test files that cover `file:line`, or [] when none do.
|
|
@@ -159,7 +176,7 @@ module Mutineer
|
|
|
159
176
|
# per-source coverage counts. record() inverts them exactly as the subprocess
|
|
160
177
|
# path does. ponytail: serial fork (one test at a time) — boot apps fork
|
|
161
178
|
# cheaply via COW and per-test isolation matters more than throughput here.
|
|
162
|
-
def run_phase_a_via_fork(
|
|
179
|
+
def run_phase_a_via_fork(after_fork:)
|
|
163
180
|
@phase_a_ran = true
|
|
164
181
|
@map = {}
|
|
165
182
|
@failed_test_files = []
|
|
@@ -169,7 +186,7 @@ module Mutineer
|
|
|
169
186
|
# Tri-state payload (KTD-1): Hash = coverage, String = error diagnostic
|
|
170
187
|
# from the child, nil = pipe gone / empty.
|
|
171
188
|
# ponytail/#9: this String diagnostic is what #9 turns into an :uncapturable status.
|
|
172
|
-
case (coverage = fork_capture(absolute(test_path), abs_sources,
|
|
189
|
+
case (coverage = fork_capture(absolute(test_path), abs_sources, after_fork))
|
|
173
190
|
when Hash then record(coverage, test_path)
|
|
174
191
|
when String
|
|
175
192
|
fail_test(test_path, @verbose ? "fork capture failed: #{coverage}" :
|
|
@@ -182,7 +199,7 @@ module Mutineer
|
|
|
182
199
|
# Fork the booted parent, run one test under the inherited Coverage, and
|
|
183
200
|
# return its per-source counts hash (or nil on failure). Reuses the same
|
|
184
201
|
# fork + Marshal-over-pipe + hard-exit! discipline as WorkerPool/Isolation.
|
|
185
|
-
def fork_capture(abs_test, abs_sources,
|
|
202
|
+
def fork_capture(abs_test, abs_sources, after_fork)
|
|
186
203
|
rd, wr = IO.pipe
|
|
187
204
|
# #19: Marshal output is binary — an un-binmoded pipe can raise
|
|
188
205
|
# Encoding::UndefinedConversionError on write, which the child's rescue then
|
|
@@ -193,7 +210,10 @@ module Mutineer
|
|
|
193
210
|
rd.close
|
|
194
211
|
payload =
|
|
195
212
|
begin
|
|
196
|
-
|
|
213
|
+
# Fork-safety hook: the in-process path reconnects AR; the daemon routes
|
|
214
|
+
# to its worker DB. Nil (non-Rails) = no-op. Injected so this file needs
|
|
215
|
+
# neither Runner (Prism) nor Rails.
|
|
216
|
+
after_fork&.call
|
|
197
217
|
Coverage.result(clear: true, stop: false) # discard pre-test delta
|
|
198
218
|
TestRunners.for(@framework).run([abs_test])
|
|
199
219
|
# lines:true yields {file => {lines: [...]}}; reduce to the counts
|
|
@@ -0,0 +1,187 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
require "json"
|
|
4
|
+
require "open3"
|
|
5
|
+
|
|
6
|
+
module Mutineer
|
|
7
|
+
# Raised when the daemon cannot be booted (bad boot path, app error, or it dies on
|
|
8
|
+
# the handshake). The CLI maps it to a runtime error.
|
|
9
|
+
class DaemonBootError < StandardError; end
|
|
10
|
+
|
|
11
|
+
# #26/#27 Phase 2a — the TOOL-side handle for the app-side daemon.
|
|
12
|
+
#
|
|
13
|
+
# Spawns `daemon_server.rb` UNDER THE APP'S BUNDLE/RUBY (cleaned env so the gem's
|
|
14
|
+
# bundler context never leaks; the daemon file is loaded by absolute path with
|
|
15
|
+
# `-r`, which bypasses the app bundle that has no mutineer), completes the ready
|
|
16
|
+
# handshake, then ships per-mutant payloads and reads structured verdicts. If the
|
|
17
|
+
# daemon dies mid-run it respawns (bounded) and marks the in-flight mutant `error`
|
|
18
|
+
# rather than corrupting the run. Reuses the cleaned-env spawn + stderr-drain proven
|
|
19
|
+
# in the spike driver and the spawn discipline of ExternalBackend.
|
|
20
|
+
class DaemonClient
|
|
21
|
+
# Absolute path to the daemon entry, loaded app-side by `-r` (bypasses the bundle).
|
|
22
|
+
DAEMON_PATH = File.expand_path("daemon_server.rb", __dir__)
|
|
23
|
+
# How many times to respawn a crashing daemon before aborting the run.
|
|
24
|
+
MAX_RESTARTS = 3
|
|
25
|
+
|
|
26
|
+
# @param boot [Hash] boot config sent to the daemon: project_root, boot,
|
|
27
|
+
# load_paths, framework, rails.
|
|
28
|
+
# @param app_root [String] directory to spawn the daemon in (the app root).
|
|
29
|
+
# @param ruby_version [String, nil] RBENV_VERSION for the app's Ruby (nil = inherit).
|
|
30
|
+
# @param gemfile [String, nil] BUNDLE_GEMFILE for the app's bundle (nil = app_root/Gemfile).
|
|
31
|
+
# @param errio [IO] where daemon stderr is drained.
|
|
32
|
+
def initialize(boot:, app_root:, ruby_version: nil, gemfile: nil, errio: $stderr)
|
|
33
|
+
@boot = boot
|
|
34
|
+
@app_root = app_root
|
|
35
|
+
@ruby_version = ruby_version
|
|
36
|
+
@gemfile = gemfile || File.join(app_root, "Gemfile")
|
|
37
|
+
@errio = errio
|
|
38
|
+
@restarts = 0
|
|
39
|
+
end
|
|
40
|
+
|
|
41
|
+
# Spawn the daemon and complete the ready handshake. Raises DaemonBootError on
|
|
42
|
+
# failure (surfaced by the CLI as a clean runtime error, not a hang).
|
|
43
|
+
#
|
|
44
|
+
# @return [self]
|
|
45
|
+
def start
|
|
46
|
+
spawn_daemon
|
|
47
|
+
self
|
|
48
|
+
end
|
|
49
|
+
|
|
50
|
+
# Run one mutant: ship the payload + covering tests, return the verdict string.
|
|
51
|
+
# On a daemon crash (EOF/dead pipe) respawn (bounded) and return `"error"` for
|
|
52
|
+
# this mutant — never a wrong verdict, never a wedged run.
|
|
53
|
+
#
|
|
54
|
+
# @param id [Integer] request id (echoed back for ordering safety).
|
|
55
|
+
# @param payload [Hash] {"code" => mutated ruby, "source_file" => path}.
|
|
56
|
+
# @param tests [Array<String>] covering test file paths.
|
|
57
|
+
# @param timeout [Numeric] per-mutant wall-clock timeout (seconds).
|
|
58
|
+
# @param worker [Integer] worker slot; the daemon routes the fork to
|
|
59
|
+
# `<db>-<worker>` (#26 isolation). Defaults to 0 (serial in U5).
|
|
60
|
+
# @return [String] one of survived/killed/error/timeout.
|
|
61
|
+
def request(id:, payload:, tests:, timeout:, worker: 0)
|
|
62
|
+
# A crash can surface on the WRITE (daemon died idle between requests →
|
|
63
|
+
# Errno::EPIPE) as well as the read (EOF), so guard both: either way, respawn
|
|
64
|
+
# for future mutants and score THIS one error (re-running a crash-causing
|
|
65
|
+
# mutant could loop). Never let a dead pipe abort the whole run.
|
|
66
|
+
reply =
|
|
67
|
+
begin
|
|
68
|
+
send_line("id" => id, "worker" => worker, "payload" => payload, "tests" => tests, "timeout" => timeout)
|
|
69
|
+
read_line
|
|
70
|
+
rescue Errno::EPIPE, IOError
|
|
71
|
+
nil
|
|
72
|
+
end
|
|
73
|
+
return reply["verdict"] if reply && reply["id"] == id
|
|
74
|
+
|
|
75
|
+
restart!
|
|
76
|
+
"error"
|
|
77
|
+
end
|
|
78
|
+
|
|
79
|
+
# #26/U7: ask the daemon to build the coverage map app-side and return it.
|
|
80
|
+
# One-shot control message (no id). Returns `{"map"=>..., "failed_test_files"=>...}`
|
|
81
|
+
# (possibly with an `"error"`), or nil if the daemon vanished — the caller then
|
|
82
|
+
# falls back to running the full test set (no narrowing) rather than mis-scoring.
|
|
83
|
+
#
|
|
84
|
+
# @return [Hash, nil] the coverage payload, or nil on a dead pipe.
|
|
85
|
+
def coverage
|
|
86
|
+
send_line("cmd" => "coverage")
|
|
87
|
+
read_line
|
|
88
|
+
rescue Errno::EPIPE, IOError
|
|
89
|
+
nil
|
|
90
|
+
end
|
|
91
|
+
|
|
92
|
+
# Graceful shutdown; leaves no orphaned daemon/child.
|
|
93
|
+
#
|
|
94
|
+
# @return [void]
|
|
95
|
+
def quit
|
|
96
|
+
return unless @stdin
|
|
97
|
+
|
|
98
|
+
send_line("cmd" => "quit") rescue nil # rubocop:disable Style/RescueModifier
|
|
99
|
+
@wait_thr&.join
|
|
100
|
+
ensure
|
|
101
|
+
close_io
|
|
102
|
+
end
|
|
103
|
+
|
|
104
|
+
private
|
|
105
|
+
|
|
106
|
+
# Cleaned environment for the app bundle: strip the gem's bundler/Ruby context so
|
|
107
|
+
# `bundle exec` resolves the APP's Gemfile under the requested Ruby.
|
|
108
|
+
def app_env
|
|
109
|
+
env = ENV.to_h.reject { |k, _| k.start_with?("BUNDLE_", "RUBY", "GEM_") }
|
|
110
|
+
env["BUNDLE_GEMFILE"] = @gemfile
|
|
111
|
+
env["RBENV_VERSION"] = @ruby_version if @ruby_version
|
|
112
|
+
env["RAILS_ENV"] ||= "test" if @boot[:rails] || @boot["rails"]
|
|
113
|
+
env
|
|
114
|
+
end
|
|
115
|
+
|
|
116
|
+
# Spawn the daemon under the app bundle and complete the ready handshake.
|
|
117
|
+
#
|
|
118
|
+
# @return [void]
|
|
119
|
+
# @raise [Mutineer::DaemonBootError] when the daemon fails to boot.
|
|
120
|
+
def spawn_daemon
|
|
121
|
+
# Plain `bundle exec ruby` — NOT `rbenv exec`, which would break CI and any
|
|
122
|
+
# non-rbenv setup. When bundler/ruby are rbenv shims, the RBENV_VERSION carried
|
|
123
|
+
# in app_env still selects the app's Ruby; otherwise the active Ruby is used.
|
|
124
|
+
@stdin, @stdout, @stderr, @wait_thr = Open3.popen3(
|
|
125
|
+
app_env, "bundle", "exec", "ruby",
|
|
126
|
+
"-r", DAEMON_PATH, "-e", "Mutineer::DaemonServer.run", chdir: @app_root
|
|
127
|
+
)
|
|
128
|
+
# Drain daemon stderr to the tool's stderr so child/boot errors are visible.
|
|
129
|
+
# Tracked (not fire-and-forget) so close_io can reclaim it on quit/respawn; the
|
|
130
|
+
# rescue swallows the benign EBADF/IOError raised when close_io closes the pipe
|
|
131
|
+
# out from under an in-flight copy_stream.
|
|
132
|
+
@drain = Thread.new do # rubocop:disable ThreadSafety/NewThread
|
|
133
|
+
IO.copy_stream(@stderr, @errio)
|
|
134
|
+
rescue IOError, Errno::EBADF
|
|
135
|
+
nil
|
|
136
|
+
end
|
|
137
|
+
|
|
138
|
+
send_line(@boot)
|
|
139
|
+
ready = read_line
|
|
140
|
+
unless ready && ready["ready"]
|
|
141
|
+
detail = ready && ready["error"] ? ready["error"] : "daemon exited before the handshake"
|
|
142
|
+
close_io
|
|
143
|
+
raise DaemonBootError, "daemon failed to boot under the app bundle: #{detail}"
|
|
144
|
+
end
|
|
145
|
+
end
|
|
146
|
+
|
|
147
|
+
# Respawn after a crash, up to MAX_RESTARTS, then hard-fail loudly.
|
|
148
|
+
def restart!
|
|
149
|
+
close_io
|
|
150
|
+
@restarts += 1
|
|
151
|
+
if @restarts > MAX_RESTARTS
|
|
152
|
+
raise DaemonBootError, "daemon crashed #{@restarts} times; aborting the run"
|
|
153
|
+
end
|
|
154
|
+
|
|
155
|
+
@errio.puts("[mutineer] daemon crashed — respawning (#{@restarts}/#{MAX_RESTARTS})")
|
|
156
|
+
spawn_daemon
|
|
157
|
+
end
|
|
158
|
+
|
|
159
|
+
# Write one JSON object as a line to the daemon.
|
|
160
|
+
#
|
|
161
|
+
# @param obj [Hash] the message to encode.
|
|
162
|
+
# @return [void]
|
|
163
|
+
def send_line(obj)
|
|
164
|
+
@stdin.puts(JSON.generate(obj))
|
|
165
|
+
@stdin.flush
|
|
166
|
+
end
|
|
167
|
+
|
|
168
|
+
# Read one JSON reply line; nil on EOF/dead pipe (caller treats as a crash).
|
|
169
|
+
def read_line
|
|
170
|
+
line = @stdout.gets
|
|
171
|
+
line && JSON.parse(line.strip)
|
|
172
|
+
rescue IOError, Errno::EPIPE, JSON::ParserError
|
|
173
|
+
nil
|
|
174
|
+
end
|
|
175
|
+
|
|
176
|
+
# Close the IPC pipes, stop the stderr-drain thread, and reap the daemon so a
|
|
177
|
+
# respawn or quit leaves no leaked fd, thread, or zombie.
|
|
178
|
+
#
|
|
179
|
+
# @return [void]
|
|
180
|
+
def close_io
|
|
181
|
+
@drain&.kill # stop the drain BEFORE closing its fd (avoids a copy_stream EBADF)
|
|
182
|
+
[@stdin, @stdout, @stderr].each { |io| io&.close rescue nil } # rubocop:disable Style/RescueModifier
|
|
183
|
+
@wait_thr&.join # reap the exited daemon so respawn/quit leaves no zombie
|
|
184
|
+
@stdin = @stdout = @stderr = @drain = @wait_thr = nil
|
|
185
|
+
end
|
|
186
|
+
end
|
|
187
|
+
end
|