active_mutator 0.1.0 → 0.2.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (36) hide show
  1. checksums.yaml +4 -4
  2. data/README.md +160 -63
  3. data/lib/active_mutator/accepted_ledger.rb +23 -8
  4. data/lib/active_mutator/atomic_file.rb +1 -1
  5. data/lib/active_mutator/baseline.rb +13 -4
  6. data/lib/active_mutator/baseline_delta.rb +67 -1
  7. data/lib/active_mutator/baseline_hooks.rb +2 -2
  8. data/lib/active_mutator/cli.rb +16 -4
  9. data/lib/active_mutator/config.rb +2 -1
  10. data/lib/active_mutator/config_file.rb +89 -0
  11. data/lib/active_mutator/coverage_map.rb +1 -1
  12. data/lib/active_mutator/defined_constants.rb +48 -0
  13. data/lib/active_mutator/edit.rb +8 -2
  14. data/lib/active_mutator/engine.rb +15 -2
  15. data/lib/active_mutator/fingerprint.rb +1 -1
  16. data/lib/active_mutator/inserter.rb +6 -3
  17. data/lib/active_mutator/operators/base.rb +2 -1
  18. data/lib/active_mutator/operators/call_swap.rb +16 -0
  19. data/lib/active_mutator/operators/literal.rb +14 -2
  20. data/lib/active_mutator/reporter/github.rb +36 -0
  21. data/lib/active_mutator/reporter/json.rb +1 -0
  22. data/lib/active_mutator/reporter/operator_stats.rb +20 -0
  23. data/lib/active_mutator/reporter/stryker_json.rb +117 -0
  24. data/lib/active_mutator/reporter/terminal.rb +11 -0
  25. data/lib/active_mutator/runner.rb +121 -17
  26. data/lib/active_mutator/scheduler.rb +64 -7
  27. data/lib/active_mutator/source_location.rb +21 -0
  28. data/lib/active_mutator/subject.rb +7 -1
  29. data/lib/active_mutator/subject_finder.rb +46 -7
  30. data/lib/active_mutator/subject_matcher.rb +23 -0
  31. data/lib/active_mutator/timeout_calibrator.rb +75 -0
  32. data/lib/active_mutator/version.rb +1 -1
  33. data/lib/active_mutator/work_item.rb +8 -1
  34. data/lib/active_mutator/worker.rb +5 -2
  35. data/lib/active_mutator.rb +8 -0
  36. metadata +13 -5
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: c401b5edbafa0a072b6672e088a9016f918d9e76ce6b264a9e2a054b1b538cc6
4
- data.tar.gz: d2a1908e950d8687e68524184efc2bce200d595be3b25f31e3a0e92a95325b3e
3
+ metadata.gz: 5729b9385c24e575708a5227a3838ae28e09dd2ddf3e40e1883e9e74f7399275
4
+ data.tar.gz: 69b41b3304c8c008f54f6c20547ff9283d26a278eea706dc7cf923466dde1b5c
5
5
  SHA512:
6
- metadata.gz: 9d42e2e4a207ab1c6ced606eeeee342389b503da5055e207d282df220f112537f90b5f967079dfd76060d740f710d5441c8422565e7bdad314fa4bc9cf5d07c6
7
- data.tar.gz: 3e226d88dc71b5b0b698f5e2a1148a38c7f6b12e0cd47411033368a0bf7d18a90d0ca8243fa3d2841ebd1b1be4cfb554c41732b4e1eacc2961f6477ddb692cc7
6
+ metadata.gz: 3e0dce87a35314cb96e68e7051e3dfa8bbc329b58a3c41beef1201c7c5e8a3205d58a28593bf99382dfd1f9a9fd52859b44def8e2ee5d57dd0879e9dfcadec37
7
+ data.tar.gz: 5ec63925341ef8356801ee171e83af5b6323feb0f15838b1430421c9e5aaf73017d0e5ebaafb653039917a97144933e3572c2d182ba0544de27fbe60a0d7d6ca
data/README.md CHANGED
@@ -1,13 +1,16 @@
1
1
  # active_mutator
2
2
 
3
+ [![Gem Version](https://badge.fury.io/rb/active_mutator.svg)](https://rubygems.org/gems/active_mutator)
4
+
3
5
  Mutation testing for Ruby, built on [Prism](https://github.com/ruby/prism).
4
- Open source, RSpec-integrated, Rails-first.
6
+ Open source, RSpec-integrated, Rails-first. Available on
7
+ [RubyGems](https://rubygems.org/gems/active_mutator).
5
8
 
6
- active_mutator mutates your code one small change at a time (`>` `>=`,
7
- `&&` `||`, delete a statement, force a condition…), runs exactly the
8
- examples that cover the mutated line, and reports every mutant your suite
9
- fails to kill. A surviving mutant is a behavior change no test notices —
10
- a precise, machine-verified test gap.
9
+ active_mutator mutates your code one small change at a time (`>` becomes `>=`,
10
+ `&&` becomes `||`, a statement gets deleted, a condition gets forced, and so
11
+ on). It runs exactly the examples that cover the mutated line, and reports
12
+ every mutant your suite fails to kill. A surviving mutant is a behavior
13
+ change no test notices: a precise, machine-verified test gap.
11
14
 
12
15
  ## A surviving mutant, in one example
13
16
 
@@ -35,31 +38,31 @@ Surviving mutants:
35
38
  + total <= 100
36
39
  ```
37
40
 
38
- Nothing in the test suite calls `discount(100)` the one input where `<`
39
- and `<=` disagree. The tests pass, coverage is green, and the boundary is
40
- still unverified. That gap is invisible to coverage and obvious to
41
+ Nothing in the test suite calls `discount(100)`, the one input where `<`
42
+ and `<=` disagree. The tests pass, and coverage is green. But the boundary
43
+ is still unverified. That gap is invisible to coverage and obvious to
41
44
  mutation testing. Add `it { expect(calc.discount(100)).to eq(0) }` and the
42
45
  mutant is killed.
43
46
 
44
47
  ## What is mutation testing?
45
48
 
46
49
  Coverage answers "did a test run this line?" Mutation testing answers "would
47
- a test *notice* if this line were wrong?" a materially different, and
48
- usually more useful, question.
50
+ a test *notice* if this line were wrong?" That is a different, and usually
51
+ more useful, question.
49
52
 
50
53
  active_mutator applies one small, syntactically valid change to your code
51
54
  (a "mutant") and re-runs only the examples that cover it. If a test fails,
52
55
  the mutant is **killed**: your tests correctly reject that wrong behavior.
53
56
  If every covering test still passes, the mutant **survived**: something
54
- changed and nothing noticed. A survivor is not a hypothetical it's the
55
- exact line, the exact before/after diff, and proof that no assertion
57
+ changed and nothing noticed. A survivor is not a hypothetical. It is the
58
+ exact line, the exact before and after diff, and proof that no assertion
56
59
  depends on the difference.
57
60
 
58
61
  Mutation score is `(killed + timeout) / (killed + timeout + survived)`.
59
- 100% is usually not the right target some mutants are behaviorally
60
- *equivalent* to the original and can never be killed by any test which is
61
- why active_mutator has a committed acceptance ledger for closing survivors
62
- out with a stated reason instead of chasing an unreachable score.
62
+ 100% is usually not the right target. Some mutants are behaviorally
63
+ *equivalent* to the original and can never be killed by any test. That is
64
+ why active_mutator has a committed acceptance ledger. It lets you close
65
+ survivors out with a stated reason instead of chasing an unreachable score.
63
66
 
64
67
  Full primer, including the origin of the technique and further reading:
65
68
  **[`docs/guides/what-is-mutation-testing.md`](docs/guides/what-is-mutation-testing.md)**.
@@ -73,7 +76,7 @@ group :development, :test do
73
76
  end
74
77
  ```
75
78
 
76
- Requires Ruby 3.2, RSpec, and a green suite. Linux/macOS (MRI fork).
79
+ Requires Ruby 3.2 or later, RSpec, and a green suite. Linux/macOS (MRI fork).
77
80
 
78
81
  ## Quick start
79
82
 
@@ -82,9 +85,10 @@ bundle install
82
85
  bundle exec active_mutator app/models/calculator.rb
83
86
  ```
84
87
 
85
- First run performs an instrumented baseline of your suite to build the
86
- coverage map (cached in `.active_mutator/`, refreshed incrementally after
87
- that see [`docs/guides/how-it-works.md`](docs/guides/how-it-works.md)).
88
+ The first run performs an instrumented baseline of your suite to build the
89
+ coverage map. The map is cached in `.active_mutator/` and refreshed
90
+ incrementally after that (see
91
+ [`docs/guides/how-it-works.md`](docs/guides/how-it-works.md)).
88
92
  Then each mutant runs in its own fork against only its covering examples.
89
93
 
90
94
  ### Reading the output
@@ -115,52 +119,84 @@ Each character on the progress line is one mutant, printed as it finishes:
115
119
 
116
120
  | Char | Status | Meaning |
117
121
  |---|---|---|
118
- | `.` | `killed` | a covering test failed — good, the mutant is dead |
119
- | `S` | `survived` | every covering test passed a test gap |
120
- | `T` | `timeout` | ran past its time budget counted as detected (likely an infinite loop) |
122
+ | `.` | `killed` | a covering test failed. Good, the mutant is dead |
123
+ | `S` | `survived` | every covering test passed. This is a test gap |
124
+ | `T` | `timeout` | ran past its time budget. Counted as detected (likely an infinite loop) |
121
125
  | `E` | `error` | the worker crashed, or the mutated code raised outside a test assertion |
122
- | `U` | `uncovered` | no test executes the mutated line at all coverage debt, worse than a survivor |
123
- | `A` | `accepted` | matches a known-equivalent entry in the acceptance ledger excluded from the score |
126
+ | `U` | `uncovered` | no test executes the mutated line at all. This is coverage debt, worse than a survivor |
127
+ | `A` | `accepted` | matches a known-equivalent entry in the acceptance ledger. Excluded from the score |
124
128
 
125
129
  `invalid` mutants (edits that don't even re-parse as valid Ruby) are
126
130
  discarded before scheduling and reported as a count only. Exit code is `1`
127
- iff unaccepted survivors exist `0` otherwise, including when there are
128
- only `uncovered`/`accepted`/`error` results.
131
+ if unaccepted survivors exist (or, with `--fail-at`, if the score is below
132
+ the threshold), `0` otherwise, including when there are only `uncovered`,
133
+ `accepted`, or `error` results. The JSON report's `exit_reason` field
134
+ reflects survivor presence, independent of the `--fail-at` gate.
135
+
136
+ When survivors exist, the summary also prints a per-operator table showing
137
+ how often each operator's mutants survive, to help spot likely-equivalent
138
+ mutant patterns.
129
139
 
130
140
  ## How it works, compactly
131
141
 
132
- 1. **Subject discovery** a Prism visitor finds every method (`def`) in
142
+ 1. **Subject discovery**: a Prism visitor finds every method (`def`) in
133
143
  your target files.
134
- 2. **Source-span edits** each operator emits byte-range text edits
135
- against the original file, not a rewritten AST; every mutant is
144
+ 2. **Source-span edits**: each operator emits byte-range text edits
145
+ against the original file, not a rewritten AST. Every mutant is
136
146
  re-parsed with Prism and discarded (`invalid`) if the edit produced
137
147
  something that doesn't parse. No unparser is ever built or maintained.
138
- 3. **Coverage-mapped test selection** one instrumented baseline run maps
139
- every source line to the examples that cover it; incremental runs
148
+ 3. **Coverage-mapped test selection**: one instrumented baseline run maps
149
+ every source line to the examples that cover it. Incremental runs
140
150
  refresh only what changed instead of re-running the whole suite.
141
- 4. **Fork-per-mutant kill runs** the parent preloads your app and spec
142
- helper once; each mutant is inserted and exercised in its own fork
151
+ 4. **Fork-per-mutant kill runs**: the parent preloads your app and spec
152
+ helper once. Each mutant is inserted and exercised in its own fork
143
153
  against just its covering examples, so results can't bleed state
144
154
  between mutants.
145
155
 
146
156
  Full architecture, including the coverage-cache format, the fork pipeline,
147
- the serial lane for browser specs, timeout budgets, and every status
157
+ the serial lane for browser specs, timeout budgets, and every status, is in
148
158
  **[`docs/guides/how-it-works.md`](docs/guides/how-it-works.md)**.
149
159
 
150
160
  ## Usage
151
161
 
152
162
  ```bash
153
163
  active_mutator # mutate app/ and lib/, full run
154
- active_mutator app/models # scope by path
164
+ active_mutator app/models # scope by path (directory)
165
+ active_mutator app/models/document.rb # scope to a single file
155
166
  active_mutator --changed # uncommitted work only (dev loop)
156
167
  active_mutator --since origin/main # PR scope (CI)
157
168
  active_mutator --subject 'Foo::Bar#baz' # one method
169
+ active_mutator --exclude 'lib/generated' # skip a subtree (repeatable)
170
+ ```
171
+
172
+ `--subject` also takes broader expressions: `Foo::Bar` (all methods of
173
+ that constant), `Foo::Bar*` (raw name prefix — matches `Foo::Bar::Qux`
174
+ and also `Foo::Barn`), `Foo::Bar#*` (instance
175
+ methods only), `Foo::Bar.*` (singleton methods only).
176
+
177
+ `--exclude PAT` is a glob relative to the project root, applied during
178
+ subject discovery, and gitignore-like: `lib/generated`, `lib/generated/`,
179
+ and `lib/generated/**` all exclude the whole subtree. File globs like
180
+ `**/legacy/*` work too.
181
+
182
+ Skip a single method by putting `# active_mutator:skip` on the line above
183
+ its `def`:
184
+
185
+ ```ruby
186
+ # active_mutator:skip
187
+ def legacy_delegator
188
+ target.call
189
+ end
158
190
  ```
159
191
 
160
- Statuses: `killed` (test failed good), `survived` (test gap), `timeout`
161
- (counts as detected), `uncovered` (no covering example coverage debt),
162
- `accepted` (known-equivalent, see ledger), `error`, `invalid` (discarded).
163
- Exit code 1 iff unaccepted survivors exist.
192
+ Statuses: `killed` (test failed, this is good), `survived` (test gap),
193
+ `timeout` (counts as detected), `uncovered` (no covering example, this is
194
+ coverage debt), `accepted` (known-equivalent, see ledger), `error`,
195
+ `invalid` (discarded).
196
+ Exit code is 1 if unaccepted survivors exist (or, with `--fail-at`, if the
197
+ score is below the threshold). Mistyped positional paths (a file that
198
+ doesn't exist, or a non-`.rb` file) are an error (exit 2) instead of a
199
+ vacuous green run.
164
200
 
165
201
  Score = (killed + timeout) / (killed + timeout + survived).
166
202
 
@@ -179,65 +215,126 @@ bundle exec active_mutator --changed --accept-survivors # records to ledger
179
215
  git add .active_mutator_accepted.json # committed state
180
216
  ```
181
217
 
182
- Acceptance takes effect on the NEXT run (the accepting run still exits 1).
218
+ Acceptance takes effect on the next run. The accepting run still exits 1.
219
+ Scoped accepting runs (`--changed`, `--subject`, path args) are safe: the
220
+ ledger only prunes entries in files fully scanned by non-narrowed runs, so
221
+ out-of-scope acceptances are never dropped.
183
222
  Agent workflow: see [`docs/skills/mutation-check.md`](docs/skills/mutation-check.md).
184
223
 
224
+ ## Reports
225
+
226
+ `--format stryker-json` writes `.active_mutator/mutation-report.json` in the
227
+ Stryker [mutation-testing-report-schema](https://github.com/stryker-mutator/mutation-testing-elements)
228
+ v2 format. Open it in the
229
+ [Stryker report viewer](https://microsoft.github.io/mutation-testing-elements/)
230
+ for per-file mutant maps with inline diffs, filterable by status.
231
+
232
+ `--format github` prints one `::warning` annotation per surviving mutant, so
233
+ survivors show inline on the PR diff. Pairs with the CI recipe:
234
+
235
+ bundle exec active_mutator --since origin/main --format github
236
+
185
237
  ## CI recipe
186
238
 
187
- - Per-PR: `active_mutator --since origin/main` (minutes)
239
+ - Per-PR: `active_mutator --since origin/main --format github` (minutes;
240
+ survivors annotate the PR diff)
188
241
  - Nightly: `active_mutator --force-baseline` (full run; also recovers the
189
- incremental baseline's newly-covering-example blind spot)
242
+ residual blind spot — constant-reference detection handles the common
243
+ newly-covering-example case since 0.2)
190
244
 
191
245
  ## Flags
192
246
 
193
247
  | Flag | Default | Meaning |
194
248
  |---|---|---|
195
249
  | `--jobs N` | half the cores | fork-pool width |
196
- | `--changed` | | mutate uncommitted + untracked work |
197
- | `--since REF` | | mutate methods changed since REF |
198
- | `--subject NAME` | | one subject, e.g. `Foo#bar` |
199
- | `--format terminal\|json` | terminal | report format |
250
+ | `--changed` | none | mutate uncommitted + untracked work |
251
+ | `--since REF` | none | mutate methods changed since REF |
252
+ | `--subject EXPR` | none | subject expression, e.g. `Foo#bar`, `Foo::Bar`, `Foo::Bar*`, `Foo#*`, `Foo.*` |
253
+ | `--exclude PAT` | none | skip files matching glob during subject discovery (repeatable, gitignore-like) |
254
+ | `--max-mutants N` | none | deterministic sample of the first N mutants (quick smoke run on huge scopes; accepted/uncovered mutants count against N) |
255
+ | `--debug-plan` | off | print planned mutants as JSON and exit without running |
256
+ | `--format terminal\|json\|stryker-json\|github` | terminal | report format |
200
257
  | `--accept-survivors` | off | record survivors to the acceptance ledger |
201
258
  | `--force-baseline` | off | ignore cached coverage map |
202
259
  | `--preload-helper FILE` / `--no-preload-helper` | auto-detect | parent spec-helper preload |
203
260
  | `--serial-pattern PAT` | `spec/system/`, `spec/features/` | covering-path prefixes forced serial |
204
261
  | `--browser-boot-seconds S` | 15 | serial-lane timeout bump |
205
262
  | `--timeout-factor F` / `--timeout-floor S` | 8 / 10 | mutation timeout budget |
206
- | `--require FILE` | | preload files (repeatable) |
263
+ | `--[no-]adaptive-timeout` | on | scale timeout budgets from observed worker wall times (median utilization, grow-only, clamped 1x–4x; `--timeout-factor`/`--timeout-floor` set the starting budget) |
264
+ | `--require FILE` | none | preload files (repeatable) |
265
+ | `--operator FILE` | none | load a custom operator file before analysis (repeatable) |
266
+ | `--fail-at SCORE` | none (strict) | exit 0 if score >= SCORE even with survivors (opt-in relaxation for gradual adoption; 0 = report-only) |
267
+
268
+ `--debug-plan` prints the planned mutant list as one JSON document
269
+ (`{"planned": [...], "pre_resolved": {...}}`) and exits without running
270
+ anything. A coverage baseline is still built or loaded, since timeouts
271
+ and covering examples come from it.
207
272
 
208
- Every active_mutator process sets `ENV["ACTIVE_MUTATOR"] = "1"` use it to
273
+ Every active_mutator process sets `ENV["ACTIVE_MUTATOR"] = "1"`. Use it to
209
274
  guard SimpleCov or other tooling in your spec helper:
210
275
 
211
276
  ```ruby
212
277
  SimpleCov.start "rails" unless ENV["ACTIVE_MUTATOR"]
213
278
  ```
214
279
 
280
+ ## Configuration file
281
+
282
+ Put team-wide settings in `.active_mutator.yml` at the project root; CLI
283
+ flags override file values (`--require` and `--exclude` add to the file's
284
+ lists; the first `--serial-pattern` replaces them). Recognized keys:
285
+ `jobs`, `format`, `timeout_factor`, `timeout_floor`,
286
+ `browser_boot_seconds`, `fail_at`, `exclude`, `serial_patterns`,
287
+ `requires`, `operators` (custom operator files, loaded before analysis; see
288
+ [Custom operators](docs/guides/custom-operators.md)),
289
+ `preload_helper` (a path, or `false` to skip preload),
290
+ `adaptive_timeout` (`true`/`false`).
291
+ Unknown keys and wrong types are errors, not silent no-ops.
292
+
293
+ ```yaml
294
+ # .active_mutator.yml
295
+ jobs: 4
296
+ exclude:
297
+ - lib/generated
298
+ serial_patterns:
299
+ - spec/system/
300
+ fail_at: 90 # legacy suite: gate on score instead of zero-survivors
301
+ ```
302
+
215
303
  ## Known limits (v1.1)
216
304
 
217
- Method bodies only (no class-macro/constant mutation) · RSpec only ·
218
- heredoc strings not mutated · `class << self` bodies and nested defs
219
- skipped · incremental baseline can miss examples that only cover changed
220
- code after the change (nightly `--force-baseline` recovers).
305
+ Method bodies only (no class-macro/constant mutation). RSpec only.
306
+ Plain heredoc bodies ARE mutated (emptied); interpolated heredocs are
307
+ skipped. `class << self` bodies are mutated as singleton subjects
308
+ (`class << obj` and top-level `class << self` are skipped). Nested defs
309
+ mutate as part of the enclosing method's body — they get no subject of
310
+ their own (a directly-inserted mutant would be reverted whenever the
311
+ outer method re-runs the `def`). The incremental baseline recovers the residual blind spot —
312
+ constant-reference detection handles the common case since 0.2, and a few
313
+ residual cases (pure indirection, partially-covering files, leaf-only or
314
+ wrapper-only references, `class ::Foo`, `Data.define`/`Struct.new` value
315
+ objects) are caught by nightly `--force-baseline`.
221
316
 
222
317
  ## Guides
223
318
 
224
- - [What is mutation testing?](docs/guides/what-is-mutation-testing.md)
225
- the concepts: kill/survive, score, equivalent mutants, further reading.
226
- - [How it works](docs/guides/how-it-works.md) architecture: subject
319
+ - [What is mutation testing?](docs/guides/what-is-mutation-testing.md):
320
+ the concepts. Kill/survive, score, equivalent mutants, further reading.
321
+ - [How it works](docs/guides/how-it-works.md): architecture. Subject
227
322
  discovery, source-span edits, the coverage map, the fork pipeline, and
228
323
  honest limits.
229
- - [Operator reference](docs/guides/operators.md) every mutation
324
+ - [Operator reference](docs/guides/operators.md): every mutation
230
325
  active_mutator can generate, with before/after examples and what a
231
326
  survivor of each one means.
232
- - [Mutation-check skill](docs/skills/mutation-check.md) the agent-facing
233
- workflow: run, read survivors, strengthen tests, or accept with a reason.
327
+ - [Custom operators](docs/guides/custom-operators.md): write and load your
328
+ own mutation operators with `--operator` / the `operators:` config key.
329
+ - [Mutation-check skill](docs/skills/mutation-check.md): the agent-facing
330
+ workflow. Run, read survivors, strengthen tests, or accept with a reason.
234
331
 
235
332
  ## Contributing
236
333
 
237
334
  Issues and pull requests welcome. Run `bundle exec rspec` before sending a
238
- change; `bundle exec active_mutator --changed` on your own diff before
239
- sending a change that touches `lib/` is a good idea for the same reason
240
- you'd want it run on any other codebase.
335
+ change. Also run `bundle exec active_mutator --changed` on your own diff
336
+ before sending a change that touches `lib/`. This is a good idea for the
337
+ same reason you'd want it run on any other codebase.
241
338
 
242
339
  ## License
243
340
 
@@ -3,8 +3,12 @@ require "set"
3
3
 
4
4
  module ActiveMutator
5
5
  # Committed, repo-root ledger of accepted (equivalent) survivors.
6
- # Deliberately NOT inside .active_mutator/ that dir is gitignored and
6
+ # Deliberately NOT inside .active_mutator/: that dir is gitignored and
7
7
  # disposable, while acceptance decisions are durable team/CI state.
8
+ #
9
+ # Entries whose file no longer exists are kept, not pruned: the file may
10
+ # still exist on another branch, so deletion is the user's call. The runner
11
+ # warns about them on every run instead (see #missing_file_entries).
8
12
  class AcceptedLedger
9
13
  FILENAME = ".active_mutator_accepted.json"
10
14
 
@@ -28,16 +32,27 @@ module ActiveMutator
28
32
 
29
33
  def accepted?(fingerprint) = @entries.include?(fingerprint)
30
34
 
31
- def stale_entries(all_current_fingerprints)
35
+ # Entries outside the scanned files can't be judged by this run, so they
36
+ # are never stale here. scanned_files: nil means "no file was fully
37
+ # scanned" (subject-level filtering active) — union only, prune nothing.
38
+ # See #24: a scoped accept run once deleted every out-of-scope entry.
39
+ def stale_entries(all_current_fingerprints, scanned_files:)
40
+ return [] if scanned_files.nil?
41
+
32
42
  current = all_current_fingerprints.to_set
33
- @entries.reject { |e| current.include?(e) }
43
+ scanned = scanned_files.to_set
44
+ @entries.reject { |e| current.include?(e) || !scanned.include?(e.file) }
34
45
  end
35
46
 
36
- # Union new acceptances in, prune anything no longer matching a current
37
- # mutant, write atomically.
38
- def accept!(new_fingerprints, all_current_fingerprints)
39
- current = all_current_fingerprints.to_set
40
- @entries = (@entries + new_fingerprints).uniq.select { |e| current.include?(e) }
47
+ # Missing is objective regardless of run scope: such entries can never
48
+ # appear in scanned_files, so without this they'd be immortal AND silent.
49
+ def missing_file_entries(root)
50
+ @entries.reject { |e| File.exist?(File.join(root, e.file)) }
51
+ end
52
+
53
+ def accept!(new_fingerprints, all_current_fingerprints, scanned_files:)
54
+ stale = stale_entries(all_current_fingerprints, scanned_files: scanned_files).to_set
55
+ @entries = (@entries + new_fingerprints).uniq.reject { |e| stale.include?(e) }
41
56
  AtomicFile.write(@path, JSON.pretty_generate(@entries.map(&:to_h)))
42
57
  nil
43
58
  end
@@ -1,6 +1,6 @@
1
1
  module ActiveMutator
2
2
  # flock-guarded write-to-temp + rename. Concurrent runs in one repo (an
3
- # agent plus a human the dev-loop case) must not corrupt cache or ledger.
3
+ # agent plus a human, the dev-loop case) must not corrupt cache or ledger.
4
4
  module AtomicFile
5
5
  def self.write(path, content)
6
6
  File.open("#{path}.lock", File::CREAT | File::RDWR, 0o644) do |lock|
@@ -42,13 +42,22 @@ module ActiveMutator
42
42
 
43
43
  private
44
44
 
45
- def run_baseline!
45
+ # The cache is disposable and must never be committed. Host projects
46
+ # rarely gitignore it themselves, so the directory ignores its own
47
+ # contents (the node_modules trick).
48
+ def prepare_cache_dir
46
49
  FileUtils.mkdir_p(@cache_dir)
50
+ ignore = File.join(@cache_dir, ".gitignore")
51
+ File.write(ignore, "*\n") unless File.exist?(ignore)
52
+ end
53
+
54
+ def run_baseline!
55
+ prepare_cache_dir
47
56
  env = baseline_env(@out_path)
48
- # out: :err the subprocess suite's progress output must not pollute
57
+ # out: :err: the subprocess suite's progress output must not pollute
49
58
  # our stdout (breaks `--format json` consumers).
50
59
  ok = system(env, "bundle", "exec", "rspec", chdir: @root, out: :err)
51
- raise BaselineFailed, "baseline suite failed fix the suite before mutating" unless ok
60
+ raise BaselineFailed, "baseline suite failed, fix the suite before mutating" unless ok
52
61
  raise BaselineFailed, "baseline produced no coverage output" unless File.exist?(@out_path)
53
62
  end
54
63
 
@@ -80,7 +89,7 @@ module ActiveMutator
80
89
  if targets.any?
81
90
  env = baseline_env(partial_out)
82
91
  ok = system(env, "bundle", "exec", "rspec", *targets, chdir: @root, out: :err)
83
- raise BaselineFailed, "partial baseline run failed fix the suite before mutating" unless ok
92
+ raise BaselineFailed, "partial baseline run failed, fix the suite before mutating" unless ok
84
93
  raise BaselineFailed, "partial baseline produced no output" unless File.exist?(partial_out)
85
94
  end
86
95
  merge_partial!(partial_out, delta)
@@ -11,6 +11,10 @@ module ActiveMutator
11
11
  FULL = Delta.new(full: true, rerun_spec_files: [], rerun_example_ids: [],
12
12
  drop_example_ids: [], drop_source_files: [])
13
13
 
14
+ # If a changed constant is referenced by more than this share of all spec
15
+ # files, a full re-run is cheaper and simpler than a giant partial one.
16
+ REFERENCE_FULL_RATIO = 0.5
17
+
14
18
  def self.compute(old_digests:, new_digests:, coverage_map:, root:)
15
19
  changed = (old_digests.keys | new_digests.keys)
16
20
  .reject { |k| old_digests[k] == new_digests[k] }
@@ -21,6 +25,13 @@ module ActiveMutator
21
25
  drop_example_ids = []
22
26
  drop_source_files = []
23
27
 
28
+ # Read the spec-file list and their contents once per compute call, not
29
+ # once per changed source file: newly_covering_candidates scans every
30
+ # spec file, so re-globbing and re-reading inside the loop was
31
+ # O(changed_files x spec_files) IO. Built lazily so a delta with no
32
+ # scannable source change pays nothing.
33
+ spec_contents = nil
34
+
24
35
  changed.each do |rel|
25
36
  added = !old_digests.key?(rel)
26
37
  deleted = !new_digests.key?(rel)
@@ -39,7 +50,17 @@ module ActiveMutator
39
50
  else
40
51
  abs = File.join(root, rel)
41
52
  drop_source_files << abs if deleted
42
- rerun_example_ids.concat(coverage_map.examples_covering_file(abs)) unless added
53
+ unless added
54
+ rerun_example_ids.concat(coverage_map.examples_covering_file(abs))
55
+ end
56
+ unless deleted
57
+ spec_contents ||= Dir[File.join(root, "spec/**/*_spec.rb")].to_h { |f| [f, File.read(f)] }
58
+ candidates = newly_covering_candidates(root: root, rel: rel, coverage_map: coverage_map,
59
+ spec_contents: spec_contents)
60
+ return FULL if candidates == :full
61
+
62
+ rerun_spec_files.concat(candidates)
63
+ end
43
64
  end
44
65
  end
45
66
 
@@ -53,5 +74,50 @@ module ActiveMutator
53
74
  def self.full_trigger?(rel)
54
75
  rel.start_with?("spec/support/") || !rel.end_with?(".rb")
55
76
  end
77
+
78
+ # #11: an unchanged spec file can START covering a changed source file
79
+ # because of the edit itself. Cheap static detection: spec files that
80
+ # textually reference a constant the changed file defines, but currently
81
+ # contribute zero coverage to it, get re-run. Files already covering it
82
+ # are handled example-by-example via rerun_example_ids.
83
+ def self.newly_covering_candidates(root:, rel:, coverage_map:, spec_contents:)
84
+ abs = File.join(root, rel)
85
+ return [] unless File.exist?(abs)
86
+
87
+ constants = DefinedConstants.in_source(File.read(abs))
88
+ return [] if constants.empty?
89
+
90
+ all_specs = spec_contents.keys
91
+
92
+ covering_specs = coverage_map.examples_covering_file(abs)
93
+ .map { |id| spec_file_of(id) }.to_a.uniq
94
+ # Escaping is required: dynamic-namespace class definitions (e.g.
95
+ # `class (a)::Baz`, `class foo.bar::Baz`) make constant_path.slice carry
96
+ # regex metachars. Unescaped, "(a)::Baz" would match the literal text
97
+ # "a::Baz" — a false candidate.
98
+ # TODO(#11, Task 10 residual gap): a top-level `class ::Foo` yields the
99
+ # slice "::Foo", and /\b::Foo\b/ can never match (no word boundary
100
+ # before ":"), so such files are silently unscanned.
101
+ pattern = /\b(?:#{constants.map { |c| Regexp.escape(c) }.join("|")})\b/
102
+ candidates = all_specs.filter_map do |spec_abs|
103
+ spec_rel = spec_abs.delete_prefix(root).delete_prefix("/")
104
+ next if covering_specs.include?(spec_rel)
105
+
106
+ spec_rel if spec_contents.fetch(spec_abs).match?(pattern)
107
+ end
108
+ if candidates.size > 1 && candidates.size > all_specs.size * REFERENCE_FULL_RATIO
109
+ # Never silently degrade: a full baseline where the user expected an
110
+ # incremental refresh must be explained, or it looks like a hang.
111
+ warn "active_mutator: constant-reference scan matched #{candidates.size} of " \
112
+ "#{all_specs.size} spec files for #{rel}; falling back to full baseline"
113
+ return :full
114
+ end
115
+
116
+ candidates
117
+ end
118
+
119
+ def self.spec_file_of(example_id)
120
+ example_id.sub(%r{\A\./}, "").sub(/\[.*\]\z/, "")
121
+ end
56
122
  end
57
123
  end
@@ -1,5 +1,5 @@
1
1
  # Loaded standalone via RUBYOPT=-ractive_mutator/baseline_hooks in the host
2
- # project's suite before rspec boots, so Coverage instruments everything
2
+ # project's suite, before rspec boots, so Coverage instruments everything
3
3
  # the suite loads (including code loaded by spec_helper). Records per-example
4
4
  # coverage diffs and writes the inverted map to ACTIVE_MUTATOR_BASELINE_OUT.
5
5
  require "json"
@@ -53,7 +53,7 @@ if ENV["ACTIVE_MUTATOR_BASELINE_OUT"]
53
53
  root = ENV.fetch("ACTIVE_MUTATOR_ROOT")
54
54
  ActiveMutator::BaselineHooks::RECORDS[example.id] =
55
55
  ActiveMutator::BaselineHooks.diff_coverage(before, after, root)
56
- # NOT example.execution_result.run_time that is nil until after
56
+ # NOT example.execution_result.run_time: that is nil until after
57
57
  # around hooks complete.
58
58
  ActiveMutator::BaselineHooks::TIMES[example.id] = elapsed
59
59
  end
@@ -22,16 +22,20 @@ module ActiveMutator
22
22
  # boot cost (RSpec setup + spec file loading).
23
23
  requires: [], timeout_factor: 8.0, timeout_floor: 10.0, force_baseline: false,
24
24
  preload_helper: nil, serial_patterns: ["spec/system/", "spec/features/"],
25
- browser_boot_seconds: 15.0, accept_survivors: false
25
+ browser_boot_seconds: 15.0, accept_survivors: false, exclude: [],
26
+ max_mutants: nil, debug_plan: false, fail_at: nil, adaptive_timeout: true,
27
+ operator_paths: []
26
28
  }
29
+ options.merge!(ConfigFile.load(Dir.pwd))
27
30
  paths = OptionParser.new do |o|
28
31
  o.banner = "Usage: active_mutator [paths] [options]"
29
32
  o.on("--since REF", "Mutate only methods changed since git REF") { |v| options[:since] = v }
30
33
  o.on("--changed", "Mutate uncommitted work (alias for --since HEAD, plus untracked files)") { options[:since] = "HEAD" }
31
- o.on("--subject NAME", "Mutate only the named subject, e.g. Foo::Bar#baz") { |v| options[:subject_filter] = v }
34
+ o.on("--subject NAME", "Mutate matching subjects: Foo::Bar#baz, Foo::Bar, Foo::Bar*, Foo::Bar#*") { |v| options[:subject_filter] = v }
32
35
  o.on("--jobs N", Integer, "Concurrent workers (default: half the CPU count)") { |v| options[:jobs] = v }
33
- o.on("--format FMT", %w[terminal json], "Output format") { |v| options[:format] = v.to_sym }
34
- o.on("--require FILE", "File to require before mutating (repeatable)") { |v| options[:requires] << v }
36
+ o.on("--format FMT", ConfigFile::FORMATS, "Output format") { |v| options[:format] = v.tr("-", "_").to_sym }
37
+ o.on("--require FILE", "File to require before mutating (repeatable; adds to config-file requires)") { |v| options[:requires] << v }
38
+ o.on("--operator FILE", "Ruby file defining a custom operator, loaded before analysis (repeatable)") { |v| options[:operator_paths] << v }
35
39
  o.on("--force-baseline", "Ignore cached coverage map") { options[:force_baseline] = true }
36
40
  o.on("--timeout-factor F", Float, "Timeout = baseline time * F + floor") { |v| options[:timeout_factor] = v }
37
41
  o.on("--timeout-floor S", Float, "Minimum timeout seconds") { |v| options[:timeout_floor] = v }
@@ -43,7 +47,15 @@ module ActiveMutator
43
47
  options[:serial_patterns] << v
44
48
  end
45
49
  o.on("--browser-boot-seconds S", Float, "Extra timeout budget for serial-lane mutants") { |v| options[:browser_boot_seconds] = v }
50
+ o.on("--[no-]adaptive-timeout", "Scale timeout budgets from observed worker wall times (default: on)") { |v| options[:adaptive_timeout] = v }
46
51
  o.on("--accept-survivors", "Record surviving mutants into the acceptance ledger") { options[:accept_survivors] = true }
52
+ o.on("--exclude PAT", "Skip files matching glob, relative to root (repeatable)") { |v| options[:exclude] << v }
53
+ o.on("--max-mutants N", Integer, "Deterministically sample the first N mutants") { |v| options[:max_mutants] = v }
54
+ o.on("--debug-plan", "Print the planned mutant list as JSON and exit") { options[:debug_plan] = true }
55
+ o.on("--fail-at SCORE", Float, "Exit 0 if mutation score >= SCORE even with survivors (default: any survivor fails)") do |v|
56
+ raise OptionParser::InvalidArgument, "--fail-at must be within 0..100" unless (0..100).cover?(v)
57
+ options[:fail_at] = v
58
+ end
47
59
  end.parse(argv)
48
60
  options.delete(:serial_patterns_replaced)
49
61
 
@@ -4,5 +4,6 @@ module ActiveMutator
4
4
  Config = Data.define(:paths, :since, :subject_filter, :jobs, :format, :requires,
5
5
  :timeout_factor, :timeout_floor, :force_baseline, :root,
6
6
  :preload_helper, :serial_patterns, :browser_boot_seconds,
7
- :accept_survivors)
7
+ :accept_survivors, :exclude, :max_mutants, :debug_plan,
8
+ :fail_at, :adaptive_timeout, :operator_paths)
8
9
  end