saboteur 0.7.0 → 0.8.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +657 -0
- data/README.md +198 -1077
- data/lib/saboteur/cli/exit_status.rb +32 -1
- data/lib/saboteur/cli/help_options.rb +2 -1
- data/lib/saboteur/cli/history_command.rb +135 -0
- data/lib/saboteur/cli/presets.rb +6 -2
- data/lib/saboteur/cli/selection_options.rb +31 -0
- data/lib/saboteur/cli/shard_options.rb +17 -0
- data/lib/saboteur/cli/source_set.rb +53 -0
- data/lib/saboteur/cli/survivors_command.rb +157 -0
- data/lib/saboteur/cli.rb +71 -54
- data/lib/saboteur/config.rb +98 -5
- data/lib/saboteur/config_file.rb +8 -0
- data/lib/saboteur/constant_reads.rb +182 -0
- data/lib/saboteur/constant_subject.rb +70 -0
- data/lib/saboteur/constant_subject_finder.rb +281 -0
- data/lib/saboteur/coverage_reads.rb +29 -0
- data/lib/saboteur/dirty_tree_warning.rb +128 -0
- data/lib/saboteur/doctor.rb +9 -0
- data/lib/saboteur/engine/baseline_guard.rb +62 -14
- data/lib/saboteur/engine/boot.rb +4 -0
- data/lib/saboteur/engine/candidate_selection.rb +19 -0
- data/lib/saboteur/engine/candidates.rb +43 -3
- data/lib/saboteur/engine/coverage_store.rb +32 -10
- data/lib/saboteur/engine/defined_order_prefix.rb +43 -0
- data/lib/saboteur/engine/diagnostics.rb +18 -0
- data/lib/saboteur/engine/distribution.rb +10 -0
- data/lib/saboteur/engine/early_stop_recording.rb +3 -2
- data/lib/saboteur/engine/gated_run.rb +35 -0
- data/lib/saboteur/engine/load_coverage.rb +45 -0
- data/lib/saboteur/engine/out_of_scope_guard.rb +21 -0
- data/lib/saboteur/engine/pipelines.rb +107 -0
- data/lib/saboteur/engine/result_cache.rb +65 -8
- data/lib/saboteur/engine/run_budget.rb +27 -0
- data/lib/saboteur/engine/sandbox_policy.rb +38 -0
- data/lib/saboteur/engine/scheduler.rb +109 -3
- data/lib/saboteur/engine/selection_announcement.rb +54 -0
- data/lib/saboteur/engine/shard.rb +69 -0
- data/lib/saboteur/engine/subject_selection.rb +295 -20
- data/lib/saboteur/engine/test_guess.rb +315 -25
- data/lib/saboteur/engine/unchanged_syntax.rb +17 -2
- data/lib/saboteur/engine.rb +100 -127
- data/lib/saboteur/execution/external_runner.rb +471 -0
- data/lib/saboteur/execution/marker_sweep.rb +79 -0
- data/lib/saboteur/execution/peak_memory.rb +28 -0
- data/lib/saboteur/execution/process_table.rb +12 -0
- data/lib/saboteur/execution/process_tree.rb +65 -5
- data/lib/saboteur/execution/reevaluator.rb +68 -12
- data/lib/saboteur/execution/runner.rb +46 -7
- data/lib/saboteur/execution/sandbox.rb +15 -2
- data/lib/saboteur/execution/seatbelt.rb +226 -0
- data/lib/saboteur/execution/supervisor.rb +27 -10
- data/lib/saboteur/execution/worker_pool.rb +23 -1
- data/lib/saboteur/frameworks/sqlite_memory_database.rb +49 -0
- data/lib/saboteur/gates.rb +43 -2
- data/lib/saboteur/help.rb +60 -9
- data/lib/saboteur/ignore_list.rb +29 -6
- data/lib/saboteur/integration/rspec.rb +135 -5
- data/lib/saboteur/mutation_source/argument_removal.rb +1 -1
- data/lib/saboteur/mutation_source/builtin.rb +26 -5
- data/lib/saboteur/mutation_source/chain_removal.rb +1 -1
- data/lib/saboteur/mutation_source/condition_negation.rb +1 -1
- data/lib/saboteur/mutation_source/constant_literals.rb +66 -0
- data/lib/saboteur/mutation_source/element_removal.rb +73 -0
- data/lib/saboteur/mutation_source/endpoint_replacement.rb +1 -1
- data/lib/saboteur/mutation_source/entry_edits.rb +44 -0
- data/lib/saboteur/mutation_source/filter_replacement.rb +1 -1
- data/lib/saboteur/mutation_source/hazard_guard.rb +108 -0
- data/lib/saboteur/mutation_source/literal_replacement.rb +1 -1
- data/lib/saboteur/mutation_source/method_swap.rb +148 -0
- data/lib/saboteur/mutation_source/operator_assignment.rb +1 -1
- data/lib/saboteur/mutation_source/operator_examples.rb +3 -1
- data/lib/saboteur/mutation_source/operator_registry.rb +4 -2
- data/lib/saboteur/mutation_source/operator_visit.rb +1 -1
- data/lib/saboteur/mutation_source/pair_removal.rb +13 -31
- data/lib/saboteur/mutation_source/range_replacement.rb +1 -1
- data/lib/saboteur/mutation_source/regex_replacement.rb +93 -0
- data/lib/saboteur/preview.rb +120 -12
- data/lib/saboteur/refinement_activations.rb +60 -0
- data/lib/saboteur/reporter/agent.rb +9 -1
- data/lib/saboteur/reporter/agent_commands.rb +10 -3
- data/lib/saboteur/reporter/agent_incomplete.rb +11 -0
- data/lib/saboteur/reporter/agent_summary.rb +4 -3
- data/lib/saboteur/reporter/findings.rb +5 -2
- data/lib/saboteur/reporter/history_policy.rb +11 -0
- data/lib/saboteur/reporter/history_reader.rb +47 -0
- data/lib/saboteur/reporter/html.rb +26 -4
- data/lib/saboteur/reporter/mutation_text.rb +6 -0
- data/lib/saboteur/reporter/stryker.rb +27 -3
- data/lib/saboteur/reporter/survivor_classification.rb +101 -0
- data/lib/saboteur/reporter/survivor_list.rb +15 -9
- data/lib/saboteur/reporter/terminal.rb +40 -7
- data/lib/saboteur/result.rb +19 -3
- data/lib/saboteur/selection/coverage_map.rb +105 -10
- data/lib/saboteur/selection/dependency_shield.rb +80 -0
- data/lib/saboteur/selection/external_test_map.rb +27 -0
- data/lib/saboteur/selection/git_diff.rb +93 -11
- data/lib/saboteur/selection/layout_only_lines.rb +80 -0
- data/lib/saboteur/selection/recorder.rb +22 -4
- data/lib/saboteur/selection/source_scope.rb +22 -0
- data/lib/saboteur/subject.rb +42 -7
- data/lib/saboteur/subject_finder.rb +173 -29
- data/lib/saboteur/version.rb +1 -1
- data/lib/saboteur/working_tree.rb +38 -15
- data/lib/saboteur.rb +13 -0
- metadata +35 -2
data/CHANGELOG.md
CHANGED
|
@@ -6,6 +6,663 @@ All notable changes are documented here. The format follows
|
|
|
6
6
|
|
|
7
7
|
## [Unreleased]
|
|
8
8
|
|
|
9
|
+
## [0.8.0] - 2026-10-18
|
|
10
|
+
|
|
11
|
+
### Added
|
|
12
|
+
|
|
13
|
+
- `--test-command` runs a mutation-visibility check before the first
|
|
14
|
+
mutant (#441). The command runs once with the first subject's file
|
|
15
|
+
replaced by a load-time error; if it still passes, the command cannot
|
|
16
|
+
see mutants in that file - a `RUBYLIB`/`-I` path or an installed gem
|
|
17
|
+
anchors it to the real tree, or the suite never loads the file - and
|
|
18
|
+
the run warns loudly that the score would read 0.0 % instead of
|
|
19
|
+
silently reporting every mutant Survived.
|
|
20
|
+
|
|
21
|
+
|
|
22
|
+
- `--shard K/N` (#470): run one deterministic slice of the planned candidate
|
|
23
|
+
inventory - `K` is the 1-based shard and `N` the number of shards. Each
|
|
24
|
+
planned mutant is assigned by a SHA-256 of its candidate identity, never by
|
|
25
|
+
position, so the split stays stable when an earlier subject gains or loses a
|
|
26
|
+
mutant. The terminal, JSON and agent outputs name the shard, the inventory
|
|
27
|
+
digest and the selected/planned counts, and a shard run is always reported
|
|
28
|
+
partial. See docs/guides/sharding.md for the CI matrix setup.
|
|
29
|
+
|
|
30
|
+
- `saboteur survivors <report.json>` (#475): a read-only terminal browser
|
|
31
|
+
for the Stryker JSON report - list and filter survivors (with the
|
|
32
|
+
ranked-findings order available via `--ranked`), and `--show DIGEST`
|
|
33
|
+
prints the diff, the exact replay command and an ignore-file entry
|
|
34
|
+
template. It never boots the project or edits the ignore file.
|
|
35
|
+
- `--flaky-check survivors|decisive` (#476): one reordered confirmation
|
|
36
|
+
rerun per new survivor (and per fresh kill in `decisive` mode). A
|
|
37
|
+
verdict the confirmation disagrees with becomes `unverified` - named
|
|
38
|
+
with its decisive tests, outside the score, never retried until it
|
|
39
|
+
passes.
|
|
40
|
+
- `saboteur badge` writes Shields endpoint JSON from the opt-in score
|
|
41
|
+
history and `saboteur trend` prints the covered-score delta against the
|
|
42
|
+
last comparable record (#474). A README badge needs no hosted service:
|
|
43
|
+
commit the written JSON and point a Shields endpoint badge at its raw
|
|
44
|
+
URL. An incomplete run is refused as badge evidence unless
|
|
45
|
+
`--mark-partial` labels the badge as partial, and the trend never
|
|
46
|
+
compares records with different selection, inventory, schema version
|
|
47
|
+
or operator policy.
|
|
48
|
+
- `c.sandbox = "seatbelt"`, an opt-in macOS sandbox for mutant children
|
|
49
|
+
(#436): each child applies a Seatbelt profile (writes denied except the
|
|
50
|
+
granted dirs, the system temp dir, and its own inherited logs) via
|
|
51
|
+
`sandbox_init`, after a forked probe proves a deny-default profile
|
|
52
|
+
really blocks a write. A macOS without Seatbelt degrades to the hazard
|
|
53
|
+
guard and canary with a one-time note; an errored probe or a profile
|
|
54
|
+
that does not enforce refuses the run (the Landlock probe's pinned
|
|
55
|
+
semantics, #178), and a child whose profile refuses to apply fails
|
|
56
|
+
closed instead of running the mutant.
|
|
57
|
+
On Linux the value behaves like `true`. Opt-in because Apple deprecates
|
|
58
|
+
`sandbox_init` without a published replacement; #436 tracks the
|
|
59
|
+
ecosystem's response.
|
|
60
|
+
- `saboteur run --dry-run --format json` prints a versioned machine-readable
|
|
61
|
+
mutant inventory without booting the suite (#469): schema version, run
|
|
62
|
+
metadata, subjects, every planned mutant with its stable digest, operator,
|
|
63
|
+
file, line and byte-range edits, and every skipped candidate with its
|
|
64
|
+
reason (inline suppression, hazard refusal or the per-method cap).
|
|
65
|
+
`--dry-run --agent` now names this flag instead of only refusing.
|
|
66
|
+
- A `regex_replacement` operator (#135): drops each regex anchor
|
|
67
|
+
(`\A`, `\z`, `\Z`, `^`, `$`) in turn and swaps each bare `+` or `*`
|
|
68
|
+
quantifier, so a validation like `/\A\d+\z/` finally gets mutants.
|
|
69
|
+
Escaped characters, character classes and lazy or possessive stacks stay
|
|
70
|
+
untouched; interpolated regexes stay out. The operator policy version
|
|
71
|
+
moves to 3: histories stamped with an older policy stay readable but
|
|
72
|
+
never trend-compare against new runs.
|
|
73
|
+
- Methods defined by a non-interpolated `class_eval`/`module_eval` heredoc
|
|
74
|
+
are mutation subjects (#544): the heredoc body is re-parsed against the
|
|
75
|
+
real file's bytes, so mutants inside it are generated and judged like
|
|
76
|
+
ordinary defs (the faraday shape that used to produce 0 subjects). An
|
|
77
|
+
interpolated heredoc still yields no subjects and keeps the
|
|
78
|
+
code-in-strings warning (#467).
|
|
79
|
+
- Per-path score gate config (#473, slice 1): `score_gate "app/payments/**/*",
|
|
80
|
+
min_score: 95` in `saboteur.rb` declares a per-subtree score floor,
|
|
81
|
+
validated at load (min_score 0-100, pattern must match an included file).
|
|
82
|
+
Enforcement lands with slice 2 (#570).
|
|
83
|
+
- Per-path score gates are enforced (#473, slice 2): each `score_gate`
|
|
84
|
+
subtree from slice 1 (#565) is scored with the run's own formula. A gate under its floor fails the run like the
|
|
85
|
+
count gates (`scoreGate` cause in the JSON report); a gate with nothing
|
|
86
|
+
executed under it passes vacuously and says so. Score gates cannot combine
|
|
87
|
+
with `--agent`, like `--threshold`.
|
|
88
|
+
- Peak-memory measurement (#202, slice 1): each worker reports its kernel
|
|
89
|
+
peak RSS (VmHWM) when it retires, and the JSON report records it under
|
|
90
|
+
`saboteur.memory` as per-worker `peakMb` values plus the supervisor's own
|
|
91
|
+
peak. No total is printed: fork copy-on-write makes per-process RSS
|
|
92
|
+
non-additive. The section is omitted where procfs is unavailable.
|
|
93
|
+
|
|
94
|
+
### Changed
|
|
95
|
+
|
|
96
|
+
- Any `--since` run whose resolved range has no changes now refuses with
|
|
97
|
+
exit 2 instead of passing with 0 subjects (#643) - scheduled or no-op
|
|
98
|
+
runs included. The common hit is `saboteur pr <merge_sha>` on a
|
|
99
|
+
checkout AT the merge commit, which diffed the merge against itself;
|
|
100
|
+
the refusal names the base and the resolved range and says to fetch
|
|
101
|
+
the merge's parent or point `--since` at the change's base.
|
|
102
|
+
- A run whose effective include set selects no Ruby file now refuses with
|
|
103
|
+
exit 2 instead of passing with 0 subjects (#447). The default include
|
|
104
|
+
targets plain gems, so a first run on a Rails-shaped tree used to be a
|
|
105
|
+
green gate that mutated nothing on every CI run; the refusal names the
|
|
106
|
+
stale pattern and points at the install generator.
|
|
107
|
+
- A `--subject` pattern that matches no subject now fails the run (exit 2)
|
|
108
|
+
instead of passing silently with zero subjects (#379). When some patterns
|
|
109
|
+
match, the run keeps them and the summary names the missing pattern and
|
|
110
|
+
the matched count.
|
|
111
|
+
|
|
112
|
+
### Fixed
|
|
113
|
+
|
|
114
|
+
- A single `using` anywhere in a file used to skip every subject in the
|
|
115
|
+
file. Refinement skips are now scope-granular (#634): a bare `using`
|
|
116
|
+
skips only subjects after it in its own class/module body (or the
|
|
117
|
+
rest of the file at the top level), so one module's refinement no
|
|
118
|
+
longer skips a sibling module's methods. A `using` inside a method
|
|
119
|
+
or block body still skips the whole file.
|
|
120
|
+
|
|
121
|
+
- `scope :name, -> { ... }` and `define_singleton_method` bodies are now
|
|
122
|
+
subjects (#194): the human-essentials dogfood showed a scope's lambda
|
|
123
|
+
going unmutated. The call is a singleton subject of the enclosing owner;
|
|
124
|
+
the lambda or block body mutates, and re-evaluating the whole call
|
|
125
|
+
re-registers the method.
|
|
126
|
+
- An arid statement (a logging or instrumentation call) whose block carries
|
|
127
|
+
real work is now mutated inside the block (#464). The walk used to skip
|
|
128
|
+
the whole statement, so a method like one wrapped in
|
|
129
|
+
`Notifications.instrument ... do |payload| ... end` produced only the
|
|
130
|
+
coarse `body_raise` and no statement_removal, boolean_logic or other
|
|
131
|
+
fine mutants on the changed lines. The block body is walked like any
|
|
132
|
+
other, and such a statement also keeps the method out of the
|
|
133
|
+
nothing-to-remove coarse check.
|
|
134
|
+
- A loaded class defining a singleton `hash` that does not return an
|
|
135
|
+
Integer (forem's `ReactionCategory`) no longer crashes the boot as an
|
|
136
|
+
internal TypeError: every `Coverage.result` read now refuses with a
|
|
137
|
+
diagnostic explaining the violated `#hash` contract and blaming the
|
|
138
|
+
application's code (#564).
|
|
139
|
+
- A `--since` diff that deletes a method no longer reports the deletion as
|
|
140
|
+
"differ only in comments or formatting" and no longer names the method the
|
|
141
|
+
deleted lines drifted onto. The run says what was deleted instead: `N
|
|
142
|
+
changed subject(s) were deleted, nothing to mutate: ...` (#466).
|
|
143
|
+
- An error-dominated run no longer prints a bare score above the "tested almost nothing" warning: the terminal reports "score n/a (N of M mutants scored, X % of scored)" so the scored share cannot be misread as a run-wide score (#557).
|
|
144
|
+
- Preflight no longer refuses a precompiled Ruby (mise, ruby-build) whose
|
|
145
|
+
binary tops the 50 MB evidence-document cap (#364): tool identity
|
|
146
|
+
digests stream with an executable-sized bound instead. The cap for JSON
|
|
147
|
+
evidence documents is unchanged.
|
|
148
|
+
- A missing `rspec-core` no longer crashes saboteur into the internal-error
|
|
149
|
+
report with a bare `LoadError`: the RSpec integration refuses with
|
|
150
|
+
"rspec is not in the bundle" and names the remedy, adding the gem to the
|
|
151
|
+
test group or pointing `c.integration` at the suite's framework (#482).
|
|
152
|
+
Only the framework's own `LoadError` converts; a `LoadError` raised by
|
|
153
|
+
the app's own files, or for a different feature such as `rspec/core_ext`,
|
|
154
|
+
keeps the crash route.
|
|
155
|
+
- A `--since` run whose changed methods were all skipped - refinements,
|
|
156
|
+
no coverage, or any other skip reason - used to exit 0 with `score n/a`,
|
|
157
|
+
a green run that tested nothing. It now exits 3 and names each skipped
|
|
158
|
+
subject with its reason, and a partially skipped diff announces each
|
|
159
|
+
skip up front instead of only in the end-of-run summary (#574).
|
|
160
|
+
|
|
161
|
+
- `saboteur badge` escapes incomplete-run reasons with `.inspect` in its
|
|
162
|
+
refusal message, so a reason carrying newlines or control characters
|
|
163
|
+
cannot spoof extra output lines (#474).
|
|
164
|
+
- `c.sandbox = "seatbelt"` grants now match real paths on macOS: the
|
|
165
|
+
kernel resolves `/var` to `/private/var` when it checks a write
|
|
166
|
+
against a Seatbelt subpath grant, so a granted dir under a symlinked
|
|
167
|
+
parent (like `Dir.mktmpdir` under `/var/folders`) silently matched
|
|
168
|
+
nothing and the child saw its own writable dir denied. Granted dirs
|
|
169
|
+
are now resolved with `File.realpath` before the profile is built,
|
|
170
|
+
the same view `fcntl(F_GETPATH)` reports back.
|
|
171
|
+
|
|
172
|
+
- The artifact dir now ignores itself (#426): a run writes logs,
|
|
173
|
+
coverage maps and reports under the cache dir, and the marker
|
|
174
|
+
`.gitignore` keeps a `git add -A` from committing them. An existing
|
|
175
|
+
`.gitignore` there is left alone.
|
|
176
|
+
- Rejected kills no longer surface as generic errors: the terminal
|
|
177
|
+
lists them in a `Rejected kills` section and annotates the error count,
|
|
178
|
+
and report.json flags each rejected mutant with `rejectedKill: true`
|
|
179
|
+
instead of carrying only the `rejectedKills` count (#375).
|
|
180
|
+
|
|
181
|
+
- A changed file the diff only deletes from no longer hard-stops a
|
|
182
|
+
--since run for want of a spec (#484): a deletion-only file yields no
|
|
183
|
+
subject and no fine-grained mutant, so test inference treats it as
|
|
184
|
+
subjectless instead of demanding a spec, and the run goes ahead.
|
|
185
|
+
- The pr preset no longer hard-stops when a changed file's coverage
|
|
186
|
+
lives in a consolidated spec the diff itself touches: a spec added
|
|
187
|
+
or changed in the diff that reads a constant the uncovered file
|
|
188
|
+
defines covers that file and runs, instead of exit 2 (#538). The
|
|
189
|
+
stop remains for a mutable change no changed spec exercises.
|
|
190
|
+
- Dependency trees are never treated as changed source on a `--since`
|
|
191
|
+
run (#504): a vendored bundle or `node_modules`, tracked or
|
|
192
|
+
untracked, git-ignored or not, is rejected at enumeration, so the
|
|
193
|
+
diff scope never lists, reads or warns on it. The `.bundle/config`
|
|
194
|
+
file itself is project configuration and stays visible.
|
|
195
|
+
- The project-local bundler path (`.bundle/config`'s `BUNDLE_PATH`) now
|
|
196
|
+
joins the dependency shield in its quoted and non-canonical forms
|
|
197
|
+
(#567): matching quotes are stripped, an inline `#` comment is ignored
|
|
198
|
+
the way Bundler::YAMLSerializer ignores it, a relative path is
|
|
199
|
+
normalized against the project root, and an absolute path (the system
|
|
200
|
+
bundle) stays outside the shield. A config that is not a regular file
|
|
201
|
+
or cannot be parsed - including a NUL byte in the value - means no
|
|
202
|
+
shield rather than a crash or a hang.
|
|
203
|
+
|
|
204
|
+
- A `--max-duration` budget that cannot cover the cold coverage map build
|
|
205
|
+
now stops the run before the build starts, instead of spending the
|
|
206
|
+
whole budget mid-build and keeping nothing - the trap where every
|
|
207
|
+
`--incremental` resume redid the baselines and died at the same point.
|
|
208
|
+
Once the baseline passes are timed, a build costing about one more
|
|
209
|
+
full pass than the budget holds is refused up front, with the
|
|
210
|
+
shortfall and the remedies (a higher budget, fewer
|
|
211
|
+
`--baseline-passes`, or one unbudgeted run to warm the coverage
|
|
212
|
+
cache) in the message (#427).
|
|
213
|
+
|
|
214
|
+
- The first-pass ETA no longer reads like a second baseline check when
|
|
215
|
+
the baseline is done (#465): with a cold coverage cache the remaining
|
|
216
|
+
pass is the coverage map build, and the message now names the coverage
|
|
217
|
+
build and any remaining baseline passes separately.
|
|
218
|
+
- Def-time macro mutants - a `deprecate`-style helper, a `before_action`
|
|
219
|
+
block, any DSL whose only caller is a class body - no longer report as
|
|
220
|
+
survived: the closure captured while the app loads keeps the pristine
|
|
221
|
+
definition, so replacing the method later cannot judge them. They now
|
|
222
|
+
report as `ran at load`, outside the score (#409).
|
|
223
|
+
- A mutant on the continuation line of a multi-line interpolated
|
|
224
|
+
statement is no longer misclassified missing_test (#498): attribution
|
|
225
|
+
now spans the whole enclosing statement, because an interpolation's own
|
|
226
|
+
statements are eager and Ruby's per-line events on continuation lines
|
|
227
|
+
are not reliable per example.
|
|
228
|
+
- `--incremental` now sees dependency-declaration edits on projects
|
|
229
|
+
that commit no lockfile (the library convention): the verdict
|
|
230
|
+
fingerprint hashed only `Gemfile.lock` / `gems.locked`, so the digest
|
|
231
|
+
was a constant and every cached verdict was reused across a `Gemfile`,
|
|
232
|
+
`gems.rb` or `*.gemspec` edit (#408). Manifests and gemspecs anywhere
|
|
233
|
+
under the project root are hashed always.
|
|
234
|
+
|
|
235
|
+
- A `--since` change inside a constant literal saboteur refuses to mutate
|
|
236
|
+
(unvettable load-time code, or an unsupported literal form) no longer
|
|
237
|
+
reads like an unexplained miss in the umbrella "no subjects covering the
|
|
238
|
+
changed lines" warning: the warning now says how many of the files sit
|
|
239
|
+
in a constant literal that is not mutated, named by the constant
|
|
240
|
+
warnings above it (#537).
|
|
241
|
+
|
|
242
|
+
- ERB generator templates (`.rb` files with ERB interpolation, as Rails
|
|
243
|
+
generators carry under `lib/generators/**/templates/`) no longer print a
|
|
244
|
+
per-file "does not parse" warning on every discovery pass: they can
|
|
245
|
+
never parse as plain Ruby, so they are skipped with one summarized note
|
|
246
|
+
per run. A real syntax error in a plain Ruby file still warns per file
|
|
247
|
+
(#372).
|
|
248
|
+
- A `c.sources` entry that looks like a source glob now fails with a
|
|
249
|
+
pointer to `c.include` (#488): `c.sources` registers custom operator
|
|
250
|
+
sources (`builtin` or `path.rb:ClassName`), not the files to mutate.
|
|
251
|
+
- A baseline stopped by a containment limit (process count or memory)
|
|
252
|
+
now names the knob and its current value in the refusal (#459): a
|
|
253
|
+
suite that forks heavily by design, or legitimately needs more
|
|
254
|
+
memory, is pointed at `c.max_processes` / `c.max_memory_mb` instead
|
|
255
|
+
of reading as a suite failure.
|
|
256
|
+
|
|
257
|
+
- A baseline that passed but leaked child processes no longer reads as a
|
|
258
|
+
suite failure quoting an empty "Failures:" section (#486): the refusal
|
|
259
|
+
says the suite passed, names each leaked process (pid and command,
|
|
260
|
+
captured before the supervisor's kill) and skips the log excerpt, which
|
|
261
|
+
only holds RSpec output.
|
|
262
|
+
- Engine setup load failures are reported clearly instead of as a bare
|
|
263
|
+
quit (#378): when RSpec dies while loading spec files, the error names
|
|
264
|
+
the files that failed to load, parsed from RSpec's own load-failure
|
|
265
|
+
output, and suggests the remedy - exclude the broken subtrees with
|
|
266
|
+
`--exclude-pattern` (for a Rails engine's embedded spec/dummy) or name
|
|
267
|
+
the suite with `--test`.
|
|
268
|
+
|
|
269
|
+
- A spec file whose later examples rely on state left by earlier ones no
|
|
270
|
+
longer turns its mutants into errors (#557). When a covering example
|
|
271
|
+
fails even with the method unchanged, saboteur retries once with the
|
|
272
|
+
file's examples in defined order, from the start through the last
|
|
273
|
+
covering example. The kill stands only if that run passes unchanged.
|
|
274
|
+
|
|
275
|
+
- Spec and test files vendored under `lib/` (capybara ships its driver
|
|
276
|
+
suite at `lib/capybara/spec/**`) are no longer swept into the default
|
|
277
|
+
mutation scope (#504): the default exclude now shields
|
|
278
|
+
`lib/**/spec/**`, `lib/**/test/**`, `lib/**/tests/**` and the
|
|
279
|
+
`*_spec.rb` / `*_test.rb` / `test_*.rb` shapes under `lib/`. A config's
|
|
280
|
+
own `c.exclude` replaces the shields wholesale, and the shields never
|
|
281
|
+
trip the stale-pattern warning.
|
|
282
|
+
|
|
283
|
+
- A --since run whose changed lines sit inside a heredoc now says so
|
|
284
|
+
(#467). Code embedded in a string - a class_eval/module_eval argument or
|
|
285
|
+
a template constant - evaluates methods the static walk cannot see, so
|
|
286
|
+
the generic "may not parse, or every change may sit outside a method"
|
|
287
|
+
hint misdiagnosed it; the warning now names the files and the
|
|
288
|
+
code-in-a-string blind spot.
|
|
289
|
+
|
|
290
|
+
- The "nothing to mutate" note on an empty selection now prints only
|
|
291
|
+
once (#468). An empty-selection run already ends with the
|
|
292
|
+
"0 subjects, 0 mutants" summary, and announcing the note again as a
|
|
293
|
+
report warning at the end of the run read like a second failure; it
|
|
294
|
+
is now a one-time inform when nothing has a subject.
|
|
295
|
+
|
|
296
|
+
- `--incremental` no longer reuses a stored verdict after the project source
|
|
297
|
+
set changes: a file added after the coverage map was built (a conditional
|
|
298
|
+
require, a Zeitwerk autoload) can load into a mutant run unseen, and the
|
|
299
|
+
reused verdict read as a false green. The set is paths only; content edits
|
|
300
|
+
to other files stay the documented cross-file tradeoff (#396).
|
|
301
|
+
- A factory assigned inside `class << self` now keeps its own scope in the
|
|
302
|
+
owner path, so a constant defined there is named after the factory's
|
|
303
|
+
singleton: `Host::W.singleton_class::V#n`. An outer factory with no
|
|
304
|
+
`class << self` of its own still opens no scope, even when a `class <<
|
|
305
|
+
self` encloses it (#438).
|
|
306
|
+
|
|
307
|
+
- A class factory assigned inside `class << self` no longer shares its
|
|
308
|
+
method ids with a same-named factory in the class body (#406). The two
|
|
309
|
+
unrelated classes each keep their own subjects: the singleton-side one is
|
|
310
|
+
now named `Host.singleton_class::Widget#doubled`, so reports, `--subject`
|
|
311
|
+
filters, ignore entries and the unchanged-syntax comparison can name it.
|
|
312
|
+
- `--test-command` keeps env-exported bundler settings the child needs:
|
|
313
|
+
`Bundler.with_unbundled_env` strips every `BUNDLE_*` key, so a
|
|
314
|
+
shell-exported `BUNDLE_PATH` (common in CI without a `.bundle/config`)
|
|
315
|
+
left the child's bundler unable to find the app's gems and the run
|
|
316
|
+
failed at baseline (#449). Values present in the pre-bundler
|
|
317
|
+
environment are re-added; bundler-activated values and anchor keys
|
|
318
|
+
stay dropped.
|
|
319
|
+
- `--test-command` no longer inherits bundler anchors (`BUNDLE_GEMFILE`,
|
|
320
|
+
`BUNDLE_BIN_PATH`, `BUNDLER_SETUP`, ...) from a `bundle exec` parent or
|
|
321
|
+
an exported shell environment (#435). The anchored child resolved the
|
|
322
|
+
project gem from the real tree, never exercised the mutated copy, and
|
|
323
|
+
reported every mutant Survived; the child now gets the unbundled
|
|
324
|
+
environment and resolves its Gemfile from the mutated copy.
|
|
325
|
+
- Test selection consults `config/saboteur_custom_map.rb` again (#369).
|
|
326
|
+
The map is the documented escape hatch for layouts the naming guess
|
|
327
|
+
cannot match, but guess-tests never called it, so a mapped file still
|
|
328
|
+
stopped the run as unmatched. A matching entry now wins over the naming
|
|
329
|
+
convention, its specs run as the inferred ones, and an empty entry marks
|
|
330
|
+
the file intentionally untested so it needs no inferred spec. The map is
|
|
331
|
+
now documented in the README. Mapped spec values that are globs are
|
|
332
|
+
expanded against the project root (the literal glob string never
|
|
333
|
+
reached RSpec), a winning entry whose specs match no file stops the
|
|
334
|
+
run with a config error naming the entry, and an empty entry's file
|
|
335
|
+
is dropped from mutation with a named diagnostic instead of running
|
|
336
|
+
the whole suite (#369 review).
|
|
337
|
+
|
|
338
|
+
- The marker sweep no longer kills a marked process it cannot identify
|
|
339
|
+
(#389). A process that started before the run's root carries the marker
|
|
340
|
+
only through an exec chain the run did not spawn, so it is left alone,
|
|
341
|
+
and an escapee is signalled only when a fresh snapshot shows the same
|
|
342
|
+
pid still owned by the same start time, so a recycled pid is never
|
|
343
|
+
killed by mistake.
|
|
344
|
+
- The spec-name guess now tries the underscored require name of a dashed
|
|
345
|
+
path: `lib/acts-as-taggable-on/x.rb` finds
|
|
346
|
+
`spec/acts_as_taggable_on/x_spec.rb` (and the `spec/lib/` form) instead
|
|
347
|
+
of stopping on an unmatched file (#383). The literal dashed name still
|
|
348
|
+
wins when both exist.
|
|
349
|
+
|
|
350
|
+
- Factory-block subjects resolve to the right owner (#352). A class factory
|
|
351
|
+
assigned inside `class << self` binds its constant on the singleton
|
|
352
|
+
class, so re-evaluation now resolves a factory constant after a sclass
|
|
353
|
+
scope there instead of raising "cannot resolve" for every mutant of its
|
|
354
|
+
methods. And a constant or keyword class inside a factory block binds on
|
|
355
|
+
the enclosing lexical scope: `FactoryProbe::Nested::RULES` and
|
|
356
|
+
`M::HELPER` are now ordinary subjects under their lexical owner instead
|
|
357
|
+
of a phantom `FactoryProbe::Outer::Nested::RULES` skipped as defined
|
|
358
|
+
inside a block.
|
|
359
|
+
- Constants assigned inside `class << self` are now named with their real
|
|
360
|
+
owner, the singleton class (`Host.singleton_class::RULES`), instead of the
|
|
361
|
+
attached class (#380, #390). A same-named constant in the class body no
|
|
362
|
+
longer collides with it or triggers a false "assigned more than once"
|
|
363
|
+
refusal. Read matching follows: reads inside `class << self` still count
|
|
364
|
+
toward the attached class's constant through singleton ancestry, and a
|
|
365
|
+
singleton-class write shadows the attached class's constant for reads
|
|
366
|
+
below it.
|
|
367
|
+
|
|
368
|
+
- A spec file that calls `exit` at load now refuses the run with a
|
|
369
|
+
clear diagnostic instead of terminating the loader mid-require (#411).
|
|
370
|
+
|
|
371
|
+
### Changed
|
|
372
|
+
|
|
373
|
+
- The dirty-tree warning splits its count (#309, #316): changes inside
|
|
374
|
+
the mutation scope (the Ruby files the configured include and
|
|
375
|
+
exclude select; source operands narrow the run after the warning),
|
|
376
|
+
outside it, and saboteur setup files (the config file the run reads, a
|
|
377
|
+
`BUNDLE_GEMFILE` trial Gemfile and its lock), each as tracked and
|
|
378
|
+
untracked, naming up to five scope and setup paths. The groups add up
|
|
379
|
+
to the count. A tree of untracked notes or trial setup files no longer
|
|
380
|
+
reads like application edits about to be mutated.
|
|
381
|
+
- A broken `.rb` symlink under `include` (dangling, a loop, or through a
|
|
382
|
+
regular file) is skipped instead of ending the run with an internal
|
|
383
|
+
error: it has no real path, so it is not mutation source (#503).
|
|
384
|
+
- The `each`/`map` method swap is no longer emitted where the call stands
|
|
385
|
+
as a non-last statement (#399): its return value is discarded there, and
|
|
386
|
+
both names run the same block, so the swap is treated as equivalent.
|
|
387
|
+
The last statement of a body and calls nested inside an
|
|
388
|
+
expression stay swappable, as does any call on a lazy receiver:
|
|
389
|
+
lazy.map defers its block, so the swap is not equivalent there (#431).
|
|
390
|
+
Only a `.lazy` in the receiver chain is detected: a lazy enumerator held
|
|
391
|
+
in a variable, parameter or instance variable is not, and its swaps are
|
|
392
|
+
suppressed.
|
|
393
|
+
The discard analysis now recurses through branch tails: an each/map call
|
|
394
|
+
that is the last expression of a discarded if/elsif/else/unless/case/when
|
|
395
|
+
branch, begin/rescue body, or while/until body is suppressed too (#440).
|
|
396
|
+
- A `--since` run where only some changed files have a guessed spec now
|
|
397
|
+
runs the mapped subset with a loud warning per unmapped file instead of
|
|
398
|
+
stopping with "so nothing ran"; the stop message and the warning name
|
|
399
|
+
same-basename specs found under `spec/**`, so the file can be passed to
|
|
400
|
+
`--test` without hunting; a basename containing glob metacharacters is
|
|
401
|
+
matched literally, never as a pattern. A run with no guesses at all
|
|
402
|
+
still stops, as
|
|
403
|
+
do full runs, `--subject` runs, and any run with a `--threshold` or a
|
|
404
|
+
`--max-*` gate, since the gate would judge a score the unmapped file
|
|
405
|
+
never joined (#370).
|
|
406
|
+
|
|
407
|
+
- A `--since` run now stops when every changed source file sits outside the
|
|
408
|
+
include scope (#483), naming the hidden files and the ways out (extend
|
|
409
|
+
`include`, or `--test` the specs). The old fallback ran the whole suite
|
|
410
|
+
against zero subjects - the slow full run the `pr` preset exists to
|
|
411
|
+
avoid - and its message read as if the diff had touched no source.
|
|
412
|
+
|
|
413
|
+
- The rerun and replay commands in the Next steps footer and the agent
|
|
414
|
+
protocol leave out an empty `--operators` or `--exclude-operators` list
|
|
415
|
+
that was not typed on the command line (#317): the config the command
|
|
416
|
+
reads again supplies the same empty list. A typed empty list stays, since
|
|
417
|
+
it may clear a configured value, and non-empty lists stay as before.
|
|
418
|
+
- `--since` no longer treats a line whose only change is layout as
|
|
419
|
+
changed (#313). Wrapping existing statements in a block used to put
|
|
420
|
+
every re-indented statement in scope; now only the new block lines and
|
|
421
|
+
real edits get line-level mutants. A line is layout-only when
|
|
422
|
+
`git diff -b` pairs it with a base line and Prism lexes both to the
|
|
423
|
+
same tokens, so whitespace inside strings, heredocs and regexps, and a
|
|
424
|
+
moved `__LINE__`, still count as changes.
|
|
425
|
+
- `--since` runs every spec file the diff adds or changes next to the
|
|
426
|
+
inferred specs (#307). A request or controller spec whose name maps to no
|
|
427
|
+
changed file used to be left out, so a mutant only it killed was reported
|
|
428
|
+
as a survivor. A spec the diff adds beyond inference runs only when the
|
|
429
|
+
configured RSpec suite loads it (its pattern, exclude pattern and default
|
|
430
|
+
path) and it is a regular file; inferred specs run as before. A changed
|
|
431
|
+
`*_spec.rb` that does not run, or that the diff deletes, is named with
|
|
432
|
+
the reason, also before a stop on an unmatched file. `--test` and
|
|
433
|
+
`--all` are unchanged.
|
|
434
|
+
|
|
435
|
+
- **Breaking change:** the pr preset no longer accepts a budget-stopped
|
|
436
|
+
run by default. It exits 4; pass `--max-duration-ok` explicitly to
|
|
437
|
+
accept the ordinary verdict. The terminal prints how many planned
|
|
438
|
+
mutants did not run, or says the count is unknown when planning did not
|
|
439
|
+
finish. (#654)
|
|
440
|
+
|
|
441
|
+
|
|
442
|
+
### Fixed
|
|
443
|
+
|
|
444
|
+
- A non-Rails suite that connects ActiveRecord to sqlite `:memory:` and
|
|
445
|
+
loads its schema at require time no longer gets a bare "Could not find
|
|
446
|
+
table" baseline failure (#387): every forked child reconnects to a fresh,
|
|
447
|
+
empty in-memory database, so the child now detects that shape and the
|
|
448
|
+
refusal names the cause and the remedies - move schema setup to
|
|
449
|
+
`before(:suite)`, or load it in an `after_fork` hook (README recipe
|
|
450
|
+
added). The Rails integration's per-child snapshot is unchanged.
|
|
451
|
+
- `saboteur run --test a_spec.rb b_spec.rb` refused: spec files in the source scope now fail as a
|
|
452
|
+
usage error pointing at repeating `--test`, instead of a green run that mutates nothing; the
|
|
453
|
+
`--since` "no subjects covering" warning names the misuse when every file is a spec file (#371).
|
|
454
|
+
- A mutant's test that daemonizes - forks an intermediate child that calls
|
|
455
|
+
`setsid` and exits at once - no longer leaves a sleeping grandchild the
|
|
456
|
+
supervisor never kills (#365). The tree poll can miss a descendant that
|
|
457
|
+
leaves the process group and loses its parent between two polls, so
|
|
458
|
+
every child now carries a unique marker in its environment (which
|
|
459
|
+
reaches exec'd descendants) and in its comm via `PR_SET_NAME` (which
|
|
460
|
+
reaches fork-only ones), and the end-of-run kill also sweeps the process
|
|
461
|
+
table for the marker. Linux only for now; a self-renaming daemon still
|
|
462
|
+
needs the CPU backstop.
|
|
463
|
+
|
|
464
|
+
- The Landlock ABI probe no longer flakes fail-closed with "Success -
|
|
465
|
+
landlock_create_ruleset probe" on runners where the first errno read
|
|
466
|
+
gets clobbered between the syscall and the read: the errno reader is
|
|
467
|
+
built before the probe, and a failure return paired with errno 0
|
|
468
|
+
(impossible from a real kernel answer) is probed once more. Genuinely
|
|
469
|
+
unknown errors still fail closed.
|
|
470
|
+
- `rspec_args` that hold only options and their values (such as
|
|
471
|
+
`["--require", "spec_helper"]`) no longer leave the suite with no
|
|
472
|
+
files under `exe/saboteur`: RSpec adds its default path only when `$0`
|
|
473
|
+
is `rspec`, and the value was taken for a path. saboteur now adds the
|
|
474
|
+
default path whenever RSpec's own parse finds no file or directory to
|
|
475
|
+
run, and a `--default-path` from `.rspec` wins over `spec`.
|
|
476
|
+
- `script/preflight` no longer loses why a git probe died: a probe
|
|
477
|
+
whose child never ran git (the spawn failed, or a signal killed
|
|
478
|
+
the forked child, as macOS does pre-exec under memory pressure)
|
|
479
|
+
is retried once and, when it still fails, the error names the
|
|
480
|
+
cause instead of printing an empty detail; a failure git
|
|
481
|
+
reported itself is raised at once with git's own words (#319).
|
|
482
|
+
|
|
483
|
+
- The whole-suite empty-coverage guard no longer passes when the map holds
|
|
484
|
+
spec-file lines only: the recorder maps the suite's spec files alongside
|
|
485
|
+
the included files, and the guard checked whole-map emptiness, so a suite
|
|
486
|
+
exercising none of the included files still recorded its own spec code
|
|
487
|
+
and the run exited 0 instead of 3. The guard now checks each included
|
|
488
|
+
file.
|
|
489
|
+
|
|
490
|
+
- Coverage-guided selection no longer drops a mutant's killing tests when a
|
|
491
|
+
`case` has a `when X` arm whose body is bare `nil`: Ruby spans that arm
|
|
492
|
+
from the `when` to the case's end, containing its sibling arms, and the
|
|
493
|
+
recorder stored the span as-is, so edits inside the sibling arms were
|
|
494
|
+
narrowed to the nil-arm's takers. Sibling arms are disjoint by
|
|
495
|
+
definition; a span containing one of them is now rejected when the map
|
|
496
|
+
is recorded (coverage map format 9).
|
|
497
|
+
|
|
498
|
+
- `script/preflight` full mode no longer rejects its own RSpec report: the
|
|
499
|
+
fd3 shim now attaches a dedicated JSON formatter to the pipe instead of
|
|
500
|
+
redirecting RSpec's output stream, so a project's progress formatter
|
|
501
|
+
(`.rspec` `--format progress`) and spec output stay on stdout and cannot
|
|
502
|
+
corrupt the report.
|
|
503
|
+
|
|
504
|
+
- `--incremental` no longer reuses a verdict when a file the covering
|
|
505
|
+
tests touched changed: the dependency files from the run's coverage map
|
|
506
|
+
join the verdict fingerprint (#197). The same holds for a dependency
|
|
507
|
+
with only method coverage (an endless def), a file that ran while the
|
|
508
|
+
app loaded, a loaded source the coverage map never observed, and a
|
|
509
|
+
helper required lazily inside a test outside the include list, which
|
|
510
|
+
the coverage build now records in the map so its changes invalidate
|
|
511
|
+
verdicts too (#344, #353; coverage map format 10).
|
|
512
|
+
- `--test-command` survivors no longer carry the `missing_test`
|
|
513
|
+
classification: the run's only recorded evidence is the pseudo-test id,
|
|
514
|
+
which is not real coverage (#305).
|
|
515
|
+
- `--test-command` on macOS no longer refuses every tree containing an
|
|
516
|
+
internal symlink: the disposable copy root is canonicalized before
|
|
517
|
+
symlink targets are judged against it (/var vs /private/var) (#305).
|
|
518
|
+
|
|
519
|
+
- Suites that turn warnings into errors (`--warnings` plus a raising
|
|
520
|
+
`Warning.warn` or `Warning.process` handler, dry-rb/hanami-style) no
|
|
521
|
+
longer error every mutant: the re-evaluation's own "method redefined"
|
|
522
|
+
notice is silenced while the mutant is installed, the same guard the
|
|
523
|
+
whole-file strategy already had (#603).
|
|
524
|
+
|
|
525
|
+
- A mutant reported Survived when the suite itself replaced its
|
|
526
|
+
definition mid-run: a `before(:each)` that re-loads the mutated source
|
|
527
|
+
file (rolify-style `load "lib/..."`, #597) puts the original back
|
|
528
|
+
before any test calls it, so every selected test passing proved
|
|
529
|
+
nothing. The child now records which definition the mutant's
|
|
530
|
+
evaluation installed and compares it with what is in place after the
|
|
531
|
+
tests; a positively identified different def marks the mutant
|
|
532
|
+
Unreachable (out of the score), never Survived, only when no test ever
|
|
533
|
+
invoked the mutant, and the warning names the re-load site. A run
|
|
534
|
+
where every mutant ends Unreachable exits 3 instead of going silently
|
|
535
|
+
green. (#662)
|
|
536
|
+
|
|
537
|
+
- A scoped (`--since`) run that skips a subject for no coverage now
|
|
538
|
+
names the loaded specs (or says none) and points at `--test`. Coverage
|
|
539
|
+
in sibling spec files may not have been loaded; the existing exit-3
|
|
540
|
+
gate is unchanged. (#636)
|
|
541
|
+
|
|
542
|
+
|
|
543
|
+
### Added
|
|
544
|
+
|
|
545
|
+
- guess-tests now tries the flattened spec root and a `spec/unit/` prefix
|
|
546
|
+
after the mirrored guesses: `lib/pundit/policy_finder.rb` finds
|
|
547
|
+
`spec/policy_finder_spec.rb` and `lib/aasm/base.rb` finds
|
|
548
|
+
`spec/unit/base_spec.rb`, so `--subject` and `--since` stop hard-stopping
|
|
549
|
+
on pundit, aasm and shoulda-matchers style layouts (#394, #370, #393).
|
|
550
|
+
- Methods defined in the four core class factories (`X = Data.define(...)`,
|
|
551
|
+
`Struct.new`, `Class.new`, `Module.new` with a block) are mutation
|
|
552
|
+
subjects instead of skipped subjects (#290): the finder names the
|
|
553
|
+
generated class as the owner, and the mutant is re-evaluated through
|
|
554
|
+
`X.class_eval`, leaving the generated class itself untouched. Defs in
|
|
555
|
+
any other block stay skipped as "defined inside a block".
|
|
556
|
+
- `define_method(:m) { ... }` calls with a literal name and a block are
|
|
557
|
+
mutation subjects (#194): the block body is mutated and the whole call is
|
|
558
|
+
re-evaluated on the enclosing owner. A computed name, an explicit
|
|
559
|
+
receiver, or a lambda argument stays unsupported, and a call in a block
|
|
560
|
+
whose owner is unknown keeps the "defined inside a block" skip reason.
|
|
561
|
+
|
|
562
|
+
- A fault-injection e2e probe now pins cache-dir removal during the
|
|
563
|
+
baseline window (#300): deleting `.saboteur` while the baseline holds
|
|
564
|
+
the suite open changes neither the exit code nor the run evidence -
|
|
565
|
+
later phases recreate the directory and the report holds the same
|
|
566
|
+
verdicts; only the in-flight baseline log is lost. Other windows are
|
|
567
|
+
not pinned to recover (a missing mutant log fails the run by design),
|
|
568
|
+
so stop the run before deleting the cache, as `saboteur help` says.
|
|
569
|
+
|
|
570
|
+
- `saboteur doctor` now documents its version scope (#318): it certifies no
|
|
571
|
+
versions of Rails, databases or application tools saboteur never invokes.
|
|
572
|
+
A mismatched tool the suite loads still surfaces through the boot check's
|
|
573
|
+
LoadError or the baseline refusal with the tool's own output; a narrower
|
|
574
|
+
doctor check first needs the tool name, the expected and actual versions
|
|
575
|
+
and the failure output, recorded on the issue.
|
|
576
|
+
- `--test-command CMD` / `c.test_command`: run an external command per
|
|
577
|
+
mutant and take its exit status as the verdict - for apps that cannot
|
|
578
|
+
load inside saboteur's Ruby. Serial, no coverage, unsandboxed. Mutants
|
|
579
|
+
are applied in a fresh disposable copy of the tree per execution - the
|
|
580
|
+
runner's own writes never touch the real files or follow symlinks - and
|
|
581
|
+
the command's process group is terminated on every exit path. One
|
|
582
|
+
budget-bounded neutral preflight checks the command on an untouched
|
|
583
|
+
copy before any mutant exists; the shell's 126/127 read as setup
|
|
584
|
+
errors, blank and non-string commands are rejected, and nested
|
|
585
|
+
saboteur runs are refused (#140, #305). The preflight runs under an
|
|
586
|
+
absolute deadline admitted after the tree copy, and a spent budget
|
|
587
|
+
stops the run as budget-exhausted rather than a command failure.
|
|
588
|
+
Internal symlinks are rebuilt as relative links inside the copy, so an
|
|
589
|
+
alias sees the mutant while nothing can link out; escaping or looping
|
|
590
|
+
links refuse the run. Copy and write failures become error outcomes
|
|
591
|
+
with bounded context instead of crashing the run. Output capture is a
|
|
592
|
+
bounded in-memory ring buffer, scrubbed to valid UTF-8 for reports,
|
|
593
|
+
and its drain is time-bounded so a session-escaped descendant holding
|
|
594
|
+
the pipe cannot stall the run. Subjects outside the project root are
|
|
595
|
+
refused.
|
|
596
|
+
- Survivor classification labels in the JSON report (#192): each survivor
|
|
597
|
+
carries `classification` - `likely_equivalent` when the mutant compiles
|
|
598
|
+
to the original's instructions, `arid` when every line an edit touches
|
|
599
|
+
matches an arid pattern, `missing_test` when a test covered the line
|
|
600
|
+
and passed. The HTML report badges each survivor with the same label
|
|
601
|
+
and the agent JSON carries it on each shown finding. Labels are
|
|
602
|
+
heuristics for triage order only, never a diagnosis: scores and the
|
|
603
|
+
exit code never change.
|
|
604
|
+
- `--since` mutates changed entries of constant hash and array literals:
|
|
605
|
+
the assignments join the diff scope as subjects, `pair_removal` edits hash
|
|
606
|
+
entries, the new default-on `element_removal` operator edits array
|
|
607
|
+
elements (in methods too, so array literals grow mutant totals the way
|
|
608
|
+
hashes do), and the tests are the specs covering a read of the constant.
|
|
609
|
+
Unsupported forms (namespaced targets, `||=`/`&&=`, non-literal values,
|
|
610
|
+
condition-guarded writes, or a constant written more than once in one
|
|
611
|
+
file counting every write) are named in the diagnostics instead of being
|
|
612
|
+
mutated or reported survived (#306).
|
|
613
|
+
- Constant replay vets the whole file: `--since` replays a constant's file
|
|
614
|
+
only when its load-time code is limited to requires, pure defs,
|
|
615
|
+
class/module nesting and literal assignments and trips no hazard denylist
|
|
616
|
+
entry; anything else needs the constant listed in `c.whole_file` (the
|
|
617
|
+
existing whole-file opt-in) or the literal moved into its own file.
|
|
618
|
+
Constants whose only reads capture the value at load time elsewhere -
|
|
619
|
+
including a capture inside a boot-time block - are dropped with a
|
|
620
|
+
diagnostic. Spec files count as consumers: the coverage map records spec
|
|
621
|
+
line hits, directory test paths and the default spec tree feed the
|
|
622
|
+
consumer scan, `element_removal` honors the arid policy, arid patterns
|
|
623
|
+
screen constant entries too, and constant mutation respects `c.sources`
|
|
624
|
+
(no `builtin`, no constant mutants) (#306, #319).
|
|
625
|
+
- Constant replay hardening from the #319 lens review: `require` and
|
|
626
|
+
`require_relative` count as vetted load code only with a plain
|
|
627
|
+
literal-string argument and no block, so `require(build_path)` no longer
|
|
628
|
+
reruns helper code on every mutant; rooted constant writes (`::RULES`,
|
|
629
|
+
`::Shop::RULES`) share one identity with the bare write, and an
|
|
630
|
+
unrooted qualified write (`Shop::RULES`) resolves through the lexical
|
|
631
|
+
scope, so it demotes every literal on its scope chain it may overwrite
|
|
632
|
+
(`Shop::RULES` inside `module Outer; module Shop` overwrites
|
|
633
|
+
`Outer::Shop::RULES`); either way a re-assigning write demotes the
|
|
634
|
+
first literal to unsupported instead of leaving it scored; the --all consumer scan reads
|
|
635
|
+
the configured integration's loaded spec files instead of guessing the
|
|
636
|
+
conventional spec tree; and an arid pattern that cannot judge an entry's
|
|
637
|
+
bytes (binary source against a UTF-8 pattern) keeps the mutant instead
|
|
638
|
+
of raising (#319).
|
|
639
|
+
- Fault-injection e2e specs pin exit 3 with a clear, test-blame-free message
|
|
640
|
+
when the cache dir is read-only, is occupied by a regular file, or holds an
|
|
641
|
+
unwritable baseline log (#251).
|
|
642
|
+
- A dropped mutation that does not parse is now named in the warnings with
|
|
643
|
+
its subject and operator instead of only counting toward invalid
|
|
644
|
+
mutations (#116).
|
|
645
|
+
- `docs/stability.md`: the agent JSON payload and the exit codes are stable
|
|
646
|
+
public APIs with a deprecation process, pinned by a new contract spec; a
|
|
647
|
+
governance note in CONTRIBUTING covers maintainer availability (#249).
|
|
648
|
+
- `saboteur doctor` says plainly on macOS that mutants run unsandboxed,
|
|
649
|
+
advising a container for filesystem containment and network denial or
|
|
650
|
+
stubbing for real-API risk, and the README carries a short per-platform
|
|
651
|
+
sandbox and network statement (#247).
|
|
652
|
+
- Ignore-list entries accept a triage `state:` - `accepted_equivalent`
|
|
653
|
+
(the default) and `wont_fix` suppress the survivor's exit failure (it
|
|
654
|
+
still counts in the score), `todo` records the decision to fix and keeps
|
|
655
|
+
the survivor counting as an unignored survivor. The HTML report badges the
|
|
656
|
+
state; in the JSON report `accepted_equivalent` keeps the plain
|
|
657
|
+
`ignored: <reason>` while `wont_fix` and `todo` are labelled
|
|
658
|
+
(`ignored: <reason> (wont_fix)`, `todo: <reason>`), and all three keep
|
|
659
|
+
status `Survived` (#155).
|
|
660
|
+
|
|
661
|
+
### Removed
|
|
662
|
+
- The stale self-deprecating benchmark claims in the README ('rerun
|
|
663
|
+
pending'): speed and memory numbers now live only with the harness and raw
|
|
664
|
+
data that reproduce them, or are framed as dated history (#243).
|
|
665
|
+
|
|
9
666
|
## [0.7.0] - 2026-10-16
|
|
10
667
|
|
|
11
668
|
### Upgrade notes
|