saboteur 0.4.0 → 0.6.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: a5c52cb7485073b569359bf940160095370103747d55a6065b7f067726a1a53c
-  data.tar.gz: 40fca0c1c0f8391de86d7f17960d908add226682d93be2610775fc888ab36f60
+  metadata.gz: 50217a40a53106f31413eb89d15146944e93d15a83de3d2670c028094ef956b3
+  data.tar.gz: 534b1a15255aa3dd4743eb6c5a813529f8b4460423858c4de6e61bc34e2a6c93
 SHA512:
-  metadata.gz: 061e0b5a97c95048048767d3724ea883ea144e7530b9a52631d52c896f4154e1a63f7b65ef7b5817053a42ab31eb97c168aa3079c87e9d9a3e8be1ae949d7bdf
-  data.tar.gz: e29bcfafdaf2fc7ecffbc4f8746349d79c908dfc2de1530998b525c691135c2bfb4ade67575374178742b1fddb29ab96a7d9ff67656d29dbd7c096ba0b50d0dc
+  metadata.gz: 599ab18ac5ea974b823c6b84d35ac721ab40ceb902155145c9f87311834a767fc66e8c53c88e69a142f35825612a5799579648894eccb833db47b389b566e591
+  data.tar.gz: a1952882b5586fa3d846fb123f4700f5b5f0085d8c81dd129afeb4a08f3437da27550f93e7ac3aeb3a59ce682d26e7b31f200734c46da03105f254ac3fa2138d

data/CHANGELOG.md

@@ -6,9 +6,208 @@ All notable changes are documented here. The format follows
 
 ## [Unreleased]
 
+## [0.6.0] - 2026-10-14
+
+### Fixed
+- `saboteur doctor`'s boot-file check accepts `require` and `hooks`
+  paths with or without the `.rb` extension, so the README's
+  `c.require = ["./config/environment"]` quickstart no longer fails
+  the check (#257).
+
+### Added
+- `allow_db_hosts` config (#258): hosts you assert are disposable test
+  databases. The Rails integration's local-database check accepts them,
+  and its refusal names the setting. `saboteur doctor` surfaces the same
+  refusal, naming the remote hosts, and after a successful boot prints
+  each test database's host. `0.0.0.0` now counts as a local host: compose
+  setups write it into database URLs, and connecting to it reaches this
+  machine.
+- `--guess-tests` now maps files in packwerk packs (any directory with a
+  `package.yml`) to the spec inside the same pack, and maps
+  `app/controllers` files to `spec/requests` specs (`users_spec.rb` or
+  `users_controller_spec.rb`). It still names the changed files it could
+  not match (#259).
+- `script/preflight`: a local pre-review check on one exact base and head
+  (#210). It runs the protected-path scope check, sign-off, SPDX headers,
+  whitespace, RuboCop and the specs (fast or full mode), writes every harness
+  output through a symlink-safe writer under `tmp/preflight/`, and takes
+  check reports through harness-owned pipes rather than checkout paths.
+  It is not CI and a pass is not an approval.
+
+### Changed
+- All built-in operators now run by default, including `range`, `operator_assignment` and `condition_negation`, which were opt-in in 0.5.0. Use `--exclude-operators` (or `c.exclude_operators`) to omit operators, or `--operators` (or `c.operators`) to select a smaller set. Exclusions win; each CLI list replaces its matching config list. This changes the default mutation set: counts, runtime, survivors and scores may change. Review CI thresholds and survivor baselines before adopting the new results. To retain the 0.5.0 default operator set during migration: `bundle exec saboteur run --exclude-operators range,operator_assignment,condition_negation`. To clear an exclusion set in config from the CLI: `bundle exec saboteur run --exclude-operators ''`.
+- Breaking selection policy change: old `core`/`extended` tier names are refused with migration guidance. Policy version 2 binds effective operators and exclusions to cache/verdict identity, reports, score-history v2 and baseline evidence. Old baselines are incompatible and need a fresh reviewed proposal; unchanged mutation IDs stay stable.
+
+## [0.5.0] - 2026-10-14
+
+### Added
+- Opt-in extended `range`, `operator_assignment` and `condition_negation` operators (#135). They run only
+  when selected with `--operators`; the default core set does gain the new `chain_removal`
+  operator (below).
+- `whole_file` config: an opt-in list of subject ids or globs (for example
+  `"Point#distance"`, `"Geometry::*"`). For those subjects the child evaluates the
+  whole mutated file instead of re-evaluating the one method, so methods
+  surgical re-evaluation refuses can be mutated. It applies to methods already
+  found as subjects. The file's top-level code runs again in the child: load-time
+  side effects repeat and constants it assigns are rebuilt. A surviving
+  mutant whose definition a later definition in the file replaces is reported as
+  unreachable, not survived. Off by default (Refs #138).
+- `--matrix` (#132) runs every covering test per mutant and lists blind and
+  redundant tests after the survivors.
+- The 30-second heartbeat of the baseline and coverage phases now shows the examples finished so far, the percentage and a rough time left, read from the phase log, instead of only the elapsed time. (#56)
+- Each finding in `--agent` output carries a `testHint` (`kind`, `text`, and for a
+  comparison swap that differs only for equal or unequal operands, a `case`
+  such as `left == right`) saying where a test could look (#154). It is a
+  suggestion, not a diagnosis: a survivor can also be an equivalent mutant.
+- `--max-timeouts N` and `--max-errors N` CI gates (count or percent of executed
+  mutants). Exceeding one exits 1 with `exitCause` `maxTimeouts` or `maxErrors`.
+  The covered score is already gated by `--threshold`. No Action input yet.
+- A run that leaves a live survivor ends with a "Next steps" footer of command
+  templates: replay (`saboteur run --id DIGEST --seed N`, to be combined with the
+  run's own selection options), an ignore entry with the survivor's subject, and
+  `saboteur run --agent` (#147).
+- **Safety:** mutant children now run under a Landlock sandbox on Linux
+  6.2+ (Landlock ABI 3) (#152): the child cannot create, write, rename,
+  unlink or truncate files in the project tree except `tmp/`, the cache dir
+  and the system temp dir, and inherited descriptors already open for
+  writing elsewhere are closed before the sandbox engages. This is the
+  third layer behind the hazard guard and the canary, and the first one
+  that stops a destructive call before it acts. Landlock covers the
+  filesystem only. On kernels 5.13-6.1 (ABI 1-2) `truncate()` cannot be
+  restricted, and on kernels without Landlock there is no sandbox at all:
+  both run with the guard and canary only, noted once. `c.sandbox = false`
+  opts out (`/dev/shm` and `/dev/tty` are denied like any other
+  ungranted path; `/dev/null` stays writable); `c.sandbox = "strict"`
+  refuses to run degraded;
+  `c.sandbox_writable` grants extra directories (suites writing `db/`,
+  `log/`, ... otherwise fail with EACCES in mutant children) and warns when
+  a grant contains the project root or another grant. The root check covers
+  the effective write set - every grant realpath'd, the cache dir and the
+  system temp dir included - so `cache_dir = "."`, a symlinked `tmp`, a
+  `/` grant or a checkout inside the system temp dir cannot silently leave
+  the tree writable; `c.sandbox = "strict"` refuses to run then. A kernel probe that
+  errors (rather than cleanly reporting Landlock absent) refuses the run.
+  The descriptor sweep also closes stdio redirected to a writable regular
+  file (terminal, pipe and `/dev/null` stdio stays), inherited files
+  already deleted from the tree, and any descriptor it cannot inspect -
+  fail closed. The C library lookup tolerates musl (soname fallbacks, then
+  the process symbol table) and degrades to the one-time unsandboxed note
+  when no libc resolves, instead of crashing.
+- Audited inline suppression comments with required reasons (#133). Inspired by mutineer
+  (github.com/davidteren/mutineer, MIT, by David Teren).
+- `--dry-run` previews planned mutants per method and the max_mutants_per_method
+  omissions without booting the app, running tests, forking or writing reports
+  (#134). A file that does not parse marks the total INCOMPLETE. A cached
+  estimate is baseline covering-test work, not a bound on mutant runtime.
+  Custom sources are still required during a dry run. `--list-operators` and
+  `--operators` list and select builtin operators. Inspired by mutineer
+  (github.com/davidteren/mutineer, MIT, by David Teren).
+- [docs/minitest.md](docs/minitest.md) records that Minitest is unsupported and how much
+  of the engine depends on RSpec (#200).
+- [docs/scores.md](docs/scores.md) defines the covered score, the overall score a
+  Stryker dashboard shows, assertion and crash kills, and what makes a run
+  incomplete (#198).
+- A startup warning on stderr when the working tree has uncommitted changes,
+  because mutants run your code and could alter or delete files. Saboteur's own
+  output paths do not count as changes.
+- Terminal report rewrite: survivors grouped by file and method with
+  evidence-scoped hints (suggestions, not diagnoses), a score table, a
+  progress counter with ETA, `--verbose`, `--[no-]color` and `--fail-fast`
+  (serial only; the run is marked incomplete with exit cause `failFast` and
+  lists the subjects it never reached). The CI progress line-break can
+  misalign by a mark when parallel workers finish at the same moment.
+- Sorbet signature text does not prune nil-producing mutants. Runtime enforcement
+  is not verified: checked levels, custom handlers and shims can allow nil to
+  reach assertions. Type-aware pruning (#153) is deferred until enforcement can
+  be established without guessing from return-type text.
+- `endpoint_replacement` swaps explicit no-argument `first` and `last` calls,
+  including safe navigation. Calls with arguments or blocks, bare calls,
+  arid statements and nested scopes are excluded.
+- The `chain_removal` operator drops one link from the middle of a method chain:
+  `a.b.c` becomes `a.c`. The first receiver and the last call stay, so the result
+  is still a call on the original object. It ships in the default core set; links
+  that hand back their receiver (`freeze`, `dup`, `clone`, `itself`, `tap`) and
+  chains that are assignment targets are left alone (#183).
+- `saboteur baseline check|propose|accept --baseline PATH`: record today's
+  survivors as a baseline file and fail only on new ones, so an existing project
+  adopts mutation testing without a cleanup week. `check --epsilon POINTS` also
+  fails when the score drops more than POINTS percentage points below the
+  baseline's, and the failure detail prints inline. The GitHub Action exposes the
+  drop as the `baseline-score-drop` output (#131, #148).
+- `--guess-tests` scopes a `--since` run to the specs named after the changed
+  files, for suites where full selection costs too much. A spec that reaches a
+  changed file only through another file does not run, so a mutant only it
+  would kill can survive, and a line no guessed spec executes is reported as
+  NoCoverage. A changed source file with no matching spec (or no changed source
+  at all) falls back to the whole suite with a warning. It cannot be combined
+  with `--test` (#185).
+- `--allow-focus` (#108): without it, a suite narrowed by a stray `fit`,
+  `fdescribe` or `:focus` filter stops the run, because mutation testing a
+  focused slice would report a score for the wrong suite.
+- Survivor findings are ranked in the JSON and `--agent` findings: coarse
+  mutants first, then more covering tests, a larger edited-line span and nearer
+  changed lines (#191). The terminal report keeps its file/method/line groups,
+  with the ranking applied within each live or accepted line group.
+- `--score-history PATH` appends each run's scores to a JSONL file. Opt-in (#145).
+- [docs/guides/reading-survivors.md](docs/guides/reading-survivors.md) is a guide
+  to reading survivors, and [docs/recipes/README.md](docs/recipes/README.md)
+  collects setups for factories, fixtures, DatabaseCleaner, Capybara and test
+  adapters.
+### Changed
+- The text run prints one `Reports:` line naming the JSON and HTML reports, with
+  paths relative to the project root when they are under it, instead of two
+  lines with a blank line between them (#150). Scripts that read the old
+  `JSON report:` and `HTML report:` lines must read `Reports:` instead.
+- `saboteur doctor` checks the environment only by default (Ruby version,
+  coverage setup and SimpleCov conflicts, boot files, the custom map and ignore
+  subjects). The two unchanged baseline suite passes are slow on a large suite
+  and now run only with `doctor --suite` (or `--test`, which implies it) (#53).
+
+### Fixed
+- `--since` with a ref that is not a commit exits 2 as a usage error, naming the
+  ref, before the target boots. It used to exit 3 (tool failure) after boot (#115).
+- Diff mode skips a method whose only changes are comments, blank lines or
+  spacing, because its syntax tree matches the merge base. The run prints how many
+  it skipped. A method that uses `__LINE__` is compared by the line it stands
+  for, so moving it or adding a line above that use keeps it in the run.
+  Ambiguous cases (a new file or method, a name defined twice, a base
+  that does not parse) still run (#115).
+- The default arid logger pattern also matches `@logger`, `self.logger` and
+  `::Rails.logger`, so those calls no longer produce noise survivors (#111).
+- The unscoped `--since` warning now says what `--test` drops: the consumer specs of a changed method no longer run, which can turn kills into survivors. (#113)
+- **Safety:** mutants that would leave a directly-written destructive call
+  running with altered inputs are suppressed (#102). A syntactic hazard
+  guard matches calls written directly - `FileUtils.rm*`/`remove_entry*`
+  (bare names included), `File.delete`/`unlink`, `Dir.rmdir`/`delete`/
+  `unlink`, `rmtree`, `system`/`exec`/`spawn` (bare or on `Kernel`/
+  `Process`), backticks/`%x`, `IO.popen`, `Open3.*`, piped `open` - and
+  suppresses mutants inside such a call or elsewhere in a method that still
+  holds one. Calls behind aliases, variables or `send` are not recognized.
+  A canary re-checks the tracked tree after each method's mutants and aborts
+  with exit 3 when a file vanishes: detection after the damage, per method,
+  with no restoration. Configure the list via `c.hazard`; `--allow-hazardous`
+  opts out.
+- Methods defined inside blocks (`Class.new do`, `Data.define do`, `included do`) are listed as
+  skipped ("defined inside a block") instead of being silently absent. They are not mutated yet. `X = Data.define` / `Struct.new` / `Class.new` / `Module.new` with a block name X as the owner; any other block keeps the enclosing owner or "(anonymous block)". (#114)
+- Endless defs (`def x = ...`) are no longer skipped as "no coverage": the
+  coverage map now also records per-test method calls, which is how Ruby
+  reports endless-def bodies. A whole-suite build that records neither a line
+  hit nor a method call still stops with a clearer cause message. (#107)
+- Mutants that compile to the same bytecode as the original method are dropped
+  before they run, instead of surviving as untestable noise (#151).
+- Mutants on branch arms that no covering test takes are skipped, using per-test
+  branch coverage, instead of surviving as findings no test could kill (#184).
+- Diff mode lists every survivor on a changed line instead of only one (#146).
+- Methods with heredoc bodies select all of the method's tests again, and edits
+  to pattern-matching arms map to the line that executes, so `--since` selects
+  their killers (#130, #110).
+- The empty-coverage error names Bootsnap only when Bootsnap is loaded (#149).
+
 ## [0.4.0] - 2026-10-12
 
 ### Changed
+- The terminal report lists every survivor on a changed line in `--since` mode, instead of one
+  plus "N more survivors on this line in the JSON report" (#146).
 - **Breaking:** configuration moved from YAML to Ruby. `.saboteur.yml` becomes
   `config/saboteur.rb` (`Saboteur.configure do |c| ... end`, same key names),
   and `.saboteur-ignore.yml` becomes `config/saboteur_ignore.rb`