saboteur 0.6.0 → 0.7.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 50217a40a53106f31413eb89d15146944e93d15a83de3d2670c028094ef956b3
-  data.tar.gz: 534b1a15255aa3dd4743eb6c5a813529f8b4460423858c4de6e61bc34e2a6c93
+  metadata.gz: c6e75e9cdd3caa4691a569db12d33348c09b05076865e1f62a96ce9664333b88
+  data.tar.gz: dc677720684aa27b4c24120695339961a7cc23327e67f187948abbde77b93e16
 SHA512:
-  metadata.gz: 599ab18ac5ea974b823c6b84d35ac721ab40ceb902155145c9f87311834a767fc66e8c53c88e69a142f35825612a5799579648894eccb833db47b389b566e591
-  data.tar.gz: a1952882b5586fa3d846fb123f4700f5b5f0085d8c81dd129afeb4a08f3437da27550f93e7ac3aeb3a59ce682d26e7b31f200734c46da03105f254ac3fa2138d
+  metadata.gz: 183c1c0f548c72bb6d6535eb35c5456a74fa0bf02dbd66dd689d5f87424b91cd048e319acfea75532a50568821ea47168508831e1fd034dc67850c3e8c1bdbfe
+  data.tar.gz: c9f52334868d552cdde9e0a404cc0b45fde089dfe77dbe30f0b252e8084cb291e121a6728f267fec98437587321d5f9fff36463bb27b19f0281f3c36e3e69fc1

data/CHANGELOG.md

@@ -6,6 +6,373 @@ All notable changes are documented here. The format follows
 
 ## [Unreleased]
 
+## [0.7.0] - 2026-10-16
+
+### Upgrade notes
+- Inferred test selection is now the default. Use `--all` or
+  `c.test_selection = :all` to keep the 0.6.0 whole-suite behavior; use
+  `--test` to name specs explicitly. Missing inferred specs stop before
+  boot with exit 2. `--guess-tests` is now a deprecated alias for the
+  default; `--no-guess-tests` warns and is ignored.
+- Three new default-on operators change the mutation set, runtime and
+  scores. Review survivor baselines and CI thresholds after upgrading;
+  use `--exclude-operators` to omit operators you do not want.
+- Harness integrations should handle budget-exhausted runs (exit 4),
+  unverified mutants outside the score, and explicit JSON report write
+  failures (exit 3). In agent mode, exit codes remain verdict-driven;
+  budget exhaustion appears in `incomplete[]` and `earlyStop` instead.
+  `--max-duration-ok` does not certify an unscored run.
+
+### Changed
+- Breaking: inferred test selection is now the default. Without `--test`,
+  a run picks the spec named after each mutated file and runs only those
+  for the baseline passes, the coverage pass and every mutant, as `--test`
+  would; with `--since` only the changed files count. A file that needs a
+  spec and has none now stops the run before it boots (exit 2) and names
+  both ways out, instead of silently running the whole suite. Use `--all`
+  or `c.test_selection = :all` to keep the old whole-suite behavior.
+  `--guess-tests` is a deprecated alias for the default and
+  `--no-guess-tests` warns and is ignored; both flags disappear in a
+  future release (#270). A `--subject` run infers only from the named
+  subjects' files, so an unrelated spec-less file no longer stops a
+  one-method run. The unmatched-file guidance names the expected specs
+  and tells you to re-run the same command with `--test` followed by your
+  existing spec paths, or to append `--all`, instead of printing a
+  reconstructed command. `--test` may stand alone (the configured include
+  names the sources), so agent rerun/replay commands stay executable; a
+  whole-suite `--all` run is recorded in the agent provenance only when
+  no tests resolved, so a configured `:all` combined with named tests
+  still produces runnable follow-up commands. Baseline evidence is
+  fingerprinted from the declared config before inference narrows the
+  test list, so `baseline propose` and `baseline check` agree again
+  (#275). Two stale ignore-list entries for
+  `Saboteur::CLI#define_execution_options` are gone: full runs no longer
+  generate those description-string mutants, so the entries could never
+  match again.
+  A `--subject` run combined with `--since` now infers specs only for
+  named subjects inside the diff scope: an unchanged subject file, one
+  whose changes sit in other methods, or one with a comment-only change
+  no longer stops the run before boot. Agent provenance records tests
+  defined in the config file, so rerun and replay commands keep their
+  scope instead of falling back to inference.
+- Report-artifact write semantics (#296): a failed HTML report or score
+  history write now warns and lets the run exit by its results, so a
+  read-only tree (a sandboxed child holding the project read-only) no
+  longer turns a good run into a tool failure. An explicitly requested
+  JSON report (`--json` / `c.json_report`) instead fails the run with
+  exit 3 - it is the completion evidence an outer harness trusts, and a
+  warning could leave a stale or partial file presented as this run's.
+  The default `.saboteur/report.json` stays best-effort. Score-history
+  soft failures are narrowed to the write's own IO errors; validation,
+  integrity, record-size and lock failures stay hard errors.
+- Crash-log fallback hardening (#296): when the cache dir cannot hold
+  `internal-error.log`, the fallback in the system temp dir is created
+  privately (`O_EXCL | O_NOFOLLOW`, mode 0600) instead of following or
+  overwriting a pre-existing path. The pid-keyed fallback path is only
+  computed after the primary write fails. Every crash log is now written private (0600) through a
+  no-follow, non-blocking open - a planted symlink cannot redirect the
+  write and a planted FIFO cannot block it - and the runner reads it
+  back the same way, accepting only a regular file owned by the same uid
+  with no group/other bits. Where the platform lacks `O_NOFOLLOW` or
+  `O_NONBLOCK`, the primary log and the excerpt are omitted rather than
+  opened unsafely; the `O_EXCL`-created temp fallback stays safe and can
+  still be written. The appended
+  excerpt is scrubbed to valid UTF-8 and labelled as freshness-matched,
+  not certainly that run's.
+
+### Added
+- Dogfood/mutation diagnostics (#296): a rejected kill now reports why the
+  killing test failed with the method unchanged - the exception class, its
+  message and up to five backtrace frames, bounded to 2000 chars -
+  instead of only the first line ("expected: 0"). Filtering frames whose
+  paths contain `/gems/rspec` prioritizes project frames; the message and
+  frame limits do not guarantee that the failing spec line survives.
+  The terminal report
+  prints the detail as indented continuation lines and the JSON report's
+  `statusReason` keeps every line; one-line reasons are unchanged. An `unverified` verdict also keeps the runner's own
+  reason (for example "the --max-duration budget is spent") rather than a
+  generic one.
+- Nested-run crash surfacing (#296): a spec that runs saboteur itself (the
+  dogfood's own cli_spec) crashes the nested CLI as exit 3, and the crash
+  lands in the run's internal-error.log. A child whose tests wrote a fresh
+  log appends its first lines to the failure detail, so a verdict check's
+  rejection includes crash context instead of only the exit-code diff.
+  The excerpt is freshness-matched and may be from another run; it is
+  not proof that this nested run wrote the log.
+- `--max-duration` budget-exhaustion semantics (#296): a verdict check or
+  timeout calibration the budget cannot admit is skipped rather than run
+  with a zero-length timeout, so genuine kills are no longer misclassified
+  as errors and cached. Admission is per check and per sequence: a check
+  admitted with time left that hits its budget-shortened cap mid-check is
+  `unverified`, not a rejected kill, later sequences never fork once the
+  budget is spent, and unverified verdicts are never cached. Affected
+  mutants report as `unverified` - outside the score - the early stop
+  lists their subjects (`unverifiedSubjects` in the JSON report), and the
+  CLI run exits 4 unless `--max-duration-ok` is set. Agent mode remains
+  verdict-driven and records the budget in `incomplete[]` and `earlyStop`.
+  Baseline checks and the coverage map build are now capped
+  at the time left in the budget, and the early-stop reason comes from the
+  actual truncation state, so mid-subject budget stops report
+  `max_duration` even with `--fail-fast`. Agent output names unverified
+  mutants in `incomplete[]`, and `--max-duration-ok` only passes a run
+  whose verified verdicts (kills, survivors, timeouts) can carry a score.
+- `saboteur pr [REF]` and `saboteur audit` presets: opinionated bundles of
+  existing flags - the PR loop (changed lines since REF, one baseline
+  check, a ten-minute budget with `--max-duration-ok`) and the audit
+  preset (`--all`, all operators, two baseline checks, JSON and HTML
+  reports). The PR preset opts out of the budget-specific exit 4, not
+  normal or incomplete verdict failures. The audit keeps configured
+  `since`, subjects, exclusions and budgets; explicit tests replace its
+  injected `--all`. The run prints the expansion up front and explicit
+  flags override it (#244).
+- `--max-duration` (config `max_duration`, e.g. `10m`) budgets the whole
+  run, baselines included: past the budget the run stops before the next
+  subject, reports which selected subjects never ran, marks the JSON and
+  agent output partial, and exits 4 in ordinary CLI mode;
+  `--max-duration-ok` leaves the exit to the plain verdict (which can
+  still fail the run). Agent exits remain verdict-driven, with budget
+  exhaustion in `incomplete[]` and `earlyStop`. Works with
+  `--jobs` by halting the worker pool (#245).
+- `baseline_passes` (config and `--baseline-passes`) sets how many times the
+  unchanged suite runs before mutation (default 2). `1` is the fast PR
+  loop; the run announces that it gives up the order-stability
+  comparison between passes, and the whole-suite cost message counts
+  the configured checks (#55).
+- Usability fixes from 0.5.0 field feedback: an unscoped whole-suite run
+  announces the example count and the real number of full passes before
+  the first one (a cached map counts as reused only when it loads clean)
+  and estimates the remaining baseline and coverage time once the first
+  baseline pass is timed; past 1,000 examples it also names `--test` as
+  the way to scope (#260). The Next steps footer prints the complete,
+  shell-safe replay command carrying the run's own selection options
+  and the CLI-only switches that change candidate availability or boot,
+  in full however long it gets, instead of asking you to reassemble
+  them (#261). `--verbose` prints one line per finished subject with its
+  result marks, so a survivor is attributable to a method before the run
+  ends (#262). The threads-alive-at-fork warning says why the two known
+  threads are expected harmless, and under which conditions, and what an
+  unknown thread can break (#263). A `chain_removal` survivor's hint
+  appends a conditional equivalence note: if the shorter chain proves
+  equivalent across the intended inputs, simplify or ignore (#264).
+- Three new default-on operators: `method_swap` swaps a method with its
+  semantic opposite (`min`/`max`, `any?`/`all?`, `map`/`each`; a receiver
+  is required, so bare `max(a, b)` is untouched) (#237),
+  `operand_replacement` replaces an arithmetic or comparison operation
+  with each operand in turn (`a + b` yields `a` and `b`; boolean
+  connectors stay with `boolean_logic`) (#238), and
+  `interpolation_removal` removes each `#{expr}` and each literal segment
+  of an interpolated string (#239). The hazard guard now also recognizes
+  `send`, `public_send` and `__send__` with a literal symbol or string
+  selector and applies the same checks as for the direct call to the
+  resolved selector, with the arguments shifted past it: a destructive
+  method name on any receiver (`helper.send(:rmtree, path)`), the
+  configured hazard patterns, `Kernel#system` / `Process#spawn`, and
+  `Kernel#open` with a "| command" argument all stay suppressed. Dynamic
+  selectors stay out of scope as documented (#240).
+- `--since` warns when a changed Ruby file in scope has no subjects
+  covering the changed lines, so nothing is mutated from it (#232). A
+  silent zero-subject scope used to pass as a clean run. Comment-only or
+  formatting-only changes within discovered methods are skipped without
+  this warning; the run reports those skipped methods. `--since` also
+  warns when a changed Ruby file is covered by an include pattern but
+  removed by `c.exclude`, naming up to five such files, so a too-wide
+  exclude can no longer dogfood as a clean empty scope (#282).
+
+### Fixed
+- `--since` no longer skips untracked files silently (#295): a never-committed
+  source file is invisible to `git diff`, so it used to contribute no changed
+  lines and every subject in it was skipped without a word - even the
+  changed-files-without-subjects warning could not name it. Untracked files
+  now union into the changed-lines map as fully changed, so their subjects
+  are selected, and one without subjects triggers the warning like any other
+  changed file. Ignored (`.gitignore`), binary and empty files stay out, and
+  only files inside the configured `include` set are considered, so a large
+  unrelated untracked file is never opened. Each unioned file is opened
+  `O_NOFOLLOW`/`O_NONBLOCK` and validated on that descriptor (a swapped-in
+  symlink or FIFO is rejected and reported as unreadable), the binary sniff reads only the first
+  few kilobytes, and the map holds a range of lines instead of one entry per
+  line, keeping memory bounded. On platforms without those open flags the
+  union is skipped with a warning.
+- `--since` no longer drops an include-eligible untracked file quietly when
+  it cannot be read (vanished, unreadable, or replaced by a link or special
+  file between the listing and the open): the run names it in a warning and
+  records it as a discovery failure, so an agent run over mixed tested and
+  omitted source reports `partial`/`incomplete` with the `unreadable` reason
+  instead of a clean result. Files that do not parse (`invalid_file_syntax`)
+  surface the same way instead of only warning (#297).
+- Include membership for untracked files is lexical first: an included path
+  swapped to a symlink is attempted, rejected by the safe open and reported,
+  instead of being filtered out silently on its resolved target. A symlinked
+  repository root stays in scope through the canonical fallback (#297).
+- Test inference under `--since` no longer enumerates a whole-file change
+  range per method when deciding whether a changed file needs a spec; it
+  answers ranges by interval, like the run's own subject selection (#297).
+- The excluded-changes warning now also names untracked files in excluded
+  territory: they never appear in the diff, so an exclude that removes one
+  used to hide the removal entirely (#297).
+- Distance and overlap checks over the whole-file ranges the untracked
+  union produces are now geometric, so subject selection, test guessing and
+  survivor ranking answer in O(1) per subject instead of scanning one entry
+  per line of a large new file (#297).
+- A baseline check or coverage map build that hit only its
+  budget-shortened timeout failed the run as a suite error (exit 3) with
+  no partial report; it now reports budget exhaustion with every selected
+  subject unreached and exits 4 in ordinary CLI mode unless
+  `--max-duration-ok` is set; agent exits remain verdict-driven (#296).
+- The unscoped-suite warning pluralizes zero baseline checks correctly
+  ("0 baseline checks", not "0 baseline check") (#296).
+- Review hardening for the 0.5.0 loop flags (#245): the `--max-duration`
+  clock starts when the run starts (boot, baselines and the coverage map
+  all spend the budget) and is checked before dispatching every subject,
+  between mutants inside a subject, and between baseline passes, in
+  serial and `--jobs` runs alike; a parallel budget halt stops handing
+  out new work and lets in-flight subjects finish and report, so their
+  completed outcomes are kept instead of abandoned as nil holes
+  mislabeled "never ran". `--max-duration-ok`
+  no longer exits 0 when the budget stopped the run before any mutant
+  was tested - there is no score to judge, so the run is incomplete
+  (exit 3), in agent mode too. Agent output records `failFast` only for
+  a real `--fail-fast` stop, `--baseline-passes 1` no longer certifies
+  the suite stable in the baseline evidence, and the rerun/replay
+  commands keep `--max-duration`, `--max-duration-ok` and
+  `--baseline-passes`. `saboteur audit` bundles the `--operators all
+  --baseline-passes 2` its help advertises, an explicit `--test`
+  replaces its injected `--all`, and `--no-max-duration-ok` overrides
+  the pr preset's budget pass.
+- macOS CI flakes: the supervisor `on_tick` cleanup spec waits for a fully
+  written pid before triggering instead of racing the file's first bytes
+  (#277), fixture repositories disable Git background auto-maintenance so
+  teardown no longer races its lock deletion (#271), and a baseline child
+  that exits without a result now has its exit status or signal in the
+  error and its dying exception on stderr, so the next flake names its
+  cause (#255). The same error carries a bounded tail of the child's
+  redirected log, the dying-child warning names its stage ("baseline
+  child died during setup/run", "coverage child died during the map
+  build"), and a broken child stderr can no longer swallow the exit
+  (#289).
+- `Saboteur::Engine#call` restores the host process's Coverage state when
+  it returns or raises: an embedded engine (a spec, the dogfood's own
+  coverage-map build) no longer leaves Coverage suspended for everything
+  the host runs afterwards, and a resume that fails because the host
+  disturbed Coverage mid-run warns instead of masking the run outcome,
+  with worker cleanup running either way (#254). A scoped run (`--since`) no longer
+  calls an ignore entry stale when its mutation exists but sits on an
+  unchanged line and was scoped out; staleness is judged against each
+  examined subject's full, pre-scope candidate digests (#284). A worker
+  start-failure storm while the pool is already draining no longer raises
+  "workers keep failing to start" over the tool errors that carry the
+  real abort cause (#286).
+- The canary no longer blames a method's last mutant for deleted files:
+  a cheap per-mutant probe (snapshot stats only, about 1.3 ms for this
+  repo's 692 watched files against a full inventory's 5 ms) runs after
+  every mutant, so the abort names the mutant whose run just ended when
+  the loss was detected as the likely cause - with the per-method check
+  as the backstop for losses outside a mutant's window (#235).
+- Parallel runs aborted by a wiped log dir now name the culprit: a
+  worker that only noticed the damage could report "an unknown mutant"
+  before the worker whose mutant did it, and the first report won the
+  abort. Tool-error reports now carry whether they name a mutant, the
+  pool stops handing out new work but lets in-flight workers finish
+  and report, and the abort ranks the reports by attribution quality:
+  a report that names its mutant first, then a canary trip, then the
+  earliest bare report. A canary trip from the end-of-subject check
+  folds into the same choice instead of overriding collected reports,
+  and the in-flight drain is bounded (a 30-second default drain
+  window, followed by worker cleanup), so a wedged worker cannot
+  stall the abort (#235).
+- `saboteur doctor` prints the coverage line once instead of twice (it is
+  still verified before and after target boot), and reports the sandbox
+  status as a kernel capability probe, not verified isolation: the
+  Landlock ABI when the kernel can deliver it, the gap when it cannot -
+  told apart as a `c.sandbox = "strict"` refusal versus the permitted
+  degradation of a non-strict run - or off when `c.sandbox = false`. A
+  probe the kernel refuses (EPERM under a seccomp policy) is an
+  actionable tool error with exit 3, not an internal-error crash report,
+  and doctor writes nothing (#236). The sandbox policy
+  no longer creates the project's `tmp/` directory to grant it: the
+  grant applies when `tmp/` already exists and only `c.sandbox_writable`
+  entries are created (#236).
+- The mutant sandbox no longer grants the cache dir: a mutant child
+  cannot delete the baseline, the reports or other children's logs,
+  and its own log is opened before the fork and inherited as a
+  descriptor, with stdio kept explicitly through the sandbox sweep.
+  A child log that cannot be opened, or that vanishes mid-run with
+  the sandbox off or degraded, aborts the run with exit 3 - the
+  canary's hazard report names the culprit when tracked files
+  vanished too, otherwise the tool error does - never a mutant
+  verdict such as "killing test also fails with the method
+  unchanged"; in parallel runs the error aborts the whole run
+  instead of degrading into per-mutant errors. The grants are
+  additive, so a cache dir placed inside
+  `tmp/`, the system temp dir or a `c.sandbox_writable` entry stays
+  writable. Note the sandbox prevents the damage, not the verdict:
+  with `--allow-hazardous`, a destructive mutant whose test swallows
+  the denial (EACCES) can still read as a survivor; by default the
+  hazard guard suppresses such mutants and reports the hazard (#234).
+- An extensionless `hooks` path now boots: it resolves its `.rb` at load
+  time, the way `require` does and matching what doctor accepts (#267).
+- Fixed two encoding failures in mutant children running under a non-UTF-8
+  locale such as `LANG=C` (seen on CI, where the Landlock sandbox engages):
+  the inherited-fd sweep compared a locale-tagged descriptor target against
+  grant patterns with string semantics, raising
+  `Encoding::CompatibilityError` before any test ran when an open file's
+  path carried non-ASCII bytes; and an exception message embedding such a
+  path made failure reporting raise the same way, turning a killed mutant
+  into an errored run. The sweep now compares raw bytes, and failure
+  messages are scrubbed before their first line is extracted; if the
+  scrub itself raises - it is mutable code, and the mutant under test
+  may have broken it - an escaped fallback keeps the failure a clean
+  kill instead of an errored run (#272).
+- Fixed `--guess-tests` falling back to the whole suite when a changed
+  file has no mutation subjects on its changed lines, such as a deletion
+  whose remaining context is blank or comment lines or a comment-only
+  change: such files no longer require a guessed spec. A changed file
+  that does need a spec but has none now stops before boot with exit 2,
+  matching the default-inference policy above (#275).
+  Guesses come only from files with a changed mutation subject, and when
+  no changed file has one the run reports that there is nothing to mutate
+  (fixes #268).
+- Fixed failures under non-UTF-8 locales such as `LANG=C` when project
+  paths contain non-ASCII characters. Git output, coverage maps, result
+  caches, baseline logs and worker messages are now decoded as UTF-8
+  regardless of the process locale. This prevents
+  `Encoding::CompatibilityError` in the canary, baseline failure excerpts
+  and worker results, and fixes `--since` silently skipping changed files
+  with non-ASCII names. Paths are identity, not display text: the canary
+  watch, `--since` diff parsing, source discovery and the run fingerprint
+  retag locale-tagged bytes losslessly, so a tracked file whose name is
+  not valid UTF-8 stays watched (its deletion still trips the canary) and
+  keeps its exact bytes through selection. The canary and diff parser
+  preserve path bytes rather than replacing invalid UTF-8 bytes; the
+  cache, coverage-map and worker-message boundaries reject invalid UTF-8
+  instead of rewriting it.
+- An unexpected internal error now prints `saboteur: internal error (...)
+  - please report it` and exits 3, instead of escaping as a bare stack
+  trace with an accidental exit 1. The crash message is normalized to one
+  bounded line first, and agent mode reports the same failure as agent
+  JSON. The full backtrace is written to
+  `<cache_dir>/internal-error.log` (default `.saboteur/`), which the
+  message names, and printed to stderr with `--verbose`; an unset
+  `cache_dir` or a failed write leaves the one-liner standing alone.
+  The executable loads the crash reporter before the CLI,
+  so a failure while it is loading, being built, or running gets
+  the full report. Coverage setup and the reporter's own load happen before
+  it, so those failures get only the bare normalized one-liner plus
+  exit 3. When agent mode was requested but the agent reporter never
+  loaded, a minimal standalone failure JSON in the agent reporter's
+  schema (status `error`, verdict `tool_failure`, exit code 3)
+  answers instead (#233).
+- The install spec reads the README as UTF-8, so the suite also passes
+  under `LANG=C`.
+- Tool and test reads of files that can carry non-ASCII text are decoded
+  as UTF-8 regardless of the process locale: the baseline manifest and
+  preview receipt, `.rspec` option files, the custom-map duplicate scan,
+  the installer's Gemfile and `.gitignore` probes (`.gitignore` is
+  rewritten losslessly), and the JSON reads in the test helpers.
+- CI now runs the suite on Ubuntu Ruby 3.4 with `LANG=C` and `LC_ALL=C`
+  to catch locale regressions exercised by the tests. The panel gate's
+  required-check policy includes this job (#281).
+
 ## [0.6.0] - 2026-10-14
 
 ### Fixed