saboteur 0.10.0 → 0.11.1

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: e7d127c4a50466eb11e40c8f1f8ac92b013621d76ddfd3f122823ec85f0ff726
-  data.tar.gz: 8768ae6add39713a3ffb0222609c096ddb8e71e7edcb174f83e0ce3719c6296f
+  metadata.gz: 74914fa96e0e644c7615013b6ad8a014c3c74b01e9b154931f2087fa0685dd31
+  data.tar.gz: 3265c82c7383a4ec2ce6355c009fc5816822e96334224e05a3cf6dac05404504
 SHA512:
-  metadata.gz: eb07cd559cf3b12dadd0e7b1a148f04e2e6aada3e961f9dfcd249f05da69cd7f59bc8a43335450df86ae4b6406be013ca7e737357ac3e07023169f6f9d434e52
-  data.tar.gz: 1cb27f6614903b63907f1ddf54872bdcfa1e1ab2c288bd40f4ed3ce9e247bed320105afb33fcb03e845a03887f7343250c36d2b82f35a1c7da61227dd83f0244
+  metadata.gz: de0a4aa5db3d0403a46d9db2390fca263b9d7b433a9bbdb5d28049402c07d38ac75f2f68221bc4576ca3bc1a27c90c42aa7c62ba9f19d67eb6183507cfeef43b
+  data.tar.gz: 9d0286667bae0a37141efacd0547cb62d7185567ca8728f39be84902905f149f7d08f8eff2df07b2601083a8118a539d9d34fed06cfd47ec3840490c8ceb65a6

data/CHANGELOG.md

@@ -5,6 +5,101 @@ All notable changes are documented here. The format follows
 [Semantic Versioning](https://semver.org/).
 
 ## [Unreleased]
+## [0.11.1] - 2026-10-22
+
+### Added
+
+- Diff runs echo the real `--since` ref in hints and name the diff
+  scope in the reports (#1235, #1218, #1219), and `survivors` reads the
+  last report by default and refuses ambiguous digest prefixes (#1235,
+  #1189, #1188, #1192).
+- `baseline propose` tolerates untested subjects: proposability is the
+  gate at propose time, while strict evidence completeness stays where
+  it is required (#1168, #1141). The baseline command names what it is
+  doing, the file it writes and the follow-up command at every step,
+  and blockers resolve to subject, operator and reason plus a
+  remediation line (#1217, #1195, #1196).
+- The GitHub Action gains an opt-in `max-duration` input passed to
+  `--max-duration`, summary lines for exits 2/3/4/5/70, an annotation
+  on exit 4, and failure messages that each name the command or
+  setting that fixes them (#1220).
+- Incremental runs report reuse feedback (#1221), and the HTML report
+  gains a copy button on survivor blocks, print rules and aria fixes
+  (#1231).
+
+### Changed
+
+- Warning notices, the baseline tail and the CLI rescue paths render
+  through the colour palette (monochrome output is byte-identical), and
+  first runs are quieter: the Ruby 3.4 fiddle stdlib notice is dropped
+  during the guarded require, Rails mode no longer warns about the
+  harmless AR Pool Reaper thread, and the no-spec stop ends with a
+  clearer custom-map pointer (#1236, #1226, #1229, #1230).
+- Survivors rank by per-statement density first (the statement with
+  the most surviving mutants leads), then by thin coverage; ranking is
+  review priority only, with no score or exit-code effect (#1163).
+- `survivors --show` prints the complete detail - diff, replay command
+  with the run's seed, and a paste-correct ignore template (#1194,
+  #1217) - and unknown-command and doctor include errors are clearer
+  (#1194).
+- First-run DX: the footer ignore template pastes into `.saboteur.rb`
+  unchanged, install prints the next step, and lifecycle warnings are
+  quieter (#1217, #1221). The survivor-confirmation skip notice names
+  the real reason (#1231, #1184, #1177).
+- The HTML report is printable and collapsible, with clearer baseline
+  errors (#1179).
+
+### Fixed
+
+- Survivor confirmation no longer crashes the run (exit 70): a
+  confirmation failure degrades to a warning (#1164, #1140).
+- README hero output shows the right subject count (#1162), and the
+  exit-code docs match the gates (#1231, #1222).
+
+
+## [0.11.0] - 2026-10-21
+
+### Added
+
+- The colour contract palette: six tokens with overrides and CI-aware
+  detection (#1112), the `c.palette` setting for text-report colour
+  overrides (#1114), and one palette per stream wired from the CLI
+  through the engine (#1117). The terminal report, score table and safe
+  output (#1115), the survivor list and next steps (#1116), and the
+  crash and failure exit lines (#1118) render with contract tokens.
+- `run --baseline PATH` with `--epsilon POINTS`: an accepted baseline
+  absorbs legacy survivors; only new survivors or a score drop beyond
+  epsilon fail the run (#1120, #1121). The run tail reports matched
+  counts, the gate verdict, and a re-baseline hint (#1122).
+- `run --ignore-empty`: a zero-mutant `--since` diff exits 0 with a
+  note (#1119).
+- Quiet output: the terminal reporter gains a quiet mode - one line per
+  finding plus the score line and verdict (#1126) - wired through the
+  CLI as `--quiet` (#1133). Startup chatter moves behind `--verbose`
+  and runs open with three quiet lines (#1134). Clean runs close with a
+  success screen: win tally, untested note, and reports line (#1137),
+  plus baseline, coverage and dots lines when phase data is present
+  (#1139). Progress streams as width-sized glyph chunks on a redrawing
+  terminal line (#1127).
+- Survivors render as per-statement blocks with fenced source and
+  killer-test details (#1136), built from prose templates and an
+  edit-span helper (#1123). Each survivor shows which covering tests
+  ran against it (#1106), survivor labels mark exception message edits
+  arid (#1095), and an opt-in pass confirms survivors against the
+  coverage map (#1089).
+- The HTML report vendors and pins the mutation-testing-elements 3.9.0
+  viewer (#1124), mounts it in the report (#1125) with an end-to-end
+  smoke test (#1128), and names the changed line each mutant relates to
+  in diff mode (#1110).
+
+### Changed
+
+- Exit-code split: a red baseline exits 5 and an internal crash exits
+  70 (both were exit 3) (#1098), documented in the Action (#1099).
+- The score is honest about untested subjects (#1104).
+- The kill split counts cause-not-recorded kills and exposes them to
+  agents (#1109).
+- The README is rewritten as a top-tier dev-tool README.
 
 ## [0.10.0] - 2026-10-21
 

data/README.md

@@ -2,244 +2,57 @@
   <img src="assets/logo.png" alt="saboteur logo: a ninja beetle with a ruby shell carrying a lit bomb" width="200">
 </p>
 
-# saboteur
+<h1 align="center">saboteur</h1>
 
-**Mutation testing for Ruby and RSpec, built for pull requests.** 🧨
-
-[![CI](https://github.com/vamsipavanmahesh/saboteur/actions/workflows/ci.yml/badge.svg)](https://github.com/vamsipavanmahesh/saboteur/actions/workflows/ci.yml)
-[![Gem Version](https://img.shields.io/gem/v/saboteur)](https://rubygems.org/gems/saboteur)
-[![License: MIT](https://img.shields.io/badge/license-MIT-blue.svg)](LICENSE)
-
-Line coverage tells you which code ran. It does not tell you if a test
-would notice when that code breaks. Mutation testing does.
-
-saboteur makes one small change to your code at a time and runs only the
-tests that touch it, so it is fast. If the tests still pass, you see the
-exact change they missed. Every kill is re-checked against the unchanged
-code, so a broken suite never counts as a kill.
+<p align="center">
+  <strong>Mutation testing for Ruby and RSpec, built for pull requests.</strong><br>
+  Line coverage tells you which code ran. saboteur tells you whether your tests would notice when it breaks.
+</p>
 
-**Docs, benchmarks and guides live on the site:
-[vamsipavanmahesh.github.io/saboteur](https://vamsipavanmahesh.github.io/saboteur/)**
+<p align="center">
+  <a href="https://github.com/vamsipavanmahesh/saboteur/actions/workflows/ci.yml"><img src="https://github.com/vamsipavanmahesh/saboteur/actions/workflows/ci.yml/badge.svg" alt="CI"></a>
+  <a href="https://rubygems.org/gems/saboteur"><img src="https://img.shields.io/gem/v/saboteur" alt="Gem Version"></a>
+  <a href="LICENSE"><img src="https://img.shields.io/badge/license-MIT-blue.svg" alt="License: MIT"></a>
+</p>
 
-<a id="30-second-quick-start"></a>
-## 30-second quick start
+```text
+$ bundle exec saboteur pr
+saboteur 0.11.0: 2 subjects, 10 mutants, wall 0.70 s, seed 1 (repeat with --seed 1)
+Score: 80.0% (8/10 killed, tested subjects)
+saboteur: 8 killed, 2 survived in 1s - score 80.0%
+```
 
-You need Ruby 3.3 or newer, RSpec, and Linux or macOS with `fork`.
+A survivor is a change to your code that your tests waved through.
+saboteur breaks your code one small change at a time, runs only the tests
+that touch it, and shows you the exact change they missed. Every kill is
+re-checked against the unchanged code, so a broken suite never counts as
+a kill. A green run here actually means something.
 
-Prerequisite: the sandbox layers use fiddle, which ships with Ruby
-(a default gem on Ruby 3.x). On Ruby 4.0+, where fiddle is a bundled
-gem, add `gem "fiddle"` to your Gemfile. If saboteur refuses a run
-because fiddle cannot load: reinstall Ruby, run `gem install fiddle`,
-or install libffi headers (libffi-dev on Debian/Ubuntu, libffi-devel
-on Fedora/RHEL) and reinstall fiddle. libffi headers are required only
-when Ruby's shipped fiddle is missing or must be installed standalone.
+## Get started in 3 steps
 
 ```sh
-# Add it to your project's Gemfile:
+# 1. Add it to your project's Gemfile
 bundle add saboteur --group test
 
-# Then:
+# 2. Run it (the first run builds a cached coverage map)
 bundle exec saboteur run
-```
-
-Outside a Bundler project, `gem install saboteur` gives the same
-executable - run it there as plain `saboteur run`, without
-`bundle exec`.
-
-The first run builds a coverage map of which tests touch which code. The
-map is cached and reused while your code and tests stay unchanged, so
-later runs start fast.
-
-If the first run stops with exit 2 and `no spec found for ...`, saboteur
-could not map a source file to its spec: it matches by path first, then
-by parent directory and basename when the layout strips directories,
-and it refuses to guess wrong. Point it at the specs yourself - set
-`test_selection = :all` in `config/saboteur.rb` to run the whole suite
-against each mutant, pass `--test spec/...` to name the specs, or
-declare a custom map
-([monorepos and the custom map](docs/guides/monorepo-and-custom-map.md)).
-
-On a branch, `bundle exec saboteur pr` mutates only the methods you
-changed relative to `main`, with one baseline pass and a ten-minute
-budget. If the budget stops the run, it exits 4 and prints `partial: X
-of Y mutants not run` (or says the count is unknown when planning did
-not finish). Pass `--max-duration-ok` to accept a partial verdict; the
-warning stays, and other failure gates still apply. Pass `--since` for a
-different base, for example `bundle exec saboteur pr --since master`.
-
-Setup, configuration and CI:
-[the quickstart](https://vamsipavanmahesh.github.io/saboteur/quickstart.html).
-
-## Why RSpec-only
-
-A mutation verdict is only worth anything if the test selection behind it
-is exact. RSpec's example structure lets saboteur map each method to the
-examples that cover it, rerun just those against each change, and re-check
-every kill against the code before the change. One framework, done
-properly - that is the bet. Minitest is not supported today;
-[why, and what it would take](docs/minitest.md).
-
-## What a run tells you
-
-```text
-saboteur 0.7.0: 1 subjects, 10 mutants, wall 0.70 s, seed 1
-killed 8  survived 2  timeout 0  resource limit 0  error 0  score 80.0 %
-```
-
-Each survivor comes with its diff, the tests that ran, and a triage label
-(`likely_equivalent`, `arid`, `missing_test`) to sort what to read first.
-The labels are heuristics for reading order, never a diagnosis, and they
-never change the score. The terminal prints one finding per statement with
-the label mix and mutant count; `--all-mutants` lists every mutant.
-
-A run on a tree with uncommitted changes opens with a warning that splits
-the count into changes inside the mutation scope, outside it, and
-saboteur's own setup files. Mutants run your code, so run on a clean,
-committed tree or a CI checkout.
-
-Code that only runs while the app loads cannot be judged by replacing the
-method later, so it reports as `ran at load`, outside the score. The same
-holds for def-time macros - a `deprecate`-style helper, a `before_action`
-block, any DSL whose only caller is a class body: the closure captured at
-load keeps the pristine definition, so the boot's def-call counters mark
-the macro itself `ran at load` instead of letting its mutants survive.
-
-## What you get
-
-- **Fast pull-request runs.** `--since main` scopes the run to the methods
-  you changed, refuses (exit 2) when the resolved range has no changes -
-  `saboteur pr $GITHUB_SHA` on a checkout at the merge commit diffs the
-  merge against itself - and fails loudly (exit 3) when the changed code
-  cannot be verified - for example when every changed method is skipped
-  because a refinement is active where it sits. A changed method skipped
-  for no coverage names the specs the run loaded; when its coverage
-  lives in a sibling spec file, pass `--test spec/models` (the directory
-  holding it) to load it. The GitHub Action runs on every pull request
-  and caches the coverage map between runs. `--shard K/N` splits one
-  planned inventory deterministically across CI jobs; each shard reports
-  partial evidence.
-- **Honest verdicts.** Every kill is verified against the unchanged
-  method; a kill the suite cannot confirm stays out of the score.
-  Suspect a flaky test? `--flaky-check survivors` confirms each new
-  survivor with one reordered rerun (`decisive` also confirms fresh
-  kills); a verdict the rerun disagrees with becomes `unverified`,
-  named with its tests, and stays out of the score. A spec
-  file whose examples depend on earlier ones is retried in defined order
-  before a kill is rejected, so its mutants are still scored.
-- **Reports for people and tools.** Terminal, HTML, a Stryker-compatible
-  JSON report, and a stable agent JSON API with a deprecation policy.
-  The GitHub Action posts surviving mutants as review comments on the
-  diff lines of the pull request.
-- **Resumable runs.** `--resume DIR` keeps verdicts across runs and
-  `--incremental` checkpoints them while the run is going, so a stopped
-  run picks up where it left off.
-- **Faster coverage builds.** `c.coverage_parts` builds the coverage map
-  in parallel parts, `partial_coverage = "use"` rebuilds only the stale
-  spec files and merges them into the saved map, and
-  `c.coverage_recorder = "trace"` builds the map with a TracePoint
-  recorder.
-- **A plan before the run.** `saboteur run --dry-run --format json` prints
-  the versioned [mutant inventory](docs/dry-run-json.md) - every planned
-  mutant with its stable digest and edits - without booting the suite.
-- **A review workflow.** An ignore list with reasons and triage states,
-  an opt-in survivor baseline, score thresholds and score history.
-  `saboteur survivors .saboteur/report.json` browses survivors in the
-  terminal straight from the JSON report - list, filter
-  (`--file`/`--operator`/`--triage`/`--ranked`), and `--show DIGEST`
-  for the diff, the exact `run --id` replay command and an ignore-file
-  entry template. It never boots your project or edits the ignore file.
-- **A local README badge.** `saboteur badge` writes Shields endpoint JSON
-  from the score history and `saboteur trend` prints the score delta -
-  no hosted service:
-
-  ```md
-  ![mutation score](https://img.shields.io/endpoint?url=<raw-url-of-the-badge-json>)
-  ```
-
-  Incomplete runs are refused as badge evidence unless `--mark-partial`
-  labels them, and the trend never compares runs with different
-  selection, inventory or operator policy.
-- **Safety controls.** A hazard guard, canary values, and time, memory
-  and process-count limits on every mutant in the default in-process
-  mode; a Landlock sandbox on Linux
-  6.2 and newer.
-- **Extension points.** Add your own mutation operators through the
-  public interface.
 
-Per-path score gates hold one subtree to a stricter floor: `c.score_gate
-"app/payments/**/*", min_score: 95` in config/saboteur.rb (repeatable). Each
-gate scores only the mutants under its pattern and fails the run under its
-floor; a gate with nothing executed under it passes vacuously and says so.
-See
-[choosing and raising a threshold](docs/best-practices.md#choosing-and-raising-a-threshold)
-for how to pick a number.
-
-## Honest limits
-
-- saboteur is RSpec-only and needs `fork`: no Windows, no Minitest.
-- A survivor is review evidence, not proof of a bug in your tests. It can
-  be a missing test, an equivalent change, a flaky test, or a limit of
-  the tool. Read it before you write a test for it.
-- `--since` and `--incremental` are iteration aids. Run the whole project
-  on main and before a release.
-- The hazard guard is syntactic, sandboxing varies by platform (Landlock
-  on Linux 6.2+; on macOS mutants run unsandboxed unless `c.sandbox = "seatbelt"`
-  opts into a probe-gated Seatbelt profile; see
-  [the macOS Seatbelt sandbox](docs/guides/macos-seatbelt-sandbox.md)), and network access is
-  never restricted. Stub real APIs and use a container when that matters.
-
-### Custom spec mapping
-
-When the naming guess does not fit the project's layout, teach the mapping
-once in `config/saboteur_custom_map.rb` instead of passing `--test` on every
-run:
-
-```ruby
-SABOTEUR_CUSTOM_MAP = {
-  "lib/aasm/core/state.rb" => ["spec/unit/state_spec.rb"],
-  "app/jobs/**/*.rb"       => ["spec/jobs/**/*_spec.rb"],
-  %r{\Aapp/jobs/(.*)\.rb\z} => ['spec/jobs/\1_spec.rb'],
-  "app/generated/**/*.rb"  => [], # intentionally untested
-}.freeze
+# 3. Put it on every pull request - mutates only what you changed vs main
+bundle exec saboteur pr
 ```
 
-A matching entry wins over the naming convention: the file stops being
-unmatched and its mapped specs run as the inferred ones. Keys are exact
-paths, globs or Regexps; an empty array marks the file intentionally
-untested, so it needs no inferred spec. The file loads before the app
-boots, so keep it plain Ruby. `saboteur doctor` validates it.
-
-### Rejected kills
-
-The terminal lists them in a `Rejected kills` section and annotates the error
-count (`error 3 (2 rejected kills)`); JSON flags each rejected mutant with
-`rejectedKill: true` and keeps the `rejectedKills` count. These are fresh
-per-mutant kill claims whose tests also fail with the method unchanged. They
-stay errors outside the score. Timeout checks, other errors and reused cached
-kills are not included. See [rejected kills](docs/rejected-kills.md) for
-details.
+You need Ruby 3.3 or newer, RSpec, and Linux or macOS with `fork`. The
+sandbox layers use fiddle, which ships with Ruby; on Ruby 4.0+, where
+fiddle is a bundled gem, add `gem "fiddle"` to your Gemfile. Outside
+a Bundler project, `gem install saboteur` gives the same executable.
+Rails app? `bundle exec rails generate saboteur:install` writes the config
+for you. Full setup:
+[the quickstart](https://vamsipavanmahesh.github.io/saboteur/quickstart.html).
 
 ### Rails apps
 
-Quickstart for a Rails + RSpec app. The defaults target a plain gem, so a
-Rails app needs a few settings; the generator writes them for you:
-
-```sh
-bundle add saboteur --group test
-bundle exec rails generate saboteur:install
-```
-
-This writes `config/saboteur.rb`, a `bin/saboteur` binstub (it sets
-`RAILS_ENV=test` for you) and a `.gitignore` entry for `.saboteur/`. It
-never overwrites an existing file unless you pass `--force`.
-
-Until a config names your sources, a run whose include selects no Ruby
-file refuses with exit 2 and points at the install generator above -
-a first CI run on an unconfigured tree fails loud instead of passing
-with 0 subjects.
-
-Prefer to do it by hand, or on a version without the generator? Put this
-in `config/saboteur.rb`:
+`bundle exec rails generate saboteur:install` writes the config for you.
+By hand, put this in `config/saboteur.rb`:
 
 ```ruby
 Saboteur.configure do |c|
@@ -250,41 +63,89 @@ Saboteur.configure do |c|
 end
 ```
 
-An engine gem whose specs live under `spec/dummy` needs an exclude for them; see
+The defaults target a plain gem, so a first run on an unconfigured Rails
+tree refuses with exit 2 and points at the generator. Engine gem with
+`spec/dummy`? See
 [Rails engines and dummy apps](docs/guides/rails-engines-and-dummy-apps.md).
 
-## SimpleCov
-
-saboteur needs Ruby's `Coverage` for itself, so leave SimpleCov off for
-saboteur runs - for example, start it only when `ENV["COVERAGE"]` is set.
-The [recipes](docs/recipes/README.md) cover this and VCR, WebMock and
-Sidekiq.
-
-<a id="dated-validation-runs"></a>
-<a id="where-it-has-run"></a>
-## Benchmarks and validation
-
-Speed and memory numbers live with the harness and the raw data that
-reproduce them, framed as dated measurements rather than standing claims:
-the [benchmarks page](https://vamsipavanmahesh.github.io/saboteur/benchmark/)
-on the site and [docs/benchmark](docs/benchmark/) in this repo.
+## Highlights
+
+- ⚡ **Fast by scope.** `--since main` mutates only the methods you
+  changed, `saboteur pr` adds one baseline pass and a ten-minute budget,
+  and `--shard K/N` splits one planned inventory deterministically across
+  CI jobs.
+- 🎯 **A score that tells the truth.** The score covers tested subjects
+  only. A method no test covers is named in a `Not scored` list instead of
+  padding the number, kills split by cause (`kills: 7 by a failed
+  assertion, 1 by a crash`), and an error-dominated run prints
+  `Score: n/a` instead of a number you would misread.
+- 🔍 **Survivors you can act on.** Survivors group per statement with a
+  triage label (`likely_equivalent`, `arid`, `missing_test`) and their
+  covering tests - split into the tests that ran against the mutant and
+  the ones that never did. `saboteur survivors` browses them read-only:
+  one digest shows the full diff and the exact replay command.
+- 🧪 **Flakiness contained.** `--flaky-check survivors` re-runs each new
+  survivor in a reordered confirmation pass; a verdict the rerun disagrees
+  with becomes `unverified` and stays out of the score. With
+  `partial_coverage = "use"`, open survivors get one budgeted confirmation
+  run against the covering specs that never ran them.
+- 📊 **Reports for people and tools.** Terminal, HTML, a Stryker-compatible
+  JSON report, and a stable agent JSON API with a deprecation policy. The
+  GitHub Action posts survivors as review comments on your PR diff lines.
+  `saboteur badge` and `saboteur trend` track the score history with no
+  hosted service.
+- 🛡️ **Safe to point at real code.** A hazard guard, canary values, and
+  time, memory and process-count limits on every mutant; a Landlock
+  sandbox on Linux 6.2+ and an opt-in Seatbelt profile on macOS.
+- 🧭 **A plan before the run.** `--dry-run --format json` prints the
+  versioned mutant inventory - every planned mutant with its stable digest
+  and edits - without booting your suite.
+- 🔁 **Built for big suites.** `--resume` and `--incremental` pick up
+  where a stopped run left off, parallel coverage builds
+  (`c.coverage_parts`) and a TracePoint recorder keep the map cheap, and
+  per-path score gates hold one subtree to a stricter floor.
+
+## Exit codes that mean something
+
+| Exit | Meaning |
+| --- | --- |
+| 0 | Clean: no unignored survivors, every gate passed |
+| 1 | Survivors to review, or a gate failed |
+| 2 | Usage or configuration error |
+| 3 | Incomplete: tested nothing, mostly errored, or the change is unverified |
+| 4 | The `--max-duration` budget stopped the run early |
+| 5 | Your unchanged suite already fails its baseline - fix your tests first |
+| 70 | An internal crash: a saboteur bug, never your code; the error names a log with the full backtrace |
+
+## Know the limits
+
+- RSpec-only, and it needs `fork`: no Windows, and no Minitest
+  ([why, and what it would take](docs/minitest.md)).
+- saboteur needs Ruby's `Coverage` for itself, so leave SimpleCov off for
+  saboteur runs - start it only when `ENV["COVERAGE"]` is set
+  ([recipes](docs/recipes/README.md)).
+- A survivor is review evidence, not proof of a bug in your tests. It can
+  be a missing test, an equivalent change, a flaky test, or a limit of
+  the tool. Read it before you write a test for it.
+- `--since` and `--incremental` are iteration aids. Run the whole project
+  on main and before a release.
+- Sandboxing varies by platform and network access is never restricted.
+  Stub real APIs and use a container when that matters.
 
-## Project
+## Docs
 
+- [Quickstart and guides](https://vamsipavanmahesh.github.io/saboteur/quickstart.html) - setup, CI, recipes
+- [All documentation](docs/README.md) - configuration, operators, recipes
 - [Changelog](CHANGELOG.md)
-- [All documentation](docs/README.md) - configuration, operators, CI, recipes
+- [Benchmarks](https://vamsipavanmahesh.github.io/saboteur/benchmark/) - dated measurements with the raw data
 - [Contributing](CONTRIBUTING.md) - every commit needs a DCO `Signed-off-by` line
-- [Design notes](docs/design.md)
 
 ## Disclaimer
 
-saboteur is provided "as is", without warranty of any kind, express or
-implied. In no event shall the authors or copyright holders be liable
-for any claim, damages or other liability arising from its use -
-including damage caused by mutating your code or by running unusual
-DSLs and metaprogramming-heavy code. Mutation testing runs your test
-suite against deliberately broken code; run it on a clean, committed
-tree or a CI checkout. The full text is in [LICENSE](LICENSE).
+saboteur is provided "as is", without warranty of any kind. Mutation
+testing runs your test suite against deliberately broken code; run it on a
+clean, committed tree or a CI checkout. The full text is in
+[LICENSE](LICENSE).
 
 ## License
 

data/exe/saboteur

@@ -8,7 +8,7 @@
 # is being required, gets the full report (the bounded one-liner, the log
 # file, the --verbose backtrace and agent JSON when the CLI loaded far
 # enough). If even the reporter cannot load, a bare normalized one-liner
-# (one line, invalid bytes scrubbed, bounded length) still exits 3.
+# (one line, invalid bytes scrubbed, bounded length) still exits 70.
 begin
   require "coverage"
   Coverage.setup(lines: true, branches: true) if Coverage.state == :idle
@@ -19,7 +19,7 @@ rescue LoadError, StandardError => e
   message = "#{message[0...160]}..." if message.length > 160
   warn "saboteur: internal error (#{e.class}: #{message}) - please report it: " \
        "https://github.com/vamsipavanmahesh/saboteur/issues"
-  exit 3
+  exit 70
 end
 
 # For `run` coverage counts from here: what executes while the app and

data/lib/saboteur/baseline/command.rb

@@ -20,6 +20,7 @@ module Saboteur
           opts.on("--config PATH", "Config file (default config/saboteur.rb)") { |value| set(:config, value) }
           opts.on("--subject EXPR", "Whole subject filter; repeatable") { |value| add_subject(value) }
           opts.on("--seed N", Integer, "Repeatable run seed") { |value| set(:seed, value) }
+          opts.on("--format FORMAT", "propose: terminal (default) or json") { |value| @format = value }
         end
       end
 
@@ -34,7 +35,7 @@ module Saboteur
           workflow = Baseline::Workflow.new(config: config, err: @err)
           baseline_action(workflow, action)
         end
-        @out.puts(JSON.generate(payload))
+        action == "propose" && @format != "json" ? print_proposal(payload) : @out.puts(JSON.generate(payload))
         payload.fetch("exit_code", CLI::EXIT_OK)
       rescue ConfigError, SystemCallError => e
         @err.puts("saboteur: #{e.message}")
@@ -44,13 +45,17 @@ module Saboteur
         CLI::EXIT_TOOL
       end
 
+      def baseline_path_required(action)
+        "--baseline PATH is required (for example: saboteur baseline #{action} --baseline baseline.json)"
+      end
+
       def parse_baseline(action)
         raise CLI::UsageError, "baseline expects check, propose or accept" unless %w[check propose
                                                                                      accept].include?(action)
 
         baseline_parser.parse!(@argv)
         raise CLI::UsageError, "unexpected arguments: #{@argv.join(' ')}" if @argv.any?
-        raise CLI::UsageError, "--baseline PATH is required" unless @baseline_path
+        raise CLI::UsageError, baseline_path_required(action) unless @baseline_path
         raise CLI::UsageError, "--proposal PATH is required" if action != "check" && !@proposal_path
         if action == "check" && (@proposal_path || @previous)
           raise CLI::UsageError,
@@ -58,6 +63,15 @@ module Saboteur
         end
         validate_epsilon(action)
         raise CLI::UsageError, "--previous needs a SHA-256" if @previous && !@previous.match?(/\A[0-9a-f]{64}\z/)
+
+        validate_format(action)
+      end
+
+      def validate_format(action)
+        unless @format.nil? || %w[terminal json].include?(@format)
+          raise CLI::UsageError, "--format must be terminal or json"
+        end
+        raise CLI::UsageError, "--format applies to propose only" if @format && action != "propose"
       end
 
       def validate_epsilon(action)
@@ -67,6 +81,36 @@ module Saboteur
         raise CLI::UsageError, "--epsilon must be between 0 and 100" unless (0.0..100.0).cover?(@epsilon)
       end
 
+      def print_proposal(payload)
+        @out.puts("baseline proposal written: #{payload.fetch('proposal')}")
+        survivors = payload.fetch("survivors")
+        @out.puts("  matched mutants: #{payload.fetch('candidates')} " \
+                  "(#{plural(survivors, 'survivor')} to reason about)")
+        gaps = payload.fetch("gaps", [])
+        unless gaps.empty?
+          names = gaps.map { |gap| gap.sub(/\A(subject_skipped|omitted):/, "") }
+          @out.puts("  proposed with #{plural(names.size, 'untested subject')}: #{names.join(', ')} - " \
+                    "the baseline cannot speak for them")
+        end
+        @out.puts("  next step: give each accepted survivor a reason in the proposal, then run: " \
+                  "saboteur baseline accept#{scope_flags} " \
+                  "--baseline #{payload.fetch('destination')} --proposal #{payload.fetch('proposal')}")
+      end
+
+      def plural(count, noun)
+        count == 1 ? "#{count} #{noun}" : "#{count} #{noun}s"
+      end
+
+      # accept re-verifies the recorded run settings, so the printed command
+      # repeats the scope flags this propose ran with.
+      def scope_flags
+        flags = +""
+        Array(options[:subjects]).each { |expr| flags << " --subject #{expr}" }
+        flags << " --seed #{options[:seed]}" if options[:seed]
+        flags << " --config #{options[:config]}" if options[:config]
+        flags
+      end
+
       def baseline_action(workflow, action)
         case action
         when "check" then workflow.check(path: @baseline_path, epsilon: @epsilon)

data/lib/saboteur/baseline/comparison.rb

@@ -62,7 +62,12 @@ module Saboteur
       end
 
       def outcome(current)
-        return "incomplete" unless current.fetch("complete") && issues.empty? && !current.fetch("candidates").empty?
+        # The proposability tier is the comparability gate: a run over soft
+        # gaps still speaks for the subjects it executed. Legacy documents
+        # without the split fall back to the strict flag and full issue list.
+        proposable = current.fetch("proposable") { current.fetch("complete") }
+        blockers = current.fetch("blockers") { current.fetch("issues") }
+        return "incomplete" unless proposable && blockers.empty? && !current.fetch("candidates").empty?
         return "incomplete" unless categories.fetch("errors").empty?
         return "review_required" if %w[new changed unmatched].any? { |name| !categories.fetch(name).empty? }
         return "review_required" if score&.fetch("exceeded")