mutineer 0.11.3 → 1.0.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: b671af802d8c032a7b9fa3e48eac7d63478930f6077ce58793007c4e8aa98de6
-  data.tar.gz: ef1ef88c9b6af5a0c634d0e9f7bf68ce9644e8c5d5a48002a5b3de1c1a4b7ffe
+  metadata.gz: 9104fa94a852cbd1257f9dce0af285865d98da1100849f5d1b8c5af12d6513fa
+  data.tar.gz: 4ca6421fdd7ef6ef862813cbd74481ec6babf176991c8d34301368861ea19272
 SHA512:
-  metadata.gz: 69ab5b4e78163653f838afba995273f156ad51d15b4613d8bd40f1d4f0571423018e5b1a8ffd59160405e81e9d2aca26fce08deba802ee896be3fd37254ff887
-  data.tar.gz: 98e011da927ed5108fec748628a9bb0a69d74e03cdf3b1f2d395b5082e38f3478ad45bbb5caa58fa7099ecb7d787086cdaebef20828fe5b670591c4fa8e0b5bf
+  metadata.gz: 66a7b0eae5bb41331d267b4c26a0b72e9023177d89d9d21cc679e712978836c4884b8b0347fd8263f55041897c2581e7117d5ae522965efbdb9af0daf5b47fee
+  data.tar.gz: cf1ed699c65b733b717ce5ed6ee95f9f50387fb2963b0ecd5516fa43d6a18bd21e0c354f2c85bde2df56cec305446cd988697f8eb5ae5b080bdfd9d2edc0fa06

data/CHANGELOG.md

@@ -6,6 +6,90 @@ All notable changes to this project are documented here. The format is based on
 
 ## [Unreleased]
 
+## [1.0.0] - 2026-09-08
+
+The GitHub Action's PR default changes in this release, which is why it is a
+new major: workflows pinned to `davidteren/mutineer@v0` keep the old full-scan
+behavior; upgrading to `@v1` opts into diff-scoped PR runs (details under
+Changed).
+
+### Added
+- **The Action reports where CI readers look**: with the default JSON format it
+  writes a score/pass-fail table (plus the baseline delta, when `baseline` is
+  set) to the job step summary, and emits one `file=…,line=…` annotation per
+  surviving mutant (up to 50; the summary lists the first 20, the full set
+  stays in the JSON report) so results land on the PR diff instead of only in a
+  collapsed log group — `error` level when the gate failed, `warning` when it
+  passed. When `output` is unset the JSON report is routed to a temp file and
+  still printed to the log, and the `report` output now exposes the report
+  path in both cases so a later step can consume the JSON without scraping
+  the log (#86).
+- **Progress during the run**: every backend prints `[mutineer] N/M mutants
+  (P%)` to stderr at each 10% step, so a long run is never silent between
+  config resolution and the report. Stdout stays byte-exact for `--format json`
+  and `--output`. `WorkerPool#run` gains an optional `on_result:` callback for
+  this (called in the parent per reaped result) (#86).
+
+### Changed
+- **PR runs scope themselves in the GitHub Action**: on `pull_request` events
+  the `since` input now defaults to the PR base, so the action grades just the
+  diff out of the box. **Migration note (the reason for the major bump)**: a
+  PR gate that previously full-scanned now scores only the PR's changed lines,
+  so `threshold` applies to fewer mutants. Stay on `@v0` to keep the old
+  default, or pass `since: none` on `@v1` for full scans. Workflows that
+  already pass a non-empty `since` are unchanged (an explicit empty string is
+  indistinguishable from unset and picks up the new default). A PR with no
+  changed source lines (docs- or test-only) scores zero mutants and passes the
+  scoped gate vacuously; keep a full-scan baseline refresh on main as the
+  backstop. With `use-bundler: true` the caller's Gemfile picks the gem, and
+  the new default needs mutineer >= 1.0.0 (the action enforces the floor with
+  a clear error). The action scopes to the PR's exact base commit from the
+  event payload (immune to the
+  base branch advancing mid-job), falling back to a fresh fetch of the base
+  branch tip; when neither can be resolved it warns and runs without an
+  action-provided `--since` (a `.mutineer.yml` `since:` key, if any, still
+  applies). The default deliberately does NOT fire on `pull_request_target`:
+  checkout there defaults to the base branch, so auto-scoping would diff the
+  base against itself and green the gate on an empty run (#86).
+- **`--baseline` on a diff-scoped run gates on new survivors only**: a
+  `--since` run's score covers only the changed-line mutants, a different
+  denominator from a full-run baseline, so comparing the two scores
+  manufactured false regressions (a 3/4-mutant PR at 75% "dropped" from a
+  92% whole-repo baseline with zero new survivors). With `--since`, the
+  score-drop half of the baseline gate is skipped; new-survivor detection by
+  stable id (and the reported before/after scores) are unchanged. The JSON
+  report records the scope in a new additive `summary.scoped` key
+  (`schema_version` 1.3), and the `baseline` block records `score_comparable`
+  so a consumer knows when not to render the two scores as a comparison
+  (`fixed_survivors` is likewise empty under a scoped side: an out-of-scope
+  survivor was never re-tested, so absence does not mean fixed). The reverse
+  direction is a hard guard: a scoped report is refused as a baseline (exit 2
+  with a regenerate hint), because survivors outside its diff would all read
+  as new regressions. A new `--no-since` flag disables diff scoping explicitly: a
+  typed no beats a `.mutineer.yml` `since:` key, and the action's
+  `since: none` passes it through (#86).
+
+## [0.11.4] - 2026-07-29
+
+### Fixed
+- **`--threshold` now gates on the run being complete, not just its score** — a
+  score is `killed / (killed + survived)`, so mutants that error, time out or come
+  back uncapturable are excluded from the denominator: a broken harness *raised*
+  the score instead of lowering it. Ninety errored mutants and ten that ran (nine
+  killed) reported 90% and exited 0, so CI could not tell a complete run from a
+  mostly-broken one. Past 10% of attempted mutants producing no verdict, a positive
+  `--threshold` now exits 1 whatever the score, and the report says which states
+  broke, on every `--format`. A single bad mutant never trips it, however small the
+  run, so a `--since` PR with a handful of mutants keeps its flake tolerance (#78).
+
+### Added
+- **`no_verdict[]` in the JSON report** (`schema_version` 1.2) — every attempted
+  mutant that produced no verdict, with its `status` and, where there is one, the
+  `details` explaining the cause. `Result#details` was built and rendered in no
+  format at all, so a daemon crash reached the user as nothing but a larger errored
+  count. `summary` gains `attempted` and `no_verdict`, the two figures the
+  completeness gate is computed from (#78).
+
 ## [0.11.3] - 2026-07-29
 
 ### Fixed
@@ -297,6 +381,8 @@ Rails hardening + CI batch (issues #8–#13), all verified Rails-free.
 - `.mutineer.yml` configuration (CLI > config > default precedence).
 - Byte-correct source handling for multibyte (UTF-8) sources.
 
+[1.0.0]: https://github.com/davidteren/mutineer/releases/tag/v1.0.0
+[0.11.4]: https://github.com/davidteren/mutineer/releases/tag/v0.11.4
 [0.11.3]: https://github.com/davidteren/mutineer/releases/tag/v0.11.3
 [0.11.2]: https://github.com/davidteren/mutineer/releases/tag/v0.11.2
 [0.11.1]: https://github.com/davidteren/mutineer/releases/tag/v0.11.1

data/README.md

@@ -47,11 +47,12 @@ mutineer run lib/calculator.rb --test test/calculator_test.rb --threshold 90
 |------|---------|
 | `--test FILE` | Test file covering the sources (repeatable) |
 | `--operators LIST` | Comma-separated operator names (default: the Tier-1 set) |
-| `--threshold FLOAT` | Exit 1 when the score is below FLOAT (default: 0 = off) |
+| `--threshold FLOAT` | Exit 1 when the score is below FLOAT, or when nothing could be scored and something broke, or more than one mutant produced no verdict and they exceed 10% of those attempted (default: 0 = off) |
 | `--only NAME` | Restrict to one fully-qualified subject, e.g. `Calculator#add` |
 | `--framework NAME` | `minitest` (default) or `rspec`; auto-detected as rspec when most `--test` files end in `_spec.rb` |
 | `--since REF` | Only mutate lines changed since git `REF` (e.g. `origin/main`) — ideal for PR CI |
-| `--baseline FILE` | Compare against a prior `--format json` run; exit 1 on new survivors / score drop (see [CI](#ci-gating)) |
+| `--no-since` | Disable diff scoping; a typed no beats a `.mutineer.yml` `since:` key |
+| `--baseline FILE` | Compare against a prior `--format json` run; exit 1 on new survivors / score drop (score drop is skipped under `--since`, whose score covers a different denominator; see [CI](#ci-gating)) |
 | `--baseline-epsilon FLOAT` | Score-drop tolerance for `--baseline` (default: 0) |
 | `--jobs N` | Parallel worker count (default: processor count; `1` under `--rails`) |
 | `--verbose` | Surface the real error when a fork capture fails (alias `--debug`) |
@@ -70,7 +71,7 @@ mutineer run lib/calculator.rb --test test/calculator_test.rb --threshold 90
 | Code | Meaning |
 |------|---------|
 | `0` | Score ≥ threshold (or no threshold set) |
-| `1` | Survivors below threshold, or a runtime error |
+| `1` | Score below threshold, nothing could be scored and something broke, or more than one mutant produced no verdict and they exceed 10% of those attempted, a `--baseline` regression, or a runtime error |
 | `2` | Usage / invalid-flag error |
 
 ### Operators
@@ -233,17 +234,32 @@ This repo ships a composite action (`action.yml`) that wraps the CLI for CI:
 
 ```yaml
 - uses: actions/checkout@v4
-  with: { fetch-depth: 0 }        # --since needs full history
 - uses: ruby/setup-ruby@v1
   with: { ruby-version: "3.4", bundler-cache: true }
-- uses: davidteren/mutineer@v0
+- uses: davidteren/mutineer@v1
   with:
     sources: app/
-    since: origin/${{ github.base_ref }}
     baseline: .mutineer/baseline.json
     threshold: "90"
 ```
 
+**Default change:** on `pull_request` events (not `pull_request_target`) the
+action scopes the run to the PR's changed lines, diffing against the PR's exact
+base commit (fetched by the action itself when the checkout is shallow; falls
+back to the base branch tip). Pass `since: none` for a full scan, or an
+explicit `since:` ref (which needs `fetch-depth: 0` on checkout).
+
+With the default JSON format the action also:
+
+- writes a score summary to the job's step summary;
+- annotates surviving mutants on the PR diff, up to 50 (`error` level when the
+  gate failed, `warning` when it passed);
+- exposes the report path via the `report` output for later steps (with
+  `format: human`/`html` this needs the `output` input).
+
+The CLI prints a progress line to the log at every 10% of the run, whatever
+the format.
+
 ## For AI agents & pipelines
 
 Mutineer is built for programmatic use — versioned JSON, stable mutant ids,

data/lib/mutineer/baseline.rb

@@ -15,12 +15,19 @@ module Mutineer
     #   new_survivors   - current Result objects whose stable id is absent from
     #                     the baseline (the regressions to name).
     #   fixed_survivors - baseline survivor hashes absent from the current run
-    #                     (informational, never gates).
+    #                     (informational, never gates). Empty when either side
+    #                     is diff-scoped: an out-of-scope baseline survivor was
+    #                     never re-tested, so absence does not mean fixed.
     #   score_drop      - current score < baseline score - epsilon. nil on
-    #                     either side skips the check (see #diff).
+    #                     either side skips the check, and a diff-scoped run
+    #                     (`scoped: true`) never sets it (see #diff).
+    #   score_comparable - the two scores share a denominator (neither side was
+    #                     diff-scoped and both are non-nil), so a consumer may
+    #                     render them side by side. False means the score-drop
+    #                     check was skipped, not that it passed.
     #   regressed       - any new survivors OR a score drop.
     Delta = Data.define(:new_survivors, :fixed_survivors,
-                        :score_before, :score_after, :score_drop, :regressed)
+                        :score_before, :score_after, :score_drop, :score_comparable, :regressed)
 
     # Load a prior --format json run. Raises ConfigError (NOT exit: a data class
     # must never kill the host) on a missing/unreadable file, unparseable JSON,
@@ -35,6 +42,15 @@ module Mutineer
       unless doc.is_a?(Hash) && doc["schema_version"] && doc["survivors"].is_a?(Array)
         raise ConfigError, "not a Mutineer JSON report: #{path}"
       end
+      # A diff-scoped report covers only that diff's mutants: used as a
+      # baseline, every survivor outside the original diff would read as a NEW
+      # regression, and its score shares no denominator with any other run.
+      # Refuse loudly (exit 2 via the CLI) rather than gate unreliably.
+      if doc.dig("summary", "scoped") == true
+        raise ConfigError, "#{path} was written by a --since run and covers only that diff's " \
+                           "mutants; regenerate the baseline from a full run " \
+                           "(use --no-since if .mutineer.yml sets since:)"
+      end
 
       new(doc)
     rescue JSON::ParserError => e
@@ -45,39 +61,61 @@ module Mutineer
 
     # Builds a baseline from a JSON document.
     #
-    # The baseline retains the survivor document and score from the JSON report.
+    # The baseline retains the survivor document, score, and scope marker from
+    # the JSON report. A report whose `summary.scoped` is true came from a
+    # `--since` run: its score covers only changed-line mutants, so later diffs
+    # must not compare a full-run score against it.
     #
     # @param doc [Hash] parsed JSON document.
     def initialize(doc)
       @survivors = doc["survivors"] || []
       @score = doc.dig("summary", "score")
+      # Strict literal true only: a malformed value (say the STRING "false" in a
+      # hand-edited baseline) must not silently disable the score-drop gate.
+      @scoped = doc.dig("summary", "scoped") == true
     end
 
     # Diff a current AggregateResult against this baseline by stable survivor id.
     # `epsilon` tolerates float jitter on the score (default 0.0 = any drop
     # gates).
     #
+    # `scoped: true` marks the current run as diff-scoped (`--since`): its score
+    # is computed over only the changed-line mutants, a different denominator
+    # from a full-run baseline, so comparing the two scores manufactures false
+    # regressions. A scoped diff keeps the new-survivor gate (stable ids compare
+    # fine across scopes) and still reports both scores, but never sets
+    # score_drop.
+    #
     # @param aggregate [Mutineer::AggregateResult] current results.
     # @param epsilon [Float] score-drop tolerance.
+    # @param scoped [Boolean] current run was diff-scoped (`--since`).
     # @return [Mutineer::Baseline::Delta] delta summary.
-    def diff(aggregate, epsilon: 0.0)
+    def diff(aggregate, epsilon: 0.0, scoped: false)
       current = aggregate.surviving_mutants
       current_ids = current.map(&:id)
       baseline_ids = @survivors.map { |h| h["id"] }
 
       new_survivors = current.reject { |r| baseline_ids.include?(r.id) }
-      fixed = @survivors.reject { |h| current_ids.include?(h["id"]) }
+      # Under a diff-scoped side an out-of-scope baseline survivor was never
+      # re-tested, so reporting it "fixed" would be false: empty is honest.
+      fixed = if scoped || @scoped
+                []
+              else
+                @survivors.reject { |h| current_ids.include?(h["id"]) }
+              end
 
       current_score = aggregate.mutation_score
       # nil-score discipline (mirrors Reporter#exit_code): a score absent on
       # either side cannot be compared. Skip the drop check, keep the new-
-      # survivor check.
-      score_drop = !@score.nil? && !current_score.nil? &&
-                   current_score < @score - epsilon
+      # survivor check. Same when EITHER side is diff-scoped (the current run
+      # via `scoped:`, or the stored baseline via its `summary.scoped` marker):
+      # the denominators differ, so the scores are not comparable.
+      comparable = !scoped && !@scoped && !@score.nil? && !current_score.nil?
+      score_drop = comparable && current_score < @score - epsilon
 
       Delta.new(new_survivors: new_survivors, fixed_survivors: fixed,
                 score_before: @score, score_after: current_score,
-                score_drop: score_drop,
+                score_drop: score_drop, score_comparable: comparable,
                 regressed: !new_survivors.empty? || score_drop)
     end
   end

data/lib/mutineer/changed_lines.rb

@@ -57,16 +57,24 @@ module Mutineer
 
     # Returns the stdout of `git -C <root> diff --unified=0 <ref> -- <file>`.
     #
+    # A failure is warned, never silent: an empty result means "no changed
+    # lines", which under `--since` removes every mutant for the file — a green
+    # gate must not be manufactured by a broken diff without a trace.
+    #
     # @param ref [String] git ref to diff against.
     # @param abs_file [String] absolute path of the file being diffed.
     # @param project_root [String] repository root for `git -C`.
-    # @return [String] diff text, or `""` on failure.
+    # @return [String] diff text, or `""` on failure (after a stderr warning).
     def git_diff(ref, abs_file, project_root)
       out, _err, status = Open3.capture3(
         "git", "-C", project_root, "diff", "--unified=0", ref, "--", abs_file
       )
-      status.success? ? out : ""
-    rescue StandardError
+      return out if status.success?
+
+      warn "[mutineer] git diff failed for #{abs_file}; its lines will not be mutated (--since)"
+      ""
+    rescue StandardError => e
+      warn "[mutineer] git diff failed for #{abs_file} (#{e.class}); its lines will not be mutated (--since)"
       ""
     end
   end

data/lib/mutineer/cli.rb

@@ -41,6 +41,7 @@ module Mutineer
         --baseline-epsilon FLOAT  Score-drop tolerance for --baseline (default: 0)
         --only NAME          Restrict to one fully-qualified subject
         --since REF          Only mutate lines changed since git REF (e.g. origin/main)
+        --no-since           Disable diff scoping (a typed no beats a .mutineer.yml since:)
         --jobs N             Parallel worker count (default: processor count)
         --strategy NAME      reload (whole-file) or redefine (surgical); default: reload
         --framework NAME     minitest or rspec (default: auto-detect from --test names)
@@ -96,6 +97,9 @@ module Mutineer
         o.on("--fail-fast") { opts[:fail_fast] = true; explicit << :fail_fast }
         o.on("--only NAME") { |v| opts[:only] = v; explicit << :only }
         o.on("--since REF") { |v| opts[:since] = v; explicit << :since }
+        # A typed "no" must beat a .mutineer.yml `since:` key (CLI-over-config
+        # precedence): marking :since explicit with a nil value blocks the fill.
+        o.on("--no-since") { opts[:since] = nil; explicit << :since }
         o.on("--test FILE") { |v| (opts[:tests] ||= []) << v }
         o.on("--operators LIST") { |v| opts[:operators] = v.split(",").map(&:strip); explicit << :operators }
         o.on("--threshold FLOAT") do |v|
@@ -465,11 +469,17 @@ module Mutineer
 
       # Diff the current run against the baseline (preflighted above) by the
       # stable survivor id. The delta is rendered inline (human section / additive
-      # json block) and gates exit independently of --threshold.
-      delta = (Baseline.load(config.baseline).diff(aggregate, epsilon: config.baseline_epsilon) if config.baseline)
+      # json block) and gates exit independently of --threshold. A --since run is
+      # scoped: its score covers a different denominator than a full-run baseline,
+      # so only the new-survivor half of the gate applies (see Baseline#diff).
+      delta = if config.baseline
+                Baseline.load(config.baseline).diff(aggregate, epsilon: config.baseline_epsilon,
+                                                               scoped: !config.since.nil?)
+              end
 
       reporter.report(out: $stdout, err: $stderr, threshold: config.threshold,
-                      format: config.format, output: config.output, baseline: delta)
+                      format: config.format, output: config.output, baseline: delta,
+                      scoped: !config.since.nil?)
 
       # Warn (stderr, so it never pollutes json/html) that an external run's score
       # is not comparable to an in-process run: no coverage narrowing (uncovered

data/lib/mutineer/config.rb

@@ -182,6 +182,11 @@ module Mutineer
       when "ignore"    then Array(value).map(&:to_s)
       when "baseline"  then value.to_s
       when "test_command" then value.to_s
+      when "since"
+        # false / empty normalize to nil ("no scoping"), so every consumer's
+        # nil-check (runner scoping, the report's scoped marker) agrees. A
+        # false left raw would skip scoping but still mark the report scoped.
+        value == false || value.to_s.empty? ? nil : value.to_s
       else value
       end
     end

data/lib/mutineer/daemon_backend.rb

@@ -4,6 +4,7 @@ require_relative "parser"
 require_relative "result"
 require_relative "coverage_map"
 require_relative "daemon_client"
+require_relative "progress"
 # No require_relative "runner" on purpose: runner.rb requires this file, and the
 # reverse edge makes Ruby warn "circular require considered harmful" on every -w
 # load. Runner is loaded first on every real path; requiring this file alone leaves
@@ -137,10 +138,12 @@ module Mutineer
       client = DaemonClient.new(boot: boot_config(config, abs_tests),
                                 app_root: config.project_root).start
       results = []
+      progress = Progress.new(jobs.size)
       begin
         jobs.each_with_index do |job, i|
           r = job_result(job, i, client, 0, config, coverage_map, abs_tests, source_map)
           results << r
+          progress.tick
           break if config.fail_fast && r.survived?
         end
       ensure
@@ -159,8 +162,9 @@ module Mutineer
     # @api private
     # @return [Array<Mutineer::Result>] one result per input job, in input order.
     def self.run_parallel(jobs, worker_count, config, abs_tests, coverage_map, source_map)
-      results = Array.new(jobs.size)
-      queue   = Queue.new
+      results  = Array.new(jobs.size)
+      progress = Progress.new(jobs.size)
+      queue    = Queue.new
       jobs.each_index { |i| queue << i }
 
       # Built one at a time so a refused spawn part-way (EMFILE under a high --jobs)
@@ -189,6 +193,7 @@ module Mutineer
               break
             end
             results[i] = job_result(jobs[i], i, client, worker, config, coverage_map, abs_tests, source_map)
+            progress.tick
           end
         rescue DaemonBootError
           # The daemon gave up for good. Stop feeding the other workers rather

data/lib/mutineer/progress.rb

@@ -0,0 +1,46 @@
+# frozen_string_literal: true
+
+module Mutineer
+  # Coarse run progress on stderr: one line per 10% step of completed mutants
+  # (which means every completion on a run smaller than ten). Shared by every
+  # backend so a long run is never silent between config resolution and the
+  # report. Thread-safe so the daemon's parallel workers can share one instance.
+  # Writes only to stderr, keeping `--format json` stdout and `--output` files
+  # byte-exact per the Reporter's stream contract.
+  class Progress
+    # Builds a progress counter for a run of `total` mutants.
+    #
+    # @param total [Integer] number of mutants that will run.
+    # @param io [IO] destination stream (defaults to stderr; injectable for tests).
+    def initialize(total, io: $stderr)
+      @total = total
+      @done = 0
+      @last_step = 0
+      @mutex = Mutex.new
+      @io = io
+    end
+
+    # Records one completed mutant and prints when a 10% boundary is crossed.
+    # A zero-total counter never prints (guards the division, and an empty run
+    # has nothing to report).
+    #
+    # @return [void]
+    def tick
+      @mutex.synchronize do
+        return if @total.zero?
+
+        @done += 1
+        step = (@done * 10) / @total
+        return if step <= @last_step
+
+        @last_step = step
+        @io.puts "[mutineer] #{@done}/#{@total} mutants (#{(@done * 100) / @total}%)"
+      end
+    rescue IOError, Errno::EPIPE
+      # Progress is best-effort: a closed stderr (piped consumer went away) must
+      # never kill the run — in the daemon's worker threads a raise here would
+      # propagate through Thread#join and abort scoring.
+      nil
+    end
+  end
+end

data/lib/mutineer/reporter.rb

@@ -14,6 +14,17 @@ module Mutineer
   # `source_map` is { file_path => raw source string }, used to extract the
   # containing source line for each survivor diff.
   class Reporter
+    # Share of attempted mutants that may produce no verdict before `--threshold`
+    # stops trusting the score. A handful of flaky mutants in a large run is noise;
+    # a mostly-broken run is not a score. Deliberately not a flag: no one has needed
+    # a different number yet.
+    BROKEN_SHARE_LIMIT = 0.10
+
+    # A share alone gives a small run no tolerance at all: on a `--since` PR that
+    # yields 8 mutants, one timeout is 12.5%. README recommends exactly that
+    # workflow, so one bad mutant never fails the gate on its own, at any size.
+    BROKEN_FLOOR = 1
+
     def initialize(aggregate, source_map)
       @agg = aggregate
       @source_map = source_map
@@ -21,11 +32,14 @@ module Mutineer
 
     # Single entry point. Branches on `format` ("human" | "json" | "html") and
     # routes the rendered report to `output` (a file, with a stderr confirmation)
-    # or to `out`. Diagnostics always go to `err`.
-    def report(out: $stdout, err: $stderr, threshold: 0.0, format: "human", output: nil, baseline: nil)
+    # or to `out`. Diagnostics always go to `err`. `scoped` marks a diff-scoped
+    # (`--since`) run; the JSON report records it so a consumer (or a later
+    # `--baseline` load) knows the score covers only the changed-line mutants.
+    def report(out: $stdout, err: $stderr, threshold: 0.0, format: "human", output: nil,
+               baseline: nil, scoped: false)
       rendered =
         if format == "json"
-          json_report(baseline)
+          json_report(baseline, scoped: scoped)
         elsif format == "html"
           html_report
         else
@@ -42,6 +56,20 @@ module Mutineer
       else
         out.print rendered
       end
+
+      # Both ways a run can fail the gate on completeness, said here rather than in
+      # the human renderer: --format json is the documented CI path, and a run that
+      # exits 1 must say why on every format, not only the one a person reads.
+      return unless threshold&.positive?
+
+      if @agg.mutation_score.nil? && broken_nil_score?
+        err.puts "[mutineer] nothing could be scored (#{broken_counts_detail}), so the " \
+                 "--threshold gate fails. See no_verdict[] in --format json for the cause of each."
+      elsif broken_share_exceeded?
+        err.puts "[mutineer] #{no_verdict_ratio}: #{broken_counts_detail}. The score covers " \
+                 "only part of the run, so the --threshold gate fails. See no_verdict[] in " \
+                 "--format json for the cause of each."
+      end
     end
 
     # Renders the human report.
@@ -85,6 +113,11 @@ module Mutineer
         return 0 # pure no_coverage / ignored / empty — gate skipped
       end
 
+      # A score computed over a small slice of what was attempted is not this
+      # suite's score. Without this, 90 errored mutants and 10 that ran (9 killed)
+      # reports 90% and exits 0, so CI cannot tell a complete run from a broken one.
+      return 1 if broken_share_exceeded?
+
       score >= threshold ? 0 : 1
     end
 
@@ -96,8 +129,10 @@ module Mutineer
     #
     # @api private
     # @param baseline [Mutineer::Baseline::Delta, nil] baseline delta.
+    # @param scoped [Boolean] the run was diff-scoped (`--since`), so its score
+    #   covers only the changed-line mutants (additive `summary.scoped` key).
     # @return [String] JSON text.
-    def json_report(baseline = nil)
+    def json_report(baseline = nil, scoped: false)
       killed = @agg.killed_count
       survived = @agg.survived_count
       # null (not 0.0) on an empty denominator, matching the nil-vs-0.0
@@ -106,7 +141,7 @@ module Mutineer
       score = @agg.mutation_score
 
       doc = {
-        schema_version: "1.1",
+        schema_version: "1.3",
         summary: {
           total: @agg.total, killed: killed, survived: survived,
           no_coverage: @agg.no_coverage_count,
@@ -114,7 +149,13 @@ module Mutineer
           skipped_invalid: @agg.skipped_invalid_count,
           errored: @agg.errored_count, timeout: @agg.timeout_count,
           ignored: @agg.ignored_count,
-          score: score
+          # The gate is computed from these two, so a consumer never re-derives them.
+          attempted: attempted_count, no_verdict: no_verdict_count,
+          score: score,
+          # Additive: true when the run was diff-scoped (--since). The score then
+          # covers only the changed-line mutants, so it is not comparable to a
+          # full-run score; Baseline#diff reads this to skip the score-drop gate.
+          scoped: scoped
         },
         survivors: @agg.surviving_mutants.map { |r| survivor_json(r) }
                        .sort_by { |h| [h[:file], h[:line], h[:operator]] },
@@ -123,6 +164,21 @@ module Mutineer
         # Same shape as no_coverage; additive key.
         uncapturable: @agg.results.select(&:uncapturable?).map { |r| no_coverage_json(r) }
                           .sort_by { |h| [h[:file], h[:line]] },
+        # Every mutant that was attempted and produced no verdict, whatever the
+        # reason — the set the --threshold completeness gate counts. Named for the
+        # condition rather than one status, because summary.errored means :error
+        # alone and a key that reconciled with neither would be worse. `details`
+        # carries the cause where there is one. Uncapturable mutants also appear in
+        # uncapturable[]; that key keeps its lean shape for existing consumers.
+        # to_s/to_i because a pre-fork failure has no subject, so its file and line
+        # are null and would not compare against a real entry. id and status extend
+        # the key to a total order: these entries collide on (file, line) far more
+        # than survivors do — several mutants on one crashy line, every pre-fork
+        # entry on ("", 0) — and sort_by is not stable, so equal keys would leave
+        # worker finish order in the output and break the byte-stability promise.
+        no_verdict: @agg.results.select { |r| r.error? || r.timeout? || r.uncapturable? }
+                        .map { |r| no_verdict_json(r) }
+                        .sort_by { |h| [h[:file].to_s, h[:line].to_i, h[:id].to_s, h[:status].to_s, h[:details].to_s] },
         # Equivalent mutants the user suppressed: emitted with their stable id so
         # the user can audit what is silenced (and copy ids for survivors they
         # want to add). Excluded from the score; never in `survivors`.
@@ -134,7 +190,7 @@ module Mutineer
                         .sort_by { |h| h[:file] }
       }
       # Additive baseline-delta block, present only with --baseline. Existing
-      # consumers ignore the extra key; schema_version stays 1.1.
+      # consumers ignore the extra key; it does not move schema_version on its own.
       doc[:baseline] = baseline_json(baseline) if baseline
       "#{JSON.generate(doc)}\n"
     end
@@ -259,6 +315,10 @@ module Mutineer
         score_before: delta.score_before,
         score_after: delta.score_after,
         score_dropped: delta.score_drop,
+        # Additive: false when the score-drop check was skipped (a nil score or
+        # a diff-scoped side), so a consumer knows not to render the two scores
+        # as a comparison.
+        score_comparable: delta.score_comparable,
         new_survivors: delta.new_survivors.map { |r| ignored_json(r) }
                             .sort_by { |h| [h[:file], h[:line], h[:operator]] },
         fixed_survivors: delta.fixed_survivors.map do |h|
@@ -358,6 +418,31 @@ module Mutineer
       }
     end
 
+    # An entry under the JSON `no_verdict:` key: an attempted mutant with no verdict.
+    # A pre-fork failure has no subject or mutation attached, so those degrade to
+    # nulls rather than dropping the entry — the count must still reconcile with
+    # `summary.no_verdict`.
+    #
+    # @api private
+    # @param result [Mutineer::Result] an errored or timed-out result.
+    # @return [Hash] no-verdict JSON object.
+    def no_verdict_json(result)
+      file = result.subject&.file
+      line =
+        if result.mutation && file
+          source = @source_map[file] || File.read(file)
+          source.byteslice(0, result.mutation.start_offset).count("\n") + 1
+        end
+      {
+        subject: result.subject&.qualified_name,
+        file: file,
+        line: line,
+        id: result.id,
+        status: result.status.to_s,
+        details: result.details
+      }
+    end
+
     # Writes the summary block.
     #
     # @param out [IO] output stream.
@@ -388,10 +473,10 @@ module Mutineer
                  "#{@agg.ignored_count} ignored excluded"
       if score.nil?
         out.puts "Mutation score: N/A  (no covered mutants)"
-        if broken_nil_score?
-          err.puts "[mutineer] no covered mutations (#{broken_nil_score_detail}); " \
-                   "threshold gate fails under a positive --threshold (broken harness)."
-        else
+        # Only the benign case here: the gate-failure explanation is emitted once
+        # from {report}, for every format, so it cannot be said twice or only to
+        # the reader of the human report.
+        unless broken_nil_score?
           err.puts "[mutineer] no covered mutations; mutation score is N/A and the threshold check is skipped."
         end
       else
@@ -408,11 +493,58 @@ module Mutineer
         (@agg.errored_count + @agg.timeout_count + @agg.uncapturable_count).positive?
     end
 
-    # Human-readable counts for a broken nil-score run.
+    # Mutants that were attempted but produced no verdict, over everything
+    # attempted. Above the limit the score describes too small a slice of the run
+    # to gate on. A few flaky mutants in a large run stay under it.
+    #
+    # @api private
+    # @return [Boolean] true when too much of the run failed to produce a verdict.
+    def broken_share_exceeded?
+      attempted = attempted_count
+      attempted.positive? && no_verdict_count > BROKEN_FLOOR &&
+        no_verdict_count > attempted * BROKEN_SHARE_LIMIT
+    end
+
+    # Mutants that were attempted and produced no verdict, whatever the reason.
+    #
+    # @api private
+    # @return [Integer] errored + timed out + uncapturable.
+    def no_verdict_count
+      @agg.errored_count + @agg.timeout_count + @agg.uncapturable_count
+    end
+
+    # Mutants that were actually run. Deliberately not `total`: no_coverage,
+    # skipped-invalid and ignored mutants were never attempted, so counting them
+    # would dilute the share and let a broken run slip under the limit.
+    #
+    # skipped-invalid is excluded from both sides: it means a mutant did not
+    # re-parse and was correctly never run, which is a validity outcome rather
+    # than a broken harness. Cost: an overwhelmingly-skipped run still scores on
+    # what little ran; that is our operator misbehaving and wants its own signal.
+    #
+    # @api private
+    # @return [Integer] killed + survived + no-verdict.
+    def attempted_count
+      @agg.killed_count + @agg.survived_count + no_verdict_count
+    end
+
+    # The sentence both the verdict line and the stderr note are built from, so a
+    # user cannot read one number in the report and a different one beside it.
+    #
+    # @api private
+    # @return [String] e.g. "90 of 100 attempted mutants produced no verdict (90.0%, limit 10%)".
+    def no_verdict_ratio
+      pct = (no_verdict_count * 100.0 / attempted_count).round(1)
+      "#{no_verdict_count} of #{attempted_count} attempted mutants produced no verdict " \
+        "(#{pct}%, limit #{(BROKEN_SHARE_LIMIT * 100).round}%)"
+    end
+
+    # Human-readable counts of the states that produced no verdict. Used by both
+    # the nil-score message and the completeness gate, so they agree.
     #
     # @api private
     # @return [String]
-    def broken_nil_score_detail
+    def broken_counts_detail
       parts = []
       parts << "#{@agg.errored_count} errored" if @agg.errored_count.positive?
       parts << "#{@agg.timeout_count} timeout" if @agg.timeout_count.positive?
@@ -458,6 +590,9 @@ module Mutineer
         out.puts "  + #{r.subject.qualified_name} (#{file}:#{line}) #{r.mutation.operator}"
       end
       out.puts "score dropped #{delta.score_before}% -> #{delta.score_after}%" if delta.score_drop
+      # An OK verdict must not imply a check that never ran: say when the score
+      # comparison was skipped (a diff-scoped side or a nil score).
+      out.puts "score-drop check skipped (scores not comparable)" unless delta.score_comparable
       out.puts(delta.regressed ? "REGRESSION vs baseline" : "OK: no regression vs baseline")
     end
 
@@ -505,13 +640,17 @@ module Mutineer
       score = @agg.mutation_score
       if score.nil?
         if broken_nil_score?
-          out.puts "FAILED: no covered mutants (#{broken_nil_score_detail}); " \
+          out.puts "FAILED: no covered mutants (#{broken_counts_detail}); " \
                    "threshold #{threshold}% cannot pass with a broken harness"
         end
         return
       end
 
-      if score >= threshold
+      # Same rule as exit_code, or the report says PASSED on a run that exits 1 —
+      # and with --output that wrong verdict is what gets archived.
+      if broken_share_exceeded?
+        out.puts "FAILED: #{no_verdict_ratio}; #{score}% covers only part of the run"
+      elsif score >= threshold
         out.puts "PASSED: #{score}% >= threshold #{threshold}%"
       else
         out.puts "FAILED: #{score}% < threshold #{threshold}%"

data/lib/mutineer/runner.rb

@@ -10,6 +10,7 @@ require_relative "coverage_map"
 require_relative "changed_lines"
 require_relative "mutator_registry"
 require_relative "worker_pool"
+require_relative "progress"
 require_relative "mutant_id"
 require_relative "file_swap"
 require_relative "external_backend"
@@ -121,7 +122,9 @@ module Mutineer
         begin
           framework = config.framework
           stop_when = config.fail_fast ? ->(r) { r.survived? } : nil
-          bare = WorkerPool.new(jobs_n).run(jobs, stop_when: stop_when) do |subject, mutation|
+          progress  = Progress.new(jobs.size)
+          bare = WorkerPool.new(jobs_n).run(jobs, stop_when: stop_when,
+                                                  on_result: ->(_r) { progress.tick }) do |subject, mutation|
             run(mutation, source_file: subject.file, coverage_map: coverage_map,
                 subject: subject, strategy: strategy, rails: config.rails, framework: framework)
           end
@@ -205,11 +208,13 @@ module Mutineer
       timeout = [[smoke_elapsed * 3, 30].max, 300].min.ceil
 
       results = []
+      progress = Progress.new(jobs.size)
       begin
         jobs.each do |subject, mutation, id|
           r = run_external(subject, mutation, config.test_command, abs_tests,
                            timeout: timeout, verbose: config.verbose)
           results << r.with(subject: subject, mutation: mutation, id: id)
+          progress.tick
           break if config.fail_fast && r.survived? # stop at the first survivor
         end
       ensure

data/lib/mutineer/version.rb

@@ -2,5 +2,5 @@
 
 module Mutineer
   # Current Mutineer release version.
-  VERSION = "0.11.3"
+  VERSION = "1.0.0"
 end

data/lib/mutineer/worker_pool.rb

@@ -28,10 +28,15 @@ module Mutineer
     # @param stop_when [Proc, nil] called with each collected Result; when it
     #   returns truthy, no further items are scheduled and the run drains and
     #   returns early (--fail-fast). Unscheduled slots stay nil.
+    # @param on_result [Proc, nil] called in the parent with each collected
+    #   Result as it is reaped, in finish order (progress reporting); its
+    #   return value is ignored. Must be fast and non-blocking: it runs on the
+    #   pool's single reap thread, so a slow callback stalls draining the other
+    #   in-flight children's pipes (the #4 deadlock discipline).
     # @yieldparam item [Array] one work item.
     # @return [Array<Mutineer::Result>] results in input order (nil for any item
     #   left unscheduled by an early stop).
-    def run(items, stop_when: nil)
+    def run(items, stop_when: nil, on_result: nil)
       results  = Array.new(items.size)
       queue    = (0...items.size).to_a
       running  = {} # pid => [index, read_io, buffer]
@@ -40,6 +45,7 @@ module Mutineer
       until queue.empty? && running.empty?
         fill(items, queue, running) { |*args| yield(*args) } unless stopping
         result = reap(results, running)
+        on_result&.call(result) if result
         if !stopping && stop_when && result && stop_when.call(result)
           stopping = true
           queue.clear # schedule no more; let in-flight workers drain

data/lib/mutineer.rb

@@ -27,6 +27,7 @@ require_relative "mutineer/mutators/regex_literal"
 require_relative "mutineer/mutators/collection_method"
 require_relative "mutineer/mutator_registry"
 require_relative "mutineer/worker_pool"
+require_relative "mutineer/progress"
 require_relative "mutineer/runner"
 require_relative "mutineer/reporter"
 require_relative "mutineer/baseline"

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: mutineer
 version: !ruby/object:Gem::Version
-  version: 0.11.3
+  version: 1.0.0
 platform: ruby
 authors:
 - David Teren
@@ -95,6 +95,7 @@ files:
 - lib/mutineer/mutators/string_literal.rb
 - lib/mutineer/pairing.rb
 - lib/mutineer/parser.rb
+- lib/mutineer/progress.rb
 - lib/mutineer/project.rb
 - lib/mutineer/rails_worker_db.rb
 - lib/mutineer/reporter.rb