rspec-signal 0.2.0 → 0.3.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 6cd660f06b5b7ecd54cdacbd562c06588c157139121debebe3a9ca2a58e6686e
-  data.tar.gz: fdd869cce815ac1b14733b54edf8eac6a3d233986aa316f340c60724c852998d
+  metadata.gz: 5b2d960f1c23decbc4bd847430d8f1df056e93bfe2823223397df48058d99217
+  data.tar.gz: b114d8d9b06efe92e7216919e383b1ac9a58c97f0e3684b0d68fc4bfe88a575b
 SHA512:
-  metadata.gz: 819b4e310c972c189c1497ea25280651d3496c876f17e306a563413b4d2ab6bd03a79bb45595e4e840f2463a641bdd8890d3e4843037567196b2616ab6e5f8b0
-  data.tar.gz: d2f3245f7e28d91cb75414ce7c8e5dd361fcefcd8bb6f5ff44036f4b0022ebbc2020fac36c50487c5e9b2cbd587643946be035e073769ca9f4335f4bad99b2db
+  metadata.gz: 8d6ec971a7db81678e54dd1ef59dccce8f3e2f87b0e37af4c254ba2b916434aa04720b9c50935c025d0f1d1c17109ebe4a93f97a7dc4656f910e0e49640c17b3
+  data.tar.gz: 893b91607a8c90ac55cc26f9e385cbfe20a9fde84f1263cb6dc1c28b10fe6189d2db81b5e35555dd458de7e837f243fa64e35ba8016ebdb69c41b85ecbb4c2d1

data/CHANGELOG.md

@@ -7,6 +7,79 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## [Unreleased]
 
+### Added
+
+- **Relationships (experimental, off by default).** With `config.causal_analysis = true`,
+  a fourth layer relates *signatures* by structural evidence only: a `before(:context)`
+  exception object shared by every example, one underlying exception (bare or wrapped)
+  at one first-party line, a missing definition read from exception attributes (`ENV`
+  key, constant, method on a first-party class), or one failing setup step. Signatures
+  are grouped as CAUSAL (always HIGH), SCOPE (concentration in one file or `type:`; no
+  confidence, no cause wording) or INDEPENDENT (no relationship found), and every
+  failure is classified into exactly one group. Printed only when it adds something;
+  published as an `analysis` block in `signal.json` marked `"experimental": true`,
+  whose fields may change. Evaluated by a corpus of real runs in `spec/causal/`.
+- **Shared code paths.** A third analysis layer that reads the *stack* rather than
+  the message, and reports the first-party lines that more than one signature runs
+  through. This relates failures whose messages have nothing in common — a `KeyError`
+  and the `expect { }.not_to raise_error` that swallowed it — which no message-based
+  extractor can do. Two rules hold it back: only the innermost first-party frames
+  outside the spec suite are indexed (`config.code_path_depth`, default 5), and a
+  line is reported only when two or more distinct signatures cross it. It claims a
+  measured fact, never a cause. Configure with `config.max_code_paths`.
+- **Run-to-run comparison.** `tmp/rspec-signal/history.json` records the last ten
+  runs as signature digests and counts — no messages, no paths. Each run then reports
+  what changed: `28 resolved, 3 new, 14 still failing, 3 changed signature`. The
+  fourth bucket is what makes the rest trustworthy: an edit that only shifts line
+  numbers reads as *changed*, never as a fix. The history survives the green run that
+  deletes every other artifact. Turn it off with `config.track_history = false`.
+- **Errors outside examples are captured, not just counted.** A spec file that fails
+  to load, or a `before(:suite)` hook that blows up, now reaches the report with its
+  exception, message, reduced trace and a rerun command. Previously only a count was
+  possible. The formatter registers RSpec's `:message` notification and re-emits
+  anything no other formatter is listening for, so nothing is lost or duplicated.
+- Stdout in quiet mode is now a triage view: counts, what changed since the last run,
+  and the top shared code paths. Enough to decide whether to open the report.
+- `signal.json` gains `run_id`, `since_last_run`, `code_paths`, `outside_examples`,
+  and per-signature `summary`, `rerun`, `rerun_ids` and `loose_signature`. The
+  contract is documented in the README.
+
+### Changed
+
+- **RSpec's `Failure/Error:` source echo is no longer part of a signature.** When an
+  error is raised inside a gem -- a factory's `save!`, a driver call -- RSpec echoes
+  each example's own call site, so one broken factory used to become one signature per
+  `create` line. The echo is still shown; it just no longer decides identity.
+  Every digest changes once as a result, so `history.json` moves to schema 2 and an
+  older file is ignored: the first run after upgrading has no "since last run" line,
+  rather than reporting every signature as resolved and new.
+- **Rerun commands now name RSpec example ids rather than locations**, shell-quoted.
+  A location selects every example defined on that line, so the printed command could
+  rerun ten examples for a loop-generated `it`, and the *same* command could be
+  printed for two different signatures. Reruns are now exact. The human-readable
+  location remains on the `Example` line above.
+- The signature index shows the exception message rather than RSpec's
+  `Failure/Error:` source echo, which was identical on every row.
+- `signal.json` is `schema: 2`. Every schema 1 field keeps its name and meaning.
+
+### Fixed
+
+- In a parallel run, every signature's loose digest was derived from a placeholder
+  message, so signatures of the same exception class shared one loose key and the
+  changed-signature bucket could pair unrelated failures. Worker payloads now carry the
+  loose digest.
+- One error outside examples repeated across many spec files -- a helper every file
+  requires failing at boot -- rendered one identical section per file. It now renders once,
+  naming every file it stopped from loading.
+- `signal.json` ignored `max_message_lines` and `max_diff_lines`, so configuring
+  them changed `signal.md` and left the JSON at the built-in defaults.
+- The parallel merger dropped `shared_group_locations`, so the "Via shared example
+  group" locator was missing from every parallel report.
+- A run whose only problem was outside every example wrote no report.
+- The terminal summary printed "0 failures" beside a report link when a spec file
+  failed to load.
+- `--dry-run` deleted the report written by the previous real run.
+
 ## [0.2.0] - 2026-08-27
 
 ### Added

data/README.md

@@ -11,11 +11,51 @@ thousands of tokens on `rspec-core/hooks.rb`, Bundler, and Thor before the agent
 reaches a single line of your code — and you pay that cost repeatedly,
 even when many failures stem from one bug.
 
-`rspec-signal` is a deterministic context-reduction layer between RSpec and a coding
-agent. It writes `tmp/rspec-signal/signal.md`: a compact, model-neutral report you
-can paste into any assistant or point an agent at.
+`rspec-signal` is a deterministic failure-intelligence layer between RSpec and a
+coding agent. It answers one question -- *what actually failed, and what is it
+really?* -- in a bounded number of tokens, with an exact command to verify each fix.
 
-In normal mode your terminal output stays as it was. In opt-in agent mode it also becomes a quiet formatter that prevents verbose failures from entering an autonomous agent's context.
+In normal mode your terminal output stays as it was. In opt-in agent mode it also
+becomes a quiet formatter that prevents verbose failures from entering an autonomous
+agent's context.
+
+No model, no network, no API key. Two runs of the same suite produce byte-identical
+reports.
+
+## Quickstart
+
+```ruby
+# Gemfile
+group :test do
+  gem "rspec-signal"
+end
+```
+
+```ruby
+# spec/spec_helper.rb (or rails_helper.rb)
+require "rspec/signal"
+```
+
+```bash
+bundle exec rspec-signal
+```
+
+That is the whole setup. Your terminal output is unchanged by the require; the
+`rspec-signal` command is the quiet wrapper for agents. Either way you get
+`tmp/rspec-signal/signal.md`.
+
+## Vocabulary
+
+Six words the report uses, defined once.
+
+| Term | Meaning |
+|------|---------|
+| **signature** | A set of failures that *are* the same failure, by an exact four-part fingerprint. Authoritative |
+| **related cluster** | Failures across more than one signature that share one strong diagnostic *symptom*. A hint, never a diagnosis |
+| **symptom** | The one characteristic a cluster is keyed on: an HTTP status, a missing selector, a route, a model, a constant |
+| **shared code path** | A line of *your* code that more than one signature runs through. A measured fact about stacks, not messages |
+| **culprit** | The innermost frame that is not test-runner plumbing: the code that actually raised |
+| **app context** | The innermost first-party frame outside your spec suite: the file to open first |
 
 ---
 
@@ -84,7 +124,7 @@ spec/models/shelf_spec.rb:5 in `block (3 levels) in <top (required)>'
 **Rerun**
 
 ```bash
-bundle exec rspec spec/models/shelf_spec.rb:4
+bundle exec rspec './spec/models/shelf_spec.rb[1:1]'
 ```
 
 **Also failing identically (11)**
@@ -104,21 +144,9 @@ failures reduces far less, and that is correct behaviour.
 
 ## Installation
 
-```ruby
-# Gemfile
-group :test do
-  gem "rspec-signal"
-end
-```
-
-Then require it from `spec_helper.rb` (or `rails_helper.rb`):
-
-```ruby
-require "rspec/signal"
-```
-
-That is the whole setup. Requiring the gem registers the formatter and puts RSpec's
-default formatter back, so your terminal output is unchanged.
+See [Quickstart](#quickstart) for the two lines. Requiring the gem registers the
+formatter and puts RSpec's default formatter back, so your terminal output is
+unchanged.
 
 If you would rather be explicit, or you need it only in CI, skip the require and
 name the formatter instead:
@@ -144,16 +172,30 @@ suite remains non-zero; a passing suite remains zero. The
 low-level `bundle exec rspec --format RSpec::Signal::Formatter` interface remains
 available after requiring `rspec/signal` from the spec helper.
 
+Stdout is a tool call's return value, so stdout is the triage view — enough to decide
+whether to open the report at all:
+
+```text
+2085 examples, 42 failures, 6 pending
+
+rspec-signal: 42 failures in 35 distinct signatures, 4 related clusters (2767 backtrace frames omitted)
+Since last run: 28 resolved, 3 new, 14 still failing (42 -> 17 failures)
+Shared code paths: app/models/reader.rb:88 (9 signatures), app/api/serializer.rb:31 (4 signatures)
+Report: tmp/rspec-signal/signal.md
+```
+
 ```text
 RSpec runs the full suite
 ↓
 verbose failure bodies and framework backtraces are not printed
 ↓
-rspec-signal writes tmp/rspec-signal/signal.md
+rspec-signal writes tmp/rspec-signal/signal.md and prints five lines of triage
 ↓
-the agent reads signal.md
+the agent reads signal.md only if the triage is not enough
 ↓
-the agent reruns individual failures from the report as needed
+the agent reruns one signature by its exact example ids
+↓
+the next run reports what that fix resolved, and what it did not
 ```
 
 ### Parallel suites (`parallel_tests`)
@@ -218,6 +260,9 @@ rspec-signal: 42 failures in 35 distinct signatures, 4 related clusters (2767 ba
 Report: tmp/rspec-signal/signal.md
 ```
 
+The parent merger owns the run comparison, because it is the only process that has
+seen the whole run.
+
 ### Human and CI modes
 
 For a human, run RSpec normally:
@@ -244,16 +289,35 @@ claude "fix the failures in tmp/rspec-signal/signal.md"
 
 ```text
 tmp/rspec-signal/
-  signal.md    primary compact report -- hand this to the agent
-  signal.json  the same data, machine readable, for CI and tooling
-               (`signatures` and `related` mirror the two grouping layers)
-  full.txt     optional original output; off by default
-  .gitignore   written automatically; artifacts can contain application data
+  signal.md     primary compact report -- hand this to the agent
+  signal.json   the same data, machine readable, for CI and tooling
+  history.json  digests and counts of recent runs, so a run can say what changed
+  full.txt      optional original output; off by default
+  .gitignore    written automatically; artifacts can contain application data
 ```
 
-A run with **no** failures deletes these files. A stale report describing failures
-you already fixed is worse than no report at all, because an agent will go and
-"fix" them again.
+At the start of a run, the wrappers delete the previous `signal.md`, `signal.json`,
+and `full.txt`. The formatter does the same when execution starts. New files are
+rendered and staged together, then supporting files are renamed into place before
+`signal.md`. Treat `signal.md` as the publication marker: if it exists, its optional
+companions belong to the same completed generation. An aborted run or rendering
+failure therefore leaves no current report rather than stale or mixed evidence.
+
+The wrapper can invalidate the default (or `RSPEC_SIGNAL_OUTPUT_DIR`) directory before
+RSpec boots. At that point Ruby configuration from a spec helper has not loaded, so
+`config.output_dir` and `config.enabled = false` cannot affect early invalidation. Set
+`RSPEC_SIGNAL_DISABLE=1` to disable the wrapper without mutating artifacts, and use
+`RSPEC_SIGNAL_OUTPUT_DIR` when a custom directory needs the early-failure guarantee.
+Dry runs also leave current artifacts alone.
+
+Plain `bundle exec rspec` cannot invalidate until rspec-signal's formatter starts; a
+failure before the gem or formatter loads can consequently leave the prior artifacts
+in place. Use `rspec-signal` when the early-failure guarantee matters.
+
+`history.json` deliberately survives invalidation — "42 failures became 0" is the
+most valuable thing a run can say, and the run that deletes the report is exactly the
+run that should be able to say it. See
+[Knowing what changed](#knowing-what-changed).
 
 The large `full.txt` artifact is off by default. Enable it only when you need the
 original unreduced formatter rendering:
@@ -268,6 +332,100 @@ Individual per-failure files are deliberately **not** written. With failures
 collapsed into a handful of signatures, `signal.md` is already the unit you want
 to paste, and a directory of near-duplicate fragments is just more to sift through.
 
+### `signal.json`
+
+`schema: 2`. Stable, and safe to build tooling on:
+
+| Key | What it is |
+|-----|-----------|
+| `run_id` | Identifies this run |
+| `summary` | Counts: examples, failures, pending, signatures, related clusters, shared code paths |
+| `since_last_run` | `resolved` / `new` / `persistent` / `changed`, each a list of signatures. Absent on a first run |
+| `signatures[]` | One per exact signature, largest first |
+| `signatures[].signature` | The digest. **This is the stable identity of a failure** — key on it |
+| `signatures[].summary` | The one line that says what went wrong |
+| `signatures[].rerun` | A copy-pasteable command that reruns the representative and nothing else |
+| `signatures[].rerun_ids` | Every affected example's RSpec id |
+| `signatures[].culprit` / `.app_context` | Where it raised, and the first-party line to open |
+| `signatures[].representative` | The one failure rendered in full: message, reduced trace, diagnostics |
+| `signatures[].affected[]` | Every example in the signature: location, id, description |
+| `related[]` | Symptom clusters |
+| `code_paths[]` | Lines of your code crossed by more than one signature |
+| `outside_examples[]` | Failures RSpec reported without an example |
+| `analysis` | Only with `config.causal_analysis = true`. **Experimental and not covered by the schema:** see [Relationships](#relationships-experimental) |
+
+Only the representative carries a full trace. That is the size trade-off the whole
+gem is built on: the other members of a signature are, by definition, the same
+failure.
+
+## Rerunning exactly what failed
+
+Every `Rerun` block names **RSpec example ids**, not locations:
+
+```bash
+bundle exec rspec './spec/models/shelf_spec.rb[1:1]'
+```
+
+A location (`spec/models/shelf_spec.rb:4`) is what a human types, but it selects
+every example defined on that line — which for a loop-generated `it` is all of them,
+and for two failures that share a line is both of them. An agent that fixes a
+signature, reruns the printed command and still sees failures cannot then tell
+whether its fix failed or whether it caught something unrelated.
+
+The ids are shell-quoted because `[` and `]` are glob metacharacters: bash passes an
+unmatched glob through unchanged, but zsh fails the command outright. The
+human-readable location is still on the `Example` line directly above.
+
+RSpec's own `--only-failures` keys on the same ids. The two go together:
+
+```bash
+# spec/spec_helper.rb
+RSpec.configure { |c| c.example_status_persistence_file_path = "spec/examples.txt" }
+```
+
+```bash
+bundle exec rspec-signal --only-failures   # just what broke last time
+bundle exec rspec-signal --next-failure    # one at a time, in defined order
+```
+
+## Knowing what changed
+
+An agent has no memory between turns. The tool has a filesystem, so it keeps one:
+
+```text
+2085 examples, 42 failures, 6 pending
+
+rspec-signal: 42 failures in 35 distinct signatures, 4 related clusters
+Since last run: 28 resolved, 3 new, 14 still failing (42 -> 17 failures)
+Shared code paths: app/models/reader.rb:88 (9 signatures)
+Report: tmp/rspec-signal/signal.md
+```
+
+Four buckets, from two keys per signature — the full fingerprint, and a **loose** key
+of exception and normalized message alone, independent of line numbers:
+
+| Bucket | Rule |
+|--------|------|
+| **still failing** | The digest is in both runs |
+| **resolved** | The digest is gone, and its loose key is gone too |
+| **new** | The digest appeared, and its loose key is new too |
+| **changed signature** | The digest changed but the loose key survived — same failure, moved or reworded |
+
+The fourth bucket is what makes the other three trustworthy. Editing the file that
+raises shifts every culprit line; without it, one such edit turns every persistent
+failure into a spurious resolved/new pair, and being told "28 resolved" when you
+resolved nothing is worse than being told nothing.
+
+`history.json` holds the last ten runs as **digests and counts only** — no messages,
+no paths, nothing the redactor exists to protect. A missing, unreadable or corrupt
+history means the run has nothing to compare against, which is exactly how every run
+behaved before this existed. CI that discards `tmp/` simply gets no comparison, and
+needs no configuration to say so.
+
+```ruby
+RSpec::Signal.configure { |config| config.track_history = false }
+```
+
 ## Filtering philosophy
 
 The goal is not the shortest possible output. It is **the minimum context that
@@ -339,7 +497,7 @@ by a deterministic fingerprint of four components:
 | Component | What it is |
 |-----------|-----------|
 | **exception class** | `Capybara::ElementNotFound`, `ActiveRecord::RecordInvalid`, ... |
-| **normalized message** | The message with volatile parts masked: object addresses, UUIDs, timestamps, record ids, temp paths. Small numbers are left alone — `expected 3, got 4` and `expected 7, got 2` are different failures. |
+| **normalized message** | The message with volatile parts masked: object addresses, UUIDs, timestamps, record ids, temp paths. Small numbers are left alone — `expected 3, got 4` and `expected 7, got 2` are different failures. RSpec's `Failure/Error:` echo of the failing source line is left out: for an error raised inside a gem it is each example's own call site. |
 | **culprit** | The innermost frame that is not test-runner plumbing: the code that actually raised. |
 | **app context** | The innermost first-party frame outside your spec suite. `nil` for a plain matcher failure; decisive when the same error comes from two different call sites. |
 
@@ -356,6 +514,8 @@ The four components are what stop over-collapsing:
 - Two identical `expect(x).to be true` failures in **different specs** stay apart —
   a matcher failure raises at the spec line, so the culprit differs.
 - The same missing record id in twenty specs collapses to one — ids are masked.
+- One broken factory `create`d from ten spec lines collapses to one — the error is
+  raised inside a gem, and the `Failure/Error:` echo of each call site is not identity.
 
 Each group renders one full trace, from the failure carrying the most first-party
 frames, plus every affected example's location. Groups are ordered largest first,
@@ -430,6 +590,110 @@ hold the line:
 Ordering is deterministic: largest cluster first, then most signatures spanned, then
 run order.
 
+## Shared code paths
+
+Both layers above read the failure **message**. Neither can relate two failures whose
+messages have nothing in common — and the most valuable question is not "which
+failures look alike" but **"which line of my code is behind the most failures"**.
+
+So a third layer reads the **stack**:
+
+```markdown
+## Shared code paths
+
+First-party lines that more than one signature runs through, most signatures first.
+A measured fact, not a diagnosis: these examples all execute this line. The
+signatures below remain authoritative.
+
+- `app/serializers/order.rb:5` in `call` -- 2 signatures, 12 examples (#1, #4)
+- `app/models/reader.rb:88` in `progress` -- 9 signatures, 21 examples (#2, #3, #5, ...)
+```
+
+A worked case. One missing hash key in a pricing module, reached through three call
+paths from three spec files, produces two signatures:
+
+```text
+KeyError: key not found: :annual
+RSpec::Expectations::ExpectationNotMetError: expected no Exception, got #<KeyError...>
+```
+
+No symptom extractor can relate those — correctly, because their messages really do
+have almost nothing in common. Both stacks run through `app/pricing.rb:4`, and that
+fact was in the backtrace all along.
+
+### What stops it over-reporting
+
+The same two rules as the cluster layer, for the same reasons:
+
+1. **Only first-party frames outside the spec suite, and only the innermost few.**
+   The outer end of a stack is shared plumbing — every request spec passes through
+   `application_controller.rb` — while the inner end is the bug. Depth is
+   `config.code_path_depth`, default 5.
+2. **Two or more distinct signatures, or it is not reported.** One signature crossing
+   a line is already reported as that signature's `app_context`; saying it twice is
+   noise.
+
+And one rule of its own: **it claims nothing about causation.** The section states a
+measured fact — these signatures execute this line — and says so in those words. When
+no line is shared, the section does not appear.
+
+Ordering is deterministic: most signatures spanned, then most examples, then run
+order.
+
+## Relationships (experimental)
+
+**Off by default.** Turn it on with `config.causal_analysis = true`.
+
+Signatures say which failures are *the same failure*. This experimental layer relates
+**signatures to each other**, using only facts it can check, and prints nothing when it
+has nothing to add. It works between signatures as they are: it does not repair a
+signature that has wrongly split one failure in two, and it never names a root cause.
+
+```text
+Relationships between signatures (experimental):
+CAUSAL · HIGH · 6 failures in 4 signatures
+  missing constant Billing::TaxRate (NameError#name)
+  same underlying NameError at app/invoice.rb:8, also raised wrapped as InvoiceJob::Failed
+INDEPENDENT · 5 failures in 5 signatures (no relationship found)
+11/11 failures classified: 6 causal, 0 scope, 5 independent
+```
+
+| Group | What it claims | Evidence it may use |
+|-------|----------------|---------------------|
+| **CAUSAL · HIGH** | These signatures share an origin | one exception object RSpec handed to every example of a failed `before(:context)`; one underlying exception (same class, message and first-party line), bare or wrapped; one missing definition read from the exception itself -- an `ENV` key (`KeyError#receiver` is `ENV`), a constant (`NameError#name`), a method on a class defined in your project; one failing setup step (`before` hook, `let`, `after` hook); one error outside examples repeated across spec files |
+| **SCOPE** | Failures are concentrated here -- *where*, never *why* | `9/11 examples failed in spec/x_spec.rb; 0/529 elsewhere`: at least two otherwise-unlinked signatures in one file or one `type:`, where at least two thirds failed and fewer than one in twenty elsewhere did. No confidence label |
+| **INDEPENDENT** | No relationship was found | Not "proven unrelated" |
+
+`n/n failures classified` means every failure was placed in exactly one of those groups.
+It is bookkeeping, not a count of root causes.
+
+What it knows that raw output does not show: which phase failed (read from the frames
+RSpec itself puts on the stack, before reduction discards them), whether examples ever
+reached their bodies, which exception object they received, and how many examples
+*passed* in the same file.
+
+What it deliberately does not infer:
+
+- a relationship from exception class, message similarity, file, timing or test order;
+- anything from a method missing on `nil` or on a core class -- two nils are not one nil;
+- a shared object outside `before(:context)`: Ruby keeps the first backtrace when an
+  exception instance is raised again, so a reused instance cannot prove anything;
+- concentration per parallel worker: `parallel_tests` assigns files to workers by size,
+  so "every failure ran on worker 2" usually just means "every failing file did".
+
+Weaker observations (one line raising several different messages; one exception instance
+reported by several examples) appear as `hints` in `signal.json` and never change the
+grouping.
+
+**`signal.json`:** when switched on, the report gains an `analysis` block marked
+`"experimental": true`. The rest of `signal.json` keeps its schema 2 contract. Fields
+inside `analysis` may change or disappear before the layer graduates. Do not build
+automation on them yet.
+
+Evaluation: `bundle exec ruby spec/causal/evaluate.rb --verbose` runs a corpus of real
+`rspec` projects with known causes, and scores this layer against the existing ones.
+See [the design notes](docs/design/causal-failure-intelligence.md).
+
 ## Large HTML responses
 
 A request spec expecting one sentence and receiving a Rails exception page produces
@@ -498,10 +762,18 @@ RSpec::Signal.configure do |config|
   config.max_clusters      = 10  # clusters rendered in full (nil = all)
   config.max_cluster_specs = 6   # spec files listed per cluster
 
+  # Shared code paths (see "Shared code paths").
+  config.code_path_depth = 5   # first-party frames indexed, from the raise site in
+  config.max_code_paths  = 5   # paths rendered in full (nil = all)
+
+  # Relationships (experimental, off by default; see "Relationships").
+  config.causal_analysis = false
+
   # Artifacts.
   config.write_json      = true
   config.write_full      = true
   config.write_gitignore = true
+  config.track_history   = true  # history.json, for "since last run"
 
   # Behaviour.
   config.enabled          = true
@@ -597,11 +869,14 @@ everything with `config.redaction_filter`, or turn scrubbing off with
 
 ## Limitations
 
-- **Errors outside examples.** A `before(:suite)` blow-up, or a spec file that fails
-  to load, produces no failed examples. RSpec reports those only through its message
-  stream, which a formatter cannot subscribe to without swallowing every other
-  message. `rspec-signal` reports the count and says where to look, but cannot
-  capture the text.
+- **Errors outside examples are recovered, but from rendered text.** A spec file that
+  fails to load, or a `before(:suite)` blow-up, produces no failed example; RSpec
+  reports it on its message stream instead. `rspec-signal` captures those, parses
+  RSpec's own rendering back apart, and reports them under **Errors outside
+  examples** with the exception, the message, the file and a rerun command — but it
+  is reading formatted text rather than an exception object, so an unusual RSpec
+  rendering could reduce it to a count. Whatever it cannot parse it still re-emits
+  verbatim.
 - **Parallel test runners.** Use `rspec-signal-parallel` for suites launched with
   `parallel_tests` (see [Parallel suites](#parallel-suites-parallel_tests)); it isolates
   each worker and merges the results deterministically. Do not set a per-worker
@@ -626,6 +901,13 @@ everything with `config.redaction_filter`, or turn scrubbing off with
 - **No source packaging.** `rspec-signal` names files and lines; it does not embed
   source code. Agents working inside a repository can open the files themselves, and
   in v1 that is a better division of labour than guessing which snippets to inline.
+- **Shared code paths are a measurement, not a diagnosis.** "These signatures execute
+  this line" is exactly what is claimed and no more. A line every failure happens to
+  pass through — a shared serializer, a base controller — will be reported, and
+  judging whether it is the cause is the reader's job.
+- **Run comparison needs the previous run's `history.json`.** CI that starts from a
+  clean `tmp/` gets no comparison. That degrades to the behaviour of every run before
+  the feature existed; it does not degrade to a wrong answer.
 - **Reduction depends on your suite.** Failures that are already distinct and already
   shallow reduce very little. That is the correct outcome, not a failure of the gem.
 
@@ -661,7 +943,13 @@ grouping and rendering stages be tested directly against synthetic fixtures in
 real `rspec` processes in generated throwaway projects.
 
 Tests deliberately assert on output *size* as well as content, so a future change
-cannot quietly let noise back in.
+cannot quietly let noise back in. `spec/integration/agent_workflow_spec.rb` runs real
+`rspec` processes through the whole loop — run, fix, rerun exactly, verify what
+changed — because a rerun command that reruns the wrong examples cannot be caught
+from inside the process that printed it. `spec/performance` is a guard rather than a
+benchmark: it fails on an order-of-magnitude regression in per-failure cost, on the
+analysis ceasing to be roughly linear, or on the report ceasing to be bounded by the
+number of problems rather than the number of failures.
 
 ## Contributing
 

data/exe/rspec-signal

@@ -2,8 +2,17 @@
 # frozen_string_literal: true
 
 ENV["RSPEC_SIGNAL_QUIET"] = "1"
+require_relative "../lib/rspec/signal/selection"
+ENV[RSpec::Signal::Selection::ENV_KEY] = JSON.generate(RSpec::Signal::Selection.from_arguments(ARGV).to_h)
 arguments = ["--require", "rspec/signal", "--format", "RSpec::Signal::Formatter", *ARGV]
 
+# The wrapper owns the invocation early enough to invalidate a previous report
+# even when RSpec aborts before it can load the formatter.
+unless ARGV.include?("--dry-run") || %w[1 true yes on].include?(ENV.fetch("RSPEC_SIGNAL_DISABLE", "").downcase)
+  require "rspec/signal"
+  RSpec::Signal::Writer.new(RSpec::Signal.configuration).invalidate_current!
+end
+
 # Run the same executable as `bundle exec rspec`. Besides avoiding a parallel
 # implementation of RSpec's CLI, this is important because the executable uses
 # Runner.invoke, which applies RSpec's default spec discovery when no paths are

data/exe/rspec-signal-parallel

@@ -5,6 +5,7 @@ require "English"
 require "fileutils"
 require "securerandom"
 require "tmpdir"
+require_relative "../lib/rspec/signal/selection"
 
 begin
   parallel_rspec = Gem.bin_path("parallel_tests", "parallel_rspec")
@@ -20,10 +21,16 @@ ENV["RUBYLIB"] = [lib, ENV.fetch("RUBYLIB", nil)].compact.join(File::PATH_SEPARA
 ENV["RSPEC_SIGNAL_RUN_ID"] = run_id
 ENV["RSPEC_SIGNAL_RUN_REGISTRY"] = registry
 ENV["RSPEC_SIGNAL_QUIET"] = "1"
+ENV[RSpec::Signal::Selection::ENV_KEY] = JSON.generate(RSpec::Signal::Selection.from_arguments(ARGV).to_h)
 ENV["PARALLEL_TEST_FIRST_IS_1"] = "true"
 signal_options = "--require rspec/signal --format RSpec::Signal::Formatter"
 ENV["SPEC_OPTS"] = [ENV.fetch("SPEC_OPTS", nil), signal_options].compact.join(" ")
 
+unless ARGV.include?("--dry-run") || %w[1 true yes on].include?(ENV.fetch("RSPEC_SIGNAL_DISABLE", "").downcase)
+  require "rspec/signal"
+  RSpec::Signal::Writer.new(RSpec::Signal.configuration).invalidate_current!
+end
+
 system(Gem.ruby, parallel_rspec, *ARGV)
 test_status = $CHILD_STATUS.exitstatus || 1
 
@@ -31,12 +38,12 @@ begin
   require "rspec/signal"
   result = RSpec::Signal::ParallelMerger.new(registry: registry).call
   report = result.report
+  writer = RSpec::Signal::Writer.new(RSpec::Signal.configuration)
+  path = writer.relative(result.write_result.summary_path) if result.write_result.summary_path
   puts
-  puts "#{report.example_count} examples, #{report.failure_count} failures across #{result.workers} workers"
-  puts "rspec-signal: #{report.group_count} exact signatures, #{report.cluster_count} related clusters, " \
-       "#{report.omitted_frames} backtrace frames omitted"
-  if result.write_result.summary_path
-    puts "Report: #{RSpec::Signal::Writer.new(RSpec::Signal.configuration).relative(result.write_result.summary_path)}"
+  terminal = RSpec::Signal::TerminalSummary.new(report, report_path: path, quiet: true, workers: result.workers)
+  terminal.lines.each do |line|
+    puts line
   end
   if result.missing.any?
     warn "rspec-signal: warning: #{result.missing.size} worker artifacts were missing; report is incomplete"