rspec-tracer 2.0.0.pre.2 → 2.0.0.rc.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (23) hide show
  1. checksums.yaml +4 -4
  2. data/CHANGELOG.md +121 -0
  3. data/README.md +204 -151
  4. data/lib/rspec_tracer/cli/blast_radius.rb +318 -0
  5. data/lib/rspec_tracer/cli/cache_clear.rb +5 -2
  6. data/lib/rspec_tracer/cli/cache_info.rb +4 -2
  7. data/lib/rspec_tracer/cli/doctor.rb +26 -3
  8. data/lib/rspec_tracer/cli/explain.rb +180 -63
  9. data/lib/rspec_tracer/cli/report_open.rb +13 -5
  10. data/lib/rspec_tracer/cli/snapshot_helpers.rb +119 -0
  11. data/lib/rspec_tracer/cli.rb +116 -13
  12. data/lib/rspec_tracer/configuration.rb +2 -3
  13. data/lib/rspec_tracer/defaults.rb +7 -1
  14. data/lib/rspec_tracer/line_stub.rb +3 -2
  15. data/lib/rspec_tracer/logger.rb +27 -8
  16. data/lib/rspec_tracer/reporters/html/package-lock.json +213 -200
  17. data/lib/rspec_tracer/reporters/html/package.json +1 -1
  18. data/lib/rspec_tracer/rspec/metadata.rb +1 -1
  19. data/lib/rspec_tracer/rspec/runner_hook.rb +19 -2
  20. data/lib/rspec_tracer/source_file.rb +3 -3
  21. data/lib/rspec_tracer/tracker/env_matcher.rb +5 -5
  22. data/lib/rspec_tracer/version.rb +1 -1
  23. metadata +11 -9
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 552b92475298cfa692f8748b7b7097b12a82a2e8b1003848d1194fe560c0b385
4
- data.tar.gz: 34a9d94756083aaf46220460c4d10002d30992703c1aad0e417640ad5b2d79ae
3
+ metadata.gz: 0b2339c836852122ba3d8e4ca1e36a430ed9fe3950de724a74cc608f1dd8a524
4
+ data.tar.gz: e76654936dae948f7ca4b20cf1feab82aa977c63608c69f0051776b36fb3b2b8
5
5
  SHA512:
6
- metadata.gz: 57b510bf3f8da2071a72ee222d598e9ce8042b5745cec84e053ea7b7db87e203e73829f890f105d934901835813cee66290ff0b24a07f278b1821c3b75febdb7
7
- data.tar.gz: 57adb9424e1bdc3775294de9573726f3b80bea35615d6a41410985724d930faa00c984bf3c9b637297565fa11bf72c938bb92ce8464b7968c9be9fa5b476b2f2
6
+ metadata.gz: 9dfec94b0a6674ac1921dae16611342df860cfd4d988b9c9761c84cd1d1b09060a40501a86cedb09fd8f87cc597aa2bda2650ba0702181756eaab906c1245869
7
+ data.tar.gz: 3adc8e2c46a2a400a03d672f15b6e706480bd83b17cc201b3e83ee9f61adde36e5cce7f76b1ae69fbd9e1ad693f2bfb5c762fedd3f7da8998e59b980a8f77d95
data/CHANGELOG.md CHANGED
@@ -1,3 +1,124 @@
1
+ ## [2.0.0.rc.1] - 2026-07-13
2
+
3
+ The first release candidate of the 2.0 line. Fixes the
4
+ duplicate-example drop discarding spec files whose examples live in
5
+ nested `describe` groups
6
+ ([#262](https://github.com/avmnu-sng/rspec-tracer/issues/262), fixed
7
+ in [#265](https://github.com/avmnu-sng/rspec-tracer/pull/265)); adds
8
+ a `blast-radius` CLI sub-command
9
+ ([#230](https://github.com/avmnu-sng/rspec-tracer/issues/230)) and an
10
+ `explain --not-run` flag
11
+ ([#231](https://github.com/avmnu-sng/rspec-tracer/issues/231)), both
12
+ shipped in [#273](https://github.com/avmnu-sng/rspec-tracer/pull/273);
13
+ and repositions the README and docs around the soundness model, CI
14
+ cost framing, and maintenance policy
15
+ ([#266](https://github.com/avmnu-sng/rspec-tracer/pull/266), closing
16
+ [#223](https://github.com/avmnu-sng/rspec-tracer/issues/223)–[#229](https://github.com/avmnu-sng/rspec-tracer/issues/229)).
17
+ CI and maintenance hardening alongside: exact dev-tool gem pins
18
+ ([#267](https://github.com/avmnu-sng/rspec-tracer/pull/267)), a lint
19
+ gate against internal planning vocabulary shipping in public files
20
+ ([#272](https://github.com/avmnu-sng/rspec-tracer/pull/272)), a weekly
21
+ non-gating Ruby-HEAD canary
22
+ ([#269](https://github.com/avmnu-sng/rspec-tracer/pull/269)), a
23
+ `release:preflight` task
24
+ ([#270](https://github.com/avmnu-sng/rspec-tracer/pull/270)), removal
25
+ of the unused 1.x sample-project tree
26
+ ([#268](https://github.com/avmnu-sng/rspec-tracer/pull/268)), and
27
+ security dependency updates across the sample Rails app and CI
28
+ actions ([#257](https://github.com/avmnu-sng/rspec-tracer/pull/257),
29
+ [#259](https://github.com/avmnu-sng/rspec-tracer/pull/259)–[#261](https://github.com/avmnu-sng/rspec-tracer/pull/261),
30
+ [#263](https://github.com/avmnu-sng/rspec-tracer/pull/263),
31
+ [#264](https://github.com/avmnu-sng/rspec-tracer/pull/264)).
32
+
33
+ The cache `schema_version` is unchanged at `5`: a pre.2 cache carries
34
+ forward warm -- no cold run on this upgrade. See
35
+ [`UPGRADING.md`](UPGRADING.md).
36
+
37
+ ### Added
38
+
39
+ - **Soundness model documentation.** New ARCHITECTURE.md section
40
+ classifying what the tracer guarantees (content digests, explicit
41
+ declarations, env snapshots), where it is deliberately
42
+ conservative, where it relies on heuristics, and what it cannot
43
+ observe at all -- with the `tracks:` DSL as the escape hatch
44
+ ([#226](https://github.com/avmnu-sng/rspec-tracer/issues/226)).
45
+ - **Cost framing for CI.** New README section with measured cold/warm
46
+ timings and a formula for estimating CI-minute and dollar savings
47
+ from your own suite's numbers and your provider's rate
48
+ ([#227](https://github.com/avmnu-sng/rspec-tracer/issues/227)).
49
+ - **Maintenance and Ruby EOL policy.** New README Maintenance
50
+ section: each Ruby is supported until at least upstream EOL plus
51
+ 6 months, with a per-version table
52
+ ([#229](https://github.com/avmnu-sng/rspec-tracer/issues/229)).
53
+ - **COOKBOOK recipes for flaky-test detection and file-to-test
54
+ dependency mapping**, placed ahead of the acceleration recipes
55
+ ([#224](https://github.com/avmnu-sng/rspec-tracer/issues/224)).
56
+ - **`blast-radius` CLI sub-command.** `bundle exec rspec-tracer
57
+ blast-radius <file> [<file> ...]` reports how many examples a
58
+ change to each file would re-run (`450 examples across 12 spec
59
+ files`), with `--list` to enumerate them (location + description)
60
+ and `--json` for tooling. Multiple files compose with a
61
+ deduplicated total, so `git diff --name-only main | xargs bundle
62
+ exec rspec-tracer blast-radius` shows what a PR invalidates;
63
+ untracked paths report as such and exit 0 to keep that pipeline
64
+ unbroken. Whole-suite invalidators and boot-set files report
65
+ `re-runs all N examples`
66
+ ([#230](https://github.com/avmnu-sng/rspec-tracer/issues/230)).
67
+ - **`explain --not-run` flag.** The skip-side view of `explain`:
68
+ whether the example was skipped on the last run (and the
69
+ derivation of why no run trigger fired), its last recorded
70
+ status, and what would make it run on the next invocation, plus
71
+ the tracked dependency files
72
+ ([#231](https://github.com/avmnu-sng/rspec-tracer/issues/231)).
73
+ - **Doctor CI-environment line.** `rspec-tracer doctor` prints an
74
+ INFO line naming the CI env var it detected and pointing at the
75
+ cache-persistence recipes in
76
+ [`docs/CI_RECIPES.md`](docs/CI_RECIPES.md); never affects the
77
+ exit status
78
+ ([#228](https://github.com/avmnu-sng/rspec-tracer/issues/228)).
79
+
80
+ ### Changed
81
+
82
+ - **README restructured as a landing page**
83
+ ([#223](https://github.com/avmnu-sng/rspec-tracer/issues/223)):
84
+ measured, attributed numbers, quick start, safety summary, and
85
+ data-stays-local positioning
86
+ ([#225](https://github.com/avmnu-sng/rspec-tracer/issues/225)) up
87
+ front; the input-taxonomy depth moved to ARCHITECTURE.md and the
88
+ eager-load precision guidance to COOKBOOK.md; per-provider CI
89
+ cache recipes ([`docs/CI_RECIPES.md`](docs/CI_RECIPES.md)) linked
90
+ from the quick start
91
+ ([#228](https://github.com/avmnu-sng/rspec-tracer/issues/228)).
92
+ - **Gem summary and description repositioned** to lead with
93
+ flaky-test detection and dependency mapping, with
94
+ re-run-only-what-changed as the closing step
95
+ ([#224](https://github.com/avmnu-sng/rspec-tracer/issues/224)).
96
+
97
+ ### Fixed
98
+
99
+ - **CLI degrades cleanly when the project config itself is broken.**
100
+ A `.rspec-tracer` that raises at load time (including a
101
+ `SyntaxError`, which is not a `StandardError`) used to crash every
102
+ `rspec-tracer` sub-command with the raw backtrace plus a second
103
+ `NoMethodError` backtrace from the library's `at_exit` hook firing
104
+ against the half-loaded module. The binary now prints a one-line
105
+ `could not load configuration` message and exits 1, and `--help` /
106
+ `--version` answer without booting the config at all
107
+ ([#273](https://github.com/avmnu-sng/rspec-tracer/pull/273)).
108
+
109
+ - **Duplicate-example drop no longer discards nested spec files**
110
+ ([#262](https://github.com/avmnu-sng/rspec-tracer/issues/262)).
111
+ With at least one pair of colliding example identities anywhere in
112
+ the suite, the drop path rebuilt RSpec's top-level group list by
113
+ requiring each top-level group to directly own surviving examples,
114
+ so every spec file whose examples live inside nested `describe`
115
+ blocks was removed wholesale (a 399-example suite with one
116
+ colliding pair ran only 68 examples; with
117
+ `fail_on_duplicates false` that under-run exited zero). The group
118
+ filter now maps each surviving example back to its top-level
119
+ group, so exactly the colliding examples are dropped — the
120
+ behavior UPGRADING.md documents.
121
+
1
122
  ## [2.0.0.pre.2] - 2026-05-16
2
123
 
3
124
  Bug-fix + interop release after the field-test pass that followed
data/README.md CHANGED
@@ -5,25 +5,92 @@
5
5
  [![codecov](https://codecov.io/gh/avmnu-sng/rspec-tracer/branch/main/graph/badge.svg)](https://codecov.io/gh/avmnu-sng/rspec-tracer)
6
6
  [![Docs](https://img.shields.io/badge/docs-online-blue)](https://avmnu-sng.github.io/rspec-tracer/)
7
7
 
8
- **RSpec Tracer** is a specs dependency analyzer, flaky-test detector,
9
- test accelerator, and coverage reporter for RSpec. It records the
10
- inputs every example consumes — Ruby files (via `Coverage`), file
11
- I/O (via prepended `File` / `IO` / `YAML` / `JSON` hooks), Rails
12
- template + AR notifications, and user-declared globs — then re-runs
13
- only the examples whose inputs changed since the last run.
8
+ **Test-dependency intelligence for RSpec: detect flaky tests, map code
9
+ coupling, and -- when you are ready -- re-run only what changed.**
10
+
11
+ rspec-tracer records the inputs it can observe for each RSpec
12
+ example -- Ruby files, file I/O, Rails templates and queries,
13
+ declared globs and ENV branches -- and turns that record into a
14
+ flaky-test detector, a per-example dependency map, and optional CI
15
+ acceleration. The flaky report and the dependency map never skip a
16
+ test: value before trust.
17
+
18
+ **Measured, not promised.** In the maintainer's May 2026 field test
19
+ (rspec-tracer 2.0.0.pre.1, Apple M2 Max), Mastodon's `spec/lib` --
20
+ 1,234 examples, Rails 8.1.3, Ruby 3.4.8 -- ran in 116.4 s cold and
21
+ 17.0 s warm: **6.85x**, with the tracer's own recording overhead
22
+ included in both runs. "Cold" here is the tracer's own first run,
23
+ not plain RSpec: recording costs roughly 1.6x plain RSpec on this
24
+ repository's Rails benchmark fixture (approximate), and no
25
+ tracer-free Mastodon baseline was measured, so your gain over a
26
+ suite without the tracer will be smaller than cold minus warm.
27
+ Suite shape matters: across the same field tests, warm-run skip
28
+ rates ranged from 31% (a suite carrying 177 pre-existing failures --
29
+ failed examples are never skipped) to 100%. Measure your own suite
30
+ before you rely on these numbers.
31
+
32
+ **Conservative by design, honest about its limits.** The tracer
33
+ never skips failed, flaky, or pending examples, nor any example
34
+ whose recorded inputs changed; when a recorded input is ambiguous,
35
+ it re-runs. Real blind spots exist -- runtime metaprogramming and
36
+ monkey-patches are invisible to it; the `tracks:` DSL is the escape
37
+ hatch -- and the [soundness model](ARCHITECTURE.md#soundness-model)
38
+ catalogues the known blind spots. Shadow mode (run everything,
39
+ report what would have been skipped) lands in v2.0.0.rc.2, so you
40
+ can verify the tracer on your own data before trusting it with a
41
+ single skip.
42
+
43
+ **Your data never leaves your infrastructure.** Everything runs inside
44
+ your test process and your CI cache -- no SaaS backend, no telemetry,
45
+ no per-seat pricing.
14
46
 
15
- It never skips:
16
- - **Failed**, **flaky**, or **pending** examples.
17
- - Examples whose dependent inputs changed (the whole point).
47
+ ## Quick start
18
48
 
19
- 📚 **Full documentation site**: [avmnu-sng.github.io/rspec-tracer](https://avmnu-sng.github.io/rspec-tracer/) — YARD API reference, sample HTML report, cookbook, and per-version coverage report.
49
+ Requires Ruby 3.1+ and rspec-core 3.12+ (Rails 7.0+ when using Rails;
50
+ Rails 8.0 needs Ruby 3.2+; JRuby 9.4 supported). Every CI-gated Ruby
51
+ is supported until at least six months past its upstream EOL -- see
52
+ [Maintenance](#maintenance).
20
53
 
21
- For a complete account of the input taxonomy and how the engine fits
22
- together, read [`ARCHITECTURE.md`](ARCHITECTURE.md).
54
+ ```ruby
55
+ # Gemfile -- 2.0 is in pre-release; pin explicitly until 2.0.0 final
56
+ gem 'rspec-tracer', '= 2.0.0.rc.1', group: :test, require: false
57
+ ```
58
+
59
+ Add the tracer's working directories to your `.gitignore` before the
60
+ first run:
61
+
62
+ ```
63
+ rspec_tracer.lock
64
+ rspec_tracer_cache/
65
+ rspec_tracer_coverage/
66
+ rspec_tracer_report/
67
+ ```
68
+
69
+ ```ruby
70
+ # Top of spec_helper.rb / rails_helper.rb, before any application
71
+ # code. With SimpleCov, start SimpleCov first -- load order is part
72
+ # of the contract.
73
+ require 'rspec_tracer'
74
+ RSpecTracer.start
75
+ ```
76
+
77
+ Run the suite twice. The second run skips the examples whose recorded
78
+ inputs are unchanged and prints a per-reason breakdown of what
79
+ re-ran; open `rspec_tracer_report/index.html` to audit every
80
+ per-example decision. Not ready to let it skip anything? Set
81
+ `RSPEC_TRACER_RUN_ALL_EXAMPLES=true` (or `run_all_examples true`
82
+ inside the `RSpecTracer.configure` block in `.rspec-tracer`) --
83
+ every example still runs while the flaky report and the dependency
84
+ maps accumulate data.
85
+
86
+ Setting up CI? [docs/CI_RECIPES.md](docs/CI_RECIPES.md) has
87
+ copy-paste cache recipes for CircleCI, GitLab CI, Buildkite, and
88
+ Heroku CI, plus a link to the canonical GitHub Actions workflow.
23
89
 
24
90
  ## Table of contents
25
91
 
26
92
  - [Quick start](#quick-start)
93
+ - [What it saves](#what-it-saves)
27
94
  - [How it works](#how-it-works)
28
95
  - [Per-example `tracks:` DSL](#per-example-tracks-dsl)
29
96
  - [Rails quick start](#rails-quick-start)
@@ -34,126 +101,71 @@ together, read [`ARCHITECTURE.md`](ARCHITECTURE.md).
34
101
  - [SimpleCov interop](#simplecov-interop)
35
102
  - [FAQ + comparison](#faq--comparison)
36
103
  - [Reports](#reports)
104
+ - [Maintenance](#maintenance)
37
105
  - [Documentation + coverage](#documentation--coverage)
38
106
  - [Help and community](#help-and-community)
39
- - [Section anchor map (1.x → 2.0)](#section-anchor-map-1x--20)
40
-
41
- ## Quick start
42
-
43
- Requires Ruby 3.1+ and rspec-core 3.12+. Rails 7.0+ when using Rails;
44
- Rails 8.0 needs Ruby 3.2+. JRuby 9.4 is supported.
45
-
46
- 1. Add the gem:
47
-
48
- ```ruby
49
- # 2.0 is in pre-release. Pin to the pre-release version explicitly;
50
- # switch to '~> 2.0' once 2.0.0 final ships.
51
- gem 'rspec-tracer', '= 2.0.0.pre.2', group: :test, require: false
52
- ```
53
-
54
- `bundle install` will resolve the pre-release version. You can
55
- also install ad-hoc with `gem install rspec-tracer --pre`.
56
-
57
- 2. Add the canonical directories to your `.gitignore`:
58
-
59
- ```
60
- rspec_tracer.lock
61
- rspec_tracer_cache/
62
- rspec_tracer_coverage/
63
- rspec_tracer_report/
64
- ```
65
-
66
- 3. Load and start at the very top of `spec_helper.rb` /
67
- `rails_helper.rb`, **before any application code:**
68
-
69
- ```ruby
70
- require 'rspec_tracer'
71
- RSpecTracer.start
72
- ```
73
-
74
- With SimpleCov, start SimpleCov first (load order is part of the
75
- contract):
76
-
77
- ```ruby
78
- require 'simplecov'
79
- SimpleCov.start
80
- require 'rspec_tracer'
81
- RSpecTracer.start
82
- ```
83
-
84
- 4. Run your suite. After the run, open
85
- `rspec_tracer_report/index.html` for the HTML report, and
86
- `rspec_tracer_report/report.json` for machine-readable output.
87
-
88
- The terminal prints a one-line summary with cache size and the
89
- reasons examples re-ran:
90
-
91
- ```
92
- rspec-tracer: 1,820 examples · 42 re-run · 1,778 skipped (97% cached)
93
- by reason: 38 Files changed · 4 Failed previously
94
- cache: rspec_tracer_cache (14.4 MiB; +6.7 KiB vs prev run)
95
- ```
107
+ - [Section anchor map (1.x -> 2.0)](#section-anchor-map-1x--20)
108
+
109
+ ## What it saves
110
+
111
+ All timings are wall-clock and measured, not projected: maintainer
112
+ field tests, May 2026, Apple M2 Max, rspec-tracer 2.0.0.pre.1. In
113
+ every row, "cold" is the tracer's own first, recording run -- it
114
+ costs more than plain RSpec (roughly 1.6x on this repository's Rails
115
+ benchmark fixture; approximate), and no tracer-free baseline was
116
+ measured for these projects -- so your savings versus a suite
117
+ without the tracer will be smaller than cold minus warm.
118
+
119
+ | Project measured | Examples | Cold run | Warm run | Warm skip rate | Saved per warm run |
120
+ |---|---|---|---|---|---|
121
+ | Mastodon `spec/lib` (Rails 8.1.3, Ruby 3.4.8) | 1,234 | 116.4 s | 17.0 s | 98% (1,212 skipped) | ~99 s |
122
+ | thoughtbot/clearance (Rails 8 engine + dummy app) | 258 | 11.1 s | 1.6 s | 100% | ~9.5 s |
123
+ | publify (carries 177 pre-existing failures; failed examples are never skipped) | 257 | 6.7 s | 4.8 s | 31% | ~1.9 s |
124
+
125
+ The Mastodon warm run is the zero-change best case; in the same
126
+ field test, editing one file re-ran 68 of 1,234 examples in 19.0 s.
127
+ Laptop rows understate CI savings: CI runners are slower than the
128
+ development machine, so the seconds saved per run are typically
129
+ larger there -- the skip ratios are what transfer.
130
+
131
+ **Worked examples** (every input is an assumption you should replace
132
+ with your own). For any suite:
133
+
134
+ `(cold seconds - warm seconds) x runs/day / 60 x $/CI-minute x 30 = $/month per full-suite job`
135
+
136
+ At $0.006/CI-minute (illustrative -- GitHub-hosted Linux list price
137
+ at the time of writing; check your provider's current rate) and 50
138
+ CI runs/day on one job:
139
+
140
+ - Best case (a Mastodon-sized suite saving ~99 s per warm run):
141
+ ~83 CI-minutes/day, about $15/month per job.
142
+ - Low-skip case (a publify-shaped suite saving ~1.9 s per warm run
143
+ at 31% skip): about $0.29/month per job.
144
+
145
+ Multiply by the number of jobs that each run the full suite (for
146
+ example, a Ruby-version matrix); jobs that split one suite across
147
+ workers share the saving rather than multiply it.
148
+
149
+ Conservative assumptions: 50 runs/day and the dollar rate are stated
150
+ assumptions, not measurements; your skip rate may be far lower (see
151
+ the 31% row -- suites with many failing examples skip less, by
152
+ design); remote-cache download time on fresh CI workers is not
153
+ subtracted (measured end-to-end remote-cache gain on clearance was
154
+ still ~4.6x).
96
155
 
97
156
  ## How it works
98
157
 
99
- Every test is a pure function of its inputs; the tracker's job is to
100
- identify every input and hash it. Inputs come from six buckets:
101
-
102
- 1. **Ruby-executed source** — observed via Ruby's `Coverage` module.
103
- 2. **File I/O** (`File.read`, `YAML.load_file`, etc.) — observed via
104
- `Module#prepend` hooks.
105
- 3. **Framework events** — Rails template renders + (opt-in) AR
106
- schema-touching queries via `ActiveSupport::Notifications`.
107
- 4. **Declared globs** — what you tell the tracker to track via
108
- `track_files` / `tracks:` / `track_rails_defaults`.
109
- 5. **Whole-suite invalidators** — `Gemfile.lock`, `.ruby-version`,
110
- `.rspec-tracer`, the rspec-tracer gem version itself.
111
- 6. **Truly unobservable inputs** — env-var branches, refinements
112
- in unexecuted files. Use `track_env` / `tracks: { env: ... }`
113
- to declare them; the tracker can't auto-detect them.
114
-
115
- A digest of every observed input is stored per-example. The next run
116
- loads the cached digest set, recomputes per-input digests, and skips
117
- examples whose inputs are unchanged.
118
-
119
- Read [`ARCHITECTURE.md`](ARCHITECTURE.md) for the full layer
120
- structure (Tracker / Storage / RemoteCache / Reporters), data flows,
121
- and extension protocols.
122
-
123
- ### Per-example precision under Rails `config.eager_load`
124
-
125
- A precision tradeoff worth knowing before you run a suite that uses
126
- Rails:
127
-
128
- - **`config.eager_load = false`** in test env — per-example precision
129
- works as designed. Files autoloaded during an example land in
130
- per-example deps; editing one re-runs only the examples that
131
- touched it.
132
- - **`config.eager_load = true`** (Rails default for CI tests to
133
- mirror prod) — all `app/` files load at boot and land in the
134
- whole-suite invalidator set. Editing any `app/` file re-runs
135
- every example. **Safe** (never misses a dep) but **coarser** than
136
- per-example attribution.
137
-
138
- If your CI suite runs with `eager_load = true` and your `app/`-edit
139
- cycle is bottlenecked by whole-suite re-runs, set `config.eager_load
140
- = false` in test env to recover per-example precision. A future 2.1
141
- enhancement (working name: `track_class_attribution`) will install
142
- class-dispatch tracking to trim the invalidator scope under
143
- eager-loaded test environments — see
144
- [`CHANGELOG.md`](CHANGELOG.md) "Deferred to 2.1" for the planned
145
- contract.
146
-
147
- **Rails engines:** a gem-loaded engine's own `lib/` files are
148
- `require`d at gem-load time via the Gemfile.lock cascade and land
149
- in the boot set **regardless of `eager_load`**. Editing them
150
- re-runs every example — same SAFE-but-coarser shape as the
151
- `eager_load = true` case above. The rationale (closing the
152
- constants-lookup blind spot) lives in
153
- [`lib/rspec_tracer/tracker/loaded_files_tracker.rb`](lib/rspec_tracer/tracker/loaded_files_tracker.rb).
154
- Teams that want tighter per-example precision and accept the blind
155
- spot can set `transitive_load_tracking false` — see
156
- [`COOKBOOK.md`](COOKBOOK.md) recipe 2 for the trade-off.
158
+ Each example's observed inputs -- executed Ruby source (via
159
+ `Coverage`), file I/O (via `Module#prepend` hooks), Rails framework
160
+ events, declared globs and env vars, and whole-suite invalidators
161
+ like `Gemfile.lock` -- are digested and stored per example. The next
162
+ run recomputes the digests and skips examples whose recorded inputs
163
+ are unchanged. The full input taxonomy lives in
164
+ [ARCHITECTURE.md](ARCHITECTURE.md#input-taxonomy); what each input
165
+ kind does and does not guarantee is the
166
+ [soundness model](ARCHITECTURE.md#soundness-model); the Rails
167
+ `config.eager_load` precision trade-off is covered by the Rails
168
+ recipe in [COOKBOOK.md](COOKBOOK.md).
157
169
 
158
170
  ## Per-example `tracks:` DSL
159
171
 
@@ -275,6 +287,13 @@ rm -f rspec_tracer.lock && bundle exec parallel_rspec
275
287
 
276
288
  ## Configuring CI
277
289
 
290
+ Copy-paste cache recipes for CircleCI, GitLab CI, Buildkite, and
291
+ Heroku CI live in [docs/CI_RECIPES.md](docs/CI_RECIPES.md); the
292
+ canonical GitHub Actions workflow is
293
+ [`.github/workflows/example-tracer-cache.yml`](.github/workflows/example-tracer-cache.yml).
294
+ The rest of this section covers the remote-cache rake flow and the
295
+ cache-key pattern all the recipes share.
296
+
278
297
  The 1.x flow is preserved bit-for-bit. In your project's `Rakefile`:
279
298
 
280
299
  ```ruby
@@ -324,10 +343,11 @@ The cache step:
324
343
  ${{ runner.os }}-
325
344
  ```
326
345
 
327
- The 4-component cache key (`runner.os` + Ruby version + the
328
- rspec-tracer gem's own version + your project's Gemfile lock)
329
- invalidates when something would make the previous run's decisions
330
- incorrect: native gem binaries differ across runner OSes, Ruby ABI
346
+ The 4-component cache key (`runner.os` + Ruby version + your
347
+ project's Gemfile lock -- which also pins the rspec-tracer gem's
348
+ version -- + a fixed name suffix) invalidates when something would
349
+ make the previous run's decisions incorrect: native gem binaries
350
+ differ across runner OSes, Ruby ABI
331
351
  changes invalidate native extensions, a tracer upgrade can change the
332
352
  cache schema, and gem-set drift is the most common cache-staleness
333
353
  trigger.
@@ -376,16 +396,17 @@ Or override per-run via env: `RSPEC_TRACER_STORAGE=sqlite`.
376
396
 
377
397
  ## Command-line tools
378
398
 
379
- `rspec-tracer` exposes five sub-commands. Run them via Bundler so the
399
+ `rspec-tracer` exposes six sub-commands. Run them via Bundler so the
380
400
  gem's executable resolves cleanly without needing `bundle binstubs
381
401
  rspec-tracer` first:
382
402
 
383
403
  ```sh
384
- bundle exec rspec-tracer doctor # diagnose config + environment
385
- bundle exec rspec-tracer cache:info # size, last run, invalidation stats
386
- bundle exec rspec-tracer cache:clear # rm cache dirs
387
- bundle exec rspec-tracer report:open # open the HTML report
388
- bundle exec rspec-tracer explain <id> # why is <example_id> scheduled to (re-)run?
404
+ bundle exec rspec-tracer doctor # diagnose config + environment
405
+ bundle exec rspec-tracer cache:info # size, last run, invalidation stats
406
+ bundle exec rspec-tracer cache:clear # rm cache dirs
407
+ bundle exec rspec-tracer report:open # open the HTML report
408
+ bundle exec rspec-tracer explain <id> # why is <example_id> scheduled to (re-)run?
409
+ bundle exec rspec-tracer blast-radius <file> # which examples re-run if <file> changes?
389
410
  ```
390
411
 
391
412
  Generated binstubs (`bin/rspec-tracer …`) work too once you've run
@@ -437,6 +458,14 @@ Compose them: shard with Knapsack, then skip with rspec-tracer.
437
458
  A composition smoke spec (`spec/regressions/knapsack_coexistence_spec.rb`)
438
459
  proves they coexist.
439
460
 
461
+ **Datadog Test Optimization / Buildkite Test Analytics / CircleCI
462
+ Test Insights?** Those are hosted services: your test metadata ships
463
+ to a vendor backend, with the pricing and procurement that implies.
464
+ rspec-tracer runs entirely inside your test process -- the cache and
465
+ reports are plain files in your repo, S3 bucket, or Redis. No SaaS
466
+ backend, no telemetry, no per-seat pricing; keeping it that way is a
467
+ standing commitment in [ROADMAP.md](ROADMAP.md)'s "Won't do" list.
468
+
440
469
  **`RSpec::Retry` / `RSpec::Rerun`?** Retries flaky failures.
441
470
  rspec-tracer detects flaky examples (same inputs, different outcomes)
442
471
  and refuses to skip them on the next run. Compose: retry to make a
@@ -455,23 +484,47 @@ taxonomy" for the rule.
455
484
  ## Reports
456
485
 
457
486
  After the run, `rspec_tracer_report/index.html` opens five HTML
458
- reports:
459
-
460
- - **All Examples** — basic test info (id, status, duration, the inputs
461
- the example consumed).
462
- - **Duplicate Examples** — pairs RSpec couldn't uniquely identify
487
+ reports. The first two deliver value even if you never let the
488
+ tracer skip a single example:
489
+
490
+ - **Flaky Examples** -- examples that failed on one run and passed on
491
+ a later one, the canonical intermittence signal: a flake list your
492
+ suite builds passively from the runs you were already doing. Once
493
+ flagged, the tracer refuses to skip them on subsequent runs.
494
+ - **Files Dependency** -- "if I change this file, which tests run?"
495
+ The file-to-test map that makes refactors, reviews, and deletions
496
+ safer -- the report unique to rspec-tracer.
497
+ - **Examples Dependency** -- the per-example inverse: "what does this
498
+ test depend on?"
499
+ - **All Examples** -- basic test info (id, status, duration, the
500
+ inputs the example consumed).
501
+ - **Duplicate Examples** -- pairs RSpec couldn't uniquely identify
463
502
  (file:line collisions; only appears when duplicates are present).
464
- - **Flaky Examples** — examples that passed after previously failing
465
- without any dependency change. The tracker refuses to skip these
466
- on subsequent runs.
467
- - **Examples Dependency** — per-example file dependency lists; the
468
- "what does this test depend on?" view.
469
- - **Files Dependency** — the inverse: "what tests run if I change
470
- this file?" The unique-to-rspec-tracer report that makes refactors
471
- safer.
472
503
 
473
504
  Plus a machine-readable `rspec_tracer_report/report.json` for CI
474
- dashboards and the terminal one-liner shown in [Quick start](#quick-start).
505
+ dashboards, and a terminal summary with a per-reason breakdown after
506
+ every run (see [Quick start](#quick-start)).
507
+
508
+ ## Maintenance
509
+
510
+ Every CI-gated Ruby is supported until **at least** six months past
511
+ its upstream end-of-life date. That is a floor, not a drop date:
512
+ versions can stay supported longer, and Ruby 3.1 currently is.
513
+ Upstream EOL dates are published on the official
514
+ [Ruby maintenance schedule](https://www.ruby-lang.org/en/downloads/branches/);
515
+ that page is authoritative for the dates below.
516
+
517
+ | Version | CI-gated | Status |
518
+ |---------|----------|--------|
519
+ | Ruby 3.1 | Yes | Supported. Upstream EOL was 2025-03-26, so the committed six-month window has lapsed -- and 3.1 is still supported today because the policy is a minimum, not a schedule. It remains the floor for the 2.0 release line. |
520
+ | Ruby 3.2 | Yes | Supported. Upstream EOL was 2026-04-01; supported until at least 2026-10-01, and potentially longer -- the policy is a minimum. |
521
+ | Ruby 3.3 | Yes | Supported. Upstream EOL is expected 2027-03-31; supported until at least six months past the published EOL date. |
522
+ | Ruby 3.4 | Yes | Supported, until at least six months past its upstream EOL date (not yet announced). |
523
+ | Ruby 4.0 | Yes | Supported, until at least six months past its upstream EOL date (not yet announced). |
524
+ | JRuby 9.4 | Yes | Supported. JRuby support tracks the JRuby project's own maintenance schedule, not the Ruby-Core EOL dates above. |
525
+
526
+ TruffleRuby and Ruby head are best-effort: no CI gate, but issue
527
+ reports are welcome and get investigated.
475
528
 
476
529
  ## Documentation + coverage
477
530