testprune 0.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +7 -0
- data/LICENSE +21 -0
- data/README.md +552 -0
- data/assets/quickstart.svg +70 -0
- data/exe/testprune +6 -0
- data/lib/testprune/adapters/minitest.rb +42 -0
- data/lib/testprune/adapters/rspec.rb +31 -0
- data/lib/testprune/analysis.rb +53 -0
- data/lib/testprune/autostart.rb +40 -0
- data/lib/testprune/baseline.rb +23 -0
- data/lib/testprune/cli.rb +136 -0
- data/lib/testprune/configuration.rb +86 -0
- data/lib/testprune/coverage_delta.rb +82 -0
- data/lib/testprune/duplication_detector.rb +203 -0
- data/lib/testprune/footprint.rb +87 -0
- data/lib/testprune/patch_writer.rb +117 -0
- data/lib/testprune/recorder.rb +102 -0
- data/lib/testprune/report.rb +127 -0
- data/lib/testprune/runner.rb +76 -0
- data/lib/testprune/safety_check.rb +45 -0
- data/lib/testprune/savings_estimator.rb +30 -0
- data/lib/testprune/semantic_map.rb +185 -0
- data/lib/testprune/test_body.rb +61 -0
- data/lib/testprune/version.rb +5 -0
- data/lib/testprune.rb +18 -0
- metadata +90 -0
checksums.yaml
ADDED
|
@@ -0,0 +1,7 @@
|
|
|
1
|
+
---
|
|
2
|
+
SHA256:
|
|
3
|
+
metadata.gz: d3484c32e59411fd94af756cfaec78df0a07c6ce0f7f7dfaf16cf82c6745d84f
|
|
4
|
+
data.tar.gz: 47e74f078c45818060d393c08473a66275fd5a002e750a24ed02ca6b47d8f412
|
|
5
|
+
SHA512:
|
|
6
|
+
metadata.gz: 0c9cc56ac84e87e994165c3abce1238a84924f3ecf980042ee876b0c0b6fa551021780d23a903494298f37bd248c546d1655c282285c98f6a2111ba37a9ab67b
|
|
7
|
+
data.tar.gz: f6c4748cde9a7e89dbaaeb67cc609e8c17de9173b75c723d0f0c4f22f84e77034b52a51187233418778920ddec58953585942002b10771a8ce726201632522fd
|
data/LICENSE
ADDED
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
MIT License
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2025 Seth MacPherson
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
+
in the Software without restriction, including without limitation the rights
|
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
+
furnished to do so, subject to the following conditions:
|
|
11
|
+
|
|
12
|
+
The above copyright notice and this permission notice shall be included in all
|
|
13
|
+
copies or substantial portions of the Software.
|
|
14
|
+
|
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
21
|
+
SOFTWARE.
|
data/README.md
ADDED
|
@@ -0,0 +1,552 @@
|
|
|
1
|
+
# testprune
|
|
2
|
+
|
|
3
|
+
> Find and remove redundant tests without opening coverage gaps.
|
|
4
|
+
|
|
5
|
+
<div align="center">
|
|
6
|
+
|
|
7
|
+

|
|
8
|
+
|
|
9
|
+
</div>
|
|
10
|
+
|
|
11
|
+
```sh
|
|
12
|
+
gem install testprune # no Gemfile change required
|
|
13
|
+
|
|
14
|
+
testprune run # runs your suite once, records per-test coverage
|
|
15
|
+
testprune report # see what's redundant — read-only
|
|
16
|
+
testprune apply # approve removals → .patch file → git apply
|
|
17
|
+
```
|
|
18
|
+
|
|
19
|
+
Works with **Minitest** and **RSpec**. No config files. No changes to your project required.
|
|
20
|
+
|
|
21
|
+
---
|
|
22
|
+
|
|
23
|
+
## What it finds
|
|
24
|
+
|
|
25
|
+
| Type | Confidence | Auto-patch? | Description |
|
|
26
|
+
|------|-----------|-------------|-------------|
|
|
27
|
+
| Identical footprint | **HIGH** | ✅ | Two tests execute the exact same set of methods/branches. Keep one. |
|
|
28
|
+
| Subset / subsumed | **HIGH** | ✅ | Test A's footprint is a strict subset of test B's. A adds no unique coverage. |
|
|
29
|
+
| Structural duplicate | **MEDIUM** | ❌ review only | Prism-normalized test bodies match; footprints overlap. Human call. |
|
|
30
|
+
| High overlap (non-subset) | **LOW** | ❌ review only | Jaccard ≥ 0.9 but neither strictly subsumes the other. Flagged for review. |
|
|
31
|
+
|
|
32
|
+
**Locality gate:** cross-file identical/subset coverage is demoted to LOW — two tests in different files that both hit the same 3-line guard are testing different things and are never auto-removed.
|
|
33
|
+
|
|
34
|
+
---
|
|
35
|
+
|
|
36
|
+
## Installation
|
|
37
|
+
|
|
38
|
+
### Option A — standalone (recommended for a one-time audit)
|
|
39
|
+
|
|
40
|
+
```sh
|
|
41
|
+
gem install testprune
|
|
42
|
+
```
|
|
43
|
+
|
|
44
|
+
testprune puts itself on the subprocess load path automatically — no Gemfile change is needed.
|
|
45
|
+
|
|
46
|
+
### Option B — in-project (for recurring use / CI)
|
|
47
|
+
|
|
48
|
+
```ruby
|
|
49
|
+
# Gemfile
|
|
50
|
+
group :development do
|
|
51
|
+
gem 'testprune', require: false
|
|
52
|
+
end
|
|
53
|
+
```
|
|
54
|
+
|
|
55
|
+
```sh
|
|
56
|
+
bundle install
|
|
57
|
+
bundle exec testprune run
|
|
58
|
+
```
|
|
59
|
+
|
|
60
|
+
---
|
|
61
|
+
|
|
62
|
+
## Usage
|
|
63
|
+
|
|
64
|
+
### Step 1 — Capture
|
|
65
|
+
|
|
66
|
+
```sh
|
|
67
|
+
# Autodetect: runs `rspec` if spec/ exists, `rake test` otherwise
|
|
68
|
+
testprune run
|
|
69
|
+
|
|
70
|
+
# Explicit command (pass after --)
|
|
71
|
+
testprune run -- bundle exec rspec spec/models
|
|
72
|
+
testprune run -- bundle exec ruby -Itest test/my_test.rb
|
|
73
|
+
testprune run -- bundle exec rails test test/controllers/
|
|
74
|
+
|
|
75
|
+
# Restrict which source files are analyzed (-s is repeatable)
|
|
76
|
+
testprune run -s app -s lib -s packs
|
|
77
|
+
```
|
|
78
|
+
|
|
79
|
+
Writes `tmp/.testprune/run.json` — per-test coverage deltas and wall times. This is the only step that boots your suite.
|
|
80
|
+
|
|
81
|
+
### Step 2 — Report
|
|
82
|
+
|
|
83
|
+
```sh
|
|
84
|
+
testprune report # grouped human-readable output
|
|
85
|
+
testprune report --json # machine-readable (for CI dashboards)
|
|
86
|
+
testprune report -s app -s lib
|
|
87
|
+
```
|
|
88
|
+
|
|
89
|
+
**Example output** (against the bundled calculator fixture):
|
|
90
|
+
|
|
91
|
+
```
|
|
92
|
+
testprune — test coverage redundancy report
|
|
93
|
+
Suite: 4 test(s), framework=minitest
|
|
94
|
+
|
|
95
|
+
HIGH confidence — safe to remove: 1
|
|
96
|
+
[identical] CalculatorTest#test_add_again
|
|
97
|
+
at: test/calculator_test.rb:16
|
|
98
|
+
reason: identical coverage to CalculatorTest#test_add
|
|
99
|
+
kept by: CalculatorTest#test_add
|
|
100
|
+
covers: Calculator#add (lib/calculator.rb:4)
|
|
101
|
+
✓ safe — every covered unit remains covered by a retained test
|
|
102
|
+
|
|
103
|
+
MEDIUM confidence — review (structural duplicates): 1
|
|
104
|
+
[structural] CalculatorTest#test_positive
|
|
105
|
+
at: test/calculator_test.rb:20
|
|
106
|
+
reason: test body structurally identical to CalculatorTest#test_nonpositive
|
|
107
|
+
· review-only — not auto-applied
|
|
108
|
+
|
|
109
|
+
Estimated CI savings:
|
|
110
|
+
1 test(s), 0.0132s (~85.7% of 0.0154s test time)
|
|
111
|
+
```
|
|
112
|
+
|
|
113
|
+
### Step 3 — Apply
|
|
114
|
+
|
|
115
|
+
```sh
|
|
116
|
+
testprune apply
|
|
117
|
+
```
|
|
118
|
+
|
|
119
|
+
The tool reprints the full report, then prompts:
|
|
120
|
+
|
|
121
|
+
```
|
|
122
|
+
Apply 1 HIGH-confidence, safety-verified removal(s) as a patch?
|
|
123
|
+
(MEDIUM/LOW review-only candidates are NOT patched automatically.) [y/N]
|
|
124
|
+
```
|
|
125
|
+
|
|
126
|
+
Answering `y` writes `tmp/.testprune/removal.patch`. **No files are modified yet.**
|
|
127
|
+
|
|
128
|
+
```sh
|
|
129
|
+
git apply --check tmp/.testprune/removal.patch # dry-run first
|
|
130
|
+
git apply tmp/.testprune/removal.patch
|
|
131
|
+
```
|
|
132
|
+
|
|
133
|
+
Removed tests are **commented out** (not deleted) with a reason annotation — you can delete or restore the commented block at your discretion:
|
|
134
|
+
|
|
135
|
+
```diff
|
|
136
|
+
- def test_add_again
|
|
137
|
+
- assert_equal 5, @calc.add(2, 3)
|
|
138
|
+
- end
|
|
139
|
+
+ # testprune: removed redundant test — identical coverage to CalculatorTest#test_add
|
|
140
|
+
+# def test_add_again
|
|
141
|
+
+# assert_equal 5, @calc.add(2, 3)
|
|
142
|
+
+# end
|
|
143
|
+
```
|
|
144
|
+
|
|
145
|
+
---
|
|
146
|
+
|
|
147
|
+
<details>
|
|
148
|
+
<summary>How it works — pipeline, confidence levels, safety guarantee, baseline subtraction</summary>
|
|
149
|
+
|
|
150
|
+
## How the pipeline works
|
|
151
|
+
|
|
152
|
+
```
|
|
153
|
+
Your test suite
|
|
154
|
+
│
|
|
155
|
+
▼
|
|
156
|
+
┌─────────────────────────────────────────────────────────────────────┐
|
|
157
|
+
│ CAPTURE (testprune run) │
|
|
158
|
+
│ │
|
|
159
|
+
│ Coverage.setup(lines:, branches:, methods:) │
|
|
160
|
+
│ │ │
|
|
161
|
+
│ ▼ │
|
|
162
|
+
│ ┌──────────┐ peek_result ┌──────────────────┐ peek_result │
|
|
163
|
+
│ │ before │ ─────────────► │ your test runs │ ─────────────► │
|
|
164
|
+
│ │ snapshot │ │ (one at a time) │ after snapshot │
|
|
165
|
+
│ └──────────┘ └──────────────────┘ │
|
|
166
|
+
│ │ │
|
|
167
|
+
│ delta = after − before │
|
|
168
|
+
│ (lines/branches/methods whose count rose) │
|
|
169
|
+
│ │ │
|
|
170
|
+
│ ▼ │
|
|
171
|
+
│ tmp/.testprune/run.json │
|
|
172
|
+
│ per-test coverage + wall time │
|
|
173
|
+
└─────────────────────────────────────────────────────────────────────┘
|
|
174
|
+
│
|
|
175
|
+
▼
|
|
176
|
+
┌─────────────────────────────────────────────────────────────────────┐
|
|
177
|
+
│ ANALYZE (testprune report / apply) │
|
|
178
|
+
│ │
|
|
179
|
+
│ Prism AST parse of each source file │
|
|
180
|
+
│ │ │
|
|
181
|
+
│ ▼ │
|
|
182
|
+
│ Coverage locations ──► Semantic units │
|
|
183
|
+
│ (line 42, col 4) "Calculator#add" │
|
|
184
|
+
│ ([if, 1, 10, 4, …]) "if then-branch (lib/x.rb:10)" │
|
|
185
|
+
│ │
|
|
186
|
+
│ │ │
|
|
187
|
+
│ ▼ │
|
|
188
|
+
│ Per-test footprint = Set of semantic unit IDs │
|
|
189
|
+
│ │
|
|
190
|
+
│ │ │
|
|
191
|
+
│ ▼ │
|
|
192
|
+
│ Baseline subtraction: units in ≥50% of tests are ambient noise; │
|
|
193
|
+
│ stripped before comparison so shared fixtures don't mask signal │
|
|
194
|
+
│ │
|
|
195
|
+
│ │ │
|
|
196
|
+
│ ▼ │
|
|
197
|
+
│ ┌──────────────┐ ┌─────────────────┐ ┌──────────────────────┐ │
|
|
198
|
+
│ │ Identical │ │ Subset/Subsumed │ │ Structural / Overlap │ │
|
|
199
|
+
│ │ footprint │ │ (A ⊊ B) │ │ (Prism body hash / │ │
|
|
200
|
+
│ │ cluster │ │ │ │ Jaccard ≥ 0.9) │ │
|
|
201
|
+
│ │ HIGH ✓ │ │ HIGH ✓ │ │ MEDIUM / LOW ○ │ │
|
|
202
|
+
│ └──────────────┘ └─────────────────┘ └──────────────────────┘ │
|
|
203
|
+
│ │
|
|
204
|
+
│ │ │
|
|
205
|
+
│ ▼ │
|
|
206
|
+
│ Coverage-safety check (cascading) │
|
|
207
|
+
│ ─ For each HIGH candidate, verify every unit it covers │
|
|
208
|
+
│ still has cover_count ≥ 2 among retained tests │
|
|
209
|
+
│ ─ Decrement counts as each removal is approved │
|
|
210
|
+
│ ─ Jointly-unsafe pairs: only one is approved │
|
|
211
|
+
│ │
|
|
212
|
+
│ │ │
|
|
213
|
+
│ ▼ │
|
|
214
|
+
│ Grouped report + estimated CI savings │
|
|
215
|
+
└─────────────────────────────────────────────────────────────────────┘
|
|
216
|
+
│
|
|
217
|
+
▼
|
|
218
|
+
┌─────────────────────────────────────────────────────────────────────┐
|
|
219
|
+
│ PATCH (testprune apply) │
|
|
220
|
+
│ │
|
|
221
|
+
│ Human reviews the report, answers y/N │
|
|
222
|
+
│ │
|
|
223
|
+
│ Prism locates each approved test's AST block by line │
|
|
224
|
+
│ ─ Comments it out with a reason annotation │
|
|
225
|
+
│ ─ Diffs via git diff --no-index │
|
|
226
|
+
│ ─ Writes tmp/.testprune/removal.patch │
|
|
227
|
+
│ │
|
|
228
|
+
│ You: git apply tmp/.testprune/removal.patch │
|
|
229
|
+
│ review the commented-out tests │
|
|
230
|
+
│ delete or restore as you see fit │
|
|
231
|
+
└─────────────────────────────────────────────────────────────────────┘
|
|
232
|
+
```
|
|
233
|
+
|
|
234
|
+
## Understanding confidence levels
|
|
235
|
+
|
|
236
|
+
### HIGH — auto-patch eligible
|
|
237
|
+
|
|
238
|
+
```
|
|
239
|
+
CalculatorTest#test_add_again
|
|
240
|
+
identical coverage to CalculatorTest#test_add
|
|
241
|
+
covers: Calculator#add (lib/calculator.rb:4)
|
|
242
|
+
✓ safe — every covered unit remains covered by a retained test
|
|
243
|
+
```
|
|
244
|
+
|
|
245
|
+
Both tests execute `Calculator#add` and nothing else distinctive. After the
|
|
246
|
+
baseline strips shared setup, their footprints are byte-identical.
|
|
247
|
+
`test_add` is kept; `test_add_again` is the candidate.
|
|
248
|
+
|
|
249
|
+
**Still a human judgment call.** Coverage measures execution, not assertion
|
|
250
|
+
strength. `test_add_again` asserts `5` where `test_add` asserts `3`. If testing
|
|
251
|
+
both values of `add` is important to you, keep both. The `✓ safe` line only
|
|
252
|
+
guarantees no code path goes uncovered — it says nothing about assertion quality.
|
|
253
|
+
|
|
254
|
+
### MEDIUM — review only (never auto-patched)
|
|
255
|
+
|
|
256
|
+
```
|
|
257
|
+
CalculatorTest#test_positive
|
|
258
|
+
test body structurally identical to CalculatorTest#test_nonpositive
|
|
259
|
+
covers: Calculator#classify; if then-branch
|
|
260
|
+
· review-only — not auto-applied
|
|
261
|
+
```
|
|
262
|
+
|
|
263
|
+
The Prism-normalized bodies match (same call sequence, different literals), but
|
|
264
|
+
`test_positive` and `test_nonpositive` hit *opposite* branch arms. testprune
|
|
265
|
+
flags the structural similarity but refuses to auto-patch because the footprints
|
|
266
|
+
differ. Human decision: are both branch arms tested elsewhere?
|
|
267
|
+
|
|
268
|
+
### LOW — review only
|
|
269
|
+
|
|
270
|
+
High-Jaccard-overlap pairs (≥90%) that are not strict subsets. Often means two
|
|
271
|
+
tests share a large Rails fixture setup but test genuinely different behavior.
|
|
272
|
+
Always review-only.
|
|
273
|
+
|
|
274
|
+
## The safety guarantee
|
|
275
|
+
|
|
276
|
+
> No semantic unit's coverage ever drops to zero as a result of a recommended removal.
|
|
277
|
+
|
|
278
|
+
It is **cascading-aware**: if tests A and B both cover unit `:x` exclusively,
|
|
279
|
+
proposing to remove both would uncover `:x`. The check evaluates candidates in
|
|
280
|
+
sorted order, decrementing `cover_count` as each removal is confirmed — so the
|
|
281
|
+
second removal fails the check (`cover_count[:x]` is now 1, not ≥ 2) and is
|
|
282
|
+
marked `✗ NOT safe (kept)`.
|
|
283
|
+
|
|
284
|
+
The same guarantee covers **ambient units** (those stripped by baseline
|
|
285
|
+
subtraction): cover_count is tracked against the original, unstripped footprints
|
|
286
|
+
so shared-setup units are protected even when they're invisible to the detector.
|
|
287
|
+
|
|
288
|
+
## Baseline subtraction
|
|
289
|
+
|
|
290
|
+
Large suites accumulate shared-setup coverage that makes unrelated tests look
|
|
291
|
+
identical. Example: a `User` fixture fires the same 12 callbacks in every test.
|
|
292
|
+
Without filtering, those 12 units appear in hundreds of footprints, creating
|
|
293
|
+
false "identical" clusters.
|
|
294
|
+
|
|
295
|
+
**Baseline** strips units executed by ≥ FRAC of tests before detection:
|
|
296
|
+
|
|
297
|
+
```sh
|
|
298
|
+
testprune report --baseline 0.5 # default: 50% threshold
|
|
299
|
+
testprune report --baseline 0.3 # more aggressive: 30%
|
|
300
|
+
testprune report --baseline 0 # disabled: trust raw coverage
|
|
301
|
+
```
|
|
302
|
+
|
|
303
|
+
The report discloses what was stripped:
|
|
304
|
+
|
|
305
|
+
```
|
|
306
|
+
Baseline: subtracted 847 shared-setup unit(s);
|
|
307
|
+
23 test(s) had no distinctive coverage and were set aside.
|
|
308
|
+
```
|
|
309
|
+
|
|
310
|
+
A test with zero distinctive coverage after stripping is **never proposed for
|
|
311
|
+
removal** — testprune can't tell what it uniquely exercises.
|
|
312
|
+
|
|
313
|
+
</details>
|
|
314
|
+
|
|
315
|
+
<details>
|
|
316
|
+
<summary>Team playbook — Rails, Spring, SimpleCov, version managers, monorepo</summary>
|
|
317
|
+
|
|
318
|
+
### Minitest project (standard layout)
|
|
319
|
+
|
|
320
|
+
```sh
|
|
321
|
+
testprune run -s app -s lib
|
|
322
|
+
testprune report -s app -s lib
|
|
323
|
+
testprune apply -s app -s lib
|
|
324
|
+
```
|
|
325
|
+
|
|
326
|
+
### RSpec project
|
|
327
|
+
|
|
328
|
+
```sh
|
|
329
|
+
testprune run -s app -s lib -- bundle exec rspec
|
|
330
|
+
testprune report -s app -s lib
|
|
331
|
+
```
|
|
332
|
+
|
|
333
|
+
### Rails app
|
|
334
|
+
|
|
335
|
+
```sh
|
|
336
|
+
# Run a specific directory (never run the whole suite without a path)
|
|
337
|
+
testprune run -s app -s lib -s packs -- bundle exec rails test test/models/
|
|
338
|
+
|
|
339
|
+
# Multiple passes — capture incrementally by domain, analyze together
|
|
340
|
+
testprune run -s app -- bundle exec rails test test/models/
|
|
341
|
+
testprune run -s app -- bundle exec rails test test/controllers/
|
|
342
|
+
# run.json is overwritten on each `testprune run`; analyze each pass separately
|
|
343
|
+
```
|
|
344
|
+
|
|
345
|
+
### Rails app with Spring
|
|
346
|
+
|
|
347
|
+
Spring preloads your app but forks the test process — the forked child doesn't
|
|
348
|
+
inherit `RUBYOPT` where testprune's autostart lives. Disable it for the run:
|
|
349
|
+
|
|
350
|
+
```sh
|
|
351
|
+
DISABLE_SPRING=1 testprune run -s app -s lib -- bundle exec rails test test/models/
|
|
352
|
+
```
|
|
353
|
+
|
|
354
|
+
### Projects using SimpleCov (or other coverage gems)
|
|
355
|
+
|
|
356
|
+
No changes needed. testprune starts `Coverage` first (via `RUBYOPT`) and keeps
|
|
357
|
+
it running before your test helper loads. SimpleCov 0.22.x (verified against
|
|
358
|
+
source) guards its `Coverage.start` call with `unless Coverage.running?` — since
|
|
359
|
+
testprune already started it, SimpleCov skips its own start and cooperates
|
|
360
|
+
automatically. SimpleCov's final `Coverage.result` call still gets the full line-coverage aggregate — testprune
|
|
361
|
+
only uses `peek_result` (non-destructive snapshots) and never calls
|
|
362
|
+
`Coverage.result` itself.
|
|
363
|
+
|
|
364
|
+
> **Older SimpleCov versions:** If SimpleCov crashes with `coverage measurement
|
|
365
|
+
> is already setup`, your version calls `Coverage.start` unconditionally. Add
|
|
366
|
+
> `require 'testprune/autostart'` as the very first line of your test_helper.rb
|
|
367
|
+
> (before any SimpleCov require) so testprune initializes Coverage first and
|
|
368
|
+
> SimpleCov finds it already running.
|
|
369
|
+
|
|
370
|
+
### Ruby version manager (rv, rbenv, asdf)
|
|
371
|
+
|
|
372
|
+
Version-manager shims strip `RUBYOPT` before the subprocess starts. Re-inject it
|
|
373
|
+
*after* the shim using `env`:
|
|
374
|
+
|
|
375
|
+
```sh
|
|
376
|
+
# rv
|
|
377
|
+
rv run --ruby 3.2 env RUBYOPT="-I$(gem contents testprune | grep autostart | xargs dirname | head -1)/.. -rtestprune/autostart" \
|
|
378
|
+
bundle exec rake test
|
|
379
|
+
|
|
380
|
+
# Simpler: install testprune under the managed ruby so RUBYOPT is not needed
|
|
381
|
+
rv run --ruby 3.2 gem install testprune
|
|
382
|
+
rv run --ruby 3.2 bundle exec testprune run
|
|
383
|
+
```
|
|
384
|
+
|
|
385
|
+
The easiest path is always to install testprune under the same Ruby that runs
|
|
386
|
+
your suite. If `TESTPRUNE_DEBUG=1 testprune run` prints nothing from `[testprune-debug]`,
|
|
387
|
+
the autostart never loaded — this is the cause.
|
|
388
|
+
|
|
389
|
+
### Monorepo / packs
|
|
390
|
+
|
|
391
|
+
Run each pack separately and analyze with its source path:
|
|
392
|
+
|
|
393
|
+
```sh
|
|
394
|
+
# Capture one pack
|
|
395
|
+
testprune run -s packs/tenancy/app -- \
|
|
396
|
+
bundle exec rails test packs/tenancy/test/
|
|
397
|
+
|
|
398
|
+
# Report for that pack
|
|
399
|
+
testprune report -s packs/tenancy/app
|
|
400
|
+
testprune apply -s packs/tenancy/app
|
|
401
|
+
```
|
|
402
|
+
|
|
403
|
+
</details>
|
|
404
|
+
|
|
405
|
+
<details>
|
|
406
|
+
<summary>CI integration</summary>
|
|
407
|
+
|
|
408
|
+
testprune is best run on-demand or on a scheduled basis, not on every push.
|
|
409
|
+
The run step is the slow one (it boots and runs your suite); report/apply are
|
|
410
|
+
fast (they read from run.json).
|
|
411
|
+
|
|
412
|
+
```yaml
|
|
413
|
+
# .github/workflows/testprune.yml — run weekly
|
|
414
|
+
name: testprune audit
|
|
415
|
+
on:
|
|
416
|
+
schedule:
|
|
417
|
+
- cron: '0 9 * * 1' # Monday 9am
|
|
418
|
+
workflow_dispatch: # manual trigger
|
|
419
|
+
|
|
420
|
+
jobs:
|
|
421
|
+
audit:
|
|
422
|
+
runs-on: ubuntu-latest
|
|
423
|
+
steps:
|
|
424
|
+
- uses: actions/checkout@v4
|
|
425
|
+
- uses: ruby/setup-ruby@v1
|
|
426
|
+
with:
|
|
427
|
+
ruby-version: '3.2'
|
|
428
|
+
bundler-cache: true
|
|
429
|
+
- run: gem install testprune
|
|
430
|
+
- run: testprune run -s app -s lib -- bundle exec rails test test/models/
|
|
431
|
+
- run: testprune report -s app -s lib --json > testprune-report.json
|
|
432
|
+
- uses: actions/upload-artifact@v4
|
|
433
|
+
with:
|
|
434
|
+
name: testprune-report
|
|
435
|
+
path: |
|
|
436
|
+
tmp/.testprune/run.json
|
|
437
|
+
testprune-report.json
|
|
438
|
+
```
|
|
439
|
+
|
|
440
|
+
To gate a PR on the report without blocking it:
|
|
441
|
+
|
|
442
|
+
```sh
|
|
443
|
+
# Exit 0 always; leave actioning the findings to a human
|
|
444
|
+
testprune report -s app -s lib || true
|
|
445
|
+
```
|
|
446
|
+
|
|
447
|
+
</details>
|
|
448
|
+
|
|
449
|
+
<details>
|
|
450
|
+
<summary>All options and environment variables</summary>
|
|
451
|
+
|
|
452
|
+
| Flag | Default | Command | Description |
|
|
453
|
+
|------|---------|---------|-------------|
|
|
454
|
+
| `-s, --source PATH` | `app`, `lib` | all | Source dir to analyze. Repeatable. Coverage outside these paths is ignored. |
|
|
455
|
+
| `-o, --output DIR` | `tmp/.testprune` | all | Where `run.json` and `removal.patch` are written. |
|
|
456
|
+
| `--baseline FRAC` | `0.5` | report, apply | Strip units in ≥ FRAC of tests as shared-setup noise before detection. `0` disables. |
|
|
457
|
+
| `--json` | off | report | Emit machine-readable JSON instead of human text. |
|
|
458
|
+
| `-h, --help` | | all | Show help. |
|
|
459
|
+
| `-v, --version` | | | Print version. |
|
|
460
|
+
|
|
461
|
+
**Environment variables:**
|
|
462
|
+
|
|
463
|
+
| Variable | Effect |
|
|
464
|
+
|----------|--------|
|
|
465
|
+
| `TESTPRUNE_ROOT` | Set the project root (default: `Dir.pwd`). Set automatically by `testprune run`. |
|
|
466
|
+
| `TESTPRUNE_SOURCE_PATHS` | Colon-separated source paths. Set automatically by `testprune run`. |
|
|
467
|
+
| `TESTPRUNE_OUTPUT_DIR` | Output directory. Set automatically by `testprune run`. |
|
|
468
|
+
| `TESTPRUNE_DEBUG` | Print adapter-load diagnostics (`[testprune-debug] autostart loaded in pid …`). Useful when capture produces no `run.json`. |
|
|
469
|
+
| `DISABLE_SPRING` | Disable Spring preloader so the test process inherits testprune's instrumentation. |
|
|
470
|
+
|
|
471
|
+
</details>
|
|
472
|
+
|
|
473
|
+
<details>
|
|
474
|
+
<summary>Caveats, requirements, and development</summary>
|
|
475
|
+
|
|
476
|
+
## Caveats
|
|
477
|
+
|
|
478
|
+
**Coverage ≠ assertion strength.** A test can execute a code path without
|
|
479
|
+
asserting anything meaningful about it. testprune flags coverage-identical tests,
|
|
480
|
+
but two tests that run the same method while asserting different properties are
|
|
481
|
+
*both* meaningful. Always review HIGH candidates before applying the patch.
|
|
482
|
+
|
|
483
|
+
**Opposite branch arms are correctly preserved.** The Prism semantic mapping
|
|
484
|
+
means `test_positive` (hitting the `then` arm) and `test_nonpositive` (hitting
|
|
485
|
+
the `else` arm) are never flagged as duplicates — even though they call the same
|
|
486
|
+
method.
|
|
487
|
+
|
|
488
|
+
**CI savings are aggregate, not wall-clock.** Reported savings = sum of removed
|
|
489
|
+
tests' wall times. Under parallel runners, actual wall-clock savings will be
|
|
490
|
+
smaller (only the critical path matters).
|
|
491
|
+
|
|
492
|
+
**Per-test `peek_result` overhead.** Snapshotting coverage around each test adds
|
|
493
|
+
overhead. On very large suites (10k+ tests) this is noticeable but acceptable —
|
|
494
|
+
it's mitigated by restricting `--source` to the paths you care about.
|
|
495
|
+
|
|
496
|
+
**run.json is machine-local.** Coverage paths are absolute. Don't run
|
|
497
|
+
`testprune run` on CI and `testprune report` on a laptop with a different home
|
|
498
|
+
directory — the paths won't match. Always run all three commands on the same
|
|
499
|
+
machine.
|
|
500
|
+
|
|
501
|
+
## Requirements
|
|
502
|
+
|
|
503
|
+
- **Ruby ≥ 3.2** — requires `Coverage.setup` + `Coverage.supported?(:branches)`.
|
|
504
|
+
(`Coverage.setup` landed in 3.1; branch and method coverage in 3.0.)
|
|
505
|
+
- **Prism ≥ 1.0, < 3** — bundled with Ruby ≥ 3.3; declared as a dependency.
|
|
506
|
+
- No changes to the target project are required. testprune injects itself via
|
|
507
|
+
`RUBYOPT` at run time.
|
|
508
|
+
|
|
509
|
+
## Development
|
|
510
|
+
|
|
511
|
+
```sh
|
|
512
|
+
git clone https://github.com/seth-macpherson/testprune
|
|
513
|
+
cd testprune
|
|
514
|
+
bundle install
|
|
515
|
+
rake test # 32 tests
|
|
516
|
+
```
|
|
517
|
+
|
|
518
|
+
### Project layout
|
|
519
|
+
|
|
520
|
+
```
|
|
521
|
+
exe/testprune CLI entry point
|
|
522
|
+
lib/testprune/
|
|
523
|
+
autostart.rb Loaded via RUBYOPT; starts Coverage + installs adapter
|
|
524
|
+
recorder.rb Per-process singleton; brackets each test
|
|
525
|
+
adapters/
|
|
526
|
+
minitest.rb before_setup / after_teardown hooks
|
|
527
|
+
rspec.rb around(:each) + after(:suite)
|
|
528
|
+
coverage_delta.rb Diff two peek_result snapshots → footprint delta
|
|
529
|
+
semantic_map.rb Prism AST → semantic unit index for one file
|
|
530
|
+
footprint.rb SemanticIndex + Footprint struct
|
|
531
|
+
baseline.rb Ambient-unit detection (shared-setup noise filter)
|
|
532
|
+
duplication_detector.rb Identical / subset / structural / overlap detection
|
|
533
|
+
safety_check.rb Cascading cover_count guard
|
|
534
|
+
analysis.rb Orchestrates: load run.json → footprints → detect
|
|
535
|
+
report.rb Human + JSON output
|
|
536
|
+
savings_estimator.rb Aggregate wall-time estimate
|
|
537
|
+
patch_writer.rb Prism-located test block → git diff patch
|
|
538
|
+
cli.rb OptionParser command dispatch
|
|
539
|
+
configuration.rb Settings + env-var round-trip
|
|
540
|
+
runner.rb Subprocess boot + RUBYOPT injection
|
|
541
|
+
test/
|
|
542
|
+
fixtures/sample_minitest/ Minimal calculator project used in integration test
|
|
543
|
+
testprune/
|
|
544
|
+
baseline_test.rb
|
|
545
|
+
duplication_detector_test.rb
|
|
546
|
+
safety_check_test.rb
|
|
547
|
+
semantic_map_test.rb
|
|
548
|
+
coverage_delta_test.rb
|
|
549
|
+
integration_test.rb Full CLI end-to-end
|
|
550
|
+
```
|
|
551
|
+
|
|
552
|
+
</details>
|
|
@@ -0,0 +1,70 @@
|
|
|
1
|
+
<svg viewBox="0 0 820 310" xmlns="http://www.w3.org/2000/svg">
|
|
2
|
+
<defs>
|
|
3
|
+
<marker id="arrow" markerWidth="8" markerHeight="8" refX="7" refY="3" orient="auto" markerUnits="strokeWidth">
|
|
4
|
+
<path d="M0,0.5 L0,5.5 L7,3 z" fill="#6366f1"/>
|
|
5
|
+
</marker>
|
|
6
|
+
</defs>
|
|
7
|
+
|
|
8
|
+
<!-- Background -->
|
|
9
|
+
<rect width="820" height="310" fill="#0d1117" rx="14"/>
|
|
10
|
+
|
|
11
|
+
<!-- Header -->
|
|
12
|
+
<text x="410" y="44" font-family="system-ui,-apple-system,sans-serif" fill="#f0f6fc" font-size="21" font-weight="700" text-anchor="middle">testprune</text>
|
|
13
|
+
<text x="410" y="66" font-family="system-ui,-apple-system,sans-serif" fill="#6e7681" font-size="13" text-anchor="middle">Audit your Ruby test suite for redundant coverage.</text>
|
|
14
|
+
|
|
15
|
+
<!-- ── STEP 1 ── -->
|
|
16
|
+
<rect x="36" y="84" width="218" height="178" fill="#161b22" rx="10" stroke="#21262d" stroke-width="1.5"/>
|
|
17
|
+
<!-- top accent -->
|
|
18
|
+
<rect x="36" y="84" width="218" height="5" fill="#6366f1" rx="3"/>
|
|
19
|
+
<rect x="36" y="86" width="218" height="3" fill="#6366f1"/>
|
|
20
|
+
|
|
21
|
+
<text x="145" y="112" font-family="system-ui,-apple-system,sans-serif" fill="#6366f1" font-size="9.5" font-weight="700" text-anchor="middle" letter-spacing="2">STEP 1</text>
|
|
22
|
+
<text x="145" y="140" font-family="ui-monospace,'SF Mono',Menlo,monospace" fill="#3fb950" font-size="15.5" font-weight="600" text-anchor="middle">testprune run</text>
|
|
23
|
+
<line x1="56" y1="153" x2="234" y2="153" stroke="#21262d" stroke-width="1"/>
|
|
24
|
+
<text x="145" y="173" font-family="system-ui,-apple-system,sans-serif" fill="#8b949e" font-size="11.5" text-anchor="middle">Runs your suite once and</text>
|
|
25
|
+
<text x="145" y="191" font-family="system-ui,-apple-system,sans-serif" fill="#8b949e" font-size="11.5" text-anchor="middle">records per-test coverage</text>
|
|
26
|
+
<text x="145" y="209" font-family="system-ui,-apple-system,sans-serif" fill="#8b949e" font-size="11.5" text-anchor="middle">data to run.json</text>
|
|
27
|
+
<text x="145" y="247" font-family="system-ui,-apple-system,sans-serif" fill="#3d444d" font-size="10.5" text-anchor="middle">one-time per audit</text>
|
|
28
|
+
|
|
29
|
+
<!-- Arrow 1 -->
|
|
30
|
+
<line x1="262" y1="173" x2="300" y2="173" stroke="#6366f1" stroke-width="1.5" marker-end="url(#arrow)"/>
|
|
31
|
+
|
|
32
|
+
<!-- ── STEP 2 ── -->
|
|
33
|
+
<rect x="302" y="84" width="218" height="178" fill="#161b22" rx="10" stroke="#21262d" stroke-width="1.5"/>
|
|
34
|
+
<rect x="302" y="84" width="218" height="5" fill="#6366f1" rx="3"/>
|
|
35
|
+
<rect x="302" y="86" width="218" height="3" fill="#6366f1"/>
|
|
36
|
+
|
|
37
|
+
<text x="411" y="112" font-family="system-ui,-apple-system,sans-serif" fill="#6366f1" font-size="9.5" font-weight="700" text-anchor="middle" letter-spacing="2">STEP 2</text>
|
|
38
|
+
<text x="411" y="140" font-family="ui-monospace,'SF Mono',Menlo,monospace" fill="#3fb950" font-size="15.5" font-weight="600" text-anchor="middle">testprune report</text>
|
|
39
|
+
<line x1="322" y1="153" x2="500" y2="153" stroke="#21262d" stroke-width="1"/>
|
|
40
|
+
<text x="411" y="173" font-family="system-ui,-apple-system,sans-serif" fill="#8b949e" font-size="11.5" text-anchor="middle">See which tests are</text>
|
|
41
|
+
<text x="411" y="191" font-family="system-ui,-apple-system,sans-serif" fill="#8b949e" font-size="11.5" text-anchor="middle">redundant, rated HIGH /</text>
|
|
42
|
+
<text x="411" y="209" font-family="system-ui,-apple-system,sans-serif" fill="#8b949e" font-size="11.5" text-anchor="middle">MEDIUM / LOW confidence</text>
|
|
43
|
+
<text x="411" y="247" font-family="system-ui,-apple-system,sans-serif" fill="#3d444d" font-size="10.5" text-anchor="middle">read-only · re-runnable</text>
|
|
44
|
+
|
|
45
|
+
<!-- Arrow 2 -->
|
|
46
|
+
<line x1="528" y1="173" x2="566" y2="173" stroke="#6366f1" stroke-width="1.5" marker-end="url(#arrow)"/>
|
|
47
|
+
|
|
48
|
+
<!-- ── STEP 3 ── -->
|
|
49
|
+
<rect x="568" y="84" width="218" height="178" fill="#161b22" rx="10" stroke="#21262d" stroke-width="1.5"/>
|
|
50
|
+
<rect x="568" y="84" width="218" height="5" fill="#6366f1" rx="3"/>
|
|
51
|
+
<rect x="568" y="86" width="218" height="3" fill="#6366f1"/>
|
|
52
|
+
|
|
53
|
+
<text x="677" y="112" font-family="system-ui,-apple-system,sans-serif" fill="#6366f1" font-size="9.5" font-weight="700" text-anchor="middle" letter-spacing="2">STEP 3</text>
|
|
54
|
+
<text x="677" y="140" font-family="ui-monospace,'SF Mono',Menlo,monospace" fill="#3fb950" font-size="15.5" font-weight="600" text-anchor="middle">testprune apply</text>
|
|
55
|
+
<line x1="588" y1="153" x2="766" y2="153" stroke="#21262d" stroke-width="1"/>
|
|
56
|
+
<text x="677" y="173" font-family="system-ui,-apple-system,sans-serif" fill="#8b949e" font-size="11.5" text-anchor="middle">Approve removals, receive</text>
|
|
57
|
+
<text x="677" y="191" font-family="system-ui,-apple-system,sans-serif" fill="#8b949e" font-size="11.5" text-anchor="middle">a .patch file. Review and</text>
|
|
58
|
+
<text x="677" y="209" font-family="system-ui,-apple-system,sans-serif" fill="#8b949e" font-size="11.5" text-anchor="middle">apply with git apply.</text>
|
|
59
|
+
<text x="677" y="247" font-family="system-ui,-apple-system,sans-serif" fill="#3d444d" font-size="10.5" text-anchor="middle">never edits files directly</text>
|
|
60
|
+
|
|
61
|
+
<!-- Footer pills -->
|
|
62
|
+
<rect x="110" y="278" width="188" height="18" fill="#161b22" rx="9" stroke="#21262d" stroke-width="1"/>
|
|
63
|
+
<text x="204" y="291" font-family="system-ui,-apple-system,sans-serif" fill="#6e7681" font-size="10" text-anchor="middle">safety check · no coverage gaps</text>
|
|
64
|
+
|
|
65
|
+
<rect x="316" y="278" width="188" height="18" fill="#161b22" rx="9" stroke="#21262d" stroke-width="1"/>
|
|
66
|
+
<text x="410" y="291" font-family="system-ui,-apple-system,sans-serif" fill="#6e7681" font-size="10" text-anchor="middle">Minitest + RSpec · Rails ready</text>
|
|
67
|
+
|
|
68
|
+
<rect x="522" y="278" width="188" height="18" fill="#161b22" rx="9" stroke="#21262d" stroke-width="1"/>
|
|
69
|
+
<text x="616" y="291" font-family="system-ui,-apple-system,sans-serif" fill="#6e7681" font-size="10" text-anchor="middle">gem install testprune · no Gemfile</text>
|
|
70
|
+
</svg>
|