rspec-sprint 0.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +7 -0
- data/LICENSE.txt +21 -0
- data/README.md +98 -0
- data/exe/rspec-sprint +6 -0
- data/lib/rspec_sprint/cli.rb +31 -0
- data/lib/rspec_sprint/collector.rb +77 -0
- data/lib/rspec_sprint/diagnosis.rb +19 -0
- data/lib/rspec_sprint/doctor.rb +43 -0
- data/lib/rspec_sprint/finding.rb +11 -0
- data/lib/rspec_sprint/formatter.rb +29 -0
- data/lib/rspec_sprint/normalizer.rb +113 -0
- data/lib/rspec_sprint/ranker.rb +22 -0
- data/lib/rspec_sprint/rules.rb +137 -0
- data/lib/rspec_sprint/version.rb +5 -0
- data/lib/rspec_sprint.rb +10 -0
- metadata +88 -0
checksums.yaml
ADDED
|
@@ -0,0 +1,7 @@
|
|
|
1
|
+
---
|
|
2
|
+
SHA256:
|
|
3
|
+
metadata.gz: f16ad949645afe84a5a8ba614a5a6e2e40f609fa76f73c2e0af72557830dbd9d
|
|
4
|
+
data.tar.gz: 8503cde0dabd4c8cccefee9d53ef55b996e2724ad21ebcb6f4f4d6b9b6eec747
|
|
5
|
+
SHA512:
|
|
6
|
+
metadata.gz: 021d7debd206e0d543a97c71692eed732b0f3bc280e13e3262c19936ee790012d3b93588f3de91f1741f19fa26ac36da322e2fd1ad87ef81f215098231f00a66
|
|
7
|
+
data.tar.gz: 0d93b9cb8eb129e2cc6451d8ad2607efcbd0a1a6108549d4c5f8a566c4fd69db39b481d1b40293462ed94bd0396b49da6a2cf573b9e38eb3c6357c22c73a9e43
|
data/LICENSE.txt
ADDED
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
MIT License
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2026 yasu551
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
+
in the Software without restriction, including without limitation the rights
|
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
+
furnished to do so, subject to the following conditions:
|
|
11
|
+
|
|
12
|
+
The above copyright notice and this permission notice shall be included in all
|
|
13
|
+
copies or substantial portions of the Software.
|
|
14
|
+
|
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
21
|
+
SOFTWARE.
|
data/README.md
ADDED
|
@@ -0,0 +1,98 @@
|
|
|
1
|
+
# rspec-sprint
|
|
2
|
+
|
|
3
|
+
> Diagnose your RSpec/Rails test-suite slowness and get the top repo-specific fixes — diagnosis **plus** prescription, not just numbers.
|
|
4
|
+
|
|
5
|
+
`rspec-sprint doctor` runs your suite once (via [test-prof](https://test-prof.evilmartians.io/)'s FactoryProf and RSpec's JSON formatter in a single run), interprets the output against opinionated heuristics, and asserts the top few things to fix in *your* repo:
|
|
6
|
+
|
|
7
|
+
```
|
|
8
|
+
$ bundle exec rspec-sprint doctor
|
|
9
|
+
factory が suite の 47% (local実測)。最上位は :user(420回, うち直接生成は 30回 = カスケード)。
|
|
10
|
+
→ 不要 association を trait へ退避 / build_stubbed / let_it_be / FactoryDefault
|
|
11
|
+
...
|
|
12
|
+
```
|
|
13
|
+
|
|
14
|
+
## Installation
|
|
15
|
+
|
|
16
|
+
Add to your `Gemfile` (development/test group):
|
|
17
|
+
|
|
18
|
+
```ruby
|
|
19
|
+
group :development, :test do
|
|
20
|
+
gem "rspec-sprint"
|
|
21
|
+
gem "test-prof"
|
|
22
|
+
end
|
|
23
|
+
```
|
|
24
|
+
|
|
25
|
+
Then run `bundle install`.
|
|
26
|
+
|
|
27
|
+
**Prerequisite:** Add to your `spec/spec_helper.rb`:
|
|
28
|
+
|
|
29
|
+
```ruby
|
|
30
|
+
require "test_prof"
|
|
31
|
+
```
|
|
32
|
+
|
|
33
|
+
## Usage
|
|
34
|
+
|
|
35
|
+
Run the doctor command to diagnose your suite and get prescriptions ranked by ROI:
|
|
36
|
+
|
|
37
|
+
```
|
|
38
|
+
$ bundle exec rspec-sprint doctor
|
|
39
|
+
```
|
|
40
|
+
|
|
41
|
+
To target a subset of specs (faster feedback during investigation):
|
|
42
|
+
|
|
43
|
+
```
|
|
44
|
+
$ bundle exec rspec-sprint doctor -- spec/models
|
|
45
|
+
```
|
|
46
|
+
|
|
47
|
+
Snapshot comparison (coming in v0.2.0):
|
|
48
|
+
|
|
49
|
+
```
|
|
50
|
+
$ bundle exec rspec-sprint doctor --no-snapshot
|
|
51
|
+
```
|
|
52
|
+
|
|
53
|
+
The command exits 0 whether or not findings are reported, so it is safe to run in CI as an advisory step.
|
|
54
|
+
|
|
55
|
+
## Why not just test-prof?
|
|
56
|
+
|
|
57
|
+
test-prof gives you the **instruments** (FactoryProf, EventProf, TagProf). `rspec-sprint` is the **judgment layer** on top: it reads those instruments, ranks the bottlenecks by ROI, and tells you which one to fix first and how. It does not reimplement any profiler.
|
|
58
|
+
|
|
59
|
+
Concretely, rspec-sprint applies three opinionated heuristics to your numbers:
|
|
60
|
+
|
|
61
|
+
- **factory_dominance** — factory time exceeds 30% of total suite time
|
|
62
|
+
- **path_group_skew** — one spec directory accounts for a disproportionate share of slowness
|
|
63
|
+
- **slow_examples_concentration** — a handful of examples dominate the tail
|
|
64
|
+
|
|
65
|
+
For each finding it emits a ranked prescription (e.g. "retire unused associations to a trait, switch to `build_stubbed`") rather than leaving interpretation to you.
|
|
66
|
+
|
|
67
|
+
Even when test-prof is not installed, FactoryBot is absent, or the suite fails partway through, `rspec-sprint doctor` degrades gracefully and tells you what it could and could not measure.
|
|
68
|
+
|
|
69
|
+
## Compatibility
|
|
70
|
+
|
|
71
|
+
| Component | Version |
|
|
72
|
+
|-----------|---------|
|
|
73
|
+
| Ruby | >= 3.0 |
|
|
74
|
+
| RSpec | ~> 3.13 |
|
|
75
|
+
| test-prof | >= 1.0 |
|
|
76
|
+
| Rails | optional (not required) |
|
|
77
|
+
|
|
78
|
+
## Status
|
|
79
|
+
|
|
80
|
+
v0.1.0 — install-day tested, dogfood confirmed.
|
|
81
|
+
|
|
82
|
+
**Dogfood result (SonicGarden/aegis, 2026-06-18):**
|
|
83
|
+
|
|
84
|
+
```
|
|
85
|
+
$ bundle exec rspec-sprint doctor -- spec/models spec/policies
|
|
86
|
+
|
|
87
|
+
1. path group spec/models が suite 時間の 92% (local実測)
|
|
88
|
+
→ spec/models を別CIジョブに分離し fast suite から切り出す(候補)
|
|
89
|
+
|
|
90
|
+
2. factory が suite の 44%。最上位は :tenant(689回)
|
|
91
|
+
→ 不要な association を trait へ退避 / build_stubbed / let_it_be
|
|
92
|
+
```
|
|
93
|
+
|
|
94
|
+
Both findings are repo-specific with concrete numbers — not generic advice.
|
|
95
|
+
|
|
96
|
+
## License
|
|
97
|
+
|
|
98
|
+
MIT
|
data/exe/rspec-sprint
ADDED
|
@@ -0,0 +1,31 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
require "thor"
|
|
4
|
+
require_relative "collector"
|
|
5
|
+
require_relative "doctor"
|
|
6
|
+
|
|
7
|
+
module RspecSprint
|
|
8
|
+
class CLI < Thor
|
|
9
|
+
def self.exit_on_failure?
|
|
10
|
+
true
|
|
11
|
+
end
|
|
12
|
+
|
|
13
|
+
desc "doctor [-- RSPEC_ARGS]", "Run your suite once and assert the top things to fix"
|
|
14
|
+
long_desc <<~DESC
|
|
15
|
+
Runs your RSpec suite a single time with FactoryProf (test-prof) + RSpec's
|
|
16
|
+
JSON formatter, then prints the top repo-specific bottlenecks and how to fix
|
|
17
|
+
them. Forward args to rspec after `--`, e.g.
|
|
18
|
+
|
|
19
|
+
bundle exec rspec-sprint doctor -- spec/models --tag focus
|
|
20
|
+
|
|
21
|
+
Requires test-prof in your bundle and `require "test_prof"` in your spec
|
|
22
|
+
helper for factory diagnosis (Rule①); without it, the other rules still run.
|
|
23
|
+
DESC
|
|
24
|
+
def doctor(*rspec_args)
|
|
25
|
+
result = Collector.new(rspec_args: rspec_args).run
|
|
26
|
+
puts Doctor.diagnose(result)
|
|
27
|
+
end
|
|
28
|
+
|
|
29
|
+
default_command :doctor
|
|
30
|
+
end
|
|
31
|
+
end
|
|
@@ -0,0 +1,77 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
require "open3"
|
|
4
|
+
require "fileutils"
|
|
5
|
+
|
|
6
|
+
module RspecSprint
|
|
7
|
+
# Runs the host project's RSpec suite ONCE with FactoryProf JSON enabled and
|
|
8
|
+
# RSpec's JSON formatter writing to a dedicated file (so the host's own
|
|
9
|
+
# formatter on stdout is not clobbered — design D10③). Returns a Result that
|
|
10
|
+
# downstream code (Diagnosis) and degraded-path handling read.
|
|
11
|
+
#
|
|
12
|
+
# The pure pieces (build_command, factory_prof_path_from) are split out so
|
|
13
|
+
# most behavior is unit-testable without spawning a real rspec process.
|
|
14
|
+
class Collector
|
|
15
|
+
DEFAULT_OUT = "tmp/rspec_sprint/rspec.json"
|
|
16
|
+
FPROF_LOG = /Profile results to JSON:\s*(\S+)/
|
|
17
|
+
|
|
18
|
+
Result = Struct.new(
|
|
19
|
+
:rspec_json_path, :factory_prof_path, :exit_status, :stdout, :stderr,
|
|
20
|
+
keyword_init: true
|
|
21
|
+
) do
|
|
22
|
+
def rspec_json?
|
|
23
|
+
!rspec_json_path.nil? && File.exist?(rspec_json_path) && !File.zero?(rspec_json_path)
|
|
24
|
+
end
|
|
25
|
+
|
|
26
|
+
def factory_prof?
|
|
27
|
+
!factory_prof_path.nil? && File.exist?(factory_prof_path)
|
|
28
|
+
end
|
|
29
|
+
|
|
30
|
+
def suite_red?
|
|
31
|
+
!exit_status.nil? && exit_status != 0
|
|
32
|
+
end
|
|
33
|
+
end
|
|
34
|
+
|
|
35
|
+
# Pure: the env + argv used to run rspec. FPROF=json makes test-prof write
|
|
36
|
+
# its FactoryProf JSON; --format json --out writes rspec's JSON to its own
|
|
37
|
+
# file without replacing the host's stdout formatter.
|
|
38
|
+
def self.build_command(out_path:, rspec_args: [])
|
|
39
|
+
env = { "FPROF" => "json" }
|
|
40
|
+
argv = ["bundle", "exec", "rspec", "--format", "json", "--out", out_path, *rspec_args]
|
|
41
|
+
[env, argv]
|
|
42
|
+
end
|
|
43
|
+
|
|
44
|
+
# Pure: pull the FactoryProf output path test-prof logs to stdout. nil means
|
|
45
|
+
# test-prof never ran (not required in the host's boot) — Rule① will be skipped.
|
|
46
|
+
def self.factory_prof_path_from(stdout)
|
|
47
|
+
match = stdout.to_s.match(FPROF_LOG)
|
|
48
|
+
match && match[1]
|
|
49
|
+
end
|
|
50
|
+
|
|
51
|
+
def initialize(rspec_args: [], out_path: DEFAULT_OUT, dir: ".")
|
|
52
|
+
@rspec_args = rspec_args
|
|
53
|
+
@out_path = out_path
|
|
54
|
+
@dir = dir
|
|
55
|
+
end
|
|
56
|
+
|
|
57
|
+
def run
|
|
58
|
+
FileUtils.mkdir_p(File.dirname(File.join(@dir, @out_path)))
|
|
59
|
+
env, argv = self.class.build_command(out_path: @out_path, rspec_args: @rspec_args)
|
|
60
|
+
stdout, stderr, status = Open3.capture3(env, *argv, chdir: @dir)
|
|
61
|
+
Result.new(
|
|
62
|
+
rspec_json_path: File.join(@dir, @out_path),
|
|
63
|
+
factory_prof_path: resolve_factory_prof_path(stdout),
|
|
64
|
+
exit_status: status.exitstatus,
|
|
65
|
+
stdout: stdout,
|
|
66
|
+
stderr: stderr
|
|
67
|
+
)
|
|
68
|
+
end
|
|
69
|
+
|
|
70
|
+
private
|
|
71
|
+
|
|
72
|
+
def resolve_factory_prof_path(stdout)
|
|
73
|
+
rel = self.class.factory_prof_path_from(stdout)
|
|
74
|
+
rel && File.join(@dir, rel)
|
|
75
|
+
end
|
|
76
|
+
end
|
|
77
|
+
end
|
|
@@ -0,0 +1,19 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
require_relative "normalizer"
|
|
4
|
+
require_relative "ranker"
|
|
5
|
+
require_relative "formatter"
|
|
6
|
+
|
|
7
|
+
module RspecSprint
|
|
8
|
+
# Pure pipeline: profiler JSON files -> normalized snapshot -> ranked findings
|
|
9
|
+
# -> rendered report. Decoupled from how the JSON was produced (a live run, a
|
|
10
|
+
# fixture, or a CI artifact), per design D3.
|
|
11
|
+
module Diagnosis
|
|
12
|
+
module_function
|
|
13
|
+
|
|
14
|
+
def from_files(rspec_json:, factory_prof_json: nil)
|
|
15
|
+
snapshot = Normalizer.new(rspec_json: rspec_json, factory_prof_json: factory_prof_json).call
|
|
16
|
+
Formatter.format(Ranker.call(snapshot))
|
|
17
|
+
end
|
|
18
|
+
end
|
|
19
|
+
end
|
|
@@ -0,0 +1,43 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
require_relative "diagnosis"
|
|
4
|
+
|
|
5
|
+
module RspecSprint
|
|
6
|
+
# Turns a Collector::Result into a human report, handling the degraded paths
|
|
7
|
+
# (design D4 / D10④): distinguish "the suite never produced readable output"
|
|
8
|
+
# (diagnosis impossible) from "the suite ran but had failures" (still
|
|
9
|
+
# diagnosable, with a warning), and from "test-prof wasn't active" (skip
|
|
10
|
+
# Rule① with a one-line setup hint).
|
|
11
|
+
module Doctor
|
|
12
|
+
module_function
|
|
13
|
+
|
|
14
|
+
def diagnose(result)
|
|
15
|
+
return diagnosis_impossible(result) unless result.rspec_json?
|
|
16
|
+
|
|
17
|
+
sections = []
|
|
18
|
+
sections << suite_red_warning(result) if result.suite_red?
|
|
19
|
+
sections << factory_prof_missing_hint unless result.factory_prof?
|
|
20
|
+
sections << Diagnosis.from_files(
|
|
21
|
+
rspec_json: result.rspec_json_path,
|
|
22
|
+
factory_prof_json: result.factory_prof? ? result.factory_prof_path : nil
|
|
23
|
+
)
|
|
24
|
+
sections.join("\n\n")
|
|
25
|
+
end
|
|
26
|
+
|
|
27
|
+
def diagnosis_impossible(result)
|
|
28
|
+
"診断不可: rspec の出力(JSON)が読めませんでした(exit #{result.exit_status})。" \
|
|
29
|
+
"suite はブートしましたか? `bundle exec rspec` を直接実行してエラーを確認してください。"
|
|
30
|
+
end
|
|
31
|
+
|
|
32
|
+
def suite_red_warning(result)
|
|
33
|
+
"注意: suite に失敗があります(exit #{result.exit_status})。" \
|
|
34
|
+
"全 example が走り切っていない場合、以下の数値は不安定なことがあります。"
|
|
35
|
+
end
|
|
36
|
+
|
|
37
|
+
def factory_prof_missing_hint
|
|
38
|
+
"注意: FactoryProf の出力がありません(test-prof 未導入、または spec_helper で require 忘れ)。" \
|
|
39
|
+
"factory 関連の診断(Rule①)をスキップしました。\n" \
|
|
40
|
+
'設定: Gemfile の development/test グループに test-prof を追加し、spec_helper で `require "test_prof"` してください。'
|
|
41
|
+
end
|
|
42
|
+
end
|
|
43
|
+
end
|
|
@@ -0,0 +1,11 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
module RspecSprint
|
|
4
|
+
# One diagnosis + prescription. `score` is the ROI/severity used by the Ranker
|
|
5
|
+
# to order findings (higher = more dominant). A rule returns a Finding when it
|
|
6
|
+
# fires, or nil when it is below threshold (signal-gated output, design D8).
|
|
7
|
+
Finding = Struct.new(
|
|
8
|
+
:rule_id, :score, :headline, :prescriptions, :expected_saving, :doc_url,
|
|
9
|
+
keyword_init: true
|
|
10
|
+
)
|
|
11
|
+
end
|
|
@@ -0,0 +1,29 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
module RspecSprint
|
|
4
|
+
# Renders ranked findings as a terminal report. Asserts the top few; when no
|
|
5
|
+
# rule fired it says so explicitly rather than padding with generic advice
|
|
6
|
+
# (design D8).
|
|
7
|
+
module Formatter
|
|
8
|
+
module_function
|
|
9
|
+
|
|
10
|
+
def format(findings)
|
|
11
|
+
return no_bottleneck if findings.empty?
|
|
12
|
+
|
|
13
|
+
lines = ["rspec-sprint doctor — 上位#{findings.size}件 (信号があった項目のみ):", ""]
|
|
14
|
+
findings.each_with_index do |f, i|
|
|
15
|
+
lines << "#{i + 1}. #{f.headline}"
|
|
16
|
+
f.prescriptions.each { |p| lines << " → #{p}" }
|
|
17
|
+
lines << " 想定削減: #{f.expected_saving}" if f.expected_saving
|
|
18
|
+
lines << " 参考: #{f.doc_url}" if f.doc_url
|
|
19
|
+
lines << ""
|
|
20
|
+
end
|
|
21
|
+
lines.join("\n")
|
|
22
|
+
end
|
|
23
|
+
|
|
24
|
+
def no_bottleneck
|
|
25
|
+
"支配的なボトルネックは検出されませんでした(各指標は閾値以下=該当なし)。" \
|
|
26
|
+
"生の数値は --verbose で出せますが、いま無理に直すべき箇所はありません。"
|
|
27
|
+
end
|
|
28
|
+
end
|
|
29
|
+
end
|
|
@@ -0,0 +1,113 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
require "json"
|
|
4
|
+
|
|
5
|
+
module RspecSprint
|
|
6
|
+
# Turns raw profiler JSON (rspec's JSON formatter + test-prof's FactoryProf
|
|
7
|
+
# JSON) into one normalized Snapshot. Input is FILE PATHS (decoupled from the
|
|
8
|
+
# live run) so golden fixtures and a future artifact-ingest mode feed the same
|
|
9
|
+
# code (design doc D3).
|
|
10
|
+
class Normalizer
|
|
11
|
+
# One factory's profile. top_level_ratio is the cascade signal: low ratio
|
|
12
|
+
# means the factory is built mostly via associations, not directly.
|
|
13
|
+
Factory = Struct.new(:name, :total_count, :top_level_count, :total_time, keyword_init: true) do
|
|
14
|
+
def top_level_ratio
|
|
15
|
+
return 0.0 if total_count.nil? || total_count.zero?
|
|
16
|
+
|
|
17
|
+
top_level_count.to_f / total_count
|
|
18
|
+
end
|
|
19
|
+
end
|
|
20
|
+
|
|
21
|
+
Example = Struct.new(:full_description, :file_path, :run_time, :status, keyword_init: true)
|
|
22
|
+
|
|
23
|
+
Snapshot = Struct.new(
|
|
24
|
+
:suite_duration, :factory_time, :factories,
|
|
25
|
+
:examples, :example_count, :failure_count,
|
|
26
|
+
keyword_init: true
|
|
27
|
+
) do
|
|
28
|
+
# Share of example-run wall time spent building factories (top-level, so
|
|
29
|
+
# cascade time is not double-counted). Denominator = summary.duration (D7).
|
|
30
|
+
def factory_time_ratio
|
|
31
|
+
return 0.0 if suite_duration.nil? || suite_duration.zero?
|
|
32
|
+
|
|
33
|
+
factory_time / suite_duration
|
|
34
|
+
end
|
|
35
|
+
|
|
36
|
+
def slow_examples
|
|
37
|
+
(examples || []).sort_by { |e| -e.run_time }
|
|
38
|
+
end
|
|
39
|
+
|
|
40
|
+
# run_time summed by the first path segment under spec/ (Rule② data).
|
|
41
|
+
# Deliberately "path group", not RSpec metadata `type:` — see design D10⑦.
|
|
42
|
+
def path_group_durations
|
|
43
|
+
(examples || []).each_with_object(Hash.new(0.0)) do |ex, acc|
|
|
44
|
+
acc[path_group(ex.file_path)] += ex.run_time
|
|
45
|
+
end
|
|
46
|
+
end
|
|
47
|
+
|
|
48
|
+
private
|
|
49
|
+
|
|
50
|
+
def path_group(file_path)
|
|
51
|
+
segments = file_path.to_s.sub(%r{\A\./}, "").split("/")
|
|
52
|
+
idx = segments.index("spec")
|
|
53
|
+
(idx && segments[idx + 1] && !segments[idx + 1].end_with?("_spec.rb") ? segments[idx + 1] : "(root)")
|
|
54
|
+
end
|
|
55
|
+
end
|
|
56
|
+
|
|
57
|
+
def initialize(rspec_json:, factory_prof_json: nil)
|
|
58
|
+
@rspec_json = rspec_json
|
|
59
|
+
@factory_prof_json = factory_prof_json
|
|
60
|
+
end
|
|
61
|
+
|
|
62
|
+
def call
|
|
63
|
+
rspec = JSON.parse(File.read(@rspec_json))
|
|
64
|
+
summary = rspec.fetch("summary", {})
|
|
65
|
+
Snapshot.new(
|
|
66
|
+
suite_duration: summary.fetch("duration"),
|
|
67
|
+
factory_time: top_level_factory_time,
|
|
68
|
+
factories: build_factories,
|
|
69
|
+
examples: build_examples(rspec.fetch("examples", [])),
|
|
70
|
+
example_count: summary.fetch("example_count", 0),
|
|
71
|
+
failure_count: summary.fetch("failure_count", 0)
|
|
72
|
+
)
|
|
73
|
+
end
|
|
74
|
+
|
|
75
|
+
private
|
|
76
|
+
|
|
77
|
+
def build_examples(raw)
|
|
78
|
+
raw.map do |e|
|
|
79
|
+
Example.new(
|
|
80
|
+
full_description: e["full_description"],
|
|
81
|
+
file_path: e["file_path"],
|
|
82
|
+
run_time: e.fetch("run_time", 0.0),
|
|
83
|
+
status: e["status"]
|
|
84
|
+
)
|
|
85
|
+
end
|
|
86
|
+
end
|
|
87
|
+
|
|
88
|
+
# FactoryProf `stats`, already sorted by total_count desc by test-prof.
|
|
89
|
+
def factory_stats
|
|
90
|
+
return [] unless @factory_prof_json && File.exist?(@factory_prof_json)
|
|
91
|
+
|
|
92
|
+
JSON.parse(File.read(@factory_prof_json)).fetch("stats", [])
|
|
93
|
+
end
|
|
94
|
+
|
|
95
|
+
def build_factories
|
|
96
|
+
factory_stats.map do |s|
|
|
97
|
+
Factory.new(
|
|
98
|
+
name: s.fetch("name"),
|
|
99
|
+
total_count: s.fetch("total_count", 0),
|
|
100
|
+
top_level_count: s.fetch("top_level_count", 0),
|
|
101
|
+
total_time: s.fetch("total_time", 0.0)
|
|
102
|
+
)
|
|
103
|
+
end
|
|
104
|
+
end
|
|
105
|
+
|
|
106
|
+
# Sum of per-stat top_level_time. test-prof's top-level total_time
|
|
107
|
+
# ("00:00.077") is a formatted STRING, so we never read that; we sum the
|
|
108
|
+
# per-stat raw floats instead (design doc Verification Findings ②).
|
|
109
|
+
def top_level_factory_time
|
|
110
|
+
factory_stats.sum { |s| s.fetch("top_level_time", 0.0) }
|
|
111
|
+
end
|
|
112
|
+
end
|
|
113
|
+
end
|
|
@@ -0,0 +1,22 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
require_relative "rules"
|
|
4
|
+
|
|
5
|
+
module RspecSprint
|
|
6
|
+
# Runs every rule against a snapshot, drops the ones that didn't fire
|
|
7
|
+
# (signal-gated, design D8), and returns the strongest few by score.
|
|
8
|
+
module Ranker
|
|
9
|
+
RULES = %i[factory_dominance path_group_skew slow_examples_concentration].freeze
|
|
10
|
+
MAX_FINDINGS = 3
|
|
11
|
+
|
|
12
|
+
module_function
|
|
13
|
+
|
|
14
|
+
def call(snapshot)
|
|
15
|
+
RULES
|
|
16
|
+
.map { |rule| Rules.public_send(rule, snapshot) }
|
|
17
|
+
.compact
|
|
18
|
+
.sort_by { |finding| -finding.score }
|
|
19
|
+
.first(MAX_FINDINGS)
|
|
20
|
+
end
|
|
21
|
+
end
|
|
22
|
+
end
|
|
@@ -0,0 +1,137 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
require_relative "finding"
|
|
4
|
+
|
|
5
|
+
module RspecSprint
|
|
6
|
+
# The judgment layer — the gem's moat. Each rule is a plain method (design D1):
|
|
7
|
+
# it reads a normalized Snapshot and returns a Finding when its signal exceeds
|
|
8
|
+
# a threshold, or nil otherwise. Thresholds and prescription wording are the
|
|
9
|
+
# opinionated, tunable heart of the tool (premise #3).
|
|
10
|
+
module Rules
|
|
11
|
+
# --- Tunable thresholds (the opinionated knobs) ---------------------------
|
|
12
|
+
FACTORY_DOMINANCE_THRESHOLD = 0.30 # fires when factory_time_ratio exceeds this
|
|
13
|
+
CASCADE_RATIO = 0.5 # top_level_ratio below this = built mostly via association
|
|
14
|
+
|
|
15
|
+
# Defaults (tune freely — these are the gem's IP):
|
|
16
|
+
PATH_GROUP_SKEW_THRESHOLD = 0.50 # one path group eating MORE than this share of suite time
|
|
17
|
+
SLOW_EXAMPLE_TOP_N = 5 # "a few" = this many slowest examples
|
|
18
|
+
SLOW_EXAMPLE_CONCENTRATION_THRESHOLD = 0.30 # top-N eating more than this share of suite time
|
|
19
|
+
SLOW_EXAMPLE_MIN_EXAMPLES = 10 # below this the "top-N" isn't a minority — skip (D8: no signal)
|
|
20
|
+
|
|
21
|
+
module_function
|
|
22
|
+
|
|
23
|
+
# Rule ①: factory build time dominates the suite.
|
|
24
|
+
def factory_dominance(snapshot)
|
|
25
|
+
ratio = snapshot.factory_time_ratio
|
|
26
|
+
return nil if ratio < FACTORY_DOMINANCE_THRESHOLD
|
|
27
|
+
|
|
28
|
+
pct = (ratio * 100).round
|
|
29
|
+
top = (snapshot.factories || []).max_by(&:total_count)
|
|
30
|
+
cascade = top && top.top_level_ratio < CASCADE_RATIO
|
|
31
|
+
|
|
32
|
+
Finding.new(
|
|
33
|
+
rule_id: :factory_dominance,
|
|
34
|
+
score: ratio,
|
|
35
|
+
headline: headline_for(pct, top, cascade),
|
|
36
|
+
prescriptions: [
|
|
37
|
+
"不要な association をデフォルト factory から外し trait へ退避",
|
|
38
|
+
"永続化が不要な spec は build_stubbed / build に置換",
|
|
39
|
+
"全 spec で使う共有データは let_it_be / FactoryDefault 化"
|
|
40
|
+
],
|
|
41
|
+
expected_saving: "factory time の一部(目安 suite の #{(pct * 0.3).round}–#{pct}%、local実測)",
|
|
42
|
+
doc_url: "https://test-prof.evilmartians.io/guide/profilers/factory_prof"
|
|
43
|
+
)
|
|
44
|
+
end
|
|
45
|
+
|
|
46
|
+
# Rule ②: one "path group" (spec/system, spec/requests, ...) dominates the
|
|
47
|
+
# suite's wall time. NOTE: call it a path group, not RSpec `type:` — they can
|
|
48
|
+
# diverge (design D10⑦). Do NOT assert "split your system specs"; offer it as
|
|
49
|
+
# a candidate.
|
|
50
|
+
#
|
|
51
|
+
# Data available on `snapshot`:
|
|
52
|
+
# snapshot.path_group_durations # => { "system" => 1.5, "models" => 0.3, ... } seconds
|
|
53
|
+
# snapshot.suite_duration # => total example-run wall time (the denominator)
|
|
54
|
+
#
|
|
55
|
+
# Return a Finding when the biggest group's share exceeds
|
|
56
|
+
# PATH_GROUP_SKEW_THRESHOLD, else nil. Score = that share (0.0..1.0).
|
|
57
|
+
#
|
|
58
|
+
def path_group_skew(snapshot)
|
|
59
|
+
total = snapshot.suite_duration.to_f
|
|
60
|
+
return nil if total.zero?
|
|
61
|
+
|
|
62
|
+
# Drop "(root)" — specs directly under spec/ aren't a coherent group you'd
|
|
63
|
+
# split into a CI job (dogfooding finding; cf. Codex #8 path-group risk).
|
|
64
|
+
candidates = (snapshot.path_group_durations || {}).reject { |g, _| g == "(root)" }
|
|
65
|
+
group, time = candidates.max_by { |_, t| t }
|
|
66
|
+
return nil if group.nil?
|
|
67
|
+
|
|
68
|
+
share = time / total
|
|
69
|
+
return nil unless share > PATH_GROUP_SKEW_THRESHOLD
|
|
70
|
+
|
|
71
|
+
pct = (share * 100).round
|
|
72
|
+
Finding.new(
|
|
73
|
+
rule_id: :path_group_skew,
|
|
74
|
+
score: share,
|
|
75
|
+
headline: "path group spec/#{group} が suite 時間の #{pct}% (local実測)",
|
|
76
|
+
prescriptions: [
|
|
77
|
+
"spec/#{group} を別CIジョブに分離し fast suite から切り出す(候補)",
|
|
78
|
+
"ロジックを下位レイヤ(model/request/service)へ寄せ #{group} spec 本数を減らせるか検討",
|
|
79
|
+
"runtime ログで #{group} を shard 分割"
|
|
80
|
+
],
|
|
81
|
+
expected_saving: "spec/#{group} を分離すれば fast suite から最大 #{pct}% 相当を外せる(local実測)",
|
|
82
|
+
doc_url: "https://test-prof.evilmartians.io/guide/profilers/tag_prof"
|
|
83
|
+
)
|
|
84
|
+
end
|
|
85
|
+
|
|
86
|
+
# Rule ③: a few slow examples concentrate most of the suite time.
|
|
87
|
+
#
|
|
88
|
+
# Data available on `snapshot`:
|
|
89
|
+
# snapshot.slow_examples # => [Example(full_description, file_path, run_time, status)] slow-first
|
|
90
|
+
# snapshot.suite_duration # => denominator
|
|
91
|
+
#
|
|
92
|
+
# Return a Finding when the top-10 examples' summed run_time / suite_duration
|
|
93
|
+
# exceeds SLOW_EXAMPLE_CONCENTRATION_THRESHOLD, naming the worst few. Score =
|
|
94
|
+
# that concentration.
|
|
95
|
+
#
|
|
96
|
+
def slow_examples_concentration(snapshot)
|
|
97
|
+
total = snapshot.suite_duration.to_f
|
|
98
|
+
return nil if total.zero?
|
|
99
|
+
|
|
100
|
+
examples = snapshot.slow_examples || []
|
|
101
|
+
count = snapshot.example_count || examples.size
|
|
102
|
+
return nil if count < SLOW_EXAMPLE_MIN_EXAMPLES # tiny suite: top-N isn't a minority (D8)
|
|
103
|
+
|
|
104
|
+
top = examples.first(SLOW_EXAMPLE_TOP_N)
|
|
105
|
+
return nil if top.empty?
|
|
106
|
+
|
|
107
|
+
concentration = top.sum(&:run_time) / total
|
|
108
|
+
return nil unless concentration > SLOW_EXAMPLE_CONCENTRATION_THRESHOLD
|
|
109
|
+
|
|
110
|
+
pct = (concentration * 100).round
|
|
111
|
+
worst = top.first
|
|
112
|
+
Finding.new(
|
|
113
|
+
rule_id: :slow_examples_concentration,
|
|
114
|
+
score: concentration,
|
|
115
|
+
headline: "上位#{top.size}例が suite 時間の #{pct}% (local実測)。最遅: \"#{worst.full_description}\" (#{(worst.run_time * 1000).round}ms)",
|
|
116
|
+
prescriptions: [
|
|
117
|
+
"最遅例に EVENT_PROF='sql.active_record' / TEST_STACK_PROF を当てて原因特定",
|
|
118
|
+
"重い setup (let! / before(:each)) を let_it_be / build_stubbed へ",
|
|
119
|
+
"外部I/O・sleep・実ブラウザ依存を切り出す"
|
|
120
|
+
],
|
|
121
|
+
expected_saving: "上位例を半減できれば最大 #{(pct * 0.5).round}% 相当(local実測)",
|
|
122
|
+
doc_url: "https://test-prof.evilmartians.io/guide/profilers/event_prof"
|
|
123
|
+
)
|
|
124
|
+
end
|
|
125
|
+
|
|
126
|
+
def headline_for(pct, top, cascade)
|
|
127
|
+
base = "factory が suite の #{pct}% (local実測)"
|
|
128
|
+
return base unless top
|
|
129
|
+
|
|
130
|
+
if cascade
|
|
131
|
+
"#{base}。最上位は :#{top.name}(#{top.total_count}回, うち直接生成 #{top.top_level_count}回 = カスケード)"
|
|
132
|
+
else
|
|
133
|
+
"#{base}。最上位は :#{top.name}(#{top.total_count}回)"
|
|
134
|
+
end
|
|
135
|
+
end
|
|
136
|
+
end
|
|
137
|
+
end
|
data/lib/rspec_sprint.rb
ADDED
|
@@ -0,0 +1,10 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
require_relative "rspec_sprint/version"
|
|
4
|
+
|
|
5
|
+
# rspec-sprint: diagnose RSpec/Rails test-suite slowness and assert the top
|
|
6
|
+
# repo-specific things to fix. Wraps test-prof + rspec JSON; it does not
|
|
7
|
+
# reimplement profilers.
|
|
8
|
+
module RspecSprint
|
|
9
|
+
class Error < StandardError; end
|
|
10
|
+
end
|
metadata
ADDED
|
@@ -0,0 +1,88 @@
|
|
|
1
|
+
--- !ruby/object:Gem::Specification
|
|
2
|
+
name: rspec-sprint
|
|
3
|
+
version: !ruby/object:Gem::Version
|
|
4
|
+
version: 0.1.0
|
|
5
|
+
platform: ruby
|
|
6
|
+
authors:
|
|
7
|
+
- yasu551
|
|
8
|
+
bindir: exe
|
|
9
|
+
cert_chain: []
|
|
10
|
+
date: 1980-01-02 00:00:00.000000000 Z
|
|
11
|
+
dependencies:
|
|
12
|
+
- !ruby/object:Gem::Dependency
|
|
13
|
+
name: test-prof
|
|
14
|
+
requirement: !ruby/object:Gem::Requirement
|
|
15
|
+
requirements:
|
|
16
|
+
- - ">="
|
|
17
|
+
- !ruby/object:Gem::Version
|
|
18
|
+
version: '1.0'
|
|
19
|
+
type: :runtime
|
|
20
|
+
prerelease: false
|
|
21
|
+
version_requirements: !ruby/object:Gem::Requirement
|
|
22
|
+
requirements:
|
|
23
|
+
- - ">="
|
|
24
|
+
- !ruby/object:Gem::Version
|
|
25
|
+
version: '1.0'
|
|
26
|
+
- !ruby/object:Gem::Dependency
|
|
27
|
+
name: thor
|
|
28
|
+
requirement: !ruby/object:Gem::Requirement
|
|
29
|
+
requirements:
|
|
30
|
+
- - ">="
|
|
31
|
+
- !ruby/object:Gem::Version
|
|
32
|
+
version: '1.0'
|
|
33
|
+
type: :runtime
|
|
34
|
+
prerelease: false
|
|
35
|
+
version_requirements: !ruby/object:Gem::Requirement
|
|
36
|
+
requirements:
|
|
37
|
+
- - ">="
|
|
38
|
+
- !ruby/object:Gem::Version
|
|
39
|
+
version: '1.0'
|
|
40
|
+
description: |
|
|
41
|
+
rspec-sprint doctor runs your suite once (via test-prof's FactoryProf + rspec's
|
|
42
|
+
JSON formatter), interprets the output against opinionated heuristics, and asserts
|
|
43
|
+
the top few things to fix — diagnosis plus concrete prescription, not just numbers.
|
|
44
|
+
It wraps existing profilers; it does not reimplement them.
|
|
45
|
+
executables:
|
|
46
|
+
- rspec-sprint
|
|
47
|
+
extensions: []
|
|
48
|
+
extra_rdoc_files: []
|
|
49
|
+
files:
|
|
50
|
+
- LICENSE.txt
|
|
51
|
+
- README.md
|
|
52
|
+
- exe/rspec-sprint
|
|
53
|
+
- lib/rspec_sprint.rb
|
|
54
|
+
- lib/rspec_sprint/cli.rb
|
|
55
|
+
- lib/rspec_sprint/collector.rb
|
|
56
|
+
- lib/rspec_sprint/diagnosis.rb
|
|
57
|
+
- lib/rspec_sprint/doctor.rb
|
|
58
|
+
- lib/rspec_sprint/finding.rb
|
|
59
|
+
- lib/rspec_sprint/formatter.rb
|
|
60
|
+
- lib/rspec_sprint/normalizer.rb
|
|
61
|
+
- lib/rspec_sprint/ranker.rb
|
|
62
|
+
- lib/rspec_sprint/rules.rb
|
|
63
|
+
- lib/rspec_sprint/version.rb
|
|
64
|
+
homepage: https://github.com/yasu551/rspec-sprint
|
|
65
|
+
licenses:
|
|
66
|
+
- MIT
|
|
67
|
+
metadata:
|
|
68
|
+
source_code_uri: https://github.com/yasu551/rspec-sprint
|
|
69
|
+
changelog_uri: https://github.com/yasu551/rspec-sprint/blob/main/CHANGELOG.md
|
|
70
|
+
rdoc_options: []
|
|
71
|
+
require_paths:
|
|
72
|
+
- lib
|
|
73
|
+
required_ruby_version: !ruby/object:Gem::Requirement
|
|
74
|
+
requirements:
|
|
75
|
+
- - ">="
|
|
76
|
+
- !ruby/object:Gem::Version
|
|
77
|
+
version: '3.0'
|
|
78
|
+
required_rubygems_version: !ruby/object:Gem::Requirement
|
|
79
|
+
requirements:
|
|
80
|
+
- - ">="
|
|
81
|
+
- !ruby/object:Gem::Version
|
|
82
|
+
version: '0'
|
|
83
|
+
requirements: []
|
|
84
|
+
rubygems_version: 4.0.10
|
|
85
|
+
specification_version: 4
|
|
86
|
+
summary: Diagnose RSpec/Rails test-suite slowness and assert the top repo-specific
|
|
87
|
+
fixes.
|
|
88
|
+
test_files: []
|