rails-hyperdrive-martian-spec 0.3.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +7 -0
- data/CHANGELOG.md +92 -0
- data/LICENSE.txt +21 -0
- data/README.md +19 -0
- data/commands/bisect-order.md +21 -0
- data/commands/event-prof.md +21 -0
- data/commands/factory-default-prof.md +22 -0
- data/commands/fprof.md +21 -0
- data/commands/mem-prof.md +20 -0
- data/commands/profile-specs.md.erb +65 -0
- data/commands/rd-prof.md +20 -0
- data/commands/slowest-specs.md +20 -0
- data/commands/stack-prof.md +26 -0
- data/commands/tps-prof.md +20 -0
- data/hyperdrive.yml +18 -0
- data/lib/rails-hyperdrive-martian-spec/hyperdrive/skills/martian-spec/SKILL.md.erb +326 -0
- data/lib/rails-hyperdrive-martian-spec/hyperdrive/skills/martian-spec/references/data-setup.md +176 -0
- data/lib/rails-hyperdrive-martian-spec/hyperdrive/skills/martian-spec/references/parallel-ci.md +77 -0
- data/lib/rails-hyperdrive-martian-spec/hyperdrive/skills/martian-spec/references/profiling.md +24 -0
- data/lib/rails-hyperdrive-martian-spec/hyperdrive/skills/martian-spec/references/structure.md +45 -0
- data/lib/rails-hyperdrive-martian-spec/version.rb +3 -0
- data/lib/rails-hyperdrive-martian-spec.rb +4 -0
- metadata +71 -0
checksums.yaml
ADDED
|
@@ -0,0 +1,7 @@
|
|
|
1
|
+
---
|
|
2
|
+
SHA256:
|
|
3
|
+
metadata.gz: 761d710c55b5709f60057b13fc4cdd6a48f720c2abdb05a7112ef9b4da93c805
|
|
4
|
+
data.tar.gz: 071ff2fe7200c95bdcdb9ea7fc436055e7a2f17e7a587852a34fc3029ee76e00
|
|
5
|
+
SHA512:
|
|
6
|
+
metadata.gz: 12fd655921158c815a0cd98331e0cb2f9c25b13f9eec9aa7b5b599e78bfa710f4659c6bfd06276762312bb03c6d8a005c1a34e71a143aec83f706f3a0f91da65
|
|
7
|
+
data.tar.gz: c4b8b868e5d97d25e925006b86171402877a3b4d615869a2991d5b037335b4a3c1f08f158418a3a1aeddea2bff42a73eeff4b6be1ecae397105cc38a581bf1d0
|
data/CHANGELOG.md
ADDED
|
@@ -0,0 +1,92 @@
|
|
|
1
|
+
# Changelog
|
|
2
|
+
|
|
3
|
+
All notable changes to this project will be documented in this file.
|
|
4
|
+
|
|
5
|
+
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
|
|
6
|
+
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
|
|
7
|
+
|
|
8
|
+
## [Unreleased]
|
|
9
|
+
|
|
10
|
+
## [0.3.1] - 2026-09-01
|
|
11
|
+
|
|
12
|
+
### Fixed
|
|
13
|
+
- Profiler slash commands are gated on both rspec-rails and test-prof. A
|
|
14
|
+
per-entry `gem:` in the manifest replaces the gem-wide rspec-rails gate, so
|
|
15
|
+
the seven profiler commands used to install into bundles that had test-prof
|
|
16
|
+
but no rspec-rails. 0.3.0 was tagged but never published to rubygems.
|
|
17
|
+
|
|
18
|
+
## [0.3.0] - 2026-09-01
|
|
19
|
+
|
|
20
|
+
### Added
|
|
21
|
+
- Slash commands as a second artifact kind, installed into `.claude/commands/`:
|
|
22
|
+
seven test-prof profilers (`/fprof`, `/event-prof`, `/rd-prof`, `/tps-prof`,
|
|
23
|
+
`/factory-default-prof`, `/mem-prof`, `/stack-prof`, gated on test-prof via
|
|
24
|
+
the manifest `commands:` section), two ungated RSpec diagnostics
|
|
25
|
+
(`/slowest-specs`, `/bisect-order`), and a `/profile-specs` entry point
|
|
26
|
+
templated for bundles with and without test-prof. Each command carries the
|
|
27
|
+
flag, result interpretation, and a usage example.
|
|
28
|
+
- `martian-spec` skill: supporting references extracted from the skill body —
|
|
29
|
+
`data-setup.md` (test-prof decision tables, checklist, `refind:`, red flags,
|
|
30
|
+
symptom table), `profiling.md` (runnable cheat-sheet commands, pointing at
|
|
31
|
+
the slash commands), `structure.md` (ordering convention with annotated
|
|
32
|
+
example) — all three conditional on test-prof — and `parallel-ci.md`
|
|
33
|
+
(runner comparison, per-worker databases, shared-cache namespacing),
|
|
34
|
+
installed unconditionally.
|
|
35
|
+
- `martian-spec` skill: `allowed-tools` frontmatter; a "Never hit real
|
|
36
|
+
external APIs" rule with WebMock/VCR notes gated on those gems; a gated
|
|
37
|
+
bcrypt cost snippet; generic flaky-spec diagnostics (factory profiling via
|
|
38
|
+
`ActiveSupport::Notifications`, time-leak and `before(:context)` checks);
|
|
39
|
+
a named-subject convention; a cache-store-aware failsafe checklist.
|
|
40
|
+
- `martian-spec` skill: when the bundle lacks test-prof, the installed skill
|
|
41
|
+
instructs the agent to suggest adding it (`bundle add test-prof --group
|
|
42
|
+
test` plus recipe requires) rather than adding it silently.
|
|
43
|
+
- `Rakefile` wiring rails-hyperdrive's author-side manifest lint
|
|
44
|
+
(`rake hyperdrive:manifest:check`) and a CI workflow running it.
|
|
45
|
+
`rails-hyperdrive` and `rake` are development-only Gemfile dependencies.
|
|
46
|
+
|
|
47
|
+
### Changed
|
|
48
|
+
- Migrated to the rails-hyperdrive 0.8.0 companion contract. The skill is a
|
|
49
|
+
standalone master template
|
|
50
|
+
(`lib/rails-hyperdrive-martian-spec/hyperdrive/skills/martian-spec/SKILL.md.erb`
|
|
51
|
+
with `references/` alongside it) rendered per-app at install; content varies
|
|
52
|
+
with the consuming app's bundle via `gem?` blocks (test-prof, graphql,
|
|
53
|
+
rspec-sidekiq, sidekiq, webmock, vcr, bcrypt). Old `gem:`/`versions:`
|
|
54
|
+
frontmatter gating moved into a gem-root `hyperdrive.yml`, where the
|
|
55
|
+
requirement rides on the target member
|
|
56
|
+
(`gems: [rspec-rails: ">= 6.0, < 9.0"]` — the `versions:` key is retired).
|
|
57
|
+
Gemspec discovery metadata keys renamed from `rails_hyperdrive_*` to
|
|
58
|
+
`hyperdrive_targets` / `hyperdrive_artifacts` (now `"skill,command"`), the
|
|
59
|
+
keys `hyperdrive:discover` reads since rails-hyperdrive 0.6.
|
|
60
|
+
- `martian-spec` skill description rewritten without third-party gem mentions;
|
|
61
|
+
the intro, layer table, and examples generalized (handler example is a plain
|
|
62
|
+
request spec; "Worker spec" is "Job spec").
|
|
63
|
+
- `martian-spec` skill: wrong-layer guidance reworded as design-smell signals
|
|
64
|
+
("flag it, put coverage where the logic lives") instead of extraction
|
|
65
|
+
directives; the duplicate wrong-layer escalation ladder was dropped.
|
|
66
|
+
- Renamed the gem from `rails-hyperdrive-rspec` to `rails-hyperdrive-martian-spec`.
|
|
67
|
+
The `RailsHyperdriveRspec` module is now `RailsHyperdriveMartianSpec` and the
|
|
68
|
+
entrypoint is `require "rails-hyperdrive-martian-spec"`. `rails-hyperdrive-rspec`
|
|
69
|
+
0.2.0 remains on rubygems but receives no further releases.
|
|
70
|
+
|
|
71
|
+
## [0.2.0] - 2026-08-06
|
|
72
|
+
|
|
73
|
+
### Added
|
|
74
|
+
- Initial scaffold of the companion gem.
|
|
75
|
+
- `martian-spec` skill targeting `rspec-rails`.
|
|
76
|
+
- `martian-spec` skill: "Suite-level speed (CI)" section covering log quieting,
|
|
77
|
+
weakening cryptographic KDFs in test, fair parallelization, and global
|
|
78
|
+
failsafes for leaky state.
|
|
79
|
+
- `martian-spec` skill: profiler cheat-sheet mapping each test-prof env flag to
|
|
80
|
+
its tool and the question it answers.
|
|
81
|
+
- Gemspec discovery metadata (`rails_hyperdrive_targets`,
|
|
82
|
+
`rails_hyperdrive_artifacts`) so `hyperdrive:discover` finds the gem on
|
|
83
|
+
rubygems before it is bundled.
|
|
84
|
+
|
|
85
|
+
### Changed
|
|
86
|
+
- `martian-spec` skill description: also triggers when implementing a new
|
|
87
|
+
feature or fixing a bug, not only when touching spec files directly.
|
|
88
|
+
|
|
89
|
+
[Unreleased]: https://github.com/izhanov/rails-hyperdrive-martian-spec/compare/v0.3.1...HEAD
|
|
90
|
+
[0.3.1]: https://github.com/izhanov/rails-hyperdrive-martian-spec/compare/v0.3.0...v0.3.1
|
|
91
|
+
[0.3.0]: https://github.com/izhanov/rails-hyperdrive-martian-spec/releases/tag/v0.3.0
|
|
92
|
+
[0.2.0]: https://github.com/izhanov/rails-hyperdrive-martian-spec/releases/tag/v0.2.0
|
data/LICENSE.txt
ADDED
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
MIT License
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2026 izhanov
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
+
in the Software without restriction, including without limitation the rights
|
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
+
furnished to do so, subject to the following conditions:
|
|
11
|
+
|
|
12
|
+
The above copyright notice and this permission notice shall be included in all
|
|
13
|
+
copies or substantial portions of the Software.
|
|
14
|
+
|
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
21
|
+
SOFTWARE.
|
data/README.md
ADDED
|
@@ -0,0 +1,19 @@
|
|
|
1
|
+
# rails-hyperdrive-martian-spec
|
|
2
|
+
|
|
3
|
+
Companion gem for [rails-hyperdrive](https://github.com/rails-hyperdrive/rails-hyperdrive) — ships the `martian-spec` skill for AI coding agents working with RSpec in Rails projects.
|
|
4
|
+
|
|
5
|
+
## Install
|
|
6
|
+
|
|
7
|
+
```ruby
|
|
8
|
+
# Gemfile
|
|
9
|
+
group :development do
|
|
10
|
+
gem "rails-hyperdrive"
|
|
11
|
+
gem "rails-hyperdrive-martian-spec"
|
|
12
|
+
end
|
|
13
|
+
```
|
|
14
|
+
|
|
15
|
+
Then run `bin/rails hyperdrive:init`. The skill is installed to `.claude/skills/martian-spec/SKILL.md` and loaded lazily by Claude Code when relevant.
|
|
16
|
+
|
|
17
|
+
## License
|
|
18
|
+
|
|
19
|
+
MIT — see [LICENSE.txt](LICENSE.txt).
|
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
---
|
|
2
|
+
description: Isolate the minimal failing example pair for an order-dependent spec
|
|
3
|
+
argument-hint: <seed> [spec file or directory]
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
Reproduce and minimize an order-dependence with RSpec's bisect (the seed comes
|
|
7
|
+
from the failing CI/local run's output — ask for it if missing):
|
|
8
|
+
|
|
9
|
+
```sh
|
|
10
|
+
bundle exec rspec --seed $ARGUMENTS --bisect
|
|
11
|
+
```
|
|
12
|
+
|
|
13
|
+
Interpret: bisect prints the minimal set of examples that must run together to
|
|
14
|
+
reproduce the failure. The *earlier* example in the pair leaks state (time,
|
|
15
|
+
mocks, DB records outside the transaction, Redis/cache); the later one is the
|
|
16
|
+
victim. Check the leak diagnostics in the `martian-spec` skill's Escalation
|
|
17
|
+
Ladder to classify and fix it.
|
|
18
|
+
|
|
19
|
+
Example: `/bisect-order 54321 spec/models/`
|
|
20
|
+
|
|
21
|
+
Report the minimal failing pair, the leaked state, and the fix.
|
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
---
|
|
2
|
+
description: Measure time spent in one event type (test-prof EventProf), e.g. SQL or factories
|
|
3
|
+
argument-hint: <spec file or directory> [event]
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
Run test-prof's EventProf against the given path (ask for one if missing).
|
|
7
|
+
The second argument is the event; default to `sql.active_record`:
|
|
8
|
+
|
|
9
|
+
```sh
|
|
10
|
+
EVENT_PROF='sql.active_record' bundle exec rspec <path> # time in SQL
|
|
11
|
+
EVENT_PROF='factory.create' bundle exec rspec <path> # time in factories
|
|
12
|
+
```
|
|
13
|
+
|
|
14
|
+
Interpret the share, not the absolute number: if SQL is ~10% of total time,
|
|
15
|
+
don't chase N+1s — profile factories and hooks instead. If factories dominate,
|
|
16
|
+
follow up with `/fprof` to find which ones.
|
|
17
|
+
|
|
18
|
+
Examples: `/event-prof spec/models/user_spec.rb`,
|
|
19
|
+
`/event-prof spec/services/ factory.create`
|
|
20
|
+
|
|
21
|
+
Report the event's share of total time and where to look next.
|
|
@@ -0,0 +1,22 @@
|
|
|
1
|
+
---
|
|
2
|
+
description: Find implicit associations that create_default could share (test-prof FactoryDefault)
|
|
3
|
+
argument-hint: <spec file or directory>
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
Run test-prof's FactoryDefault profiler against the given path (ask for one if
|
|
7
|
+
missing):
|
|
8
|
+
|
|
9
|
+
```sh
|
|
10
|
+
FACTORY_DEFAULT_PROF=1 bundle exec rspec $ARGUMENTS
|
|
11
|
+
```
|
|
12
|
+
|
|
13
|
+
Interpret: the report counts how often each factory was created as an implicit
|
|
14
|
+
association — high counts are records `create_default` could share instead of
|
|
15
|
+
re-creating. Caveat: `create_default` only intercepts top-level factory
|
|
16
|
+
associations, not associations defined inside traits — for trait cascades,
|
|
17
|
+
eliminate the trait instead (see the `martian-spec` skill's
|
|
18
|
+
`references/data-setup.md`).
|
|
19
|
+
|
|
20
|
+
Example: `/factory-default-prof spec/models/`
|
|
21
|
+
|
|
22
|
+
Report the top shareable associations and whether `create_default` applies.
|
data/commands/fprof.md
ADDED
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
---
|
|
2
|
+
description: Count factory runs per factory (test-prof FactoryProf) to find cascades
|
|
3
|
+
argument-hint: <spec file or directory>
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
Run test-prof's FactoryProf against the given path (ask for one if missing —
|
|
7
|
+
whole-suite runs drown the signal):
|
|
8
|
+
|
|
9
|
+
```sh
|
|
10
|
+
FPROF=1 bundle exec rspec $ARGUMENTS
|
|
11
|
+
```
|
|
12
|
+
|
|
13
|
+
Interpret the table: `top-level` is factories the specs called explicitly,
|
|
14
|
+
`total` includes records spawned through associations and traits. `total` ≫
|
|
15
|
+
`top-level` for a factory means a cascade — trace the gap to an association or
|
|
16
|
+
trait and inline or share it (see the `martian-spec` skill's
|
|
17
|
+
`references/data-setup.md`).
|
|
18
|
+
|
|
19
|
+
Example: `/fprof spec/services/billing/`
|
|
20
|
+
|
|
21
|
+
Report the top offending factories, the cascade source for each, and the fix.
|
|
@@ -0,0 +1,20 @@
|
|
|
1
|
+
---
|
|
2
|
+
description: Find examples contributing most to GC time (test-prof memory profiler)
|
|
3
|
+
argument-hint: <spec file or directory>
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
Run test-prof's memory profiler against the given path (ask for one if
|
|
7
|
+
missing):
|
|
8
|
+
|
|
9
|
+
```sh
|
|
10
|
+
TEST_MEM_PROF=gc bundle exec rspec $ARGUMENTS
|
|
11
|
+
```
|
|
12
|
+
|
|
13
|
+
Interpret: examples at the top allocate the most and drive GC pauses — usual
|
|
14
|
+
suspects are giant fixtures/payloads built per example, `create_list` with big
|
|
15
|
+
counts, and loading large files in setup. Share or shrink the allocation
|
|
16
|
+
rather than tuning GC.
|
|
17
|
+
|
|
18
|
+
Example: `/mem-prof spec/services/import/`
|
|
19
|
+
|
|
20
|
+
Report the top allocating examples and what they allocate.
|
|
@@ -0,0 +1,65 @@
|
|
|
1
|
+
---
|
|
2
|
+
description: Profile RSpec files to find where the time goes (factories, SQL, setup)
|
|
3
|
+
argument-hint: <spec file or directory>
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
Profile the specs at `$ARGUMENTS` and report where the time goes. If no path
|
|
7
|
+
was given, ask for one — profiling the whole suite drowns the signal in noise.
|
|
8
|
+
|
|
9
|
+
Usage examples:
|
|
10
|
+
|
|
11
|
+
```
|
|
12
|
+
/profile-specs spec/models/user_spec.rb
|
|
13
|
+
/profile-specs spec/services/
|
|
14
|
+
```
|
|
15
|
+
|
|
16
|
+
Run the steps in order and interpret each result before moving to the next.
|
|
17
|
+
<%- if gem?("test-prof") -%>
|
|
18
|
+
|
|
19
|
+
1. **Find the slowest examples**: `bundle exec rspec --profile 10 $ARGUMENTS`.
|
|
20
|
+
If the top entries share one feature (e.g. all touch password hashing),
|
|
21
|
+
suspect expensive global setup, not factories.
|
|
22
|
+
2. **Count factory runs**: `FPROF=1 bundle exec rspec $ARGUMENTS`.
|
|
23
|
+
`total` ≫ `top-level` for a factory means a cascade — associations or
|
|
24
|
+
traits creating records the specs never asked for.
|
|
25
|
+
3. **Measure SQL share**: `EVENT_PROF='sql.active_record' bundle exec rspec $ARGUMENTS`.
|
|
26
|
+
If SQL is a small fraction of total time, don't chase N+1s — look at
|
|
27
|
+
factories and hooks instead.
|
|
28
|
+
4. **Split hook time from example time**: `RD_PROF=1 bundle exec rspec $ARGUMENTS`.
|
|
29
|
+
Heavy `before`-hook time with light example bodies points at setup that
|
|
30
|
+
should be shared (`let_it_be` / `before_all`).
|
|
31
|
+
|
|
32
|
+
The full flag-by-flag cheat-sheet (including `TPS_PROF`, `TEST_MEM_PROF`,
|
|
33
|
+
`TEST_STACK_PROF`) lives in the `martian-spec` skill's
|
|
34
|
+
`references/profiling.md`.
|
|
35
|
+
<%- end -%>
|
|
36
|
+
<%- if !gem?("test-prof") -%>
|
|
37
|
+
|
|
38
|
+
1. **Find the slowest examples**: `bundle exec rspec --profile 10 $ARGUMENTS`.
|
|
39
|
+
If the top entries share one feature (e.g. all touch password hashing),
|
|
40
|
+
suspect expensive global setup, not factories.
|
|
41
|
+
2. **Count factory runs** with a temporary `ActiveSupport::Notifications`
|
|
42
|
+
subscriber in `spec/rails_helper.rb` (remove it after diagnosing):
|
|
43
|
+
|
|
44
|
+
```ruby
|
|
45
|
+
counts = Hash.new(0)
|
|
46
|
+
ActiveSupport::Notifications.subscribe("factory_bot.run_factory") do |_, _, _, _, payload|
|
|
47
|
+
counts[[payload[:name], payload[:strategy]]] += 1
|
|
48
|
+
end
|
|
49
|
+
at_exit { counts.sort_by { |_, n| -n }.first(20).each { |k, n| puts "#{n}\t#{k.join(" ")}" } }
|
|
50
|
+
```
|
|
51
|
+
|
|
52
|
+
A factory with a high count that the file never calls explicitly is a
|
|
53
|
+
cascade — associations or traits creating records the specs never asked for.
|
|
54
|
+
|
|
55
|
+
This project does not use test-prof, which packages these diagnostics as
|
|
56
|
+
one-env-var profilers (`FPROF=1`, `EVENT_PROF=...`) and adds `let_it_be` /
|
|
57
|
+
`before_all` to fix what they find. Suggest adding it — don't add it silently:
|
|
58
|
+
|
|
59
|
+
```sh
|
|
60
|
+
bundle add test-prof --group test
|
|
61
|
+
```
|
|
62
|
+
<%- end -%>
|
|
63
|
+
|
|
64
|
+
Finish with a short report: the top time sinks, whether the bottleneck is
|
|
65
|
+
factories, SQL, hooks, or example bodies, and the single highest-leverage fix.
|
data/commands/rd-prof.md
ADDED
|
@@ -0,0 +1,20 @@
|
|
|
1
|
+
---
|
|
2
|
+
description: Split before-hook time from example-body time (test-prof RSpecDissect)
|
|
3
|
+
argument-hint: <spec file or directory>
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
Run test-prof's RSpecDissect against the given path (ask for one if missing):
|
|
7
|
+
|
|
8
|
+
```sh
|
|
9
|
+
RD_PROF=1 bundle exec rspec $ARGUMENTS
|
|
10
|
+
```
|
|
11
|
+
|
|
12
|
+
Interpret: heavy `before`-hook time with light example bodies means the setup
|
|
13
|
+
should be shared — `let_it_be` for data, `before_all` for setup (check the
|
|
14
|
+
pre-conversion checklist in the `martian-spec` skill's
|
|
15
|
+
`references/data-setup.md` first). Heavy example bodies point at the code
|
|
16
|
+
under test or over-asserting examples, not setup.
|
|
17
|
+
|
|
18
|
+
Example: `/rd-prof spec/requests/`
|
|
19
|
+
|
|
20
|
+
Report the hook/body split for the slowest groups and the conversion candidates.
|
|
@@ -0,0 +1,20 @@
|
|
|
1
|
+
---
|
|
2
|
+
description: Print the slowest examples and groups in a spec run
|
|
3
|
+
argument-hint: <spec file or directory>
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
Run RSpec's built-in profiler against the given path (or the whole suite —
|
|
7
|
+
this one is cheap enough):
|
|
8
|
+
|
|
9
|
+
```sh
|
|
10
|
+
bundle exec rspec --profile 10 $ARGUMENTS
|
|
11
|
+
```
|
|
12
|
+
|
|
13
|
+
Interpret: this says *which* examples are slow, not *why*. If the top entries
|
|
14
|
+
cluster around one feature (e.g. everything touches password hashing), suspect
|
|
15
|
+
expensive global setup. Follow up with the profiling steps in the
|
|
16
|
+
`martian-spec` skill's Escalation Ladder to find the cause.
|
|
17
|
+
|
|
18
|
+
Example: `/slowest-specs spec/`
|
|
19
|
+
|
|
20
|
+
Report the slowest examples and the pattern they share, if any.
|
|
@@ -0,0 +1,26 @@
|
|
|
1
|
+
---
|
|
2
|
+
description: CPU flamegraph of the hottest code paths in a spec run (test-prof + stackprof)
|
|
3
|
+
argument-hint: <spec file or directory>
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
Run test-prof's StackProf integration against the given path (ask for one if
|
|
7
|
+
missing; requires the `stackprof` gem):
|
|
8
|
+
|
|
9
|
+
```sh
|
|
10
|
+
TEST_STACK_PROF=1 bundle exec rspec $ARGUMENTS
|
|
11
|
+
```
|
|
12
|
+
|
|
13
|
+
Then inspect the generated dump (path is printed at the end of the run):
|
|
14
|
+
|
|
15
|
+
```sh
|
|
16
|
+
bundle exec stackprof tmp/test_prof/stack-prof-report-*.dump --text | head -30
|
|
17
|
+
```
|
|
18
|
+
|
|
19
|
+
Interpret: this answers *what code* burns CPU when factories and SQL are
|
|
20
|
+
already ruled out — crypto/KDF calls (bcrypt, Argon2) in auth-heavy specs are
|
|
21
|
+
the classic finding; weaken their cost in test config, don't optimize specs
|
|
22
|
+
around them.
|
|
23
|
+
|
|
24
|
+
Example: `/stack-prof spec/requests/auth/`
|
|
25
|
+
|
|
26
|
+
Report the hottest frames and whether the fix is config, code, or specs.
|
|
@@ -0,0 +1,20 @@
|
|
|
1
|
+
---
|
|
2
|
+
description: Find files with the most shared-setup overhead per example (best let_it_be candidates)
|
|
3
|
+
argument-hint: <spec directory>
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
Run test-prof's TPS profiler against the given directory (ask for one if
|
|
7
|
+
missing):
|
|
8
|
+
|
|
9
|
+
```sh
|
|
10
|
+
TPS_PROF=1 bundle exec rspec $ARGUMENTS
|
|
11
|
+
```
|
|
12
|
+
|
|
13
|
+
Interpret: files at the top pay the most repeated setup per example — they are
|
|
14
|
+
the best `let_it_be`/`before_all` conversion candidates, where one conversion
|
|
15
|
+
saves the most total time. Verify each against the pre-conversion checklist in
|
|
16
|
+
the `martian-spec` skill's `references/data-setup.md` before converting.
|
|
17
|
+
|
|
18
|
+
Example: `/tps-prof spec/services/`
|
|
19
|
+
|
|
20
|
+
Report the top files and the estimated win for converting each.
|
data/hyperdrive.yml
ADDED
|
@@ -0,0 +1,18 @@
|
|
|
1
|
+
gems:
|
|
2
|
+
- rspec-rails: ">= 6.0, < 9.0"
|
|
3
|
+
skills:
|
|
4
|
+
martian-spec:
|
|
5
|
+
conditional:
|
|
6
|
+
references/data-setup.md: { gem: test-prof }
|
|
7
|
+
references/profiling.md: { gem: test-prof }
|
|
8
|
+
references/structure.md: { gem: test-prof }
|
|
9
|
+
commands:
|
|
10
|
+
# Per-entry gem: REPLACES the gem-wide rspec-rails gate, so the profiler
|
|
11
|
+
# commands must re-state it: they need both RSpec and test-prof.
|
|
12
|
+
fprof.md: { gem: { all: [rspec-rails, test-prof] } }
|
|
13
|
+
event-prof.md: { gem: { all: [rspec-rails, test-prof] } }
|
|
14
|
+
rd-prof.md: { gem: { all: [rspec-rails, test-prof] } }
|
|
15
|
+
tps-prof.md: { gem: { all: [rspec-rails, test-prof] } }
|
|
16
|
+
factory-default-prof.md: { gem: { all: [rspec-rails, test-prof] } }
|
|
17
|
+
mem-prof.md: { gem: { all: [rspec-rails, test-prof] } }
|
|
18
|
+
stack-prof.md: { gem: { all: [rspec-rails, test-prof] } }
|
|
@@ -0,0 +1,326 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: martian-spec
|
|
3
|
+
description: Write, review, optimize, and debug RSpec specs in Rails projects. Use when implementing a feature or fixing a bug in a Rails app — the specs that come with it are part of the change; when creating new spec files or adjusting existing ones; when reviewing test code, PRs, or diffs for anti-patterns and wrong-layer tests; when optimizing slow tests or CI queue time; and when debugging flaky or order-dependent specs. Triggers on "flaky spec", "slow specs", "spec layer", "service/model/request spec", "handler spec", "spec review", "test setup", "test performance", "parallel CI", "test database setup", "RSpec best practices".
|
|
4
|
+
allowed-tools:
|
|
5
|
+
- Grep
|
|
6
|
+
- Glob
|
|
7
|
+
- Read
|
|
8
|
+
- Edit
|
|
9
|
+
- Write
|
|
10
|
+
- Bash
|
|
11
|
+
- Task
|
|
12
|
+
---
|
|
13
|
+
|
|
14
|
+
# RSpec for Rails
|
|
15
|
+
|
|
16
|
+
A skill for writing fast, well-layered, non-flaky RSpec specs in Rails projects.
|
|
17
|
+
|
|
18
|
+
## Thinking Prompts
|
|
19
|
+
|
|
20
|
+
Before writing a test context, ask yourself:
|
|
21
|
+
|
|
22
|
+
- "Could I test this with a simpler object at a lower layer?" -- if yes, do that instead
|
|
23
|
+
- "Am I creating records just to satisfy the entry point, not the logic?" -- wrong layer
|
|
24
|
+
- "Would this test break if I changed the HTTP/GraphQL wrapper?" -- logic is too high
|
|
25
|
+
- "Am I stubbing >3 collaborators just to isolate what I'm testing?" -- testing wiring, not behavior
|
|
26
|
+
- "Is this setup expensive because the logic lives far from where I'm testing it?" -- push logic down
|
|
27
|
+
|
|
28
|
+
When reviewing existing specs, scan for performance and layer issues:
|
|
29
|
+
|
|
30
|
+
- "How many records are created per example vs per file?" -- the #1 speed lever
|
|
31
|
+
<%- if gem?("test-prof") -%>
|
|
32
|
+
- "Is there a `before` that could be `before_all`?" -- quick wins first
|
|
33
|
+
<%- end -%>
|
|
34
|
+
- "Are any traits used only for their attributes, not their associations?" -- trait cascades are the #1 hidden waste source
|
|
35
|
+
- "Are any contexts testing a layer above where the logic lives?" -- structural issue, see Red Flags
|
|
36
|
+
|
|
37
|
+
Then annotate contexts with `[ok]`/`[warning]`/`[error]`:
|
|
38
|
+
|
|
39
|
+
- `[ok]` -- belongs in this layer
|
|
40
|
+
- `[warning]` -- borderline, signals a design smell
|
|
41
|
+
- `[error]` -- wrong layer, should be tested elsewhere
|
|
42
|
+
|
|
43
|
+
If a handler (controller action, GraphQL resolver, job `perform`) has > 3 conditional branches (simple guards don't count), that logic is hard to cover at the handler layer -- flag it as a design smell. Don't compensate with a fat handler spec: keep one smoke test at the handler and put full coverage where the logic ends up living.
|
|
44
|
+
|
|
45
|
+
Before writing a spec with > 5 contexts, sketch the describe/context tree and annotate each with `[ok]`/`[warning]`/`[error]`. If > 30% are `[warning]` or `[error]`, the code needs refactoring before it needs more tests.
|
|
46
|
+
|
|
47
|
+
### Layer Selection
|
|
48
|
+
|
|
49
|
+
| What you're testing | Spec layer | Subject pattern |
|
|
50
|
+
|---------------------|------------|-----------------|
|
|
51
|
+
| Business rule / computation | Service spec | `described_class.call(...)` |
|
|
52
|
+
| DB scope / validation / callback | Model spec | `model.method` or `Model.scope` |
|
|
53
|
+
<%- if gem?("graphql") -%>
|
|
54
|
+
| GraphQL field (thin delegation) | Type spec | `MySchema.execute(query, ...)` |
|
|
55
|
+
| GraphQL field with >3 branches | Service spec + type smoke test | Extract first |
|
|
56
|
+
<%- end -%>
|
|
57
|
+
| Background job execution | Job spec | `described_class.new.perform(...)` |
|
|
58
|
+
| Job gets enqueued by caller | Caller's spec | <% if gem?("rspec-sidekiq") %>`have_enqueued_sidekiq_job` / <% end %>`have_enqueued_job` |
|
|
59
|
+
| HTTP endpoint (REST API) | Request spec | `post "/api/path", params:, as: :json` |
|
|
60
|
+
|
|
61
|
+
### When logic is already at the wrong layer
|
|
62
|
+
|
|
63
|
+
| Situation | Action |
|
|
64
|
+
|-----------|--------|
|
|
65
|
+
| Writing spec for a handler with > 3 branches | Flag the design smell; one smoke test at the handler, full coverage where the logic ends up living |
|
|
66
|
+
| Reviewing spec with `[error]`-layer contexts | Flag it, don't add more wrong-layer tests |
|
|
67
|
+
| Adding a feature to code that already has wrong-layer tests | Don't compound it — cover what you're touching at the right layer |
|
|
68
|
+
| Can't restructure now (time-pressured) | Minimal handler spec + `# TODO` marking where the logic belongs; when next touching this code, move coverage first |
|
|
69
|
+
|
|
70
|
+
## Data Setup
|
|
71
|
+
|
|
72
|
+
- Create the minimum data that exercises the logic: persist only what the code under test actually reads; keep objects in memory when the DB is never touched.
|
|
73
|
+
- Split setup by volatility: data shared by all examples goes in the outermost scope where it's still correct, per-context variations in `let` overrides, mutable state and mocks in `before`.
|
|
74
|
+
- Declare setup in declaration blocks at the top of the `describe`/`context` — never create records inside examples (see Red Flags).
|
|
75
|
+
- Think in totals: every record in shared setup is paid once per example, so a file with 40 examples pays 40× for each one. Trimming shared setup is the highest-leverage speed fix.
|
|
76
|
+
<%- if gem?("test-prof") -%>
|
|
77
|
+
|
|
78
|
+
With test-prof, static data can be created once per *file* instead of once per
|
|
79
|
+
example — `let_it_be` for records, `before_all` for setup; the single biggest
|
|
80
|
+
speed win. The decision tables, pre-conversion checklist, `refind:` modifiers,
|
|
81
|
+
optimization strategies, red flags, and the broken-conversion symptom table
|
|
82
|
+
live in [references/data-setup.md](references/data-setup.md).
|
|
83
|
+
<%- end -%>
|
|
84
|
+
<%- if !gem?("test-prof") -%>
|
|
85
|
+
|
|
86
|
+
This project does not use [test-prof](https://test-prof.evilmartians.io). When
|
|
87
|
+
optimizing slow specs, suggest adding it to the user: `let_it_be`/`before_all`
|
|
88
|
+
create shared data once per file instead of once per example (often the single
|
|
89
|
+
biggest suite speedup), and its profilers (`FPROF`, `EVENT_PROF`) pinpoint
|
|
90
|
+
factory and SQL hotspots. Propose the dependency — don't add it silently. If
|
|
91
|
+
the user agrees:
|
|
92
|
+
|
|
93
|
+
```sh
|
|
94
|
+
bundle add test-prof --group test
|
|
95
|
+
```
|
|
96
|
+
|
|
97
|
+
then enable the recipes in `spec/rails_helper.rb`:
|
|
98
|
+
|
|
99
|
+
```ruby
|
|
100
|
+
require "test_prof/recipes/rspec/let_it_be"
|
|
101
|
+
require "test_prof/recipes/rspec/before_all"
|
|
102
|
+
```
|
|
103
|
+
<%- end -%>
|
|
104
|
+
|
|
105
|
+
## Spec Ordering Convention
|
|
106
|
+
|
|
107
|
+
Within each `describe`/`context` block, declarations go from stable to
|
|
108
|
+
volatile: `subject`, then data declarations (`let`/`let!`), then setup hooks
|
|
109
|
+
(`before`), then examples, then nested contexts. Data before hooks, hooks
|
|
110
|
+
before examples — a reader should learn *what exists* before *what happens*.
|
|
111
|
+
|
|
112
|
+
Name the subject (`subject(:result) { ... }`) whenever examples reference it;
|
|
113
|
+
bare `subject` is fine only for `is_expected` one-liners.
|
|
114
|
+
<%- if gem?("test-prof") -%>
|
|
115
|
+
|
|
116
|
+
With test-prof the same principle splits each tier by lifetime: `let_it_be`
|
|
117
|
+
(static data) before `let`/`let!` (per-context overrides), `before_all`
|
|
118
|
+
(static setup) before `before` (mocks, mutable state). The full convention
|
|
119
|
+
with an annotated example lives in
|
|
120
|
+
[references/structure.md](references/structure.md).
|
|
121
|
+
<%- end -%>
|
|
122
|
+
|
|
123
|
+
## Critical Rules
|
|
124
|
+
|
|
125
|
+
### Scope expensive operations narrowly
|
|
126
|
+
Materialized view refreshes, search index reindexing, external data syncs, file generation — scope these to only the contexts that need them. Don't pay the cost for every example.
|
|
127
|
+
|
|
128
|
+
### Don't over-create records
|
|
129
|
+
To test a `MAX_LIMIT` cap, create `MAX_LIMIT + 1` records, not 600. Extract math into a testable unit and test with plain numbers when possible.
|
|
130
|
+
|
|
131
|
+
### Make retry delays configurable
|
|
132
|
+
If the service under test has retry logic with `sleep`, make the delay configurable and pass `retry_delay: 0` (or `0.01`) in specs. Never let specs sleep for real.
|
|
133
|
+
|
|
134
|
+
### Never hit real external APIs
|
|
135
|
+
Specs must not depend on the network: real calls are slow, flaky, and rate-limited. Stub at a boundary you own — wrap third-party calls in a client/adapter class and mock it like any collaborator (`allow(ExternalApi::Client).to receive(:call)`).
|
|
136
|
+
<%- if gem?("webmock") -%>
|
|
137
|
+
|
|
138
|
+
With WebMock, stub at the HTTP boundary instead: `stub_request(:post, %r{api\.example\.com}).to_return(status: 200, body: ...)`. Disable stray connections globally — `WebMock.disable_net_connect!(allow_localhost: true)` in spec setup — so an unstubbed call fails loudly. Re-enabling net connect inside individual specs is a red flag.
|
|
139
|
+
<%- end -%>
|
|
140
|
+
<%- if gem?("vcr") -%>
|
|
141
|
+
|
|
142
|
+
With VCR, record real interactions once into cassettes (`record: :once`) and replay them deterministically. Watch for stale cassettes drifting from the live API, and filter secrets before committing (`filter_sensitive_data`).
|
|
143
|
+
<%- end -%>
|
|
144
|
+
|
|
145
|
+
### One canonical spec per shared resolver / action
|
|
146
|
+
When two endpoints share a resolver or controller action (e.g., a public and an admin endpoint both expose the same `#summary`), test the logic fully in one spec. The other gets a single smoke test with a comment pointing to the canonical spec.
|
|
147
|
+
|
|
148
|
+
### Handler specs: mock services, don't retest logic
|
|
149
|
+
|
|
150
|
+
```ruby
|
|
151
|
+
# WRONG -- 9 contexts testing eligibility logic through full HTTP execution
|
|
152
|
+
describe "POST /api/resources/:id/actions" do
|
|
153
|
+
context "when already performed" do ... end # business logic
|
|
154
|
+
context "when owner" do ... end # business logic
|
|
155
|
+
# 7 more contexts...
|
|
156
|
+
end
|
|
157
|
+
|
|
158
|
+
# RIGHT -- handler spec delegates, service spec has full coverage
|
|
159
|
+
describe "POST /api/resources/:id/actions" do
|
|
160
|
+
subject(:perform_request) { post "/api/resources/#{resource.id}/actions", as: :json }
|
|
161
|
+
|
|
162
|
+
let(:service_result) { { allowed: true } }
|
|
163
|
+
|
|
164
|
+
before { allow(EligibilityService).to receive(:call).and_return(service_result) }
|
|
165
|
+
|
|
166
|
+
it "delegates to the service" do
|
|
167
|
+
perform_request
|
|
168
|
+
expect(EligibilityService).to have_received(:call).with(
|
|
169
|
+
user: current_user, resource: resource
|
|
170
|
+
)
|
|
171
|
+
end
|
|
172
|
+
|
|
173
|
+
it "returns the service result" do
|
|
174
|
+
perform_request
|
|
175
|
+
expect(response.parsed_body).to eq("allowed" => true)
|
|
176
|
+
end
|
|
177
|
+
end
|
|
178
|
+
```
|
|
179
|
+
|
|
180
|
+
NEVER add more contexts to a handler spec for delegated logic — if you need to test different service inputs/outputs, those tests belong in the service spec.
|
|
181
|
+
|
|
182
|
+
## Red Flags
|
|
183
|
+
|
|
184
|
+
### Layer & coverage
|
|
185
|
+
|
|
186
|
+
| Pattern | Why | Fix |
|
|
187
|
+
|---------|-----|-----|
|
|
188
|
+
| Business logic tested through GraphQL/HTTP | Slow, brittle, duplicates service coverage | Mock service in handler spec, test logic in service spec |
|
|
189
|
+
| >3 `create` calls to set up a single handler spec context | Handler has business logic that belongs in a service | Flag the smell; cover the logic in a service spec, mock it in the handler spec |
|
|
190
|
+
| Inline job execution in a service spec (`perform_enqueued_jobs`<% if gem?("sidekiq") %>, `sidekiq: :inline`<% end %>) | Exercises 4+ layers in one test, blocks CI queue | Test each layer independently |
|
|
191
|
+
| Testing privates via `send` | Signals class should be decomposed | Extract to smaller public objects |
|
|
192
|
+
| Shared example generating >5 contexts | Test concern at a lower layer instead | Extract and test independently |
|
|
193
|
+
| `skip_callbacks` or `update_columns` in test setup | Fighting the framework to work around side effects | Extract logic to a service/model that doesn't trigger callbacks |
|
|
194
|
+
|
|
195
|
+
### Factory & mock hygiene
|
|
196
|
+
|
|
197
|
+
| Pattern | Why | Fix |
|
|
198
|
+
|---------|-----|-----|
|
|
199
|
+
| `create_list` with count >50 | Over-population for boundary test | Use threshold + 1, or test math separately |
|
|
200
|
+
| `FactoryBot.create` inside `subject` or `it` block | Hides setup from the declaration block | Move to `let` / `before` declarations |
|
|
201
|
+
| Factory implicitly creating association trees (cascade) | Every `create` spawns redundant records the test doesn't need | Inline associations in factory definitions or pass shared records explicitly |
|
|
202
|
+
| Trait used only for its attribute defaults, not its associations | Trait associations cascade silently — often the #1 factory waste source | Replace trait with explicit attributes; check model scopes to know which attributes are required |
|
|
203
|
+
| Optional associations defined in factory default | Every `create` pays for associations the test doesn't need | Remove from factory; add explicitly in tests that need them |
|
|
204
|
+
| `allow_any_instance_of(Klass)` | Fragile, unclear which instance is stubbed | Inject dependency or stub on the specific object |
|
|
205
|
+
| Mocking internal code | Over-mocking hides real bugs; only external APIs need isolation | Stub at the external boundary (see Never hit real external APIs); let internal code run |
|
|
206
|
+
|
|
207
|
+
### General
|
|
208
|
+
|
|
209
|
+
| Pattern | Why | Fix |
|
|
210
|
+
|---------|-----|-----|
|
|
211
|
+
| Spec file >1,000 lines | Skews CI queue — one worker stuck while others idle | Split by domain |
|
|
212
|
+
| `sleep` in specs | Wastes real wall-clock time; usually masks retry backoff or async waiting | Retry: make the delay configurable, pass 0 (see Critical Rules). Async: poll for the condition, don't wait a fixed time |
|
|
213
|
+
| `before` block with >5 stubs | Service has too many responsibilities | Decompose service |
|
|
214
|
+
| `rubocop:disable Rails/Output` | Specs must be automated, not human-verified | Remove debug output |
|
|
215
|
+
|
|
216
|
+
## Escalation Ladder: flaky or slow spec
|
|
217
|
+
|
|
218
|
+
Diagnostics, roughly in this order:
|
|
219
|
+
|
|
220
|
+
- **Reproduce ordering issues**: `bundle exec rspec --seed <seed> --bisect` to isolate the minimal failing pair.
|
|
221
|
+
- **Profile factory usage**: factory_bot instruments every factory run — tally the counts with a plain `ActiveSupport::Notifications` subscriber. A factory with a high count that the file never calls explicitly is a cascade from associations or traits.
|
|
222
|
+
|
|
223
|
+
```ruby
|
|
224
|
+
# ad hoc in rails_helper.rb while diagnosing
|
|
225
|
+
counts = Hash.new(0)
|
|
226
|
+
ActiveSupport::Notifications.subscribe("factory_bot.run_factory") do |_, _, _, _, payload|
|
|
227
|
+
counts[[payload[:name], payload[:strategy]]] += 1
|
|
228
|
+
end
|
|
229
|
+
at_exit { counts.sort_by { |_, n| -n }.first(20).each { |k, n| puts "#{n}\t#{k.join(" ")}" } }
|
|
230
|
+
```
|
|
231
|
+
<%- if gem?("test-prof") -%>
|
|
232
|
+
- **With test-prof profiling is one env var**: `FPROF=1 bundle exec rspec spec/path/to/file.rb` shows top-level vs total factory counts (a large gap = cascade); `EVENT_PROF='sql.active_record' bundle exec rspec` surfaces SQL hotspots.
|
|
233
|
+
Cheat-sheet in [references/profiling.md](references/profiling.md).
|
|
234
|
+
<%- end -%>
|
|
235
|
+
- **Find slow examples**: `--profile 10` prints the slowest 10.
|
|
236
|
+
- **State-leak suspect?** Check for Redis/cache writes, `update_columns`, or model callbacks that bypass the transaction.
|
|
237
|
+
- **Time-leak suspect?** `travel_to` / `freeze_time` without a matching `travel_back` (use the block form, or an `after` hook) leaks frozen time into later examples.
|
|
238
|
+
- **Leaking `before(:context)`?** Records created there bypass the per-example transaction, and RSpec forbids mocks in it — move both to `before`.
|
|
239
|
+
<%- if gem?("test-prof") -%>
|
|
240
|
+
- **Using `before_all` / `let_it_be`?** The same time/mock/state leaks have test-prof-specific symptoms — see the symptom table in [references/data-setup.md](references/data-setup.md).
|
|
241
|
+
<%- end -%>
|
|
242
|
+
|
|
243
|
+
## Suite-level speed (CI)
|
|
244
|
+
|
|
245
|
+
Everything above tunes individual examples. These levers work on the whole
|
|
246
|
+
suite at once — three cut CI time and one guards the parallel setup. Do them
|
|
247
|
+
roughly in this order — the first is almost free and often the biggest.
|
|
248
|
+
<%- if gem?("test-prof") -%>
|
|
249
|
+
|
|
250
|
+
Even a suite fully converted to `let_it_be`/`before_all` still pays these
|
|
251
|
+
costs — suite-level levers are the next step after data-setup tuning, not a
|
|
252
|
+
replacement for it.
|
|
253
|
+
<%- end -%>
|
|
254
|
+
|
|
255
|
+
### 1. Quiet the logs (do this first)
|
|
256
|
+
|
|
257
|
+
The default test `log_level` is `:debug`, so every SQL statement is written to
|
|
258
|
+
`log/test.log` — a suite emitting tens of thousands of queries pays real I/O for
|
|
259
|
+
output nobody reads. In `config/environments/test.rb`:
|
|
260
|
+
|
|
261
|
+
```ruby
|
|
262
|
+
config.log_level = :fatal
|
|
263
|
+
config.active_record.verbose_query_logs = false # no per-query backtrace
|
|
264
|
+
config.active_record.query_log_tags_enabled = false # no SQL comment tagging
|
|
265
|
+
```
|
|
266
|
+
|
|
267
|
+
Usually the biggest cheap win in a suite that has never been tuned: in Evil
|
|
268
|
+
Martians' write-up, quieting the verbose query logs together with a Sentry
|
|
269
|
+
logger fix took a single-process run from ~25min to ~12min, with the query
|
|
270
|
+
logging doing the heavy lifting. Costs nothing and never changes behavior.
|
|
271
|
+
|
|
272
|
+
### 2. Weaken expensive global setup in test
|
|
273
|
+
|
|
274
|
+
Cryptographic KDFs (Argon2id, bcrypt, scrypt, PBKDF2), blind indexes, and
|
|
275
|
+
password hashing are CPU bottlenecks that dominate the *slowest examples* —
|
|
276
|
+
they're designed to be slow. Use minimal cost in test only: expose the cost
|
|
277
|
+
parameters (`t_cost`/`m_cost`/`p_cost`, iteration counts) as env-aware config
|
|
278
|
+
and use the cheapest values in the test environment.
|
|
279
|
+
<%- if gem?("bcrypt") -%>
|
|
280
|
+
|
|
281
|
+
```ruby
|
|
282
|
+
BCrypt::Engine.cost = BCrypt::Engine::MIN_COST # in test setup
|
|
283
|
+
```
|
|
284
|
+
<%- end -%>
|
|
285
|
+
|
|
286
|
+
Find them: they're the top entries in `--profile`. If your slowest specs are all
|
|
287
|
+
one crypto/auth feature, this is the fix, not factory work.
|
|
288
|
+
|
|
289
|
+
### 3. Parallelize fairly (runner choice matters)
|
|
290
|
+
|
|
291
|
+
Running across CPU cores is the biggest wall-clock lever once the above are
|
|
292
|
+
done, but the runner matters: prefer a *fair queue* (`test-queue` — one boot,
|
|
293
|
+
examples handed to free workers) over static file splitting
|
|
294
|
+
(`parallel_tests` — the slowest file strands a worker), and give each worker
|
|
295
|
+
its own database. The runner comparison, per-runner database setup (including
|
|
296
|
+
the `test-queue` custom-runner snippet), CI core counts, and how to tell an
|
|
297
|
+
isolation bug from a runner problem live in
|
|
298
|
+
[references/parallel-ci.md](references/parallel-ci.md).
|
|
299
|
+
|
|
300
|
+
### 4. Global failsafes for leaky state
|
|
301
|
+
|
|
302
|
+
Leaked global state (frozen time, singletons, cache entries) causes flakes
|
|
303
|
+
even in a single process; a parallel runner just widens the blast radius to
|
|
304
|
+
unrelated specs in other workers. Add belt-and-suspenders resets in
|
|
305
|
+
`spec/rails_helper.rb`:
|
|
306
|
+
|
|
307
|
+
```ruby
|
|
308
|
+
config.after { travel_back } # undo any stray travel_to/freeze_time
|
|
309
|
+
# reset any singletons / global clients / thread-locals the app sets
|
|
310
|
+
```
|
|
311
|
+
|
|
312
|
+
For the cache, check what `config/environments/test.rb` actually sets before
|
|
313
|
+
adding a failsafe:
|
|
314
|
+
|
|
315
|
+
- **`:null_store`** (the Rails default in test) — cache writes are no-ops,
|
|
316
|
+
nothing to clear. But if the app talks to Redis *directly* (job queues,
|
|
317
|
+
custom clients, counters), those writes survive the DB transaction — clear
|
|
318
|
+
or namespace them in an `after` hook instead.
|
|
319
|
+
- **`:memory_store`** — per-process; `config.after { Rails.cache.clear }` is
|
|
320
|
+
cheap and sufficient.
|
|
321
|
+
- **A shared store** (`:redis_cache_store`, memcached) — one worker's `clear`
|
|
322
|
+
wipes every other worker's entries: namespace the store per worker first,
|
|
323
|
+
then clear — setup in [references/parallel-ci.md](references/parallel-ci.md).
|
|
324
|
+
|
|
325
|
+
Prefer fixing the leak at its source (the time-leak / mock-leak red flags
|
|
326
|
+
above); the global `after` is the safety net for what slips through.
|
data/lib/rails-hyperdrive-martian-spec/hyperdrive/skills/martian-spec/references/data-setup.md
ADDED
|
@@ -0,0 +1,176 @@
|
|
|
1
|
+
# Data Setup (test-prof)
|
|
2
|
+
|
|
3
|
+
`let_it_be` and `before_all` create data once per spec file instead of once
|
|
4
|
+
per example — the single biggest per-example speed win. This reference holds
|
|
5
|
+
the decision rules, the safety red flags, and the recovery ladder.
|
|
6
|
+
|
|
7
|
+
## Critical Rules
|
|
8
|
+
|
|
9
|
+
### `let_it_be` by default
|
|
10
|
+
Use `let_it_be` for all static data (see Quick Reference above). This is the single biggest speed win — heavy specs often drop from hundreds of factory creates to a handful.
|
|
11
|
+
|
|
12
|
+
### `before_all` for static setup
|
|
13
|
+
If a `before` block only creates records or sets up static state (no mocks), use `before_all`.
|
|
14
|
+
|
|
15
|
+
## Quick Reference
|
|
16
|
+
|
|
17
|
+
### Use `let_it_be` when:
|
|
18
|
+
|
|
19
|
+
| Situation | Variant | Why |
|
|
20
|
+
|-----------|---------|-----|
|
|
21
|
+
| Data never changes between examples | `let_it_be(:x) { create(...) }` | Created once, shared across examples |
|
|
22
|
+
| Record mutated by code under test | `let_it_be(:x, refind: true)` | `refind: true` does a full `find` after each example (fresh Ruby object); use `let` if mutation is destructive (soft-delete, discard) |
|
|
23
|
+
| Static setup (no mocks involved) | `before_all { create(...) }` | Runs once, not per example |
|
|
24
|
+
|
|
25
|
+
### Keep as `let` when (NOT `let_it_be`):
|
|
26
|
+
|
|
27
|
+
| Situation | Why |
|
|
28
|
+
|-----------|-----|
|
|
29
|
+
| Value overridden in nested contexts | `let_it_be` is frozen at creation; nested `let` overrides won't affect it |
|
|
30
|
+
| ANY dependency in the chain is overridden in nested contexts | Transitive: if the root is overridden, every dependent that references it must ALL stay as `let` — `let_it_be` records were created with the original dependency |
|
|
31
|
+
| Nested context creates records with unique constraints on a shared object | `let_it_be` reuses the same DB record; a nested `create` hitting a unique index raises `RecordNotUnique` |
|
|
32
|
+
| Factory or setup writes to Redis/cache | Transaction savepoints only roll back DB state; cache/Redis writes persist across examples |
|
|
33
|
+
| References another `let` that varies | Depends on per-context data that `let_it_be` can't see |
|
|
34
|
+
| Setup involves mocks, stubs, or test doubles (`double`, `instance_double`, `spy`) | Mocks and doubles are scoped to one example; `let_it_be` leaks them into subsequent examples causing "leaked into another example" errors |
|
|
35
|
+
| Time-sensitive logic | `before_all` / `let_it_be` freezes time across all examples and leaks to other files; use `before { travel_to(time) }` |
|
|
36
|
+
| `before` block creates records referencing data that sibling contexts override | `before_all` records persist across ALL sibling contexts; if a sibling redefines a dependency, the `before_all` records still reference the original — keep as `before` |
|
|
37
|
+
|
|
38
|
+
### Pre-conversion checklist: `let` → `let_it_be`
|
|
39
|
+
|
|
40
|
+
Before converting, verify ALL of these:
|
|
41
|
+
- [ ] No nested `let` override of any dependency in the chain
|
|
42
|
+
- [ ] No unique constraints on the shared object in nested contexts
|
|
43
|
+
- [ ] No Redis/cache writes in the factory or setup
|
|
44
|
+
- [ ] Declaration appears AFTER all its `let_it_be` dependencies in source order
|
|
45
|
+
- [ ] No model callbacks (`before_commit`, `after_commit`) on created records that implicitly create other persistent records (these survive savepoint rollback and cause unique constraint violations)
|
|
46
|
+
- [ ] For `before` → `before_all`: no sibling context overrides any data the block references (records leak across siblings)
|
|
47
|
+
|
|
48
|
+
**`let_it_be` modifiers:** Prefer `refind: true` over `reload: true` — `reload: true` is a half-measure that re-reads attributes but keeps the same Ruby object, while `refind: true` does a full `Model.find` returning a completely fresh object. When neither modifier works (soft-delete, `discard`), fall back to `let`.
|
|
49
|
+
|
|
50
|
+
**`refind: true` side-effect:** `refind` issues a `SELECT *` per example. If the spec asserts on SQL events, filter out the reload query.
|
|
51
|
+
|
|
52
|
+
### Optimization strategies (don't leave speed on the table)
|
|
53
|
+
|
|
54
|
+
**Use `refind: true` to unlock cascading conversions.** When a record is mutated in nested contexts (e.g., `user.save!`, `record.update!`) but NOT destroyed, use `let_it_be(:record, refind: true)` instead of falling back to `let`. This lets all dependents also become `let_it_be`, saving dozens of factory calls. Only fall back to `let` when the mutation is destructive (soft-delete, discard).
|
|
55
|
+
|
|
56
|
+
**Use `refind: true` when sibling contexts load the same association.** Without `refind`, `let_it_be` reuses the same Ruby object across all contexts. If context A's `before` creates child records and the example loads the association (e.g., `user.posts`), the association cache persists on the Ruby object even after the records are rolled back. Context B then gets stale cached data instead of querying the DB. `refind: true` returns a fresh Ruby object per example, with no stale association cache.
|
|
57
|
+
|
|
58
|
+
```ruby
|
|
59
|
+
# BEFORE: user mutated in contexts → entire chain stays `let`
|
|
60
|
+
let(:user) { create(:user) }
|
|
61
|
+
let!(:post) { create(:post, user: user) }
|
|
62
|
+
let!(:comment) { create(:comment, post: post) }
|
|
63
|
+
let!(:reaction) { create(:reaction, user: user) }
|
|
64
|
+
|
|
65
|
+
# AFTER: refind on the root unlocks the whole chain
|
|
66
|
+
let_it_be(:user, refind: true) { create(:user) }
|
|
67
|
+
let_it_be(:post) { create(:post, user: user) }
|
|
68
|
+
let_it_be(:comment) { create(:comment, post: post) }
|
|
69
|
+
let_it_be(:reaction) { create(:reaction, user: user) }
|
|
70
|
+
```
|
|
71
|
+
|
|
72
|
+
**Convert nested single-context declarations.** A `let` inside a context with 1-2 examples and no nested overrides is safe to convert to `let_it_be`. Don't skip these — a file with 20 contexts × 2 examples = 40 unnecessary factory calls if you only optimize the top level. Scan every `describe`/`context` block.
|
|
73
|
+
|
|
74
|
+
**Hoist duplicate `let` declarations.** When multiple sibling contexts define identical `let(:x) { create(...) }`, hoist the declaration to the shared parent scope. If the parent uses `let_it_be`, the hoisted declaration can too — one factory call instead of N.
|
|
75
|
+
|
|
76
|
+
**Convert pure-Ruby declarations.** `let` blocks that build non-DB objects (anonymous classes, schema definitions, static hashes) are always safe to convert to `let_it_be` — no DB rollback concerns.
|
|
77
|
+
|
|
78
|
+
**Break factory cascades by passing shared objects.** When a factory implicitly creates associations, each `create` spawns redundant records. Pass existing `let_it_be` objects to short-circuit the cascade.
|
|
79
|
+
|
|
80
|
+
```ruby
|
|
81
|
+
# BEFORE: 3 contexts × create(:post) = 3 users + 3 categories created implicitly
|
|
82
|
+
let(:post) { create(:post) }
|
|
83
|
+
|
|
84
|
+
# AFTER: share the parent objects, pass them explicitly
|
|
85
|
+
let_it_be(:user) { create(:user) }
|
|
86
|
+
let_it_be(:category) { create(:category) }
|
|
87
|
+
let_it_be(:post) { create(:post, user: user, category: category) }
|
|
88
|
+
```
|
|
89
|
+
|
|
90
|
+
**Eliminate expensive traits that the spec doesn't need.** Factory traits with associations are hidden cascade sources — each trait association spawns its own dependency tree. Profile factories (e.g., test-prof's `FPROF=1`) and look for factories where `total` >> `top-level`; trace the gap to a trait, then replace the trait with only the explicit attributes the test actually requires.
|
|
91
|
+
|
|
92
|
+
```ruby
|
|
93
|
+
# BEFORE: :with_full_profile creates address + avatar + preferences,
|
|
94
|
+
# each cascading to additional records
|
|
95
|
+
create(:user, :with_full_profile)
|
|
96
|
+
|
|
97
|
+
# AFTER: replace trait with the attributes the code under test actually checks
|
|
98
|
+
create(:user, status: "active", role: "admin")
|
|
99
|
+
```
|
|
100
|
+
|
|
101
|
+
**Check model scopes when replacing traits.** When you remove a trait, verify which attributes the code under test actually requires. If the service calls `User.visible` and that scope requires `status: "active"`, you must set `status` explicitly — the trait was setting it silently. Read the scope definition before choosing replacement attributes.
|
|
102
|
+
|
|
103
|
+
**`create_default` only works for top-level factory associations.** `create_default(:user)` sets a thread-local default so that any factory calling `user` (top-level association) reuses the existing record. However, it does NOT intercept associations defined inside traits — if a trait defines `address` which itself has a `user` association, `create_default(:user)` won't prevent the cascade. For trait cascades, eliminate the trait instead.
|
|
104
|
+
|
|
105
|
+
## Red Flags
|
|
106
|
+
|
|
107
|
+
### `let_it_be` / `before_all` safety
|
|
108
|
+
|
|
109
|
+
| Pattern | Why | Fix |
|
|
110
|
+
|---------|-----|-----|
|
|
111
|
+
| `let(:x) { create(...) }` for static data | N examples = N unnecessary DB writes | `let_it_be` (check pre-conversion checklist above) |
|
|
112
|
+
| `before { create(...) }` for static setup | Same — runs per example | `before_all` |
|
|
113
|
+
| `let_it_be` redefined in nested context | Evaluated once at load time — redefining is fragile and may not override as expected | Use `let` for values that vary per context |
|
|
114
|
+
| `let_it_be(:x)` where a dependency is overridden via `let` in nested contexts | Record was created with the ORIGINAL dependency — nested overrides are invisible, causing silent wrong-data bugs | Keep the entire dependency chain as `let` |
|
|
115
|
+
| `let_it_be` object referenced by nested `create` with a unique constraint | Shared object persists across contexts; nested creates hit duplicate-entry errors on the unique index | Use `let` for the parent object in those contexts |
|
|
116
|
+
| Top-level `let!` → `let_it_be` when nested contexts redefine the same name with different associations | The `let_it_be` record persists in DB and is still associated with shared parents; nested `let(:name)` shadows the Ruby variable but the DB record is still there, polluting queries that count or filter by association | Keep as `let!` when the name is redefined in nested contexts AND the record is associated with a shared parent |
|
|
117
|
+
| `before_all` or `let_it_be` factory that writes to Redis/cache | DB transaction savepoints don't roll back cache — later examples see stale state | Use `before`/`let`, or add explicit cache cleanup in `after` |
|
|
118
|
+
| `before_all` creating records that reference data overridden in sibling contexts | `before_all` records persist across sibling contexts; if a sibling overrides a dependency, the `before_all` records still reference the original — causing wrong counts or wrong associations (see example below) | Keep as `before` when sibling contexts override any data used in the setup |
|
|
119
|
+
| `let_it_be(:record)` without `refind: true` when sibling contexts load associations on the same object | Association cache persists on the Ruby object even after DB records are rolled back — next context gets stale cached data instead of querying DB | Use `let_it_be(:record, refind: true)` to get a fresh Ruby object per example |
|
|
120
|
+
| `before_all` / `let_it_be` factory on a model with `before_commit`/`after_commit` callbacks that create OTHER records | Callback-created records are committed outside the savepoint — they persist across examples and cause `RecordNotUnique` when a nested `let_it_be` tries to create the same record | Find the callback-created record instead of creating a duplicate: `Model.find_by!(...).tap { \|m\| m.update_columns(...) }` |
|
|
121
|
+
| `travel_to` or `freeze_time` inside `before_all` | Time stays frozen across all examples and leaks to other spec files | Use `before { travel_to(...) }` or block form |
|
|
122
|
+
|
|
123
|
+
**`before_all` sibling leak example:**
|
|
124
|
+
|
|
125
|
+
```ruby
|
|
126
|
+
# WRONG — before_all records leak into sibling context that overrides the parent
|
|
127
|
+
context "when filtering posts" do
|
|
128
|
+
let_it_be(:user) { create(:user, :active) }
|
|
129
|
+
|
|
130
|
+
before_all do
|
|
131
|
+
create_list(:post, 2, user: user) # created with :active user
|
|
132
|
+
end
|
|
133
|
+
|
|
134
|
+
it "returns 2 posts" do ... end # passes
|
|
135
|
+
|
|
136
|
+
context "when user is suspended" do
|
|
137
|
+
let(:user) { create(:user, :suspended) }
|
|
138
|
+
# before_all posts still point to the ORIGINAL :active user!
|
|
139
|
+
it "returns 0 posts for suspended user" do ... end # FAILS
|
|
140
|
+
end
|
|
141
|
+
end
|
|
142
|
+
|
|
143
|
+
# RIGHT — keep as `before` so each context gets its own records
|
|
144
|
+
before do
|
|
145
|
+
create_list(:post, 2, user: user)
|
|
146
|
+
end
|
|
147
|
+
```
|
|
148
|
+
|
|
149
|
+
**Mock-leak example:**
|
|
150
|
+
|
|
151
|
+
```ruby
|
|
152
|
+
# WRONG — mock set up in before_all persists past its example,
|
|
153
|
+
# raising "leaked into another example" errors
|
|
154
|
+
before_all do
|
|
155
|
+
create(:subscription, user: user)
|
|
156
|
+
allow(PaymentGateway).to receive(:charge).and_return(success)
|
|
157
|
+
end
|
|
158
|
+
|
|
159
|
+
# RIGHT — records in before_all, mocks in before
|
|
160
|
+
before_all { create(:subscription, user: user) }
|
|
161
|
+
before { allow(PaymentGateway).to receive(:charge).and_return(success) }
|
|
162
|
+
```
|
|
163
|
+
|
|
164
|
+
## Escalation Ladder
|
|
165
|
+
|
|
166
|
+
`let_it_be` → `let_it_be(refind: true)` → `let` → `before { create(...) }` (last resort)
|
|
167
|
+
|
|
168
|
+
**Conversion broke tests? Diagnose by symptom:**
|
|
169
|
+
|
|
170
|
+
| Symptom | Cause | Fix |
|
|
171
|
+
|---------|-------|-----|
|
|
172
|
+
| `RecordNotUnique` / duplicate entry | Model callback (`before_commit`, `after_commit`) implicitly creates records that survive the savepoint | Find the callback-created record instead of creating a duplicate: `Model.find_by!(...).tap { \|m\| m.update_columns(...) }` |
|
|
173
|
+
| Stale data / wrong attribute values | Same Ruby object reused across examples; mutations or association cache persist | Add `refind: true` to get a fresh object per example |
|
|
174
|
+
| Wrong associations / unexpected counts | Dependency in the chain is overridden in nested contexts but `let_it_be` record was created with original | Keep entire dependency chain as `let` |
|
|
175
|
+
| `leaked into another example` error | Mocks or doubles used inside `let_it_be` / `before_all` | Move mocks to `before`, keep `let_it_be` for DB records only |
|
|
176
|
+
| Unique constraint in nested context | Nested `create` conflicts with persistent `let_it_be` record on same unique index | Use `let` for the parent object in those contexts |
|
data/lib/rails-hyperdrive-martian-spec/hyperdrive/skills/martian-spec/references/parallel-ci.md
ADDED
|
@@ -0,0 +1,77 @@
|
|
|
1
|
+
# Parallel CI: runner choice and setup
|
|
2
|
+
|
|
3
|
+
## Runner choice matters
|
|
4
|
+
|
|
5
|
+
- **`parallel_tests`** splits *files* across processes up front (static). The
|
|
6
|
+
slowest file becomes the bottleneck and each worker boots the app separately.
|
|
7
|
+
- **`test-queue`** (preferred) forks workers after one boot and hands out
|
|
8
|
+
examples from a shared queue — a free worker grabs the next spec, so slow
|
|
9
|
+
files can't strand a worker, and there's no per-worker re-boot. This is what
|
|
10
|
+
Rails' own Minitest `parallelize` does (fork + distribute), which is why a
|
|
11
|
+
Minitest suite often looks "faster" than a single-process RSpec — it's the
|
|
12
|
+
parallelism, not the framework.
|
|
13
|
+
|
|
14
|
+
## Per-worker databases
|
|
15
|
+
|
|
16
|
+
Give each worker its own database so workers never contend — but the setup
|
|
17
|
+
differs by runner:
|
|
18
|
+
|
|
19
|
+
- **`test-queue`** does NOT set `TEST_ENV_NUMBER` (its env vars, e.g.
|
|
20
|
+
`TEST_QUEUE_WORKERS`, control the queue, not worker identity) — it forks
|
|
21
|
+
after boot, so workers inherit the parent's DB connection. The stock
|
|
22
|
+
`rspec-queue` binary has no per-worker hooks; wire a custom runner and
|
|
23
|
+
reconnect in `after_fork`:
|
|
24
|
+
|
|
25
|
+
```ruby
|
|
26
|
+
#!/usr/bin/env ruby
|
|
27
|
+
# bin/test-queue — run as: bin/test-queue spec
|
|
28
|
+
require "test_queue"
|
|
29
|
+
require "test_queue/runner/rspec"
|
|
30
|
+
|
|
31
|
+
class Runner < TestQueue::Runner::RSpec
|
|
32
|
+
def after_fork(num)
|
|
33
|
+
db = ActiveRecord::Base.connection_db_config.configuration_hash
|
|
34
|
+
ActiveRecord::Base.establish_connection(db.merge(database: "#{db[:database]}_#{num}"))
|
|
35
|
+
end
|
|
36
|
+
end
|
|
37
|
+
|
|
38
|
+
Runner.new.execute
|
|
39
|
+
```
|
|
40
|
+
|
|
41
|
+
Create/load the per-worker schemas once in CI setup before the run.
|
|
42
|
+
|
|
43
|
+
- **`parallel_tests`** sets `TEST_ENV_NUMBER` per process: suffix the database
|
|
44
|
+
name with it in `database.yml` (SQLite: one file per number), and create the
|
|
45
|
+
databases with `rake parallel:create parallel:load_schema`.
|
|
46
|
+
|
|
47
|
+
## Shared cache stores under parallel workers
|
|
48
|
+
|
|
49
|
+
With a shared store (`:redis_cache_store`, memcached) in test, one worker's
|
|
50
|
+
`Rails.cache.clear` wipes every other worker's entries. Namespace the store
|
|
51
|
+
per worker first, then clear:
|
|
52
|
+
|
|
53
|
+
```ruby
|
|
54
|
+
# config/environments/test.rb
|
|
55
|
+
config.cache_store = :redis_cache_store,
|
|
56
|
+
{ namespace: "test#{ENV.fetch("TEST_ENV_NUMBER", "")}" }
|
|
57
|
+
|
|
58
|
+
# spec/rails_helper.rb
|
|
59
|
+
config.after { Rails.cache.clear }
|
|
60
|
+
```
|
|
61
|
+
|
|
62
|
+
Under `test-queue` nothing sets `TEST_ENV_NUMBER` — assign it in the custom
|
|
63
|
+
runner's `after_fork` above (`ENV["TEST_ENV_NUMBER"] = num.to_s`) so the same
|
|
64
|
+
namespacing works there.
|
|
65
|
+
|
|
66
|
+
## CI sizing
|
|
67
|
+
|
|
68
|
+
Know your core count first (GitHub-hosted **private** repos get 2 vCPUs,
|
|
69
|
+
public get 4) — cache the runtime log for runtime-based balancing.
|
|
70
|
+
|
|
71
|
+
## Fair queues expose isolation bugs — that's a feature
|
|
72
|
+
|
|
73
|
+
**A fair queue exposes every latent isolation bug** — specs that only passed
|
|
74
|
+
because another file ran first in the same process. It finds real
|
|
75
|
+
order-dependence. When a spec fails only under parallelism, run it *alone*
|
|
76
|
+
(`rspec path/to/x_spec.rb`) — if it fails there too, it's a self-sufficiency
|
|
77
|
+
bug (a missing `require`/constant, or leaked global state), not the runner.
|
|
@@ -0,0 +1,24 @@
|
|
|
1
|
+
# Profiler Cheat-sheet (test-prof)
|
|
2
|
+
|
|
3
|
+
Run against one file (or a directory) — profiling a whole suite drowns the signal:
|
|
4
|
+
|
|
5
|
+
| Command | Tool | Answers |
|
|
6
|
+
|---|---|---|
|
|
7
|
+
| `FPROF=1 bundle exec rspec <path>` | FactoryProf | per-factory create counts; `total` ≫ `top-level` = cascade |
|
|
8
|
+
| `FACTORY_DEFAULT_PROF=1 bundle exec rspec <path>` | FactoryDefault prof | which implicit associations `create_default` could share |
|
|
9
|
+
| `EVENT_PROF='sql.active_record' bundle exec rspec <path>` | EventProf | time spent in SQL (is it even the bottleneck?) |
|
|
10
|
+
| `EVENT_PROF='factory.create' bundle exec rspec <path>` | EventProf | share of time in factories |
|
|
11
|
+
| `RD_PROF=1 bundle exec rspec <path>` | RSpecDissect | `before`-hook time vs example-body time, slowest groups |
|
|
12
|
+
| `TPS_PROF=1 bundle exec rspec <path>` | TPS profiler | files with the most shared-setup overhead per example (best `let_it_be` candidates) |
|
|
13
|
+
| `TEST_MEM_PROF=gc bundle exec rspec <path>` | Memory profiler | examples contributing most to GC time |
|
|
14
|
+
| `TEST_STACK_PROF=1 bundle exec rspec <path>` | StackProf | CPU flamegraph of the hottest code paths |
|
|
15
|
+
| `bundle exec rspec --profile 10 <path>` | RSpec | slowest 10 examples |
|
|
16
|
+
| `bundle exec rspec --seed <N> --bisect <path>` | RSpec | minimal failing example pair for an order-dependence |
|
|
17
|
+
|
|
18
|
+
Start with `EVENT_PROF`/`FPROF` to confirm *where* time goes before optimizing —
|
|
19
|
+
if SQL is ~10% and factories are a third, don't chase N+1s.
|
|
20
|
+
|
|
21
|
+
Each profiler also ships as a slash command the user can invoke directly
|
|
22
|
+
(`/fprof`, `/event-prof`, `/rd-prof`, `/tps-prof`, `/factory-default-prof`,
|
|
23
|
+
`/mem-prof`, `/stack-prof`, plus `/slowest-specs` and `/bisect-order`) — when
|
|
24
|
+
suggesting a profiling step to the user, point them at the command.
|
|
@@ -0,0 +1,45 @@
|
|
|
1
|
+
# Spec Ordering Convention (test-prof)
|
|
2
|
+
|
|
3
|
+
Within each `describe`/`context` block, declarations follow this order:
|
|
4
|
+
|
|
5
|
+
1. `subject` — named (`subject(:result) { ... }`) whenever examples reference it; bare only for `is_expected` one-liners
|
|
6
|
+
2. `let_it_be` — static data (created once)
|
|
7
|
+
3. `let` / `let!` — per-context overrides
|
|
8
|
+
4. `before_all` — static setup (no mocks)
|
|
9
|
+
5. `before` — per-example setup (mocks, stubs)
|
|
10
|
+
6. Examples (`it` / `specify`)
|
|
11
|
+
7. Nested `context` blocks
|
|
12
|
+
|
|
13
|
+
```ruby
|
|
14
|
+
RSpec.describe MyService, type: :service do
|
|
15
|
+
subject(:result) { described_class.call(user: user, params: params) }
|
|
16
|
+
|
|
17
|
+
# 1. Static data (let_it_be)
|
|
18
|
+
let_it_be(:user) { create(:user) }
|
|
19
|
+
let_it_be(:account) { create(:account) }
|
|
20
|
+
|
|
21
|
+
# 2. Per-context data (let) — only when overridden below
|
|
22
|
+
let(:params) { { name: "test" } }
|
|
23
|
+
|
|
24
|
+
# 3. Static setup (before_all)
|
|
25
|
+
before_all do
|
|
26
|
+
create(:membership, user:, account:, role: "owner")
|
|
27
|
+
end
|
|
28
|
+
|
|
29
|
+
# 4. Per-example setup (before) — only for mocks or mutable state
|
|
30
|
+
before do
|
|
31
|
+
allow(ExternalApi::Client).to receive(:call).and_return(success_response)
|
|
32
|
+
end
|
|
33
|
+
|
|
34
|
+
# 5. Tests
|
|
35
|
+
it "does the thing" do
|
|
36
|
+
expect(result).to ...
|
|
37
|
+
end
|
|
38
|
+
|
|
39
|
+
context "when condition varies" do
|
|
40
|
+
let(:params) { { name: "" } } # override is why this uses let
|
|
41
|
+
|
|
42
|
+
it "handles the edge case" do ...
|
|
43
|
+
end
|
|
44
|
+
end
|
|
45
|
+
```
|
metadata
ADDED
|
@@ -0,0 +1,71 @@
|
|
|
1
|
+
--- !ruby/object:Gem::Specification
|
|
2
|
+
name: rails-hyperdrive-martian-spec
|
|
3
|
+
version: !ruby/object:Gem::Version
|
|
4
|
+
version: 0.3.1
|
|
5
|
+
platform: ruby
|
|
6
|
+
authors:
|
|
7
|
+
- izhanov
|
|
8
|
+
bindir: bin
|
|
9
|
+
cert_chain: []
|
|
10
|
+
date: 1980-01-02 00:00:00.000000000 Z
|
|
11
|
+
dependencies: []
|
|
12
|
+
description: |
|
|
13
|
+
Companion gem for rails-hyperdrive. Ships the `martian-spec` skill — a procedural,
|
|
14
|
+
model-invoked guide for writing RSpec specs in Rails projects. Installed lazily
|
|
15
|
+
by `bin/rails hyperdrive:init` into `.claude/skills/martian-spec/SKILL.md`.
|
|
16
|
+
email:
|
|
17
|
+
- aibek.izhanov@evilmartians.com
|
|
18
|
+
executables: []
|
|
19
|
+
extensions: []
|
|
20
|
+
extra_rdoc_files: []
|
|
21
|
+
files:
|
|
22
|
+
- CHANGELOG.md
|
|
23
|
+
- LICENSE.txt
|
|
24
|
+
- README.md
|
|
25
|
+
- commands/bisect-order.md
|
|
26
|
+
- commands/event-prof.md
|
|
27
|
+
- commands/factory-default-prof.md
|
|
28
|
+
- commands/fprof.md
|
|
29
|
+
- commands/mem-prof.md
|
|
30
|
+
- commands/profile-specs.md.erb
|
|
31
|
+
- commands/rd-prof.md
|
|
32
|
+
- commands/slowest-specs.md
|
|
33
|
+
- commands/stack-prof.md
|
|
34
|
+
- commands/tps-prof.md
|
|
35
|
+
- hyperdrive.yml
|
|
36
|
+
- lib/rails-hyperdrive-martian-spec.rb
|
|
37
|
+
- lib/rails-hyperdrive-martian-spec/hyperdrive/skills/martian-spec/SKILL.md.erb
|
|
38
|
+
- lib/rails-hyperdrive-martian-spec/hyperdrive/skills/martian-spec/references/data-setup.md
|
|
39
|
+
- lib/rails-hyperdrive-martian-spec/hyperdrive/skills/martian-spec/references/parallel-ci.md
|
|
40
|
+
- lib/rails-hyperdrive-martian-spec/hyperdrive/skills/martian-spec/references/profiling.md
|
|
41
|
+
- lib/rails-hyperdrive-martian-spec/hyperdrive/skills/martian-spec/references/structure.md
|
|
42
|
+
- lib/rails-hyperdrive-martian-spec/version.rb
|
|
43
|
+
homepage: https://github.com/izhanov/rails-hyperdrive-martian-spec
|
|
44
|
+
licenses:
|
|
45
|
+
- MIT
|
|
46
|
+
metadata:
|
|
47
|
+
homepage_uri: https://github.com/izhanov/rails-hyperdrive-martian-spec
|
|
48
|
+
source_code_uri: https://github.com/izhanov/rails-hyperdrive-martian-spec
|
|
49
|
+
changelog_uri: https://github.com/izhanov/rails-hyperdrive-martian-spec/blob/main/CHANGELOG.md
|
|
50
|
+
allowed_push_host: https://rubygems.org
|
|
51
|
+
rubygems_mfa_required: 'true'
|
|
52
|
+
hyperdrive_targets: rspec-rails
|
|
53
|
+
hyperdrive_artifacts: skill,command
|
|
54
|
+
rdoc_options: []
|
|
55
|
+
require_paths:
|
|
56
|
+
- lib
|
|
57
|
+
required_ruby_version: !ruby/object:Gem::Requirement
|
|
58
|
+
requirements:
|
|
59
|
+
- - ">="
|
|
60
|
+
- !ruby/object:Gem::Version
|
|
61
|
+
version: 3.2.0
|
|
62
|
+
required_rubygems_version: !ruby/object:Gem::Requirement
|
|
63
|
+
requirements:
|
|
64
|
+
- - ">="
|
|
65
|
+
- !ruby/object:Gem::Version
|
|
66
|
+
version: '0'
|
|
67
|
+
requirements: []
|
|
68
|
+
rubygems_version: 4.0.6
|
|
69
|
+
specification_version: 4
|
|
70
|
+
summary: 'Rails Hyperdrive companion gem: RSpec skill for AI coding agents.'
|
|
71
|
+
test_files: []
|