test-prof 0.4.9 → 0.5.0

data/guides/factory_doctor.md

@@ -1,119 +0,0 @@
-# Factory Doctor
-
-One common bad pattern that slows our tests down is unnecessary database manipulation. Consider a _bad_ example:
-
-```ruby
-it 'validates name presence' do
-  user = create(:user)
-  user.name = ''
-  expect(user).not_to be_valid
-end
-```
-
-Here we create a new user record, run all callbacks and validations and save it to the database. We don't need all these! Here is a _good_ example:
-
-```ruby
-it 'validates name presence' do
-  user = build_stubbed(:user)
-  user.name = ''
-  expect(user).not_to be_valid
-end
-```
-
-Read more about [`build_stubbed`](https://robots.thoughtbot.com/use-factory-girls-build-stubbed-for-a-faster-test).
-
-FactoryDoctor is a tool that helps you identify such _bad_ tests, i.e. tests that perform unnecessary database queries.
-
-Example output:
-
-```sh
-[TEST PROF INFO] FactoryDoctor report
-
-Total (potentially) bad examples: 2
-Total wasted time: 00:13.165
-
-User (./spec/models/user_spec.rb:3)
-  validates name (./spec/user_spec.rb:8) – 1 record created, 00:00.114
-  validates email (./spec/user_spec.rb:8) – 2 records created, 00:00.514
-```
-
-**NOTE**: have you noticed the "potentially" word? Unfortunately, FactoryDoctor is not a
-magician (it's still learning) and sometimes it produces false negatives and false positives too.
-
-Please, submit an [issue](https://github.com/palkan/test-prof/issues) if you found a case which makes FactoryDoctor fail.
-
-You can also tell FactoryDoctor to ignore specific examples/groups. Just add the `:fd_ignore` tag to it:
-
-```ruby
-# won't be reported as offense
-it 'is ignored', :fd_ignore do
-  user = create(:user)
-  user.name = ''
-  expect(user).not_to be_valid
-end
-```
-
-## Instructions
-
-Currently, FactoryDoctor works only with FactoryGirl/FactoryBot.
-
-## RSpec
-
-To activate FactoryDoctor use `FDOC` environment variable:
-
-```sh
-FDOC=1 rspec ...
-```
-
-## Using with RSpecStamp
-
-FactoryDoctor can be used with [RSpec Stamp](https://github.com/palkan/test-prof/tree/master/guides/rspec_stamp.md) to automatically mark _bad_ examples with custom tags. For example:
-
-```sh
-FDOC=1 FDOC_STAMP="fdoc:consider" rspec ...
-```
-
-After running the command above all _potentially_ bad examples would be marked with the `fdoc: :consider` tag.
-
-## Minitest
-
-To activate FactoryDoctor use `FDOC` environment variable:
-
-```sh
-FDOC=1 ruby ...
-```
-
-or use CLI option as shown below:
-
-```sh
-ruby ... --factory-doctor
-```
-
-The same option to force Factory Doctor to ignore specific examples is also available for Minitest.
-Just use `fd_ignore` inside your example:
-
-```ruby
-# won't be reported as offense
-it 'is ignored' do
-  fd_ignore
-
-  @user.name = ''
-  refute @user.valid?
-end
-```
-
-## Using with Minitest::Reporters
-
-If you're using `Minitest::Reporters` in your project you have to explicitly declare it
-in your test helper file:
-
-```sh
-require 'minitest/reporters'
-Minitest::Reporters.use! [YOUR_FAVORITE_REPORTERS]
-```
-#### NOTICE
-When you have `minitest-reporters` installed as a gem but not declared in your `Gemfile`
-make sure to always prepend your test run command with `bundle exec` (but we sure that you always do it).
-Otherwise, you'll get an error caused by Minitest plugin system, which scans all the entries in the
-`$LOAD_PATH` for any `minitest/*_plugin.rb`, thus initialization of `minitest-reporters` plugin which is
-available in that case doesn't happens correctly.

data/guides/factory_prof.md

@@ -1,86 +0,0 @@
-# FactoryProf
-
-FactoryProf tracks your factories usage statistics, i.e. how often each factory has been used.
-
-Example output:
-
-```sh
-[TEST PROF INFO] Factories usage
-
- total      top-level             name
-
-     8              4             user
-     5              3             post
-```
-
-
-It shows both the total number of the factory runs and the number of _top-level_ runs, i.e. not during another factory invocation (e.g. when using associations.)
-
-**NOTE**: FactoryProf only tracks the database-persisted factories. In case of FactoryGirl/FactoryBot these are the factories
-provided by using `create` strategy. In case of Fabrication - objects that created using `create` method.
-
-## Instructions
-
-FactoryProf can be used with FactoryGirl/FactoryBot or Fabrication - application can be bundled with both gems at the same time.
-
-To activate FactoryProf use `FPROF` environment variable:
-
-```sh
-# Simple profiler
-FPROF=1 rspec
-
-# or
-FPROF=1 bundle exec rake test
-```
-
-## Factory Flamegraph
-
-The most useful feature of FactoryProf is the _FactoryFlame_ report. That's the special interpretation of Brendan Gregg's [flame graphs](http://www.brendangregg.com/flamegraphs.html) which allows you to identify _factory cascades_.
-
-To generate FactoryFlame report set `FPROF` environment variable to `flamegraph`:
-
-```sh
-FPROF=flamegraph rspec
-
-# or
-FPROF=flamegraph bundle exec rake test
-```
-
-That's how a report looks like:
-
-![](https://s3-us-west-1.amazonaws.com/vladem/test-prof/factory-flame.gif)
-
-How to read this?
-
-Every column represents a _factory stack_ or _cascade_, that is a sequence of recursive `#create` method calls. Consider an example:
-
-```ruby
-factory :comment do
-  answer
-  author
-end
-
-factory :answer do
-  question
-  author
-end
-
-factory :question do
-  author
-end
-
-create(:comment) #=> creates 5 records
-
-# And the corresponding stack is:
-# [:comment, :answer, :question, :author, :author, :author]
-```
-
-The wider column the more often this stack appears.
-
-The `root` cell shows the total number of `create` calls.
-
-## Acknowledgements
-
-- Thanks to [Martin Spier](https://github.com/spiermar) for [d3-flame-graph](https://github.com/spiermar/d3-flame-graph)
-
-- Thanks to [Sam Saffron](https://github.com/SamSaffron) for his [flame graphs implementation](https://github.com/SamSaffron/flamegraph).

data/guides/let_it_be.md

@@ -1,97 +0,0 @@
-# Let It Be!
-
-Let's bring a little bit of magic and introduce a new way to setup a _shared_ test data.
-
-Suppose you have the following setup:
-
-```ruby
-describe BeatleWeightedSearchQuery do
-  let!(:paul) { create(:beatle, name: 'Paul') }
-  let!(:ringo) { create(:beatle, name: 'Ringo') }
-  let!(:george) { create(:beatle, name: 'George') }
-  let!(:john) { create(:beatle, name: 'John') }
-
-  specify { expect(subject.call('john')).to contain_exactly(john) }
-
-  # and more examples here
-end
-```
-
-We don't need to re-create the Fab Four for every example, do we?
-
-We already have [`before_all`](https://github.com/palkan/test-prof/tree/master/guides/before_all.md) to solve the problem of _repeatable_ data:
-
-```ruby
-describe BeatleWeightedSearchQuery do
-  before_all do
-    @paul = create(:beatle, name: 'Paul')
-    # ...
-  end
-
-  specify { expect(subject.call('joh')).to contain_exactly(@john) }
-
-  # ...
-end
-```
-
-That technique works pretty good but requires us to use instance variables and define everything at once. Thus it's not easy to refactor existing tests which use `let/let!` instead.
-
-With `let_it_be` you can do the following:
-
-```ruby
-describe BeatleWeightedSearchQuery do
-  let_it_be(:paul) { create(:beatle, name: 'Paul') }
-  let_it_be(:ringo) { create(:beatle, name: 'Ringo') }
-  let_it_be(:george) { create(:beatle, name: 'George') }
-  let_it_be(:john) { create(:beatle, name: 'John') }
-
-  specify { expect(subject.call('john')).to contain_exactly(john) }
-
-  # and more examples here
-end
-```
-
-That's it! Just replace `let!` with `let_it_be`. That's equal to the `before_all` approach but requires less refactoring.
-
-**NOTE**: requires RSpec >= 3.2.0.
-
-## Instructions
-
-In your `spec_helper.rb`:
-
-```ruby
-require 'test_prof/recipes/rspec/let_it_be'
-```
-
-In your tests:
-
-```ruby
-describe MySuperDryService do
-  let_it_be(:user) { create(:user) }
-
-  # ...
-end
-```
-
-## Caveats
-
-If you modify objects generated within a `let_it_be` block in your examples, you maybe have to re-initiate them.
-We have a built-in support for that:
-
-
-```ruby
-# Use reload: true option to reload user object (assuming it's an instance of ActiveRecord)
-# for every example
-let_it_be(:user, reload: true) { create(:user) }
-
-# it is almost equal to
-before_all { @user = create(:user) }
-let(:user) { @user.reload }
-
-# You can also specify refind: true option to hard-reload the record
-let_it_be(:user, refind: true) { create(:user) }
-
-# it is almost equal to
-before_all { @user = create(:user) }
-let(:user) { User.find(@user.id) }
-```

data/guides/rspec_dissect.md

@@ -1,60 +0,0 @@
-# RSpecDissect
-
-Do you know how much time you spend in `before` hooks? Or in memoization helpers such as `let`? Usually, the most of the whole test suite time.
-
-_RSpecDissect_ provides this kind of information and also shows you the worst example groups. The main purpose of RSpecDissect is to identify these slow groups and refactor them using [`before_all`](https://github.com/palkan/test-prof/tree/master/guides/before_all.md) or [`let_it_be`](https://github.com/palkan/test-prof/tree/master/guides/let_it_be.md) recipes.
-
-Example output:
-
-```sh
-[TEST PROF INFO] RSpecDissect enabled
-
-Total time: 25:14.870
-Total `before(:each)` time: 14:36.482
-Total `let` time: 19:20.259
-
-Top 5 slowest suites (by `before(:each)` time):
-
-Webhooks::DispatchTransition (./spec/services/webhooks/dispatch_transition_spec.rb:3) – 00:29.895 of 00:33.706 (327)
-FunnelsController (./spec/controllers/funnels_controller_spec.rb:3) – 00:22.117 of 00:43.649 (133)
-ApplicantsController (./spec/controllers/applicants_controller_spec.rb:3) – 00:21.220 of 00:41.407 (222)
-BookedSlotsController (./spec/controllers/booked_slots_controller_spec.rb:3) – 00:15.729 of 00:27.893 (50)
-Analytics::Wor...rsion::Summary (./spec/services/analytics/workflow_conversion/summary_spec.rb:3) – 00:15.383 of 00:15.914 (12)
-
-
-Top 5 slowest suites (by `let` time):
-
-FunnelsController (./spec/controllers/funnels_controller_spec.rb:3) – 00:38.532 of 00:43.649 (133)
-ApplicantsController (./spec/controllers/applicants_controller_spec.rb:3) – 00:33.252 of 00:41.407 (222)
-Webhooks::DispatchTransition (./spec/services/webhooks/dispatch_transition_spec.rb:3) – 00:30.320 of 00:33.706 (327)
-BookedSlotsController (./spec/controllers/booked_slots_controller_spec.rb:3) – 00:25.710 of 00:27.893 (50)
-AvailableSlotsController (./spec/controllers/available_slots_controller_spec.rb:3) – 00:18.481 of 00:23.366 (85)
-```
-
-**NOTE**: Tracking `let` time only supported in RSpec >= 3.3.0.
-
-## Instructions
-
-RSpecDissect can only be used with RSpec (which is clear from the name).
-
-To activate RSpecDissect use `RD_PROF` environment variable:
-
-```sh
-RD_PROF=1 rspec ...
-```
-
-You can also specify the number of top slow groups through `RD_PROF_TOP` variable:
-
-```sh
-RD_PROF=1 RD_PROF_TOP=10 rspec ...
-```
-
-## Using with RSpecStamp
-
-RSpecDissect can be used with [RSpec Stamp](https://github.com/palkan/test-prof/tree/master/guides/rspec_stamp.md) to automatically mark _slow_ examples with custom tags. For example:
-
-```sh
-RD_PROF=1 RD_PROF_STAMP="slow" rspec ...
-```
-
-After running the command above the slowest example groups would be marked with the `:slow` tag.

data/guides/rspec_stamp.md

@@ -1,52 +0,0 @@
-# RSpecStamp
-
-RSpecStamp is a tool to automatically _tag_ failed examples with custom tags.
-
-It _literally_ adds tags to your examples (i.e. rewrites them).
-
-The main purpose of RSpecStamp is to make testing codebase refactoring easy. Changing global configuration may cause a lot of failures. You can patch failing spec by adding a shared context. And here comes RSpecStamp.
-
-## Example Use Case: Sidekiq Inline
-
-Using `Sidekiq::Testing.inline!` may be considered a _bad practice_ (see [here](https://github.com/mperham/sidekiq/issues/3495)) due to its negative performance impact. But it's still widely used.
-
-How to migrate from `inline!` to `fake!`?
-
-Step 0. Make sure that all your tests pass.
-
-Step 1. Create a shared context to conditionally turn on `inline!` mode:
-
-```ruby
-shared_context 'sidekiq:inline', sidekiq: :inline do
-  around(:each) { |ex| Sidekiq::Testing.inline!(&ex) }
-end
-```
-
-Step 2. Turn on `fake!` mode globally.
-
-Step 3. Run `RSTAMP=sidekiq:inline rspec`.
-
-The output of the command above contains information about the _stamping_ process:
-
-- How many files have been affected?
-
-- How many patches were made?
-
-- How many patches failed?
-
-- How many files have been ignored?
-
-Now all (or almost all) failing specs are tagged with `sidekiq: :inline`. Run the whole suite again and check it there are any failures left.
-
-There is also a `dry-run` mode (activated by `RSTAMP_DRY_RUN=1` env variable) which prints out patches instead of re-writing files.
-
-## Configuration
-
-By default, RSpecStamp ignores examples located in `spec/support` directory (typical place to put shared examples in).
-You can add more _ignore_ patterns:
-
-```ruby
-TestProf::RSpecStamp.configure do |config|
-  config.ignore_files << %r{spec/my_directory}
-end
-```

data/guides/rubocop.md

@@ -1,48 +0,0 @@
-# Custom RuboCop Cops
-
-TestProf comes with the [RuboCop](https://github.com/bbatsov/rubocop) cops that help you write more performant tests.
-
-To enable them:
-
-- Require `test_prof/rubocop` in your RuboCop configuration:
-
-```yml
-# .rubocop.yml
-require:
- - 'test_prof/rubocop'
-```
-
-- Enable cops:
-
-```yml
-RSpec/AggregateFailures:
-  Enabled: true
-  Include:
-    - 'spec/**/*.rb'
-```
-
-## RSpec/AggregateFailures
-
-This cop encourages you to use one of the greatest features of the recent RSpec – aggregating failures within an example.
-
-Instead of writing one example per assertion, you can group _independent_ assertions together, thus running all setup hooks only once. That can dramatically increase your performance (by reducing the total number of examples).
-
-Consider an example:
-
-```ruby
-# bad
-it { is_expected.to be_success }
-it { is_expected.to have_header('X-TOTAL-PAGES', 10) }
-it { is_expected.to have_header('X-NEXT-PAGE', 2) }
-
-# good
-it 'returns the second page', :aggregate_failures do
-  is_expected.to be_success
-  is_expected.to have_header('X-TOTAL-PAGES', 10)
-  is_expected.to have_header('X-NEXT-PAGE', 2)
-end
-```
-
-This cop supports auto-correct feature, so you can automatically refactor you legacy tests!
-
-**NOTE**: auto-correction may break your tests (especially the ones using block-matchers, such as `change`).

data/guides/ruby_prof.md

@@ -1,63 +0,0 @@
-# Profiling with RubyProf
-
-Easily integrate the power of [ruby-prof](https://github.com/ruby-prof/ruby-prof) into your test suite.
-
-## Instructions
-
-Install `ruby-prof` gem (>= 0.16):
-
-```ruby
-# Gemfile
-group :development, :test do
-  gem 'ruby-prof', '>= 0.16.0', require: false
-end
-```
-
-RubyProf profiler has two modes: `global` and `per-example`.
-
-You can activate the global profiling using the environment variable `TEST_RUBY_PROF`:
-
-```sh
-TEST_RUBY_PROF=1 bundle exec rake test
-
-# or for RSpec
-TEST_RUBY_PROF=1 rspec ...
-```
-
-Or in your code:
-
-```ruby
-TestProf::RubyProf.run
-```
-
-TestProf provides a built-in shared context for RSpec to profile examples individually:
-
-```ruby
-it 'is doing heavy stuff', :rprof do
-  # ...
-end
-```
-
-### Configuration
-
-The most useful configuration option is `printer` – it allows you to specify a RubyProf [printer](https://github.com/ruby-prof/ruby-prof#printers).
-
-You can specify a printer through environment variable `TEST_RUBY_PROF`:
-
-```sh
-TEST_RUBY_PROF=call_stack bundle exec rake test
-```
-
-Or in your code:
-
-```ruby
-TestProf::RubyProf.configure do |config|
-  config.printer = :call_stack
-end
-```
-
-By default, we use `FlatPrinter`.
-
-Also, you can specify RubyProf mode (`wall`, `cpu`, etc) through `TEST_RUBY_PROF_MODE` env variable.
-
-See [ruby_prof.rb](https://github.com/palkan/test-prof/tree/master/lib/test_prof/ruby_prof.rb) for all available configuration options and their usage.

data/guides/stack_prof.md

@@ -1,47 +0,0 @@
-# Profiling with StackProf
-
-[StackProf](https://github.com/tmm1/stackprof) is a sampling call-stack profiler for ruby.
-
-## Instructions
-
-Install `stackprof` gem (>= 0.2.9):
-
-```ruby
-# Gemfile
-group :development, :test do
-  gem 'stackprof', '>= 0.2.9', require: false
-end
-```
-
-StackProf profiler has 2 modes: `global` and `per-example`.
-
-You can activate global profiling using the environment variable `TEST_STACK_PROF`:
-
-```sh
-TEST_STACK_PROF=1 bundle exec rake test
-
-# or for RSpec
-TEST_STACK_PROF=1 rspec ...
-```
-
-Or in your code:
-
-```ruby
-TestProf::StackProf.run
-```
-
-TestProf provides a built-in shared context for RSpec to profile examples individually:
-
-```ruby
-it 'is doing heavy stuff', :sprof do
-  # ...
-end
-```
-
-### Configuration
-
-You can change StackProf mode (which is `wall` by default) through `TEST_STACK_PROF_MODE` env variable.
-
-If you want to generate flame graphs you should collect _raw_ data. Turn _raw_ collection on by passing `TEST_STACK_PROF=raw`.
-
-See [stack_prof.rb](https://github.com/palkan/test-prof/tree/master/lib/test_prof/stack_prof.rb) for all available configuration options and their usage.

data/guides/tag_prof.md

@@ -1,51 +0,0 @@
-# TagProf
-
-TagProf is a simple profiler which collects examples statistics grouped by a provided tag value.
-
-That's pretty useful in conjunction with `rspec-rails` built-in feature – `infer_spec_types_from_location!` – which automatically adds `type` to examples metadata.
-
-Example output:
-
-```sh
-[TEST PROF INFO] TagProf report for type
-
-       type          time   total  %total   %time           avg
-
-    request     00:04.808      42   33.87   54.70     00:00.114
- controller     00:02.855      42   33.87   32.48     00:00.067
-      model     00:01.127      40   32.26   12.82     00:00.028
-```
-
-
-It shows both the total number of examples in each group and the total time spent (as long as percentages and average values).
-
-
-## Instructions
-
-TagProf can only be used with RSpec.
-
-To activate TagProf use `TAG_PROF` environment variable:
-
-```sh
-# Group by type
-TAG_PROF=type rspec
-```
-
-## Pro-Tip: More Types
-
-By default, RSpec only infers types for default Rails app entities (such as controllers, models, mailers, etc.).
-Modern Rails applications typically contain other abstractions too (e.g. services, forms, presenters, etc.), but RSpec is not aware of them and doesn't add any metadata.
-
-That's the quick workaround:
-
-```ruby
-RSpec.configure do |config|
-  # ...
-  config.define_derived_metadata(file_path: %r{/spec/}) do |metadata|
-    # do not overwrite type if it's already set
-    next if metadata.key?(:type)
-    match = metadata[:location].match(%r{/spec/([^/]+)/})
-    metadata[:type] = match[1].singularize.to_sym
-  end
-end
-```

data/guides/tests_sampling.md

@@ -1,24 +0,0 @@
-# Tests Sampling
-
-Sometimes it's useful to run profilers against randomly chosen tests. Unfortunetaly, test frameworks don's support such functionality. That's why we've included small patches for RSpec and Minitest in TestProf.
-
-
-## Instructions
-
-Require the corresponding patch:
-
-```ruby
-# For RSpec in your spec_helper.rb
-require 'test_prof/recipes/rspec/sample'
-
-# For Minitest in your test_helper.rb
-require 'test_prof/recipes/minitest/sample'
-```
-
-And then just add `SAMPLE` env variable with the number of example groups (or suites) you want to run:
-
-```sh
-SAMPLE=10 rspec
-```
-
-That's it. Enjoy!