test-prof 0.11.1 → 0.12.2

data/LICENSE.txt

@@ -1,6 +1,6 @@
 The MIT License (MIT)
 
-Copyright (c) 2017-2019 palkan
+Copyright (c) 2017-2020 Vladimir Dementyev
 
 Permission is hereby granted, free of charge, to any person obtaining a copy
 of this software and associated documentation files (the "Software"), to deal

data/README.md

@@ -1,7 +1,7 @@
-[![Cult Of Martians](http://cultofmartians.com/assets/badges/badge.svg)](http://cultofmartians.com)
-[![Gem Version](https://badge.fury.io/rb/test-prof.svg)](https://rubygems.org/gems/test-prof) [![Build](https://github.com/palkan/test-prof/workflows/Build/badge.svg)](https://github.com/palkan/test-prof/actions)
-[![JRuby Build](https://github.com/palkan/test-prof/workflows/JRuby%20Build/badge.svg)](https://github.com/palkan/test-prof/actions)
-[![Code Triagers Badge](https://www.codetriage.com/palkan/test-prof/badges/users.svg)](https://www.codetriage.com/palkan/test-prof)
+[![Cult Of Martians](http://cultofmartians.com/assets/badges/badge.svg)](https://cultofmartians.com)
+[![Gem Version](https://badge.fury.io/rb/test-prof.svg)](https://rubygems.org/gems/test-prof) [![Build](https://github.com/test-prof/test-prof/workflows/Build/badge.svg)](https://github.com/test-prof/test-prof/actions)
+[![JRuby Build](https://github.com/test-prof/test-prof/workflows/JRuby%20Build/badge.svg)](https://github.com/test-prof/test-prof/actions)
+[![Code Triagers Badge](https://www.codetriage.com/test-prof/test-prof/badges/users.svg)](https://www.codetriage.com/test-prof/test-prof)
 [![Documentation](https://img.shields.io/badge/docs-link-brightgreen.svg)](https://test-prof.evilmartians.io)
 
 # Ruby Tests Profiling Toolbox
@@ -47,11 +47,11 @@ TestProf toolbox aims to help you identify bottlenecks in your test suite. It co
 ## Who uses TestProf
 
 - [Discourse](https://github.com/discourse/discourse) reduced [~27% of their test suite time](https://twitter.com/samsaffron/status/1125602558024699904)
-- [Gitlab](https://gitlab.com/gitlab-org/gitlab-ce) reduced [39% of their API tests time](https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/14370)
+- [Gitlab](https://gitlab.com/gitlab-org/gitlab-ce) reduced [39% of their API tests time](https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/14370) and [improved factories usage](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/26810)
 - [CodeTriage](https://github.com/codetriage/codetriage)
 - [Dev.to](https://github.com/thepracticaldev/dev.to)
 - [Open Project](https://github.com/opf/openproject)
-- [...and others](https://github.com/palkan/test-prof/issues/73)
+- [...and others](https://github.com/test-prof/test-prof/issues/73)
 
 ## Resources
 
@@ -83,7 +83,7 @@ And that's it)
 
 Supported Ruby versions:
 
-- Ruby (MRI) >= 2.4.0 (**NOTE:** for Ruby 2.2 use TestProf < 0.7.0 or Ruby 2.3 use TestProf ~> 0.7.0)
+- Ruby (MRI) >= 2.5.0 (**NOTE:** for Ruby 2.2 use TestProf < 0.7.0, Ruby 2.3 use TestProf ~> 0.7.0, Ruby 2.4 use TestProf <0.12.0)
 
 - JRuby >= 9.1.0.0 (**NOTE:** refinements-dependent features might require 9.2.7+)
 
@@ -95,16 +95,12 @@ Check out our [docs][].
 
 ## What's next?
 
-Have an idea? [Propose](https://github.com/palkan/test-prof/issues/new) a feature request!
+Have an idea? [Propose](https://github.com/test-prof/test-prof/issues/new) a feature request!
 
-Already using TestProf? [Share your story!](https://github.com/palkan/test-prof/issues/73)
+Already using TestProf? [Share your story!](https://github.com/test-prof/test-prof/issues/73)
 
 ## License
 
 The gem is available as open source under the terms of the [MIT License](http://opensource.org/licenses/MIT).
 
 [docs]: https://test-prof.evilmartians.io
-
-## Security Contact
-
-To report a security vulnerability, please use the [Tidelift security contact](https://tidelift.com/security). Tidelift will coordinate the fix and disclosure.

data/config/default.yml

@@ -6,7 +6,7 @@ AllCops:
     - "(?:^|/)spec/"
 
 RSpec/AggregateExamples:
-  Description: Checks if example groups contain two or more aggregatable examples.
+  Description: Checks if example group contains two or more aggregatable examples.
   Enabled: true
   StyleGuide: https://rspec.rubystyle.guide/#expectation-per-example
   AddAggregateFailuresMetadata: true
@@ -21,8 +21,8 @@ RSpec/AggregateExamples:
 
 # TODO: remove this one we hit 1.0
 RSpec/AggregateFailures:
-  Description: Checks if example groups contain two or more aggregatable examples.
-  Enabled: true
+  Description: Checks if example group contains two or more aggregatable examples.
+  Enabled: false
   StyleGuide: https://rspec.rubystyle.guide/#expectation-per-example
   AddAggregateFailuresMetadata: true
   MatchersWithSideEffects:

data/lib/minitest/test_prof_plugin.rb

@@ -12,6 +12,7 @@ module Minitest # :nodoc:
         opts[:top_count] = ENV["EVENT_PROF_TOP"].to_i if ENV["EVENT_PROF_TOP"]
         opts[:per_example] = true if ENV["EVENT_PROF_EXAMPLES"]
         opts[:fdoc] = true if ENV["FDOC"]
+        opts[:sample] = true if ENV["SAMPLE"] || ENV["SAMPLE_GROUPS"]
       end
     end
   end
@@ -39,5 +40,7 @@ module Minitest # :nodoc:
 
     reporter << TestProf::EventProfReporter.new(options[:io], options) if options[:event]
     reporter << TestProf::FactoryDoctorReporter.new(options[:io], options) if options[:fdoc]
+
+    ::TestProf::MinitestSample.call if options[:sample]
   end
 end

data/lib/test_prof/before_all.rb

@@ -24,10 +24,6 @@ module TestProf
         yield
       end
 
-      def within_transaction
-        yield
-      end
-
       def rollback_transaction
         raise AdapterMissing if adapter.nil?
 

data/lib/test_prof/cops/inject.rb

@@ -2,22 +2,24 @@
 
 # This is shamelessly borrowed from RuboCop RSpec
 # https://github.com/rubocop-hq/rubocop-rspec/blob/master/lib/rubocop/rspec/inject.rb
-module RuboCop
-  # Because RuboCop doesn't yet support plugins, we have to monkey patch in a
-  # bit of our configuration.
-  module Inject
-    PROJECT_ROOT = Pathname.new(__dir__).parent.parent.parent.expand_path.freeze
-    CONFIG_DEFAULT = PROJECT_ROOT.join("config", "default.yml").freeze
+module TestProf
+  module Cops
+    # Because RuboCop doesn't yet support plugins, we have to monkey patch in a
+    # bit of our configuration.
+    module Inject
+      PROJECT_ROOT = Pathname.new(__dir__).parent.parent.parent.expand_path.freeze
+      CONFIG_DEFAULT = PROJECT_ROOT.join("config", "default.yml").freeze
 
-    def self.defaults!
-      path = CONFIG_DEFAULT.to_s
-      hash = ConfigLoader.send(:load_yaml_configuration, path)
-      config = Config.new(hash, path)
-      puts "configuration from #{path}" if ConfigLoader.debug?
-      config = ConfigLoader.merge_with_default(config, path)
-      ConfigLoader.instance_variable_set(:@default_configuration, config)
+      def self.defaults!
+        path = CONFIG_DEFAULT.to_s
+        hash = RuboCop::ConfigLoader.send(:load_yaml_configuration, path)
+        config = RuboCop::Config.new(hash, path)
+        puts "configuration from #{path}" if RuboCop::ConfigLoader.debug?
+        config = RuboCop::ConfigLoader.merge_with_default(config, path)
+        RuboCop::ConfigLoader.instance_variable_set(:@default_configuration, config)
+      end
     end
   end
 end
 
-RuboCop::Inject.defaults!
+TestProf::Cops::Inject.defaults!

data/lib/test_prof/cops/rspec/aggregate_examples.rb

@@ -12,7 +12,7 @@ module RuboCop
     module RSpec
       # Checks if example groups contain two or more aggregatable examples.
       #
-      # @see https://github.com/rubocop-hq/rspec-style-guide#expectations-per-example
+      # @see https://github.com/rubocop-hq/rspec-style-guide#expectation-per-example
       #
       # This cop is primarily for reducing the cost of repeated expensive
       # context initialization.
@@ -108,7 +108,7 @@ module RuboCop
       #     expect(number).to be_odd
       #   end
       #
-      class AggregateExamples < Cop
+      class AggregateExamples < ::RuboCop::Cop::Cop
         include LineRangeHelpers
         include MetadataHelpers
         include NodeMatchers

data/lib/test_prof/cops/rspec/aggregate_examples/its.rb

@@ -3,7 +3,7 @@
 module RuboCop
   module Cop
     module RSpec
-      class AggregateExamples < Cop
+      class AggregateExamples < ::RuboCop::Cop::Cop
         # @example `its`
         #
         #   # Supports regular `its` call with an attribute/method name,

data/lib/test_prof/cops/rspec/aggregate_examples/line_range_helpers.rb

@@ -3,7 +3,7 @@
 module RuboCop
   module Cop
     module RSpec
-      class AggregateExamples < Cop
+      class AggregateExamples < ::RuboCop::Cop::Cop
         # @internal Support methods for keeping newlines around examples.
         module LineRangeHelpers
           include RangeHelp

data/lib/test_prof/cops/rspec/aggregate_examples/matchers_with_side_effects.rb

@@ -5,7 +5,7 @@ require_relative "../language"
 module RuboCop
   module Cop
     module RSpec
-      class AggregateExamples < Cop
+      class AggregateExamples < ::RuboCop::Cop::Cop
         # When aggregated, the expectations will fail when not supposed to or
         # have a risk of not failing when expected to. One example is
         # `validate_presence_of :comment` as it leaves an empty comment after

data/lib/test_prof/cops/rspec/aggregate_examples/metadata_helpers.rb

@@ -3,7 +3,7 @@
 module RuboCop
   module Cop
     module RSpec
-      class AggregateExamples < Cop
+      class AggregateExamples < ::RuboCop::Cop::Cop
         # @internal
         #   Support methods for example metadata.
         #   Examples with similar metadata are grouped.

data/lib/test_prof/cops/rspec/aggregate_examples/node_matchers.rb

@@ -5,7 +5,7 @@ require_relative "../language"
 module RuboCop
   module Cop
     module RSpec
-      class AggregateExamples < Cop
+      class AggregateExamples < ::RuboCop::Cop::Cop
         # @internal
         #   Node matchers and searchers.
         module NodeMatchers

data/lib/test_prof/cops/rspec/aggregate_failures.rb

@@ -4,14 +4,12 @@ module RuboCop
   module Cop
     module RSpec
       class AggregateExamples
-        def self.inherited(subclass)
-          superclass.registry.enlist(subclass)
+        def self.registry
+          RuboCop::Cop::Cop.registry
         end
       end
 
       class AggregateFailures < AggregateExamples
-        raise "Remove me" if TestProf::VERSION >= "1.0"
-
         def initialize(*)
           super
           self.class.just_once { warn "`AggregateFailures` cop has been renamed to `AggregateExamples`." }

data/lib/test_prof/event_prof/instrumentations/active_support.rb

@@ -4,13 +4,31 @@ module TestProf::EventProf
   module Instrumentations
     # Wrapper over ActiveSupport::Notifications
     module ActiveSupport
+      class Subscriber
+        attr_reader :block, :started_at
+
+        def initialize(block)
+          @block = block
+        end
+
+        def start(*)
+          @started_at = TestProf.now
+        end
+
+        def publish(_name, started_at, finished_at, *)
+          block.call(finished_at - started_at)
+        end
+
+        def finish(*)
+          block.call(TestProf.now - started_at)
+        end
+      end
+
       class << self
-        def subscribe(event)
+        def subscribe(event, &block)
           raise ArgumentError, "Block is required!" unless block_given?
 
-          ::ActiveSupport::Notifications.subscribe(event) do |_event, start, finish, *_args|
-            yield (finish - start)
-          end
+          ::ActiveSupport::Notifications.subscribe(event, Subscriber.new(block))
         end
 
         def instrument(event)

data/lib/test_prof/ext/active_record_3.rb

@@ -5,8 +5,8 @@ module TestProf
   module ActiveRecord3Transactions
     refine ::ActiveRecord::ConnectionAdapters::AbstractAdapter do
       def begin_transaction(joinable: true)
-        increment_open_transactions
         if open_transactions > 0
+          increment_open_transactions
           create_savepoint
         else
           begin_db_transaction

data/lib/test_prof/recipes/minitest/before_all.rb

@@ -8,20 +8,21 @@ module TestProf
     # store instance variables
     module Minitest # :nodoc: all
       class Executor
-        attr_reader :active
+        attr_reader :active, :block, :captured_ivars
 
         alias active? active
 
         def initialize(&block)
           @block = block
+          @captured_ivars = []
         end
 
-        def activate!(test_class)
-          return if active?
+        def activate!(test_object)
+          return restore_ivars(test_object) if active?
           @active = true
-          @examples_left = test_class.runnable_methods.size
+          @examples_left = test_object.class.runnable_methods.size
           BeforeAll.begin_transaction do
-            capture!
+            capture!(test_object)
           end
         end
 
@@ -33,16 +34,21 @@ module TestProf
           BeforeAll.rollback_transaction
         end
 
-        def capture!
-          instance_eval(&@block)
+        def capture!(test_object)
+          before_ivars = test_object.instance_variables
+
+          test_object.instance_eval(&block)
+
+          (test_object.instance_variables - before_ivars).each do |ivar|
+            captured_ivars << [ivar, test_object.instance_variable_get(ivar)]
+          end
         end
 
-        def restore_to(test_object)
-          instance_variables.each do |ivar|
-            next if ivar == :@block
+        def restore_ivars(test_object)
+          captured_ivars.each do |(ivar, val)|
             test_object.instance_variable_set(
               ivar,
-              instance_variable_get(ivar)
+              val
             )
           end
         end
@@ -62,8 +68,7 @@ module TestProf
 
           prepend(Module.new do
             def setup
-              self.class.before_all_executor.activate!(self.class)
-              self.class.before_all_executor.restore_to(self)
+              self.class.before_all_executor.activate!(self)
               super
             end
 

data/lib/test_prof/recipes/minitest/sample.rb

@@ -35,18 +35,14 @@ module TestProf
           end
         end
       end
-    end
 
-    # Overrides Minitest.run
-    def run(*)
-      if ENV["SAMPLE"]
-        MinitestSample.sample_examples(ENV["SAMPLE"].to_i)
-      elsif ENV["SAMPLE_GROUPS"]
-        MinitestSample.sample_groups(ENV["SAMPLE_GROUPS"].to_i)
+      def call
+        if ENV["SAMPLE"]
+          ::TestProf::MinitestSample.sample_examples(ENV["SAMPLE"].to_i)
+        elsif ENV["SAMPLE_GROUPS"]
+          ::TestProf::MinitestSample.sample_groups(ENV["SAMPLE_GROUPS"].to_i)
+        end
       end
-      super
     end
   end
 end
-
-Minitest.singleton_class.prepend(TestProf::MinitestSample)

data/lib/test_prof/recipes/rspec/before_all.rb

@@ -9,7 +9,7 @@ module TestProf
       def before_all(&block)
         raise ArgumentError, "Block is required!" unless block_given?
 
-        return within_before_all(&block) if within_before_all?
+        return before(:all, &block) if within_before_all?
 
         @__before_all_activated__ = true
 
@@ -24,14 +24,6 @@ module TestProf
         end
       end
 
-      def within_before_all(&block)
-        before(:all) do
-          BeforeAll.within_transaction do
-            instance_eval(&block)
-          end
-        end
-      end
-
       def within_before_all?
         instance_variable_defined?(:@__before_all_activated__)
       end

data/lib/test_prof/recipes/rspec/let_it_be.rb

@@ -22,6 +22,10 @@ module TestProf
 
         LetItBe.modifiers[key] = block
       end
+
+      def default_modifiers
+        @default_modifiers ||= {}
+      end
     end
 
     class << self
@@ -75,6 +79,8 @@ module TestProf
     # And we love cats!)
     PREFIX = RUBY_ENGINE == "jruby" ? "@__jruby_is_not_cat_friendly__" : "@😸"
 
+    FROZEN_ERROR_HINT = "\nIf you are using `let_it_be`, you may want to pass `reload: true` or `refind: true` modifier to it."
+
     def self.define_let_it_be_alias(name, **default_args)
       define_method(name) do |identifier, **options, &blk|
         let_it_be(identifier, **default_args.merge(options), &blk)
@@ -83,20 +89,20 @@ module TestProf
 
     def let_it_be(identifier, **options, &block)
       initializer = proc do
-        instance_variable_set(:"#{PREFIX}#{identifier}", instance_exec(&block))
+        instance_variable_set(:"#{TestProf::LetItBe::PREFIX}#{identifier}", instance_exec(&block))
+      rescue FrozenError => e
+        e.message << TestProf::LetItBe::FROZEN_ERROR_HINT
+        raise
       end
 
-      if within_before_all?
-        within_before_all(&initializer)
-      else
-        before_all(&initializer)
-      end
+      default_options = LetItBe.config.default_modifiers.dup
+      default_options.merge!(metadata[:let_it_be_modifiers]) if metadata[:let_it_be_modifiers]
 
-      define_let_it_be_methods(identifier, **options)
-    end
+      options = default_options.merge(options)
+
+      before_all(&initializer)
 
-    def define_let_it_be_methods(identifier, **modifiers)
-      let_accessor = LetItBe.wrap_with_modifiers(modifiers) do
+      let_accessor = LetItBe.wrap_with_modifiers(options) do
         instance_variable_get(:"#{PREFIX}#{identifier}")
       end
 
@@ -114,16 +120,78 @@ module TestProf
 
       let(identifier, &let_accessor)
     end
+
+    module Freezer
+      # Stoplist to prevent freezing objects and theirs associations that are defined
+      # with `let_it_be`'s `freeze: false` options during deep freezing.
+      #
+      # To only keep track of objects that are available in current example group,
+      # `begin` adds a new layer, and `rollback` removes a layer of unrelated objects
+      # along with rolling back the transaction where they were created.
+      #
+      # Stoplist holds records declared with `freeze: false` (so we do not freeze them even if they're used as
+      # associated records for frozen objects)
+      module Stoplist
+        class << self
+          def stop?(record)
+            @stoplist.any? { |layer| layer.include?(record) }
+          end
+
+          def stop!(record)
+            @stoplist.last.push(record)
+          end
+
+          def begin
+            @stoplist.push([])
+          end
+
+          def rollback
+            @stoplist.pop
+          end
+        end
+
+        # Stack of example group-related variable definitions
+        @stoplist = []
+      end
+
+      class << self
+        # Rerucsively freezes the object to detect modifications
+        def deep_freeze(record)
+          return if record.frozen?
+          return if Stoplist.stop?(record)
+
+          record.freeze
+
+          # Support `let_it_be` with `create_list`
+          return record.each { |rec| deep_freeze(rec) } if record.respond_to?(:each)
+
+          # Freeze associations as well.
+          return unless defined?(::ActiveRecord::Base)
+          return unless record.is_a?(::ActiveRecord::Base)
+
+          record.class.reflections.keys.each do |reflection|
+            # But only if they are already loaded. If not yet loaded, they weren't
+            # created by factories, and it's ok to mutate them.
+
+            next unless record.association(reflection.to_sym).loaded?
+
+            target = record.association(reflection.to_sym).target
+            deep_freeze(target) if target.is_a?(::ActiveRecord::Base) || target.respond_to?(:each)
+          end
+        end
+      end
+    end
   end
 end
 
-if defined?(::ActiveRecord)
+if defined?(::ActiveRecord::Base)
   require "test_prof/ext/active_record_refind"
   using TestProf::Ext::ActiveRecordRefind
 
   TestProf::LetItBe.configure do |config|
     config.register_modifier :reload do |record, val|
       next record unless val
+
       next record.reload if record.is_a?(::ActiveRecord::Base)
 
       if record.respond_to?(:map)
@@ -136,6 +204,7 @@ if defined?(::ActiveRecord)
 
     config.register_modifier :refind do |record, val|
       next record unless val
+
       next record.refind if record.is_a?(::ActiveRecord::Base)
 
       if record.respond_to?(:map)
@@ -145,7 +214,35 @@ if defined?(::ActiveRecord)
       end
       record
     end
+
+    config.register_modifier :freeze do |record, val|
+      if val == false
+        TestProf::LetItBe::Freezer::Stoplist.stop!(record)
+        next record
+      end
+
+      TestProf::LetItBe::Freezer.deep_freeze(record)
+      record
+    end
   end
 end
 
 RSpec::Core::ExampleGroup.extend TestProf::LetItBe
+
+TestProf::BeforeAll.configure do |config|
+  config.before(:begin) do
+    TestProf::LetItBe::Freezer::Stoplist.begin
+  end
+
+  config.after(:rollback) do
+    TestProf::LetItBe::Freezer::Stoplist.rollback
+  end
+end
+
+RSpec.configure do |config|
+  config.after(:example) do |example|
+    if example.exception&.is_a?(FrozenError)
+      example.exception.message << TestProf::LetItBe::FROZEN_ERROR_HINT
+    end
+  end
+end