rspec_profiling 0.0.6 → 0.0.7

data/README.md

@@ -5,6 +5,7 @@ with interesting attributes. For example, find the slowest specs, or the
 spec which issues the most queries.
 
 Collected attributes include:
+
 - git commit SHA (or SVN revision) and date
 - example file, line number and description
 - example status (i.e. passed or failed)
@@ -21,29 +22,36 @@ RspecProfiling should work with Rails >= 3.2 and RSpec >= 2.14.
 
 Add this line to your application's Gemfile:
 
-```
+```ruby
 gem 'rspec_profiling'
 ```
 
 And then execute:
 
-```
+```bash
 bundle
 ```
 
 Require the gem to your `spec_helper.rb`.
 
-```
+```ruby
 require "rspec_profiling/rspec"
 ```
 
 Lastly, run the installation rake tasks to initialize an empty database in
 which results will be collected.
 
-```
+```bash
 bundle exec rake rspec_profiling:install
 ```
 
+If you are planning on using `sqlite` or `pg` ensure to add the dependency to your gemfile
+
+```ruby
+  gem 'sqlite', require: false
+  gem 'pg', require: false
+```
+
 ## Usage
 
 ### Choose a version control system
@@ -74,19 +82,51 @@ RspecProfiling.configure do |config|
 end
 ```
 
+#### Custom Event Subscriptions
+
+```Ruby
+RspecProfiling.configure do |config|
+  config.events = %w[event1 event2]
+end
+```
+
+Note that custom events are only currently reported by the CSV collector.
+
+#### Custom Event Recording
+
+It is possible to record the event metadata for a spec.
+
+```Ruby
+  describe 'Records all active record queries', record_events: %w[sql.active_record] do
+    it 'Records Rails deprecations', record_events: %w[deprecation.rails] do
+      ...
+    end
+    it 'Records nothing' do
+      ...
+    end
+  end
+```
+
 ### Choose a results collector
 
 Results are collected just by running the specs.
 
 #### SQLite3
 
-By default, profiles are collected in an SQL database. Make sure you've
-run the installation rake task before attempting.
+Make sure you've run the installation rake task before attempting.
+
+You can configure `RspecProfiling` to collect results in a SQL database in `config/initializers/rspec_profiling.rb`:
+
+```Ruby
+RspecProfiling.configure do |config|
+  config.collector = RspecProfiling::Collectors::SQL
+end
+```
 
 You can review results by running the RspecProfiling console.
 The console has a preloaded `results` variable.
 
-```
+```bash
 bundle exec rake rspec_profiling:console
 
 > results.count
@@ -95,14 +135,14 @@ bundle exec rake rspec_profiling:console
 
 You can find the spec that runs the most queries:
 
-```
+```ruby
 > results.order(:query_count).last.to_s
 => "Updating my account - ./spec/features/account_spec.rb:15"
 ```
 
 Or find the spec that takes the most time:
 
-```
+```ruby
 > results.order(:time).last.to_s
 => "Updating my account - ./spec/features/account_spec.rb:15"
 ```
@@ -112,7 +152,7 @@ debugging, such as `exception` and `status`.
 
 #### CSV
 
-You can configure `RspecProfiling` to collect results in a CSV in `config/initializers/rspec_profiling.rb`:
+By default, profiles are collected in an a CSV file. You can configure `RspecProfiling` to collect results in a CSV in `config/initializers/rspec_profiling.rb`:
 
 ```Ruby
 RspecProfiling.configure do |config|
@@ -177,16 +217,18 @@ To remove the results database, run `bundle exec rake rspec_profiling:uninstall`
 
 ## Contributing
 
-1. Fork it
-2. Create your feature branch (`git checkout -b my-new-feature`)
-3. Commit your changes (`git commit -am 'Add some feature'`)
-4. Push to the branch (`git push origin my-new-feature`)
-5. Create new Pull Request
+Please read [CONTRIBUTING.md](CONTRIBUTING.md) for details on our code of conduct, and the process for submitting pull requests to us.
 
-## About Breakwater
+## Local Development
 
-![Breakwater Logo](https://images.squarespace-cdn.com/content/5d084fe43b0b620001239437/1565926359217-2LQ1BOFAO5846OAYAGZV/breakwater.png?content-type=image%2Fpng)
+Local tools needed:
 
-Breakwater builds exciting web and mobile apps in Louisville, CO. Our work powers a wide variety of businesses with many different needs. We love open source software, and we're proud to contribute where we can. Interested to learn more? [Contact us today](https://www.breakwaterltd.com/contact).
+- docker
+- docker-compose
+- ruby
 
-This project is maintained by Breakwater. The names and logos of Breakwater are fully owned and copyright Breakwater Limited.
+To run the specs:
+
+```bash
+make spec
+```

data/SECURITY.md

@@ -0,0 +1,20 @@
+# Security Policy
+
+## Supported Versions
+
+Ruby versions that are currently being supported with security updates.
+
+| Version | Supported          |
+| ------- | ------------------ |
+| <=2.6   | :x:                |
+| 2.7     | :white_check_mark: |
+| 3.0     | :white_check_mark: |
+| 3.1     | :white_check_mark: |
+| 3.2     | :white_check_mark: |
+| 3.3     | :white_check_mark: |
+
+## Reporting a Vulnerability
+
+Please click the `Report a vulnerability` button [here](https://github.com/procore-oss/rspec_profiling/security) to report a vulnerability.
+
+A maintainer will respond to you as soon as possible and discuss the process to get the vulnerability fixed.

data/docker-compose.yml

@@ -0,0 +1,13 @@
+version: '3.9'
+
+services:
+  postgres:
+    image: postgres:16-alpine
+    ports:
+      - 5432:5432
+    volumes:
+      - ~/apps/postgres:/var/lib/postgresql/data
+    environment:
+      - POSTGRES_HOST=localhost
+      - POSTGRES_USER=myuser
+      - POSTGRES_PASSWORD=mypassword

data/lib/rspec_profiling/collectors/csv.rb

@@ -6,6 +6,7 @@ module RspecProfiling
       HEADERS = %w{
         branch
         commit_hash
+        seed
         date
         file
         line_number
@@ -17,6 +18,8 @@ module RspecProfiling
         query_time
         request_count
         request_time
+        start_memory
+        end_memory
       }
 
       def self.install
@@ -31,24 +34,44 @@ module RspecProfiling
         # no op
       end
 
-      def initialize
-        RspecProfiling.config.csv_path ||= 'tmp/spec_benchmarks.csv'
+      def initialize(config=RspecProfiling.config)
+        config.csv_path ||= 'tmp/spec_benchmarks.csv'
+
+        @config = config
       end
 
       def insert(attributes)
-        output << HEADERS.map do |field|
-          attributes.fetch(field.to_sym)
-        end
+        output << static_cells(attributes) + event_cells(attributes)
       end
 
       private
 
+      attr_reader :config
+
       def output
-        @output ||= ::CSV.open(path, "w").tap { |csv| csv << HEADERS }
+        @output ||= ::CSV.open(path, "w").tap { |csv| csv << HEADERS + event_headers }
       end
 
       def path
-        RspecProfiling.config.csv_path.call
+        config.csv_path.call
+      end
+
+      def static_cells(attributes)
+        HEADERS.map do |field|
+          attributes.fetch(field.to_sym)
+        end
+      end
+
+      def event_headers
+        config.events.flat_map do |event|
+          ["#{event}_count", "#{event}_time", "#{event}_events"]
+        end
+      end
+
+      def event_cells(attributes)
+        config.events.flat_map do |event|
+          [attributes[:event_counts][event], attributes[:event_times][event], attributes[:event_events][event].to_json]
+        end
       end
     end
   end

data/lib/rspec_profiling/collectors/json.rb

@@ -0,0 +1,70 @@
+module RspecProfiling
+  module Collectors
+    class JSON
+      KEYS = %w{
+        branch
+        commit_hash
+        seed
+        date
+        file
+        line_number
+        description
+        status
+        exception
+        time
+        query_count
+        query_time
+        request_count
+        request_time
+        start_memory
+        end_memory
+      }
+
+      def self.install
+        # no op
+      end
+
+      def self.uninstall
+        # no op
+      end
+
+      def self.reset
+        # no op
+      end
+
+      def initialize(config=RspecProfiling.config)
+        config.output_file_path ||= 'tmp/spec_benchmarks.json'
+
+        @config = config
+      end
+
+      def insert(attributes)
+        output << merge_attributes_and_events(attributes) + "\n"
+      end
+
+      private
+
+      attr_reader :config
+
+      def output
+        @output ||= ::File.open(path, "w")
+      end
+
+      def path
+        config.output_file_path.call
+      end
+
+      def merge_attributes_and_events(attributes)
+        config.events.flat_map do |event|
+          attributes["#{event}_counts"] = attributes[:event_counts][event]
+          attributes["#{event}_times"] = attributes[:event_times][event]
+          attributes["#{event}_events"] = attributes[:event_events][event]
+        end
+
+        attributes.merge!(config.additional_data)
+
+        attributes.except(:event_counts, :event_times, :event_events, :events).to_json
+      end
+    end
+  end
+end

data/lib/rspec_profiling/collectors/psql.rb

@@ -47,7 +47,7 @@ module RspecProfiling
       end
 
       def insert(attributes)
-        results.create!(attributes.except(:created_at))
+        results.create!(attributes.except(:created_at, :events, :event_counts, :event_times, :event_events))
       end
 
       def results

data/lib/rspec_profiling/collectors/sql.rb

@@ -47,7 +47,7 @@ module RspecProfiling
       end
 
       def insert(attributes)
-        results.create!(attributes.except(:created_at))
+        results.create!(attributes.except(:created_at, :events, :event_counts, :event_times, :event_events))
       end
 
       def results

data/lib/rspec_profiling/config.rb

@@ -5,9 +5,11 @@ module RspecProfiling
 
   def self.config
     @config ||= OpenStruct.new({
-      collector:  RspecProfiling::Collectors::SQL,
+      collector:  RspecProfiling::Collectors::CSV,
       vcs:        RspecProfiling::VCS::Git,
-      table_name: 'spec_profiling_results'
+      table_name: 'spec_profiling_results',
+      events:     [],
+      additional_data: {}
     })
   end
 end

data/lib/rspec_profiling/example.rb

@@ -21,6 +21,9 @@ module RspecProfiling
     def initialize(example)
       @example = example
       @counts  = Hash.new(0)
+      @event_counts = Hash.new(0)
+      @event_times = Hash.new(0)
+      @event_events = Hash.new()
     end
 
     def file
@@ -38,7 +41,7 @@ module RspecProfiling
     def status
       execution_result.status
     end
-    
+
     def exception
       execution_result.exception
     end
@@ -63,6 +66,8 @@ module RspecProfiling
       counts[:request_time]
     end
 
+    attr_reader :event_counts, :event_times, :event_events
+
     def log_query(query, start, finish)
       unless query[:sql] =~ IGNORED_QUERIES_PATTERN
         counts[:query_count] += 1
@@ -75,6 +80,19 @@ module RspecProfiling
       counts[:request_time] += request[:view_runtime].to_f
     end
 
+    def log_event(event_name, event, start, finish)
+      event_counts[event_name] += 1
+      event_times[event_name] += (finish - start)
+      event_events[event_name] ||= []
+      if verbose_record_event?(event_name)
+        begin
+          event_events[event_name] << event.as_json
+        rescue => e
+          # no op
+        end
+      end
+    end
+
     private
 
     attr_reader :example, :counts
@@ -90,5 +108,9 @@ module RspecProfiling
     def metadata
       example.metadata
     end
+
+    def verbose_record_event?(event_name)
+      metadata[:record_events].to_a.include?(event_name)
+    end
   end
 end

data/lib/rspec_profiling/rspec.rb

@@ -1,7 +1,7 @@
 require "rspec_profiling"
 
 RSpec.configure do |config|
-  runner = RspecProfiling::Run.new(RspecProfiling.config.collector.new, 
+  runner = RspecProfiling::Run.new(RspecProfiling.config.collector.new,
                                    RspecProfiling.config.vcs.new)
 
   config.reporter.register_listener(

data/lib/rspec_profiling/run.rb

@@ -1,26 +1,31 @@
+require "get_process_mem"
 require "rspec_profiling/example"
 require "rspec_profiling/vcs/git"
 require "rspec_profiling/vcs/svn"
 require "rspec_profiling/vcs/git_svn"
-require "rspec_profiling/collectors/sql"
-require "rspec_profiling/collectors/psql"
 require "rspec_profiling/collectors/csv"
+require "rspec_profiling/collectors/json"
 
 module RspecProfiling
   class Run
     def initialize(collector = RspecProfiling.config.collector.new,
-                   vcs = RspecProfiling.config.vcs.new)
+                   vcs = RspecProfiling.config.vcs.new,
+                   events = RspecProfiling.config.events)
 
       @collector = collector
       @vcs       = vcs
+      @events    = events
+      @seed      = RSpec.configuration.seed
     end
 
     def start(*args)
       start_counting_queries
       start_counting_requests
+      start_counting_events
     end
 
     def example_started(example)
+      start_recording_memory
       example = example.example if example.respond_to?(:example)
       @current_example = Example.new(example)
     end
@@ -29,6 +34,7 @@ module RspecProfiling
       collector.insert({
         branch:        vcs.branch,
         commit_hash:   vcs.sha,
+        seed:          @seed,
         date:          vcs.time,
         file:          @current_example.file,
         line_number:   @current_example.line_number,
@@ -39,7 +45,13 @@ module RspecProfiling
         query_count:   @current_example.query_count,
         query_time:    @current_example.query_time,
         request_count: @current_example.request_count,
-        request_time:  @current_example.request_time
+        request_time:  @current_example.request_time,
+        events:        @events,
+        event_counts:  @current_example.event_counts,
+        event_times:   @current_example.event_times,
+        event_events:  @current_example.event_events,
+        start_memory:  @start_memory,
+        end_memory:    end_memory
       })
     end
 
@@ -48,7 +60,15 @@ module RspecProfiling
 
     private
 
-    attr_reader :collector, :vcs
+    attr_reader :collector, :vcs, :events, :seed, :start_memory
+
+    def end_memory
+      GetProcessMem.new.mb
+    end
+
+    def start_recording_memory
+      @start_memory = GetProcessMem.new.mb
+    end
 
     def start_counting_queries
       ActiveSupport::Notifications.subscribe("sql.active_record") do |name, start, finish, id, query|
@@ -61,5 +81,13 @@ module RspecProfiling
         @current_example.try(:log_request, request, start, finish)
       end
     end
+
+    def start_counting_events
+      events.each do |event_name|
+        ActiveSupport::Notifications.subscribe(event_name) do |name, start, finish, id, event|
+          @current_example.try(:log_event, event_name, event, start, finish)
+        end
+      end
+    end
   end
 end

data/lib/rspec_profiling/vcs/git.rb

@@ -12,7 +12,7 @@ module RspecProfiling
       end
 
       def time
-        Time.parse `git show -s --format=%ci #{sha}`
+        Time.parse `git show -s --format=%ci #{sha}`.chomp
       end
     end
   end

data/lib/rspec_profiling/version.rb

@@ -1,3 +1,3 @@
 module RspecProfiling
-  VERSION = "0.0.6"
+  VERSION = "0.0.7"
 end

data/lib/rspec_profiling.rb

@@ -4,12 +4,24 @@ require "rspec_profiling/config"
 require "rspec_profiling/version"
 require "rspec_profiling/run"
 require "rspec_profiling/collectors/csv"
-require "rspec_profiling/collectors/sql"
-require "rspec_profiling/collectors/psql"
 require "rspec_profiling/vcs/git"
 require "rspec_profiling/vcs/svn"
 require "rspec_profiling/vcs/git_svn"
 
+begin
+  require "rspec_profiling/collectors/sql"
+rescue LoadError
+  #no op
+end
+
+begin
+  require "rspec_profiling/collectors/psql"
+rescue LoadError
+  #no op
+end
+
+
+
 module RspecProfiling
   class Railtie < Rails::Railtie
     railtie_name :rspec_profiling

data/rspec_profiling.gemspec

@@ -6,24 +6,27 @@ require 'rspec_profiling/version'
 Gem::Specification.new do |spec|
   spec.name          = "rspec_profiling"
   spec.version       = RspecProfiling::VERSION
-  spec.authors       = ["Ben Eddy"]
-  spec.email         = ["bae@foraker.com"]
+  spec.authors       = ["Procore Technologies, Inc."]
+  spec.email         = ["opensource@procore.com"]
   spec.description   = %q{Profile RSpec test suites}
   spec.summary       = %q{Profile RSpec test suites}
-  spec.homepage      = "https://github.com/foraker/rspec_profiling"
+  spec.homepage      = "https://github.com/procore-oss/rspec_profiling"
   spec.license       = "MIT"
 
-  spec.files         = `git ls-files`.split($/)
+  spec.files         = `git ls-files -z`.split("\x0").reject { |f| f.match(%r{^(test|spec|features)/}) }
   spec.executables   = spec.files.grep(%r{^bin/}) { |f| File.basename(f) }
   spec.test_files    = spec.files.grep(%r{^(test|spec|features)/})
   spec.require_paths = ["lib"]
 
-  spec.add_dependency "sqlite3"
   spec.add_dependency "activerecord"
-  spec.add_dependency "pg"
+  spec.add_dependency "get_process_mem"
   spec.add_dependency "rails"
 
   spec.add_development_dependency "bundler", "> 1.3"
+  spec.add_development_dependency "pry"
   spec.add_development_dependency "rake"
   spec.add_development_dependency "rspec"
+
+  spec.metadata['allowed_push_host'] = 'https://rubygems.org'
+  spec.metadata['rubygems_mfa_required'] = 'true'
 end