coverband 3.0.0 → 4.0.0

data/changes.md

@@ -23,27 +23,20 @@ Will be the fully modern release that drops maintenance legacy support in favor
 - Support for file versions
   - md5 or release tags
   - add coverage timerange support
-- Drop Simplecov dependency 
-
-### Coverband 3.X
-
-Will be a stable and fast release that drops maintenance legacy support in favor of increased performance and maintainability.
-
-- expects to drop Tracepoint collection engine
-- expects to drop anything below Ruby 2.3
-- release begins to simplify ease of use
-  - drop collectors adapter
-  - reduced configuration options
+- Drop Simplecov dependency
 - improved web reporting
   - no longer relying directly on HTML in S3 but dynamically generated from any adapter
   - lists current config options
   - eventually allow updating remote config
   - full theming
-  - list redis data dump for debugging
+  - list redis data dump for debugging 
 - additional adapters: Memcache, S3, and ActiveRecord
-- add additional config / protection options on Coverage clear
-- add memory benchmarks showing memory overhead of coverband
 - add articles / podcasts like prontos readme https://github.com/prontolabs/pronto
+- Add detailed Gem usage report, if we collect and send gem usage we can give percentage of gem code used, which should help application developers know when to remove gem dependencies (0%) or perhaps inline single methods for little usage (using <= 5%) for example.
+- add additional config / protection options on Coverage clear
+- add meta data information first seen last recorded to the coverage report views (probably need to drop simplecov for that).
+   - more details in this issue: https://github.com/danmayer/coverband/issues/118 
+- Make good video on setup, install, usage 
 
 ### Coverband_jam_session
 
@@ -65,16 +58,47 @@ Feature Ideas:
 - tagging of reported Coverage
 - allow only to collect coverage based on route (limiting or scoped coverage)
 - coverage over some other variable like a set of alpha users
+- document how to use this on staging / selenium test suite runs
+   - possible add API to pull report at end of run 
 
 # Alpha
 
-### Coverband 3.0.1
-
-* ??? ;)
+### Coverband 4.0.0
 
+* Add support for Railties integration
+* Reduce configuration options
+* Default to background reporting vs middleware reporting
+* Resolves issue requiring submitting initial coverage data pre-fork
+* Simplified setup with just works sensible defaults for configuration out of the box
+* Fixes on the pilot release of background reporting in 3.0.1
+* Rake tasks automatically configured
+* Updated and simplified documentation
 
 # Released
 
+### Coverband 3.X
+
+Will be a stable and fast release that drops maintenance legacy support in favor of increased performance and maintainability.
+
+- expects to drop Tracepoint collection engine
+- drop anything below Ruby 2.3
+- release begins to simplify ease of use
+  - drop collectors adapter
+  - reduced configuration options
+- add memory benchmarks showing memory overhead of coverband
+- use full stack tests to prove no memory leaks when used in Rails 
+
+### Coverband 3.0.1
+
+* update documentation around verification steps (https://github.com/danmayer/coverband/issues/135), thanks @kbaum
+* resolve coverage drift issue, https://github.com/danmayer/coverband/issues/118, thanks for MD5 hash ideas @dnasseri and @kbaum
+* first version of background thread coverage reporting https://github.com/danmayer/coverband/pull/138, thanks @kbaum
+* auto-detection of Rack & Rails thanks @kbaum
+* improved tests allowing exceptions to raise in tests @kbaum
+* add support for both aws-sdk 1.x and 2.x thanks @jared
+* adds memory test ensuring no memory leaks
+* full stack Rails tests for Rails 4 and 5 thanks @kbaum
+
 ### Coverband 3.0.0
 
 * drops Tracepoint

data/coverband.gemspec

@@ -19,22 +19,34 @@ Gem::Specification.new do |spec|
   spec.test_files    = spec.files.grep(%r{^(test|spec|features)/})
   spec.require_paths = ['lib']
 
+  # to test support for sdk 1, uncomment this line
+  # spec.add_development_dependency 'aws-sdk', '~> 1'
+  # to test sdk 2 use this one
   spec.add_development_dependency 'aws-sdk', '~> 2'
+  spec.add_development_dependency 'benchmark-ips'
   spec.add_development_dependency 'bundler', '~> 1.3'
+  spec.add_development_dependency 'm'
+  spec.add_development_dependency 'memory_profiler'
   spec.add_development_dependency 'mocha', '~> 0.14.0'
   spec.add_development_dependency 'rack'
   spec.add_development_dependency 'rack-test'
   spec.add_development_dependency 'rake'
   spec.add_development_dependency 'test-unit'
-  spec.add_development_dependency 'redis'
-  spec.add_development_dependency 'benchmark-ips'
+
   # used for benchmarking and tests
   spec.add_development_dependency 'classifier-reborn'
   # add when debugging
   # require 'byebug'; byebug
-  spec.add_development_dependency 'byebug'
+  spec.add_development_dependency 'pry-byebug'
 
-  # todo make an optional dependency for simplecov reports
+  # TODO: Remove when other production adapters exist
+  # because the default configuration of redis store, we really do require
+  # redis now. I was reluctant to add this, but until we offer another production
+  # quality adapter, I think this is more honest about requirements and reduces confusion
+  # without this there was a race condition on calling coverband configure before redis was loaded
+  spec.add_runtime_dependency 'redis'
+  # TODO: make an optional dependency for simplecov reports
   # also likely should just require simplecov-html not the whole lib
+  # I tried this but it was harder than I thought
   spec.add_runtime_dependency 'simplecov', '> 0.11.1'
 end

data/lib/coverband/adapters/base.rb

@@ -4,36 +4,70 @@ module Coverband
   module Adapters
     class Base
       def initialize
-        raise 'abstract'
+        @file_hash_cache = {}
       end
 
       def clear!
         raise 'abstract'
       end
 
-      def save_report(_report)
-        raise 'abstract'
+      # Note: This could lead to slight race on redis
+      # where multiple processes pull the old coverage and add to it then push
+      # the Coverband 2 had the same issue,
+      # and the tradeoff has always been acceptable
+      def save_report(report)
+        data = report.dup
+        merge_reports(data, get_report)
+        save_coverage(data)
       end
 
       def coverage
-        raise 'abstract'
+        simple_report(get_report)
       end
 
       def covered_files
+        coverage.keys || []
+      end
+
+      def covered_lines_for_file(file)
+        coverage[file] || []
+      end
+
+      protected
+
+      def save_coverage
         raise 'abstract'
       end
 
-      def covered_lines_for_file(_file)
+      def get_report
         raise 'abstract'
       end
 
-      protected
+      def file_hash(file)
+        @file_hash_cache[file] ||= Digest::MD5.file(file).hexdigest
+      end
+
+      def expand_report(report)
+        report_time = Time.now.to_i
+        report.each_pair do |key, line_data|
+          extended_data = {
+            'first_updated_at' => report_time,
+            'last_updated_at' => report_time,
+            'file_hash' => file_hash(key),
+            'data' => line_data
+          }
+          report[key] = extended_data
+        end
+      end
 
       def merge_reports(new_report, old_report)
+        new_report = expand_report(new_report)
         keys = (new_report.keys + old_report.keys).uniq
         keys.each do |file|
-          new_report[file] = if new_report[file] && old_report[file]
-                               array_add(new_report[file], old_report[file])
+          new_report[file] = if new_report[file] &&
+                                old_report[file] &&
+                                new_report[file]['file_hash'] == old_report[file]['file_hash']
+                               merge_expanded_data(new_report[file], old_report[file])
                              elsif new_report[file]
                                new_report[file]
                              else
@@ -43,9 +77,24 @@ module Coverband
         new_report
       end
 
+      def merge_expanded_data(new_expanded, old_expanded)
+        {
+          'first_updated_at' => old_expanded['first_updated_at'],
+          'last_updated_at' => new_expanded['last_updated_at'],
+          'file_hash' => new_expanded['file_hash'],
+          'data' => array_add(new_expanded['data'], old_expanded['data'])
+        }
+      end
+
       def array_add(latest, original)
         latest.map.with_index { |v, i| (v && original[i]) ? v + original[i] : nil }
       end
+
+      def simple_report(report)
+        report.each_with_object({}) do |(key, extended_data), simple|
+          simple[key] = extended_data['data']
+        end
+      end
     end
   end
 end

data/lib/coverband/adapters/file_store.rb

@@ -8,9 +8,8 @@ module Coverband
     # Not recommended for production deployment
     ###
     class FileStore < Base
-      attr_accessor :path
-
       def initialize(path, _opts = {})
+        super()
         @path = path
 
         config_dir = File.dirname(@path)
@@ -21,32 +20,15 @@ module Coverband
         File.delete(path) if File.exist?(path)
       end
 
-      def save_report(report)
-        merge_reports(report, coverage)
-        save_coverage(report)
-      end
-
-      def coverage
-        existing_data(path)
-      end
-
-      def covered_files
-        report = existing_data(path)
-        existing_data(path).merge(report).keys || []
-      end
-
-      def covered_lines_for_file(file)
-        report = existing_data(path)
-        report[file] || []
-      end
-
       private
 
+      attr_accessor :path
+
       def save_coverage(report)
         File.open(path, 'w') { |f| f.write(report.to_json) }
       end
 
-      def existing_data(path)
+      def get_report
         if File.exist?(path)
           JSON.parse(File.read(path))
         else

data/lib/coverband/adapters/redis_store.rb

@@ -6,9 +6,10 @@ module Coverband
     # RedisStore store a merged coverage file to redis
     ###
     class RedisStore < Base
-      BASE_KEY = 'coverband3'
+      BASE_KEY = 'coverband3_1'
 
       def initialize(redis, opts = {})
+        super()
         @redis           = redis
         @ttl             = opts[:ttl]
         @redis_namespace = opts[:redis_namespace]
@@ -18,27 +19,6 @@ module Coverband
         @redis.del(base_key)
       end
 
-      def save_report(report)
-        # Note: This could lead to slight races
-        # where multiple processes pull the old coverage and add to it then push
-        # the Coverband 2 had the same issue,
-        # and the tradeoff has always been acceptable
-        merge_reports(report, coverage)
-        save_coverage(base_key, report)
-      end
-
-      def coverage
-        get_report(base_key)
-      end
-
-      def covered_files
-        coverage.keys
-      end
-
-      def covered_lines_for_file(file)
-        coverage[file]
-      end
-
       private
 
       attr_reader :redis
@@ -47,13 +27,13 @@ module Coverband
         @base_key ||= [BASE_KEY, @redis_namespace].compact.join('.')
       end
 
-      def save_coverage(key, data)
-        redis.set key, data.to_json
-        redis.expire(key, @ttl) if @ttl
+      def save_coverage(data)
+        redis.set base_key, data.to_json
+        redis.expire(base_key, @ttl) if @ttl
       end
 
-      def get_report(key)
-        data = redis.get key
+      def get_report
+        data = redis.get base_key
         data ? JSON.parse(data) : {}
       end
     end

data/lib/coverband/collectors/coverage.rb

@@ -17,23 +17,22 @@ module Coverband
       def reset_instance
         @project_directory = File.expand_path(Coverband.configuration.root)
         @file_line_usage = {}
-        @ignored_files = Set.new
         @ignore_patterns = Coverband.configuration.ignore + ['internal:prelude', 'schema.rb']
         @reporting_frequency = Coverband.configuration.reporting_frequency
         @store = Coverband.configuration.store
         @verbose  = Coverband.configuration.verbose
         @logger   = Coverband.configuration.logger
         @current_thread = Thread.current
+        @test_env = Coverband.configuration.test_env
+        @@previous_results = nil
         Thread.current[:coverband_instance] = nil
         self
       end
 
       def report_coverage(force_report = false)
         return if !ready_to_report? && !force_report
-        unless @store
-          @logger.debug 'no store set, no-op'
-          return
-        end
+        raise 'no Coverband store set' unless @store
+
         new_results = get_new_coverage_results
         add_filtered_files(new_results)
         @store.save_report(files_with_line_usage)
@@ -44,6 +43,7 @@ module Coverband
           @logger.error "error: #{err.inspect} #{err.message}"
           @logger.error err.backtrace
         end
+        raise err if @test_env
       end
 
       protected
@@ -58,7 +58,6 @@ module Coverband
 
       def add_filtered_files(new_results)
         new_results.each_pair do |file, line_counts|
-          next if @ignored_files.include?(file)
           next unless track_file?(file)
           add_file(file, line_counts)
         end
@@ -69,14 +68,12 @@ module Coverband
       end
 
       def get_new_coverage_results
-        coverage_results = nil
-        @semaphore.synchronize { coverage_results = new_coverage(::Coverage.peek_result.dup) }
-        coverage_results
+        @semaphore.synchronize { new_coverage(::Coverage.peek_result.dup) }
       end
 
       def files_with_line_usage
         @file_line_usage.select do |_file_name, coverage|
-          coverage.any? { |value| value && value.nonzero? }
+          coverage.any? { |value| value&.nonzero? }
         end
       end
 

data/lib/coverband/configuration.rb

@@ -2,25 +2,28 @@
 
 module Coverband
   class Configuration
-    attr_accessor :redis, :root_paths, :root,
+    attr_accessor :root_paths, :root,
                   :ignore, :additional_files, :verbose,
                   :reporter, :reporting_frequency,
-                  :disable_on_failure_for,
                   :redis_namespace, :redis_ttl,
-                  :safe_reload_files
+                  :safe_reload_files, :background_reporting_enabled,
+                  :background_reporting_sleep_seconds, :test_env
 
     attr_writer :logger, :s3_region, :s3_bucket, :s3_access_key_id, :s3_secret_access_key
 
     def initialize
       @root = Dir.pwd
       @root_paths = []
-      @ignore = []
+      @ignore = %w(vendor .erb$ .slim$)
       @additional_files = []
       @reporting_frequency = 0.0
       @verbose = false
       @reporter = 'scov'
-      @logger = Logger.new(STDOUT)
+      @logger = nil
       @store = nil
+      @background_reporting_enabled = true
+      @background_reporting_sleep_seconds = 30
+      @test_env = nil
 
       # TODO: should we push these to adapter configs
       @s3_region = nil
@@ -32,7 +35,11 @@ module Coverband
     end
 
     def logger
-      @logger ||= Logger.new(STDOUT)
+      @logger ||= if defined?(Rails)
+                    Rails.logger
+                  else
+                    Logger.new(STDOUT)
+                  end
     end
 
     def s3_bucket
@@ -52,19 +59,26 @@ module Coverband
     end
 
     def store
-      return @store if @store
-      raise 'no valid store configured'
+      @store ||= Coverband::Adapters::RedisStore.new(Redis.new(url: redis_url), redis_store_options)
     end
 
     def store=(store)
       if store.is_a?(Coverband::Adapters::Base)
         @store = store
-      elsif defined?(Redis) && redis && redis.is_a?(Redis)
-        @store = Coverband::Adapters::RedisStore.new(redis, ttl: Coverband.configuration.redis_ttl,
-                                                            redis_namespace: Coverband.configuration.redis_namespace)
-      elsif store.is_a?(String)
-        @store = Coverband::Adapters::FileStore.new(store)
+      else
+        raise 'please pass in an subclass of Coverband::Adapters for supported stores'
       end
     end
+
+    private
+
+    def redis_url
+      ENV['COVERBAND_REDIS_URL'] || ENV['REDIS_URL']
+    end
+
+    def redis_store_options
+      { ttl: Coverband.configuration.redis_ttl,
+        redis_namespace: Coverband.configuration.redis_namespace }
+    end
   end
 end

data/lib/coverband/integrations/background.rb

@@ -2,6 +2,31 @@
 
 module Coverband
   class Background
-    # TODO: stub to implement background thread recording
+    @semaphore = Mutex.new
+
+    def self.start
+      return if @background_reporting_running
+
+      logger = Coverband.configuration.logger
+      @semaphore.synchronize do
+        return if @background_reporting_running
+        logger&.debug('Coverband: Starting background reporting')
+
+        @background_reporting_running = true
+        sleep_seconds = Coverband.configuration.background_reporting_sleep_seconds
+        Thread.new do
+          loop do
+            Coverband::Collectors::Coverage.instance.report_coverage(true)
+            logger&.debug("Coverband: Reported coverage via thread. Sleeping #{sleep_seconds}s") if Coverband.configuration.verbose
+            sleep(sleep_seconds)
+          end
+        end
+
+        at_exit do
+          Coverband::Collectors::Coverage.instance.report_coverage(true)
+          logger&.debug('Coverband: Reported coverage before exit')
+        end
+      end
+    end
   end
 end

data/lib/coverband/integrations/middleware.rb

@@ -9,7 +9,11 @@ module Coverband
     def call(env)
       @app.call(env)
     ensure
-      Coverband::Collectors::Coverage.instance.report_coverage
+      if Coverband.configuration.background_reporting_enabled
+        Coverband::Background.start
+      else
+        Coverband::Collectors::Coverage.instance.report_coverage
+      end
     end
   end
 end

data/lib/coverband/integrations/rack_server_check.rb

@@ -0,0 +1,27 @@
+# frozen_string_literal: true
+
+module Coverband
+  class RackServerCheck
+    def self.running?
+      new(Kernel.caller_locations).running?
+    end
+
+    def initialize(stack)
+      @stack = stack
+    end
+
+    def running?
+      rack_server? || rails_server?
+    end
+
+    def rack_server?
+      @stack.any? { |line| line.path.include?('lib/rack/') }
+    end
+
+    def rails_server?
+      @stack.any? do |location|
+        location.path.include?('rails/commands/commands_tasks.rb') && location.label == 'server'
+      end
+    end
+  end
+end

data/lib/coverband/reporters/simple_cov_report.rb

@@ -38,13 +38,7 @@ module Coverband
           Coverband.configuration.logger.info "report is ready and viewable: open #{SimpleCov.coverage_dir}/index.html"
         end
 
-        s3_writer_options = {
-          region: Coverband.configuration.s3_region,
-          access_key_id: Coverband.configuration.s3_access_key_id,
-          secret_access_key: Coverband.configuration.s3_secret_access_key
-        }
-        Coverband::Utils::S3ReportWriter.new(Coverband.configuration.s3_bucket,
-                                                s3_writer_options).persist! if Coverband.configuration.s3_bucket
+        Coverband::Utils::S3Report.instance.persist! if Coverband.configuration.s3_bucket
       end
     end
   end

data/lib/coverband/reporters/web.rb

@@ -71,7 +71,7 @@ module Coverband
       end
 
       def show
-        html = s3.get_object(bucket: Coverband.configuration.s3_bucket, key: 'coverband/index.html').body.read
+        html = Coverband::Utils::S3Report.instance.retrieve
         # HACK: the static HTML assets to link to the path where this was mounted
         html = html.gsub("src='", "src='#{base_path}")
         html = html.gsub("href='", "href='#{base_path}")
@@ -93,7 +93,7 @@ module Coverband
       end
 
       def collect_coverage
-        Coverband::Collectors::Coverage.instance.report_coverage
+        Coverband::Collectors::Coverage.instance.report_coverage(true)
         notice = 'coverband coverage collected'
         [301, { 'Location' => "#{base_path}?notice=#{notice}" }, []]
       end
@@ -126,30 +126,12 @@ module Coverband
 
       # This method should get the root mounted endpoint
       # for example if the app is mounted like so:
-      # mount Coverband::S3Web, at: '/coverage'
+      # mount Coverband::Web, at: '/coverage'
       # "/coverage/collect_coverage?" become:
       # /coverage/
       def base_path
         request.path.match("\/.*\/") ? request.path.match("\/.*\/")[0] : '/'
       end
-
-      def s3
-        begin
-          require 'aws-sdk'
-        rescue StandardError
-          Coverband.configuration.logger.error "coverband requires 'aws-sdk' in order use S3ReportWriter."
-          return
-        end
-        @s3 ||= begin
-          client_options = {
-            region: Coverband.configuration.s3_region,
-            access_key_id: Coverband.configuration.s3_access_key_id,
-            secret_access_key: Coverband.configuration.s3_secret_access_key
-          }
-          client_options = {} if client_options.values.any?(&:nil?)
-          Aws::S3::Client.new(client_options)
-        end
-      end
     end
   end
 end

data/lib/coverband/utils/railtie.rb

@@ -0,0 +1,17 @@
+# frozen_string_literal: true
+
+module Coverband
+  class Railtie < Rails::Railtie
+    initializer 'coverband.configure' do |app|
+      app.middleware.use Coverband::Middleware
+    end
+
+    config.after_initialize do
+      Coverband::Collectors::Coverage.instance.report_coverage(true)
+    end
+
+    rake_tasks do
+      load 'coverband/utils/tasks.rb'
+    end
+  end
+end