coverband 4.1.1 → 4.2.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 0e029eb4a5ce229ca0ce9dd7ac6638bc3b9776e07a724269a495b2832228119d
-  data.tar.gz: 0f37d401fafe727024665b096e337138e53f8be139521dea9e149695b44f39ce
+  metadata.gz: 1e214b1bd6119046657daa64ed6d69f77f821c6c670b72e98d307ede5f08e61f
+  data.tar.gz: 469d9470799f90b5b52d53d3a630fae75b3b6bff0390375987902331878f530e
 SHA512:
-  metadata.gz: c0d51d0263c37369abffc5839a367cdebd5fad7d8132362044163614fca33e2bc8617770bb36d3dd268eb34a4f0243e5fe4a1fad516f724a9da122aa5739a8be
-  data.tar.gz: db40dfd936b39d159e7d92b470b186761cfea6ce944e55ff8079d70279af8c84a3602c5c873317c58d76b08aa0a654e653eb10df8a608ba646e4bd37b006105b
+  metadata.gz: f1b0f503941ab9dbc2dee6a184fe6d6d937a5f505a5c9112ae7343197372586d6cb42df758f24effde0ee30fb359c0e2e651c64a1f5ee4c9a588a4499683dc73
+  data.tar.gz: ee700abfe4c730011dcc5c1c8521f55a2cc7ec5149e44b08187c8d09419023160a0f7c8aaba4b2aa0bb405eb13f4aeee1d8ce9cabce4860738d4ea94cac63c04

data/.gitignore

@@ -5,6 +5,8 @@
 .yardoc
 .idea/
 .ruby-version
+assets/.DS_Store
+.DS_Store
 Gemfile*lock
 InstalledFiles
 _yardoc

data/README.md

@@ -1,3 +1,5 @@
+<img src="https://raw.github.com/danmayer/coverband/master/docs/assets/logo/heads.svg?sanitize=true" width='300'>
+
 # Coverband
 
 [![Build Status](https://travis-ci.org/danmayer/coverband.svg?branch=master)](https://travis-ci.org/danmayer/coverband)
@@ -15,24 +17,24 @@
 
 A gem to measure production code usage, showing a counter for the number of times each line of code that is executed. Coverband allows easy configuration to collect and report on production code usage. It reports in the background via a thread or can be used as Rack middleware, or manually configured to meet any need.
 
-__Note:__ Coverband is not intended for test code coverage, for that we recommended using [SimpleCov](https://github.com/colszowka/simplecov).
+**Note:** Coverband is not intended for test code coverage, for that we recommended using [SimpleCov](https://github.com/colszowka/simplecov).
 
 ## Key Features
 
 The primary goal of Coverband is giving deep insight into your production runtime usage of your application code, while having the least impact on performance possible.
 
-* Low performance overhead
-* Very simple setup and configuration
-* Out of the box support for all standard code execution paths (web, cron, background jobs, rake tasks, etc)
-* Easy to understand actionable insights from the report
-* Development mode, offers deep insight of code usage details (number of LOC execution during single request, etc) during development.
-* Mountable web interface to easily share reports
+- Low performance overhead
+- Very simple setup and configuration
+- Out of the box support for all standard code execution paths (web, cron, background jobs, rake tasks, etc)
+- Easy to understand actionable insights from the report
+- Development mode, offers deep insight of code usage details (number of LOC execution during single request, etc) during development.
+- Mountable web interface to easily share reports
 
 # Installation
 
 ## Redis
 
-Coverband stores coverage data in Redis. The Redis endpoint is looked for in this order: 
+Coverband stores coverage data in Redis. The Redis endpoint is looked for in this order:
 
 1. `ENV['COVERBAND_REDIS_URL']`
 2. `ENV['REDIS_URL']`
@@ -54,7 +56,6 @@ If [tracking gem usage](#collecting-gem--library-usage), be sure to include cove
 
 The Railtie integration means you shouldn't need to do anything anything else other than ensure coverband is required after rails within your Gemfile.
 
-
 ## Sinatra
 
 For the best coverage you want this loaded as early as possible. I have been putting it directly in my `config.ru` but you could use an initializer, though you may end up missing some boot up coverage. To start collection require Coverband as early as possible.
@@ -77,7 +78,7 @@ Rails.application.routes.draw do
 end
 ```
 
-But don't forget to *protect your source code with proper authentication*. Something like this when using devise:
+But don't forget to _protect your source code with proper authentication_. Something like this when using devise:
 
 ```ruby
 Rails.application.routes.draw do
@@ -92,11 +93,12 @@ end
 The web endpoint is a barebones endpoint that you can either expose direct (after authentication) or you can just link to the actions you wish to expose. The index is intended as a example to showcase all the features.
 
 ![image](https://raw.github.com/danmayer/coverband/master/docs/coverband_web_update.png)
+
 > The web index as available on the Coverband Demo site
 
-* __force coverage collection:__ This triggers coverage collection on the current webserver process
-* __reload Coverband files:__ This has Coverband reload files as configured (force reload of some files that might not capture Coverage on boot). This can be used to reload files on demand.
-* __clear coverage report:__ This will clear the coverage data. This wipes out all collected data (__dangerous__)
+- **force coverage collection:** This triggers coverage collection on the current webserver process
+- **reload Coverband files:** This has Coverband reload files as configured (force reload of some files that might not capture Coverage on boot). This can be used to reload files on demand.
+- **clear coverage report:** This will clear the coverage data. This wipes out all collected data (**dangerous**)
 
 ### Rake Tasks
 
@@ -118,17 +120,16 @@ Details on an example Sinatra app
 
 # Verify Correct Installation
 
-* boot up your application
-* run app and hit a controller (via a web request, at least one request must complete)
-* run `rake coverband:coverage` this will show app initialization coverage
-* make another request, or enough that your reporting frequency will trigger
-* run `rake coverband:coverage` and you should see coverage increasing for the endpoints you hit.
+- boot up your application
+- run app and hit a controller (via a web request, at least one request must complete)
+- run `rake coverband:coverage` this will show app initialization coverage
+- make another request, or enough that your reporting frequency will trigger
+- run `rake coverband:coverage` and you should see coverage increasing for the endpoints you hit.
 
 # Coverband Demo
 
 Take Coverband for a spin on the live Heroku deployed [Coverband Demo](https://coverband-demo.herokuapp.com/). The [full source code for the demo](https://github.com/danmayer/coverband_demo) is available to help with installation, configuration, and understanding of basic usage.
 
-
 ### Example apps
 
 - [Rails 5.2.x App](https://github.com/danmayer/coverband_demo)
@@ -137,14 +138,12 @@ Take Coverband for a spin on the live Heroku deployed [Coverband Demo](https://c
 
 # Advanced Config
 
-
 If you need to configure coverband, this can be done by creating a `config/coverband.rb` file relative to your project root.
 
-* See [lib/coverband/configuration.rb](https://github.com/danmayer/coverband/blob/master/lib/coverband/configuration.rb) for all options
-* By default Coverband will try to stored data to Redis
-	* Redis endpoint is looked for in this order: `ENV['COVERBAND_REDIS_URL']`, `ENV['REDIS_URL']`, or `localhost`
+- See [lib/coverband/configuration.rb](https://github.com/danmayer/coverband/blob/master/lib/coverband/configuration.rb) for all options
+- By default Coverband will try to stored data to Redis \* Redis endpoint is looked for in this order: `ENV['COVERBAND_REDIS_URL']`, `ENV['REDIS_URL']`, or `localhost`
 
- Below is an example config file for a Rails 5 app:
+Below is an example config file for a Rails 5 app:
 
 ```ruby
 #config/coverband.rb
@@ -170,14 +169,12 @@ end
 Sometimes you have files that are known to be valuable perhaps in other environments or something that is just run very infrequently. Opposed to having to mentally filter them out of the report, you can just have them ignored in the Coverband reporting by using `config.ignore` as shown below. Ignore takes a string but can also match with regex rules see how below ignores all rake tasks as an example.
 
 ```
-config.ignore =	['config/application.rb',
+config.ignore +=  ['config/application.rb',
                    'config/boot.rb',
                    'config/puma.rb',
                    'config/schedule.rb',
-                   'config/environments/test.rb',
-                   'config/environments/development.rb',
-                   'config/environments/staging.rb',
-                   'config/environments/production.rb',
+                   'bin/*'
+                   'config/environments/*',
                    'lib/tasks/*']
 ```
 
@@ -208,8 +205,7 @@ Between the release of 4.0 and 4.1 our data format changed. This resets all your
 
 `rake coverband:migrate`
 
-* We will be working to support migrations going forward, when possible
-
+- We will be working to support migrations going forward, when possible
 
 ### Clear Coverage
 
@@ -250,8 +246,8 @@ Coverband.configure do |config|
   config.safe_reload_files = ['config/coverband.rb']
 end
 ```
-By adding any files above you will get reporting on those files as part of your coverage runtime report. The files are reloaded when Coverband first starts, you can also trigger a reload via the web interface.
 
+By adding any files above you will get reporting on those files as part of your coverage runtime report. The files are reloaded when Coverband first starts, you can also trigger a reload via the web interface.
 
 ### Collecting Gem / Library Usage
 
@@ -278,6 +274,17 @@ end
 
 This flag exposes line by line usage of gem files. Unfortunately due to the way the coverband report is currently rendered, enabling `gem_details` slows down viewing of the coverage report in the browser and is not yet recommended.
 
+### Manually Starting Coverband
+
+Coverband starts on require of the the library which is usually done within the Gemfile. This can be disabled by setting the `COVERBAND_DISABLE_AUTO_START` environment variable. This environment variable can be useful to toggle coverband on and off in certain environments.
+
+In order to start coverband manually yourself when this flag is enabled, call `Coverband.configure` followed by `Coverband.start`.
+
+```ruby
+Coverband.configure
+Coverband.start
+```
+
 ### Verbose Debug / Development Mode
 
 Note: To debug issues getting Coverband working. I recommend running in development mode, by turning verbose logging on `config.verbose = true` and passing in the Rails.logger `config.logger = Rails.logger` to the Coverband config. This makes it easy to follow in development mode. Be careful to not leave these on in production as they will affect performance.
@@ -304,8 +311,8 @@ If you are trying to debug locally wondering what code is being run during a req
 
 # Prerequisites
 
-* Coverband 3.0.X+ requires Ruby 2.3+
-* Coverband currently requires Redis for production usage
+- Coverband 3.0.X+ requires Ruby 2.3+
+- Coverband currently requires Redis for production usage
 
 # Contributing To Coverband
 
@@ -322,18 +329,18 @@ If you are working on adding features, PRs, or bugfixes to Coverband this sectio
 
 If you submit a change please make sure the tests and benchmarks are passing.
 
-* run tests: 
-   * `bundle exec rake` 
-   * `BUNDLE_GEMFILE=Gemfile.rails4 bundle exec rake` (Same tests using rails 4 instead of 5)
-* view test coverage: `open coverage/index.html`
-* run the benchmarks before and after your change to see impact
-   * `rake benchmarks` 
+- run tests:
+  - `bundle exec rake`
+  - `BUNDLE_GEMFILE=Gemfile.rails4 bundle exec rake` (Same tests using rails 4 instead of 5)
+- view test coverage: `open coverage/index.html`
+- run the benchmarks before and after your change to see impact
+  - `rake benchmarks`
 
 ### Known Issues
 
-* __total fail__ on front end code, because of the precompiled template step basically coverage doesn't work well for `erb`, `slim`, and the like.
-  * related it will try to report something, but the line numbers reported for `ERB` files are often off and aren't considered useful. I recommend filtering out .erb using the `config.ignore` option. The default configuration excludes these files
-* coverage doesn't show for Rails `config/application.rb` or `config/boot.rb` as they get loaded when loading the Rake environment prior to starting the `Coverage` library. See [reload files section](#forcing-coverband-to-track-coverage-on-files-loaded-during-boot-safe_reload_files).
+- **total fail** on front end code, because of the precompiled template step basically coverage doesn't work well for `erb`, `slim`, and the like.
+  - related it will try to report something, but the line numbers reported for `ERB` files are often off and aren't considered useful. I recommend filtering out .erb using the `config.ignore` option. The default configuration excludes these files
+- coverage doesn't show for Rails `config/application.rb` or `config/boot.rb` as they get loaded when loading the Rake environment prior to starting the `Coverage` library. See [reload files section](#forcing-coverband-to-track-coverage-on-files-loaded-during-boot-safe_reload_files).
 
 ### Debugging Redis Store
 

data/changes.md

@@ -37,6 +37,16 @@ Will be the fully modern release that drops maintenance legacy support in favor
 - See if we can add support for views / templates
   - using this technique https://github.com/ioquatix/covered
 - Better default grouping (could use groups features for gems for rails controllers, models, lib, etc)
+- Improved logging for easier debugging and development
+  - drop the verbose mode and better support standard logger levels
+- Possibly setup a build assets system
+  - my JS rules expanded the compressed JS at the top of application.js, basically we want to stitch together JS
+  - I guess we could also load multiple JS files as most of the JS is just default compressed JS and a tiny amount of actual app JS.
+- lazy load for Coverband results
+- view layer file coverage
+- move all code to work with relative paths leaving only stdlib Coverage working on full paths
+- add gem_safe_lists to track only some gems
+- add gem_details_safe list to report on details on some gems
 
 ### Coverband_jam_session
 
@@ -63,14 +73,31 @@ Feature Ideas:
 
 # Alpha
 
-### Coverband 4.2.0.rc
+### Coverband 4.3.0?
 
-- loadtime vs runtime coverage
-- view coverage
-- perf speedup on gem details views
+- further improvements on eager_loading detection
+- ?
 
 # Released
 
+### Coverband 4.2.0
+
+**NOTE:** This release introduces load time and runtime Coverage. To get the full benifit of this feature you need to reset you coverage data (`rake coverband:clear` or clear via the web UI). Until you clear the combined result is still correct, but your runtime data will be mixed with previous data. This is due to an additive change in how we store coverage. We didn't reset by default as the coverage history for some folks may be more important than runtime breakdown.
+
+- loadtime vs runtime coverage
+  - This fixes the coverage last seen date to exclude just load time data
+  - now you can see boot time coverage vs code hit during production execution
+  - list view shows runtime percentage and runtime hits
+  - file view shows line hits load, runtime, and combined
+- perf speedup on gem details views
+- ajax load code views for more responsive pages on large projects
+- fix for issues with COVERBAND_DISABLE_AUTO_START
+- silence warnings
+- move from thread based reporter instance to process based instance
+- thanks Karl Baum for much of the above mentioned work
+- clear coverage on individual files
+- we have a logo, thanks Dave Woodall
+
 ### Coverband 4.1.1
 
 - fix bug on info page when using namespaces

data/lib/coverband/adapters/base.rb

@@ -4,6 +4,7 @@ module Coverband
   module Adapters
     class Base
       include Coverband::Utils::FilePathHelper
+      attr_accessor :type
 
       def initialize
         @file_hash_cache = {}
@@ -13,6 +14,10 @@ module Coverband
         raise 'abstract'
       end
 
+      def clear_file!(_file)
+        raise 'abstract'
+      end
+
       def migrate!
         raise 'abstract'
       end
@@ -39,6 +44,11 @@ module Coverband
         get_report
       end
 
+      def get_coverage_report
+        data = Coverband.configuration.store.split_coverage(Coverband::TYPES)
+        data.merge(Coverband::MERGED_TYPE => Coverband.configuration.store.merged_coverage(Coverband::TYPES))
+      end
+
       def covered_files
         coverage.keys || []
       end
@@ -50,6 +60,18 @@ module Coverband
 
       protected
 
+      def split_coverage(types)
+        types.reduce({}) do |data, type|
+          data.update(type => get_report(type))
+        end
+      end
+
+      def merged_coverage(types)
+        types.reduce({}) do |data, type|
+          merge_reports(data, get_report(type), skip_expansion: true)
+        end
+      end
+
       def save_coverage
         raise 'abstract'
       end
@@ -65,10 +87,11 @@ module Coverband
       def expand_report(report)
         expanded = {}
         report_time = Time.now.to_i
+        updated_time = self.type == Coverband::EAGER_TYPE ? nil : report_time
         report.each_pair do |key, line_data|
           extended_data = {
             'first_updated_at' => report_time,
-            'last_updated_at' => report_time,
+            'last_updated_at' => updated_time,
             'file_hash' => file_hash(key),
             'data' => line_data
           }

data/lib/coverband/adapters/redis_store.rb

@@ -22,7 +22,17 @@ module Coverband
       end
 
       def clear!
-        @redis.del(base_key)
+        Coverband::TYPES.each do |type|
+          @redis.del(type_base_key(type))
+        end
+      end
+
+      def clear_file!(filename)
+        Coverband::TYPES.each do |type|
+          data = get_report(type)
+          data.delete(filename)
+          save_coverage(data, type)
+        end
       end
 
       def size
@@ -51,6 +61,11 @@ module Coverband
         save_coverage(merge_reports(get_report, relative_path_report, skip_expansion: true))
       end
 
+      def type=(type)
+        super
+        reset_base_key
+      end
+
       private
 
       attr_reader :redis
@@ -60,16 +75,22 @@ module Coverband
       end
 
       def base_key
-        @base_key ||= [@format_version, @redis_namespace].compact.join('.')
+        @base_key ||= [@format_version, @redis_namespace, type].compact.join('.')
+      end
+
+      def type_base_key(local_type)
+        [@format_version, @redis_namespace, local_type].compact.join('.')
       end
 
-      def save_coverage(data)
-        redis.set base_key, data.to_json
-        redis.expire(base_key, @ttl) if @ttl
+      def save_coverage(data, local_type = nil)
+        local_type ||= type
+        redis.set type_base_key(local_type), data.to_json
+        redis.expire(type_base_key(local_type), @ttl) if @ttl
       end
 
-      def get_report
-        data = redis.get base_key
+      def get_report(local_type = nil)
+        local_type ||= type
+        data = redis.get type_base_key(local_type)
         data ? JSON.parse(data) : {}
       end
     end

data/lib/coverband/at_exit.rb

@@ -4,15 +4,32 @@ module Coverband
   class AtExit
     @semaphore = Mutex.new
 
+    @at_exit_registered = nil
     def self.register
+      return if ENV['COVERBAND_DISABLE_AT_EXIT']
       return if @at_exit_registered
+
       @semaphore.synchronize do
         return if @at_exit_registered
+
         @at_exit_registered = true
         at_exit do
           ::Coverband::Background.stop
-          Coverband.report_coverage(true)
-          Coverband.configuration.logger&.debug('Coverband: Reported coverage before exit')
+
+          #####
+          # TODO: This is is brittle and not a great solution to avoid deploy time
+          # actions polluting the 'runtime' metrics
+          #
+          # * should we skip /bin/rails webpacker:compile ?
+          # * Perhaps detect heroku deployment ENV var opposed to tasks?
+          #####
+          default_heroku_tasks = ['assets:clean', 'assets:precompile']
+          if defined?(Rake) && Rake.respond_to?(:application) && (Rake&.application&.top_level_tasks || []).any? { |task| default_heroku_tasks.include?(task) }
+            # skip reporting
+          else
+            Coverband.report_coverage(true)
+            #Coverband.configuration.logger&.debug('Coverband: Reported coverage before exit')
+          end
         end
       end
     end

data/lib/coverband/collectors/coverage.rb

@@ -1,48 +1,60 @@
 # frozen_string_literal: true
+require 'singleton'
+require_relative 'delta'
 
 module Coverband
   module Collectors
     ###
-    # TODO: look at alternatives to semaphore
-    # StandardError seems line be better option
+    # StandardError seems like be better option
     # coverband previously had RuntimeError here
     # but runtime error can let a large number of error crash this method
     # and this method is currently in a ensure block in middleware and threads
     ###
     class Coverage
-      def self.instance
-        Thread.current[:coverband_instance] ||= Coverband::Collectors::Coverage.new
-      end
+      include Singleton
 
       def reset_instance
         @project_directory = File.expand_path(Coverband.configuration.root)
-        @file_line_usage = {}
         @ignore_patterns = Coverband.configuration.ignore + ['internal:prelude', 'schema.rb']
         @reporting_frequency = Coverband.configuration.reporting_frequency
         @store = Coverband.configuration.store
         @verbose  = Coverband.configuration.verbose
         @logger   = Coverband.configuration.logger
-        @current_thread = Thread.current
         @test_env = Coverband.configuration.test_env
         @track_gems = Coverband.configuration.track_gems
-        @@previous_results = nil
-        Thread.current[:coverband_instance] = nil
+        Delta.reset
         self
       end
 
+      def runtime!
+        @store.type = nil
+      end
+
+      def eager_loading!
+        @store.type = Coverband::EAGER_TYPE
+      end
+
       def report_coverage(force_report = false)
         return if !ready_to_report? && !force_report
         raise 'no Coverband store set' unless @store
 
-        new_results = get_new_coverage_results
-        add_filtered_files(new_results)
-        @store.save_report(files_with_line_usage)
-        @file_line_usage.clear
+        original_previous_set = Delta.previous_results
+        files_with_line_usage = filtered_files(Delta.results)
+
+        ###
+        # Hack to prevent processes and threads from reporting first Coverage hit
+        # when we are in runtime collection mode, which do not have a cache of previous
+        # coverage to remove the initial stdlib Coverage loading data
+        ###
+        if ((original_previous_set.nil? && @store.type == Coverband::EAGER_TYPE) ||
+           (original_previous_set && @store.type != Coverband::EAGER_TYPE))
+          @store.save_report(files_with_line_usage)
+        end
       rescue StandardError => err
         if @verbose
-          @logger.error 'coverage failed to store'
-          @logger.error "error: #{err.inspect} #{err.message}"
-          @logger.error err.backtrace
+          @logger&.error 'coverage failed to store'
+          @logger&.error "error: #{err.inspect} #{err.message}"
+          @logger&.error err.backtrace
         end
         raise err if @test_env
       end
@@ -59,81 +71,27 @@ module Coverband
         @ignore_patterns.none? do |pattern|
           file.include?(pattern)
         end && (file.start_with?(@project_directory) ||
-         (@track_gems &&
-          Coverband.configuration.gem_paths.any? { |path| file.start_with?(path) }))
+                (@track_gems &&
+                 Coverband.configuration.gem_paths.any? { |path| file.start_with?(path) }))
       end
 
       private
 
-      def add_filtered_files(new_results)
-        new_results.each_pair do |file, line_counts|
-          next unless track_file?(file)
-          add_file(file, line_counts)
-        end
+      def filtered_files(new_results)
+        new_results.each_with_object({}) do |(file, line_counts), file_line_usage|
+          file_line_usage[file] = line_counts if track_file?(file)
+        end.select { |_file_name, coverage| coverage.any? { |value| value&.nonzero? } }
       end
 
       def ready_to_report?
         (rand * 100.0) >= (100.0 - @reporting_frequency)
       end
 
-      def get_new_coverage_results
-        @semaphore.synchronize { new_coverage(::Coverage.peek_result.dup) }
-      end
-
-      def files_with_line_usage
-        @file_line_usage.select do |_file_name, coverage|
-          coverage.any? { |value| value&.nonzero? }
-        end
-      end
-
-      def array_diff(latest, original)
-        latest.map.with_index do |v, i|
-          if (v && original[i])
-            [0, v - original[i]].max
-          else
-            nil
-          end
-        end
-      end
-
-      def previous_results
-        @@previous_results
-      end
-
-      def add_previous_results(val)
-        @@previous_results = val
-      end
-
-      def new_coverage(current_coverage)
-        if previous_results
-          new_results = {}
-          current_coverage.each_pair do |file, line_counts|
-            if previous_results[file]
-              new_results[file] = array_diff(line_counts, previous_results[file])
-            else
-              new_results[file] = line_counts
-            end
-          end
-        else
-          new_results = current_coverage
-        end
-
-        add_previous_results(current_coverage)
-        new_results.dup
-      end
-
-      def add_file(file, line_counts)
-        @file_line_usage[file] = line_counts
-      end
-
       def initialize
         if Gem::Version.new(RUBY_VERSION) < Gem::Version.new('2.3.0')
           raise NotImplementedError, 'not supported until Ruby 2.3.0 and later'
         end
-        unless defined?(::Coverage)
-          # puts 'loading coverage'
-          require 'coverage'
-        end
+        require 'coverage'
         if Gem::Version.new(RUBY_VERSION) >= Gem::Version.new('2.5.0')
           ::Coverage.start unless ::Coverage.running?
         else
@@ -144,8 +102,6 @@ module Coverband
             load safe_file
           end
         end
-        @semaphore = Mutex.new
-        @@previous_results = nil
         reset_instance
       end
     end