coverband 1.3.1 → 1.5.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA1:
-  metadata.gz: e98fd5cb692315012eaf942729c5dda8733dc1f2
-  data.tar.gz: f5aa2c9ed5c11d804a5849f17044c557545adb02
+  metadata.gz: 4e4f7c810900bee83c3516d394e950fde6cb5186
+  data.tar.gz: ba60c4f523b5d50e145cf179113dd6431fd6be4c
 SHA512:
-  metadata.gz: 520f5f7d174cbb4839b8d2aa8a763c5810efcc5102d32e0891ca1923a7cdf2a87183d6267cd24576159001f1829b7ac8eed4eddbc1d3f2eb196285d78adc80ea
-  data.tar.gz: 1398b2283ba65698d8ce5040b69742e365c3cf9c26a52300322baff70fad708e65152531487dec576ec4a8030fc6b1e1542645f21bb3c32f1d868d3eb16431d2
+  metadata.gz: ca67d3de1ce546e6d5d3050bde171f70ef258b0792a2e871d8d4c4b93c5609f71e85afcb7d39b9a08d582381e6476fe13c791c45d764726eccaada3170dfd25a
+  data.tar.gz: a1543e402bcc3ef6a0c68c478235d34d1794f695ea9c3099eb31f7c4b54b56015622f1fb920804e0c20614118516fabcf12364e5c97a220108e820e53c29683b

data/.gitignore

@@ -3,6 +3,7 @@
 .bundle
 .config
 .yardoc
+.idea/
 Gemfile.lock
 InstalledFiles
 _yardoc

data/README.md

@@ -13,12 +13,31 @@ A gem to measure production code usage, showing each line of code that is execut
 __Notes:__ Latest versions of Coverband drop support for anything less than Ruby 2.0, and Ruby 2.1+ is recommended.
 
 ###### Success:
+
 After running in production for 30 minutes, we were able very easily delete 2000 LOC after looking through the data. We expect to be able to clean up much more after it has collected more data.
 
 ###### Performance Impact
 
 The performance impact on Ruby 2.1+ is fairly small and no longer requires a C-extension. Look at the benchmark rake task for specific details.
 
+## How I use Coverband
+
+* install coverband.
+* take baseline: `rake coverband:baseline`
+* validate baseline with  `rake coverband:coverage`
+* test setup in development (hit endpoints and generate new report)
+* deploy to staging and verify functionality
+* deploy to production and verify functionality
+* every 2 weeks or with major releases
+   * clear old coverage: `rake coverband:clear`
+   * take new baseline: `rake coverband:baseline`
+   * deploy and verify coverage is matching expectations
+* __COVERAGE DRIFT__
+   * if you never clear you have lines of code drift from when they were recorded
+   * if you clear on every deploy you don't capture as useful of data
+   * there is a tradeoff on accuracy and data value
+   * I recommend clearing after major code changes, significant releases, or some regular schedule.
+
 ## Installation
 
 Add this line to your application's Gemfile:
@@ -82,10 +101,16 @@ You need to configure cover band you can either do that passing in all configura
 #config/coverband.rb
 Coverband.configure do |config|
   config.root              = Dir.pwd
+  
   if defined? Redis
     config.redis           = Redis.new(:host => 'redis.host.com', :port => 49182, :db => 1)
   end
-  config.coverage_baseline = Coverband.parse_baseline
+  # don't want to use redis, store to file system ;)
+  # config.coverage_file           = './tmp/coverband_coverage.json'
+  
+  # DEPRECATED now will use redis or file store
+  # config.coverage_baseline = Coverband.parse_baseline
+  
   config.root_paths        = ['/app/'] # /app/ is needed for heroku deployments
   # regex paths can help if you are seeing files duplicated for each capistrano deployment release
   #config.root_paths       = ['/server/apps/my_app/releases/\d+/']
@@ -103,6 +128,7 @@ Coverband.configure do |config|
   if defined? Statsd
     config.stats           = Statsd.new('statsd.host.com', 8125)
   end
+  
   # config options false, true, or 'debug'. Always use false in production
   # true and debug can give helpful and interesting code usage information
   # they both increase the performance overhead of the gem a little.
@@ -260,7 +286,8 @@ coverband.sample {
 
 ### Clearing Line Coverage Data
 
-After a deploy where code has changed.
+After a deploy where code has changed significantly.
+
 The line numbers previously recorded in Redis may no longer match the current state of the files.
 If being slightly out of sync isn't as important as gathering data over a long period,
 you can live with minor inconsistency for some files.
@@ -300,6 +327,8 @@ If you are trying to debug locally wondering what code is being run during a req
 
 If you are clearing data on every deploy. You might want to write the data out to a file first. Then you could merge the data into the final results later.
 
+__note:__ I don't actually recommend clearing on every deploy, but only following significant releases where many line numbers would be off. If you follow that approach you don't need to merge data over time as this example shows how.
+
 ```ruby
 data = JSON.generate Coverband::Reporter.get_current_scov_data
 File.write("blah.json", data)
@@ -321,20 +350,24 @@ If you are working on adding features, PRs, or bugfixes to Coverband this sectio
 
 ### Known issues
 
+* __total fail__ on front end code, because of the precompiled template step basically coverage doesn't work well for `erb`, `slim`, and the like.
 * If you don't have a baseline recorded your coverage can look odd like you are missing a bunch of data. It would be good if Coverband gave a more actionable warning in this situation.
 * If you have SimpleCov filters, you need to clear them prior to generating your coverage report. As the filters will be applied to Coverband as well and can often filter out everything we are recording.
 * the line numbers reported for `ERB` files are often off and aren't considered useful. I recommend filtering out .erb using the `config.ignore` option.
+* coverage doesn't show for Rails `config/application.rb` or `config/boot.rb` as they get loaded when loading the Rake environment prior to starting to record the baseline..
 
 ## TODO
 
+* graphite adapters (it would allow passing in date ranges on usage)
+* perf test for array vs hash
+* redis pipeline around hash (or batch get then push)
+* pass in namespace to redis (coverage vs baseline)
+* what about having baseline a onetime recording into redis no merge later
+* move to SimpleCov console out, or make similar console tabular output
 * Fix network performance by logging to files that purge later (like NR) (far more time lost in TracePoint than sending files, hence not a high priority, but would be cool)
 * Add support for [zadd](http://redis.io/topics/data-types-intro) so one could determine single call versus multiple calls on a line, letting us determine the most executed code in production.
 * Possibly add ability to record code run for a given route
 * Improve client code api, around manual usage of sampling (like event usage)
-* Provide a better lighter example app, to show how to use Coverband.
-  * blank rails app
-  * blank Sinatra app
-* report on Coverband files that haven't recorded any coverage (find things like events and crons that aren't recording, or dead files)
 * ability to change the Coverband config at runtime by changing the config pushed to the Redis hash. In memory cache around the changes to only make that call periodically.
 * Opposed to just showing code usage on a route allow 'tagging' events which would record line coverage for that tag (this would allow tagging all code that modified an ActiveRecord model for example
 * mountable rack app to view coverage similar to flipper-ui

data/changes.md

@@ -0,0 +1,23 @@
+### 1.5.0
+
+This is a major release with significant refactoring a stepping stone for a 2.0 release.
+
+* staging a changes.md document!
+* refactored out full abstraction for stores
+* supports hit counts vs binary covered / not covered for lines
+  * this will let you find density of code usage just not if it was used
+  * this is a slight performance hit, so you can fall back to the old system if you want `redisstore.new(@redis, array: true)`
+  * this is the primary new feature in 1.5.0
+* Redis has configurable base name, so I can safely change storage formats between releases
+* improved documentation
+* supports `SimpleCov.root`
+* show files that were never touched
+* apply coverband filters to ignore files in report not just collection
+* improved test coverage
+* improved benchmarks including support for multiple stores ;)
+
+### 1.3.1
+
+* This was a small fix release addressing some issues
+* mostly readme updates
+* last release prior to having a changes document!

data/coverband.gemspec

@@ -26,8 +26,9 @@ Gem::Specification.new do |spec|
   spec.add_development_dependency "test-unit"
   spec.add_development_dependency 'sinatra'
   spec.add_development_dependency 'classifier-reborn'
+  spec.add_development_dependency 'aws-sdk', '~> 2'
   spec.add_runtime_dependency "simplecov"
   spec.add_runtime_dependency "json"
+  # TODO make redis optional dependancy as we add additional adapters
   spec.add_runtime_dependency "redis"
-  spec.add_runtime_dependency 'aws-sdk', '~> 2'
 end

data/lib/coverband.rb

@@ -1,30 +1,31 @@
-require 'redis'
 require 'logger'
-require 'aws-sdk'
+require 'json'
+# todo move to only be request if using redis store
+require 'redis'
 
 require 'coverband/version'
 require 'coverband/configuration'
-require 'coverband/redis_store'
-require 'coverband/memory_cache_store'
+require 'coverband/adapters/redis_store'
+require 'coverband/adapters/memory_cache_store'
+require 'coverband/adapters/file_store'
 require 'coverband/base'
-require 'coverband/reporter'
+require 'coverband/baseline'
+require 'coverband/reporters/base'
+require 'coverband/reporters/simple_cov_report'
+require 'coverband/reporters/console_report'
 require 'coverband/middleware'
 require 'coverband/s3_report_writer'
 
 module Coverband
-
   CONFIG_FILE = './config/coverband.rb'
 
   class << self
     attr_accessor :configuration_data
   end
 
+  # this method is left for backwards compatibility with existing configs
   def self.parse_baseline(baseline_file = './tmp/coverband_baseline.json')
-    baseline = if File.exist?(baseline_file)
-      JSON.parse(File.read(baseline_file))
-    else
-      {}
-    end
+    Coverband::Baseline.parse_baseline(baseline_file)
   end
 
   def self.configure(file = nil)

data/lib/coverband/adapters/file_store.rb

@@ -0,0 +1,59 @@
+module Coverband
+  module Adapters
+    class FileStore
+      attr_accessor :path
+
+      def initialize(path, opts = {})
+        @path = path
+
+        config_dir = File.dirname(@path)
+        Dir.mkdir config_dir unless File.exist?(config_dir)
+      end
+
+      def clear!
+        if File.exist?(path)
+          File.delete(path)
+        end
+      end
+
+      def save_report(report)
+        results = existing_data(path)
+        report.each_pair do |file, values|
+          if results.has_key?(file)
+            # convert the keys to "3" opposed to 3
+            values = JSON.parse(values.to_json)
+            results[file].merge!( values ){|k, old_v, new_v| old_v.to_i + new_v.to_i}
+          else
+            results[file] = values
+          end
+        end
+        File.open(path, 'w') { |f| f.write(results.to_json) }
+      end
+
+      def coverage
+        existing_data(path)
+      end
+
+      def covered_files
+        report = existing_data(path)
+        existing_data(path).merge(report).keys || []
+      end
+
+      def covered_lines_for_file(file)
+        report = existing_data(path)
+        report[file] || []
+      end
+
+      private
+
+      def existing_data(path)
+        if File.exist?(path)
+          JSON.parse(File.read(path))
+        else
+          {}
+        end
+      end
+
+    end
+  end
+end

data/lib/coverband/adapters/memory_cache_store.rb

@@ -0,0 +1,46 @@
+module Coverband
+  module Adapters
+    class MemoryCacheStore
+      attr_accessor :store
+
+      def initialize(store)
+        @store = store
+      end
+
+      def self.reset!
+        files_cache.clear
+      end
+
+      def clear!
+        self.class.reset!
+      end
+
+      def save_report(files)
+        filtered_files = filter(files)
+        store.save_report(filtered_files) if filtered_files.any?
+      end
+
+      private
+
+      def self.files_cache
+        @files_cache ||= Hash.new
+      end
+
+      def files_cache
+        self.class.files_cache
+      end
+
+      def filter(files)
+        files.each_with_object(Hash.new) do |(file, lines), filtered_file_hash|
+          #first time we see a file, we pre-init the in memory cache to whatever is in store(redis)
+          line_cache = files_cache[file] ||= Set.new(store.covered_lines_for_file(file))
+          lines.reject! do |line|
+            line_cache.include?(line) ? true : (line_cache << line and false)
+          end
+          filtered_file_hash[file] = lines if lines.any?
+        end
+      end
+
+    end
+  end
+end

data/lib/coverband/adapters/redis_store.rb

@@ -0,0 +1,101 @@
+module Coverband
+  module Adapters
+    class RedisStore
+      BASE_KEY = 'coverband1'
+
+      def initialize(redis, opts = {})
+        @redis = redis
+        #remove check for coverband 2.0
+        @_sadd_supports_array = recent_gem_version? && recent_server_version?
+        #possibly drop array storage for 2.0
+        @store_as_array = opts.fetch(:array){ false }
+      end
+
+      def clear!
+        @redis.smembers(BASE_KEY).each { |key| @redis.del("#{BASE_KEY}.#{key}") }
+        @redis.del(BASE_KEY)
+      end
+
+      def save_report(report)
+        if @store_as_array
+          redis.pipelined do
+            store_array(BASE_KEY, report.keys)
+
+            report.each do |file, lines|
+              store_array("#{BASE_KEY}.#{file}", lines.keys)
+            end
+          end
+        else
+          store_array(BASE_KEY, report.keys)
+
+          report.each do |file, lines|
+            store_map("#{BASE_KEY}.#{file}", lines)
+          end
+        end
+      end
+
+      def coverage
+        data = {}
+        redis.smembers(BASE_KEY).each do |key|
+          data[key] = covered_lines_for_file(key)
+        end
+        data
+      end
+
+      def covered_files
+        redis.smembers(BASE_KEY)
+      end
+
+      def covered_lines_for_file(file)
+        if @store_as_array
+          @redis.smembers("#{BASE_KEY}.#{file}").map(&:to_i)
+        else
+          @redis.hgetall("#{BASE_KEY}.#{file}")
+        end
+      end
+
+      private
+
+      attr_reader :redis
+
+      def sadd_supports_array?
+        @_sadd_supports_array
+      end
+
+      def store_map(key, values)
+        unless values.empty?
+          existing = redis.hgetall(key)
+          #in redis all keys are strings
+          values = Hash[values.map{|k,val| [k.to_s,val] } ]
+          values.merge!( existing ){|k, old_v, new_v| old_v.to_i + new_v.to_i}
+          redis.mapped_hmset(key, values)
+        end
+      end
+
+      def store_array(key, values)
+        if sadd_supports_array?
+          redis.sadd(key, values) if (values.length > 0)
+        else
+          values.each do |value|
+            redis.sadd(key, value)
+          end
+        end
+        values
+      end
+
+      def recent_server_version?
+        info_data = redis.info
+        if info_data.is_a?(Hash)
+          Gem::Version.new(info_data['redis_version']) >= Gem::Version.new('2.4')
+        else
+          #guess supported
+          true
+        end
+      end
+
+      def recent_gem_version?
+        Gem::Version.new(Redis::VERSION) >= Gem::Version.new('3.0')
+      end
+    end
+  end
+end

data/lib/coverband/base.rb

@@ -45,10 +45,8 @@ module Coverband
       @ignore_patterns = Coverband.configuration.ignore + ["internal:prelude"]
       @ignore_patterns += ['gems'] unless Coverband.configuration.include_gems
       @sample_percentage = Coverband.configuration.percentage
-      if Coverband.configuration.redis
-        @reporter = Coverband::RedisStore.new(Coverband.configuration.redis)
-        @reporter = Coverband::MemoryCacheStore.new(@reporter) if Coverband.configuration.memory_caching
-      end
+      @store = Coverband.configuration.store
+      @store = Coverband::Adapters::MemoryCacheStore.new(@store) if Coverband.configuration.memory_caching
       @stats    = Coverband.configuration.stats
       @verbose  = Coverband.configuration.verbose
       @logger   = Coverband.configuration.logger
@@ -95,12 +93,12 @@ module Coverband
         end
       end
 
-      if @reporter
+      if @store
         if @stats
           @before_time = Time.now
           @stats.count "coverband.files.recorded_files", @file_line_usage.length
         end
-        @reporter.store_report(@file_line_usage)
+        @store.save_report(@file_line_usage)
         if @stats
           @time_spent = Time.now - @before_time
           @stats.timing "coverband.files.recorded_time", @time_spent
@@ -123,7 +121,6 @@ module Coverband
       !@ignore_patterns.any?{ |pattern| file.include?(pattern) } && file.start_with?(@project_directory)
     end
 
-
     def set_tracer
       unless @tracer_set
         @trace.enable