coverband 1.1 → 1.2

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA1:
-  metadata.gz: cc5c300ba7c487716b6a46f455757351caaaafea
-  data.tar.gz: 6a8509f0b16f807930bcb15a409f273bbdcdd241
+  metadata.gz: f8438c95988521edfcb5911828e1b75b4a83d625
+  data.tar.gz: 8a92bc7880429005fb1ba837b273e4968bda8a7d
 SHA512:
-  metadata.gz: fb3eeb08c1b3e00de3e9c19ceed3e3d3275c9e43b8e9b38bce8725788b00a951923bec225349eb8df1178810de31ccd914fe5663821f095f7fdcbc2c2692edc7
-  data.tar.gz: b7794879639fb64bab184ceca7763c716db6384cb36d814db091b6a2e2d77375fb51f47f2df6321b68f27c4f513318c509a5fd974a0d2ac54a2a5f06b8031cd2
+  metadata.gz: 78922ea8ccdad5cc0b71ff9096d5e2928777b3ba58156bb3a637c46fe5fd43faa011d9f575e193c380209eebdf1920cd29cb3c958167f1d262dd830b92f2d782
+  data.tar.gz: b75a40fac79fb36c18bed97bb4f6f8e345ad94c21204d003ddc5cc03f25846ea6265b86b026e07bbe655b4ee999799e89fc265c4e143070128215dfde2c9c8c4

data/README.md

@@ -2,21 +2,22 @@
 
 [![Build Status](https://travis-ci.org/danmayer/coverband.svg?branch=master)](https://travis-ci.org/danmayer/coverband)
 
-A gem to measure production code coverage. Coverband allows easy configuration to collect and report on production code coverage. It can be used as Rack middleware, wrapping a block with sampling, or manually configured to meet any need (like coverage on background jobs).
+A gem to measure production code usage, showing each line of code that is executed. Coverband allows easy configuration to collect and report on production code usage. It can be used as Rack middleware, wrapping a block with sampling, or manually configured to meet any need (like usage during background jobs). I like to think of this as production code coverage, but that implies test coverage to some folks, so being more explicit to say that it shows when a line of code is executed in a given environment is the most accurate way to describe it.
 
 * Allow sampling to avoid the performance overhead on every request.
 * Ignore directories to avoid overhead data collection on vendor, lib, etc.
-* Take a baseline to get initial app loading coverage.
-
-coverband 1.1+ supports ruby 2.0+.  For ruby 1.9 use coverband 1.0.X.
+* Take a baseline to get initial app execution during app initialization. (the baseline is important because some code is executed during app load, but might not be invoked during any requests, think prefetching, initial cache builds, setting constants, etc...)
+* Development mode for additional code usage details (number of LOC execution during single request, etc).
+* Coverband is not intended for test code coverage, for that just check out [SimpleCov](https://github.com/colszowka/simplecov).
 
+__Notes:__ Latest versions of Coverband drop support for anything less than Ruby 2.0, and Ruby 2.1+ is recommended.
 
 ###### Success:
 After running in production for 30 minutes, we were able very easily delete 2000 LOC after looking through the data. We expect to be able to clean up much more after it has collected more data.
 
 ###### Performance Impact
 
-Because coverband 2.0+ uses TracePoint which is much speedier than the ruby 1.9 set_trace_func, the performance impact is reasonable and there is no need for the c-ext [coverband_ext](https://github.com/danmayer/coverband_ext).  If you are stuck on ruby 1.9 and coverband 1.0.X, the [coverband_ext](https://github.com/danmayer/coverband_ext) is a must.
+The performance impact on Ruby 2.1+ is fairly small and no longer requires a C-extension. Look at the benchmark rake task for specific details.
 
 ## Installation
 
@@ -50,13 +51,7 @@ Details on a example Sinatra app
 
 ## Notes
 
-* Coverband has been running in production on Ruby 1.9.3, 2.x, 2.1.x on Sinatra, Rails 2.3.x, Rails 3.0.x, Rails 3.1.x, and Rails 3.2.x
-* No 1.8.7 support, Coverband requires Ruby 1.9.3+
-* There is a performance impact which is why the gem supports sampling. On low traffic sites I am running a sample rate of 20% and on very high traffic sites I am sampling at 1%, which still gives useful data
-    * The impact with the pure Ruby coverband can't be rather significant on sampled requests
-    * Most of the overhead is in the Ruby coverage collector, you can now use [coverband_ext](https://github.com/danmayer/coverband_ext) to run a C extension collector which is MUCH faster.
-    * Using Redis 2.x gem, while supported, is slow and not recommended. It will have a larger impact on overhead performance. Although the Ruby collection dwarfs the redis time, so it likely doesn't matter much.
-    * Make sure to ignore any folders like `vendor` and possibly `lib` as it can help reduce performance overhead. Or ignore specific frequently hit in app files for better perf.
+* Coverband has been used on large scale production websites without large impacts on performance. Adjusting the samplerate to achieve an acceptable trade-off on detailed information vs performance impact.
 
 ## Configuration
 
@@ -66,16 +61,12 @@ You need to configure cover band you can either do that passing in all configura
 
 ```ruby
 #config/coverband.rb
-require 'json'
-
-baseline = Coverband.parse_baseline
-
 Coverband.configure do |config|
   config.root              = Dir.pwd
   if defined? Redis
     config.redis           = Redis.new(:host => 'redis.host.com', :port => 49182, :db => 1)
   end
-  config.coverage_baseline = baseline
+  config.coverage_baseline = Coverband.parse_baseline
   config.root_paths        = ['/app/'] # /app/ is needed for heroku deployments
   # regex paths can help if you are seeing files duplicated for each capistrano deployment release
   #config.root_paths       = ['/server/apps/my_app/releases/\d+/']
@@ -83,6 +74,7 @@ Coverband.configure do |config|
   # Since rails and other frameworks lazy load code. I have found it is bad to allow
   # initial requests to record with coverband. This ignores first 15 requests
   config.startup_delay     = Rails.env.production? ? 15 : 2
+  # Percentage of requests recorded
   config.percentage        = Rails.env.production? ? 30.0 : 100.0
 
   config.logger            = Rails.logger
@@ -101,7 +93,7 @@ end
 
 ### 2. Configuring Rake
 
-Either add the below to your `Rakefile` or to a file included in your Rakefile such as `lib/tasks/coverband` if you want to break it up that way.
+Either add the below to your `Rakefile` or to a file included in your Rakefile such as `lib/tasks/coverband.rake` if you want to break it up that way.
 
 ```ruby
 require 'coverband'
@@ -117,7 +109,7 @@ rake coverband:clear         # reset coverband coverage data
 rake coverband:coverage      # report runtime coverband code coverage
 ```
 
-The default Coverband baseline task will try to detect your app as either Rack or Rails environment. It will load the app to take a baseline reading. If the baseline task doesn't load your app well you can override the default baseline to create a better baseline yourself. Below for example is how I take a baseline on a pretty simple Sinatra app.
+The default Coverband baseline task will try to detect your app as either Rack or Rails environment. It will load the app to take a baseline reading. The baseline coverage is important because some code is executed during app load, but might not be invoked during any requests, think prefetching, initial cache builds, setting constants, etc. If the baseline task doesn't load your app well you can override the default baseline to create a better baseline yourself. Below for example is how I take a baseline on a pretty simple Sinatra app.
 
 ```ruby
 namespace :coverband do
@@ -155,10 +147,10 @@ run ActionController::Dispatcher.new
 
 #### For Rails apps
 
-Create an initializer file
+Create an initializers file
 
 ```ruby
-# config/initializes/coverband_middleware.rb
+# config/initializers/coverband_middleware.rb
 
 # Configure the Coverband Middleware
 require 'coverband'
@@ -182,7 +174,7 @@ module MyApplication
 end
 ```
 
-Note: To debug in development mode, I recommend turning verbose logging on `config.verbose = true` and passing in the Rails.logger `config.logger = Rails.logger` to the Coverband config. This makes it easy to follow in development mode. Be careful to not leave these on in production as they will effect performance.
+Note: To debug in development mode, I recommend turning verbose logging on `config.verbose = true` and passing in the Rails.logger `config.logger = Rails.logger` to the Coverband config. This makes it easy to follow in development mode. Be careful to not leave these on in production as they will affect performance.
 
 ## Usage
 
@@ -190,11 +182,12 @@ Note: To debug in development mode, I recommend turning verbose logging on `conf
 2. Hit your development server exercising the endpoints you want to verify Coverband is recording (you should see debug outputs in your server console)
 3. Run `rake coverband:coverage` again, previously it should have only shown the baseline data of your app initializing. After using it in development it should show increased coverage from the actions you have exercised.
 
-Note: if you use `rails s` and data aren't reccorded, make sure it is using your `config.ru`.
+Note: if you use `rails s` and data aren't recorded, make sure it is using your `config.ru`.
 
 ## Example apps
 
 - [Rails app](https://github.com/arnlen/rails-coverband-example-app)
+- [Rails app with Coverband 1.1](https://github.com/danmayer/covered_rails)
 - [Sinatra app](https://github.com/danmayer/churn-site)
 - [Non rack ruby app](https://github.com/danmayer/coverband_examples)
 
@@ -295,10 +288,12 @@ data = JSON.parse(File.read("blah.json"))
 Coverband::Reporter.report :additional_scov_data => [data]
 ```
 
+You can also pass a `:additional_scov_data => [data]` option to `Coverband::Reporter.get_current_scov_data` to write out merged data.
+
 ### Known issues
 
-* If you don't have a baseline recorded your coverage can look odd like you are missing a bunch of data. It would be good if coverband gave a more actionable warning in this situation.
-* If you have simplecov filters, you need to clear them prior to generating your coverage report. As the filters will be applied to coverband as well and can often filter out everything we are recording.
+* If you don't have a baseline recorded your coverage can look odd like you are missing a bunch of data. It would be good if Coverband gave a more actionable warning in this situation.
+* If you have SimpleCov filters, you need to clear them prior to generating your coverage report. As the filters will be applied to Coverband as well and can often filter out everything we are recording.
 * the line numbers reported for `ERB` files are often off and aren't considered useful. I recommend filtering out .erb using the `config.ignore` option.
 
 ## TODO
@@ -311,7 +306,9 @@ Coverband::Reporter.report :additional_scov_data => [data]
   * blank rails app
   * blank Sinatra app
 * report on Coverband files that haven't recorded any coverage (find things like events and crons that aren't recording, or dead files)
-* ability to change the coverband config at runtime by changing the config pushed to the Redis hash. In memory cache around the changes to only make that call periodically.
+* ability to change the Coverband config at runtime by changing the config pushed to the Redis hash. In memory cache around the changes to only make that call periodically.
+* Opposed to just showing code usage on a route allow 'tagging' events which would record line coverage for that tag (this would allow tagging all code that modified an ActiveRecord model for example
+* mountable rack app to view coverage similar to flipper-ui
 
 ## Contributing
 
@@ -319,11 +316,12 @@ Coverband::Reporter.report :additional_scov_data => [data]
 2. Create your feature branch (`git checkout -b my-new-feature`)
 3. Commit your changes (`git commit -am 'Add some feature'`)
 4. Push to the branch (`git push origin my-new-feature`)
-5. Create new Pull Request
+5. Make sure all tests are passing (run `bundle install`, make sure Redis is running, and then execute `bundle exec rake test`)
+6. Create new Pull Request
 
 ## Resources
 
-These notes of kind of for myself, but if anyone is seriously interested in contributing to the project, these resorces might be helpfu. I learned a lot looking at various existing projects and open source code.
+These notes of kind of for myself, but if anyone is seriously interested in contributing to the project, these resources might be helpful. I learned a lot looking at various existing projects and open source code.
 
 ##### Ruby Std-lib Coverage
 
@@ -338,8 +336,8 @@ These notes of kind of for myself, but if anyone is seriously interested in cont
 * [erb syntax](http://stackoverflow.com/questions/7996695/rails-erb-syntax) parse out and mark lines as important
 * [ruby 2 tracer](https://github.com/brightbox/deb-ruby2.0/blob/master/lib/tracer.rb)
 * [coveralls hosted code coverage tracking](https://coveralls.io/docs/ruby) currently for test coverage but might be a good partner for production coverage
-* [simplecov walk through](http://www.tamingthemindmonkey.com/2011/09/27/ruby-code-coverage-using-simplecov) copy some of the syntax sugar setup for cover band
-* [Jruby coverage bug](http://jira.codehaus.org/browse/JRUBY-6106?page=com.atlassian.jira.plugin.system.issuetabpanels:changehistory-tabpanel)
+* [simplecov usage example](http://www.cakesolutions.net/teamblogs/brief-introduction-to-rspec-and-simplecov-for-ruby) copy some of the syntax sugar setup for cover band
+* [Jruby coverage bug](https://github.com/jruby/jruby/issues/1196)
 * [learn from oboe ruby code](https://github.com/appneta/oboe-ruby#writing-custom-instrumentation)
 * [learn from stackprof](https://github.com/tmm1/stackprof#readme)
 * I believe there are possible ways to get even better data using the new [Ruby2 TracePoint API](http://www.ruby-doc.org/core/TracePoint.html)

data/coverband.gemspec

@@ -22,8 +22,11 @@ Gem::Specification.new do |spec|
   spec.add_development_dependency "rake"
   spec.add_development_dependency "mocha", "~> 0.14.0"
   spec.add_development_dependency "rack"
+  spec.add_development_dependency "rack-test"
   spec.add_development_dependency "test-unit"
   spec.add_runtime_dependency "simplecov"
   spec.add_runtime_dependency "json"
   spec.add_runtime_dependency "redis"
+  spec.add_runtime_dependency 'aws-sdk', '~> 2'
+  spec.add_runtime_dependency 'sinatra'
 end

data/lib/coverband.rb

@@ -1,17 +1,20 @@
 require 'redis'
 require 'logger'
+require 'aws-sdk'
 
 require 'coverband/version'
 require 'coverband/configuration'
 require 'coverband/redis_store'
+require 'coverband/memory_cache_store'
 require 'coverband/base'
 require 'coverband/reporter'
 require 'coverband/middleware'
+require 'coverband/s3_report_writer'
 
 module Coverband
 
   CONFIG_FILE = './config/coverband.rb'
-  
+
   class << self
     attr_accessor :configuration_data
   end
@@ -41,5 +44,5 @@ module Coverband
   def self.configuration
     self.configuration_data ||= Configuration.new
   end
-  
+
 end

data/lib/coverband/base.rb

@@ -45,7 +45,10 @@ module Coverband
       @ignore_patterns = Coverband.configuration.ignore + ["internal:prelude"]
       @ignore_patterns += ['gems'] unless Coverband.configuration.include_gems
       @sample_percentage = Coverband.configuration.percentage
-      @reporter = Coverband::RedisStore.new(Coverband.configuration.redis) if Coverband.configuration.redis
+      if Coverband.configuration.redis
+        @reporter = Coverband::RedisStore.new(Coverband.configuration.redis)
+        @reporter = Coverband::MemoryCacheStore.new(@reporter) if Coverband.configuration.memory_caching
+      end
       @stats    = Coverband.configuration.stats
       @verbose  = Coverband.configuration.verbose
       @logger   = Coverband.configuration.logger
@@ -87,8 +90,10 @@ module Coverband
 
       @files.reject!{|file, lines| !track_file?(file) }
 
+      #make lines uniq
+      @files.each{|file, lines| lines.uniq!}
+
       if @verbose
-        @file_usage.reject!{|file, line_count| !track_file?(file) }
         @logger.info "coverband file usage: #{@file_usage.sort_by {|_key, value| value}.inspect}"
         if @verbose=="debug"
           output_file_line_usage
@@ -162,7 +167,7 @@ module Coverband
             @file_line_usage[file][line] += 1
           end
           file_lines = (@files[file] ||= [])
-          file_lines << line
+          file_lines.push(line) unless file_lines.include?(line)
         end
       end
     end

data/lib/coverband/configuration.rb

@@ -1,6 +1,10 @@
 module Coverband
   class Configuration
-    attr_accessor :redis, :coverage_baseline, :root_paths, :root, :ignore, :percentage, :verbose, :reporter, :stats, :logger, :startup_delay, :baseline_file, :trace_point_events, :include_gems
+     
+    attr_accessor :redis, :coverage_baseline, :root_paths, :root, 
+                  :ignore, :percentage, :verbose, :reporter, :stats,
+                  :logger, :startup_delay, :baseline_file, :trace_point_events, 
+                  :include_gems, :memory_caching, :s3_bucket
 
     def initialize
       @root = Dir.pwd
@@ -17,6 +21,7 @@ module Coverband
       @logger = Logger.new(STDOUT)
       @startup_delay = 0
       @trace_point_events = [:line]
+      @memory_caching = false
     end
 
     def logger

data/lib/coverband/memory_cache_store.rb

@@ -0,0 +1,42 @@
+module Coverband
+  class MemoryCacheStore
+
+    attr_accessor :store
+
+
+    def self.files_cache
+      @files_cache ||= Hash.new
+    end
+
+    def self.reset!
+      files_cache.clear
+    end
+
+    def initialize(store)
+      @store = store
+    end
+
+    def store_report files
+      filtered_files = filter(files)
+      store.store_report(filtered_files) if filtered_files.any?
+    end
+
+    private
+
+    def files_cache
+      self.class.files_cache
+    end
+
+    def filter(files)
+      files.each_with_object(Hash.new) do |(file, lines), filtered_file_hash|
+        #first time we see a file, we pre-init the in memory cache to whatever is in store(redis)
+        line_cache = files_cache[file] ||= Set.new(store.covered_lines_for_file(file))
+        lines.reject! do |line|
+          line_cache.include?(line) ? true : (line_cache << line and false)
+        end
+        filtered_file_hash[file] = lines if lines.any?
+      end
+    end
+
+  end
+end

data/lib/coverband/redis_store.rb

@@ -15,6 +15,12 @@ module Coverband
       end
     end
 
+
+    def covered_lines_for_file(file)
+      @redis.smembers("coverband.#{file}").map(&:to_i)
+    end
+
+
     def sadd_supports_array?
       @_sadd_supports_array
     end

data/lib/coverband/reporter.rb

@@ -7,7 +7,7 @@ module Coverband
       yield
       @project_directory = File.expand_path(Coverband.configuration.root)
       results = Coverage.result
-      results = results.reject{|key, val| !key.match(@project_directory) || Coverband.configuration.ignore.any?{|pattern| key.match(/#{pattern}/)} }
+      results = results.reject { |key, val| !key.match(@project_directory) || Coverband.configuration.ignore.any? { |pattern| key.match(/#{pattern}/) } }
 
       if Coverband.configuration.verbose
         Coverband.configuration.logger.info results.inspect
@@ -15,7 +15,7 @@ module Coverband
 
       config_dir = File.dirname(Coverband.configuration.baseline_file)
       Dir.mkdir config_dir unless File.exists?(config_dir)
-      File.open(Coverband.configuration.baseline_file, 'w') {|f| f.write(results.to_json) }
+      File.open(Coverband.configuration.baseline_file, 'w') { |f| f.write(results.to_json) }
     end
 
     def self.report(options = {})
@@ -28,27 +28,27 @@ module Coverband
       redis = Coverband.configuration.redis
       roots = get_roots
       existing_coverage = Coverband.configuration.coverage_baseline
-      open_report = options.fetch(:open_report){ true }
+      open_report = options.fetch(:open_report) { true }
 
       if Coverband.configuration.verbose
         Coverband.configuration.logger.info "fixing root: #{roots.join(', ')}"
       end
 
-      if  Coverband.configuration.reporter=='scov'
-        additional_scov_data = options.fetch(:additional_scov_data){ [] }
+      if Coverband.configuration.reporter == 'scov'
+        additional_scov_data = options.fetch(:additional_scov_data) { [] }
         if Coverband.configuration.verbose
           print additional_scov_data
         end
         report_scov(redis, existing_coverage, additional_scov_data, roots, open_report)
       else
-        lines = redis.smembers('coverband').map{|key| report_line(redis, key) }
+        lines = redis.smembers('coverband').map{ |key| report_line(redis, key) }
         Coverband.configuration.logger.info lines.join("\n")
       end
     end
 
     def self.clear_coverage(redis = nil)
       redis ||= Coverband.configuration.redis
-      redis.smembers('coverband').each{|key| redis.del("coverband.#{key}")}
+      redis.smembers('coverband').each { |key| redis.del("coverband.#{key}") }
       redis.del("coverband")
     end
 
@@ -66,7 +66,7 @@ module Coverband
 
     def self.fix_file_names(report_hash, roots)
       fixed_report = {} #normalize names across servers
-      report_hash.each_pair do |key, values| 
+      report_hash.each_pair do |key, values|
         filename = filename_from_key(key, roots)
         fixed_report[filename] = values
       end
@@ -79,47 +79,54 @@ module Coverband
     # [0,0,1,0,1]
     def self.merge_arrays(first, second)
       merged = []
-      longest = if first.length > second.length 
-        first
-      else
-        second
+      longest = first.length > second.length ? first : second
+
+      longest.each_with_index do |line, index|
+        if first[index] || second[index]
+          merged[index] = (first[index].to_i + second[index].to_i >= 1 ? 1 : 0)
+        else
+          merged[index] = nil
+        end
       end
-     longest.each_with_index do |line, index|
-              if first[index] || second[index]
-                merged[index] = (first[index].to_i + second[index].to_i >= 1 ? 1 : 0)
-              else
-                merged[index] = nil
-              end
-            end
-     merged
+
+      merged
     end
 
-    
-    def self.get_current_scov_data
-      get_current_scov_data_imp(Coverband.configuration.redis, get_roots)
+
+    def self.get_current_scov_data(options = {})
+      additional_scov_data = options.fetch(:additional_scov_data) { [] }
+
+      if (additional_scov_data)
+        report_scov_with_additional_data(Coverband.configuration.redis, Coverband.configuration.coverage_baseline, additional_scov_data, get_roots)
+      else
+        get_current_scov_data_imp(Coverband.configuration.redis, get_roots)
+      end
     end
 
     def self.get_current_scov_data_imp(redis, roots)
       scov_style_report = {}
+
       redis.smembers('coverband').each do |key|
-                                   next if Coverband.configuration.ignore.any?{ |i| key.match(i)}
-                                   line_data = line_hash(redis, key, roots)
-                                   
-                                   if line_data
-                                     line_key = line_data.keys.first
-                                     previous_line_hash = scov_style_report[line_key]
-                                     if previous_line_hash
-                       
-                                       line_data[line_key] = merge_arrays(line_data[line_key], previous_line_hash)
-                                     end
-                                     scov_style_report.merge!(line_data)
-                                   end
-                                 end
+        next if Coverband.configuration.ignore.any?{ |i| key.match(i) }
+        line_data = line_hash(redis, key, roots)
+
+        if line_data
+          line_key = line_data.keys.first
+          previous_line_hash = scov_style_report[line_key]
+
+          if previous_line_hash
+            line_data[line_key] = merge_arrays(line_data[line_key], previous_line_hash)
+          end
+
+          scov_style_report.merge!(line_data)
+        end
+      end
+
       scov_style_report = fix_file_names(scov_style_report, roots)
       scov_style_report
     end
-      
-    def self.report_scov(redis, existing_coverage, additional_scov_data, roots, open_report)
+
+    def self.report_scov_with_additional_data(redis, existing_coverage, additional_scov_data, roots)
       scov_style_report = get_current_scov_data_imp redis, roots
       existing_coverage = fix_file_names(existing_coverage, roots)
       scov_style_report = merge_existing_coverage(scov_style_report, existing_coverage)
@@ -127,25 +134,33 @@ module Coverband
       additional_scov_data.each do |data|
         scov_style_report = merge_existing_coverage(scov_style_report, data)
       end
-      
+
+      scov_style_report
+    end
+
+    def self.report_scov(redis, existing_coverage, additional_scov_data, roots, open_report)
+      scov_style_report = report_scov_with_additional_data(redis, existing_coverage, additional_scov_data, roots)
+
       if Coverband.configuration.verbose
         Coverband.configuration.logger.info "report: "
         Coverband.configuration.logger.info scov_style_report.inspect
       end
-      
+
       SimpleCov::Result.new(scov_style_report).format!
       if open_report
         `open #{SimpleCov.coverage_dir}/index.html`
       else
         Coverband.configuration.logger.info "report is ready and viewable: open #{SimpleCov.coverage_dir}/index.html"
       end
+      S3ReportWriter.new(Coverband.configuration.s3_bucket).persist! if Coverband.configuration.s3_bucket
     end
 
+
     def self.merge_existing_coverage(scov_style_report, existing_coverage)
       existing_coverage.each_pair do |key, lines|
         if current_lines = scov_style_report[key]
           lines.each_with_index do |line, index|
-            if line.nil? && current_lines[index].to_i==0
+            if line.nil? && current_lines[index].to_i == 0
               current_lines[index] = nil
             else
               current_lines[index] = current_lines[index] ? (current_lines[index].to_i + line.to_i) : nil
@@ -169,7 +184,7 @@ module Coverband
     def self.line_members(redis, key)
       redis.smembers("coverband.#{key}").inspect
     end
-                                                            
+
     def self.filename_from_key(key, roots)
       filename = key
       roots.each do |root|
@@ -188,13 +203,13 @@ module Coverband
       filename = filename_from_key(key, roots)
       if File.exists?(filename)
         lines_hit = redis.smembers("coverband.#{key}")
-        count = File.foreach(filename).inject(0) {|c, line| c+1}
+        count = File.foreach(filename).inject(0) { |c, line| c + 1 }
         if filename.match(/\.erb/)
           line_array = Array.new(count, nil)
         else
           line_array = Array.new(count, 0)
         end
-        line_array.each_with_index{|line,index| line_array[index]=1 if lines_hit.include?((index+1).to_s) }
+        line_array.each_with_index{|line,index| line_array[index] = 1 if lines_hit.include?((index + 1).to_s) }
         {filename => line_array}
       else
         Coverband.configuration.logger.info "file #{filename} not found in project"

data/lib/coverband/s3_report_writer.rb

@@ -0,0 +1,30 @@
+class S3ReportWriter
+
+  def initialize(bucket_name)
+    @bucket_name = bucket_name
+  end
+
+  def persist!
+    object.put(body: coverage_content)
+  end
+
+  private
+
+  def coverage_content
+    File.read("#{SimpleCov.coverage_dir}/index.html").gsub("./assets/#{Gem::Specification.find_by_name('simplecov-html').version.version}/", '')
+  end
+
+  def object
+    bucket.object('coverband/index.html')
+  end
+
+  def s3
+    Aws::S3::Resource.new
+  end
+
+  def bucket
+    s3.bucket(@bucket_name)
+  end
+
+
+end

data/lib/coverband/s3_web.rb

@@ -0,0 +1,21 @@
+require 'sinatra/base'
+
+module Coverband
+
+  class S3Web < Sinatra::Base
+
+    set :public_folder, proc { File.expand_path('public', Gem::Specification.find_by_name('simplecov-html').full_gem_path) }
+
+    get '/' do
+      s3.get_object(bucket: Coverband.configuration.s3_bucket, key:'coverband/index.html').body.read
+    end
+
+    private
+
+    def s3
+      @s3 ||= Aws::S3::Client.new
+    end
+
+  end
+
+end

data/lib/coverband/version.rb

@@ -1,3 +1,3 @@
 module Coverband
-  VERSION = "1.1"
+  VERSION = "1.2"
 end

data/test/benchmarks/benchmark.rake

@@ -1,5 +1,6 @@
 require 'coverband'
 require 'redis'
+require File.join(File.dirname(__FILE__), 'dog')
 
 namespace :benchmarks do
 
@@ -21,12 +22,14 @@ namespace :benchmarks do
     require 'classifier-reborn'
 
     Coverband.configure do |config|
-      config.redis             = Redis.new
-      config.root              = Dir.pwd
-      config.startup_delay     = 0
-      config.percentage        = 100.0
-      config.logger            = $stdout
-      config.verbose           = false
+      config.redis              = Redis.new
+      config.root               = Dir.pwd
+      config.startup_delay      = 0
+      config.percentage         = 100.0
+      config.logger             = $stdout
+      config.verbose            = false
+      #config.memory_caching     = true
+      #config.trace_point_events = [:call]
     end
 
   end
@@ -57,6 +60,9 @@ namespace :benchmarks do
       bayes_classification
       lsi_classification
     end
+
+    #simulate many calls to the same line
+    10_000.times { Dog.new.bark }
   end
 
 

data/test/benchmarks/dog.rb

@@ -0,0 +1,7 @@
+class Dog
+
+  def bark
+    "bark"
+  end
+
+end

data/test/unit/base_test.rb

@@ -3,6 +3,19 @@ require File.expand_path('./dog', File.dirname(__FILE__))
 
 class BaseTest < Test::Unit::TestCase
 
+  test 'defaults to a redis store' do
+    coverband = Coverband::Base.instance.reset_instance
+    assert_equal Coverband::RedisStore, coverband.instance_variable_get('@reporter').class
+  end
+
+
+  test 'configure memory caching' do
+    Coverband.configuration.memory_caching = true
+    coverband = Coverband::Base.instance.reset_instance
+    assert_equal Coverband::MemoryCacheStore, coverband.instance_variable_get('@reporter').class
+    Coverband.configuration.memory_caching = false
+  end
+
   test "start should enable coverage" do
     coverband = Coverband::Base.instance.reset_instance
     assert_equal false, coverband.instance_variable_get("@enabled")
@@ -65,7 +78,17 @@ class BaseTest < Test::Unit::TestCase
     store.expects(:store_report).once.with(has_entries(dog_file => [3]) )
     assert_equal false, coverband.instance_variable_get("@enabled")
     coverband.start
-    Dog.new.bark
+    5.times { Dog.new.bark }
+    coverband.stop
+    coverband.save
+  end
+
+  test "tracer should collect uniq line numbers" do
+    dog_file = File.expand_path('./dog.rb', File.dirname(__FILE__))
+    coverband = Coverband::Base.instance.reset_instance
+    coverband.start
+    100.times { Dog.new.bark }
+    assert_equal 1, coverband.instance_variable_get("@files")[dog_file].select{ |i| 3 == i }.count
     coverband.stop
     coverband.save
   end

data/test/unit/memory_cache_store_test.rb

@@ -0,0 +1,67 @@
+require File.expand_path('../test_helper', File.dirname(__FILE__))
+
+module Coverband
+  class MemoryCacheStoreTest < Test::Unit::TestCase
+
+    def setup
+      MemoryCacheStore.reset!
+      @store = mock('store')
+      @memory_store = MemoryCacheStore.new(@store)
+    end
+
+
+    test 'it passes data into store' do
+      data = {
+        'file1' => [ 3, 5 ],
+        'file2' => [1, 2]
+      }
+      @store.expects(:store_report).with data
+      @store.expects(:covered_lines_for_file).with('file1').returns []
+      @store.expects(:covered_lines_for_file).with('file2').returns []
+      @memory_store.store_report data
+    end
+
+    test 'it passes data into store only once' do
+      data = {
+        'file1' => [ 3, 5 ],
+        'file2' => [1, 2]
+      }
+      @store.expects(:store_report).once.with data
+      @store.expects(:covered_lines_for_file).with('file1').returns []
+      @store.expects(:covered_lines_for_file).with('file2').returns []
+      2.times { @memory_store.store_report data }
+    end
+
+    test 'it only passes files and lines we have not hit yet' do
+      first_data = {
+        'file1' => [ 3, 5 ],
+        'file2' => [1, 2]
+      }
+      second_data = {
+        'file1' => [ 3, 5, 10 ],
+        'file2' => [1, 2]
+      }
+      @store.expects(:covered_lines_for_file).with('file1').returns []
+      @store.expects(:covered_lines_for_file).with('file2').returns []
+      @store.expects(:store_report).once.with first_data
+      @store.expects(:store_report).once.with(
+        'file1' => [10]
+      )
+      @memory_store.store_report first_data
+      @memory_store.store_report second_data
+    end
+
+    test 'it initializes cache with what is in store' do
+      data = {
+        'file1' => [ 3, 5 ],
+        'file2' => [1, 2]
+      }
+      @store.expects(:covered_lines_for_file).with('file1').returns [3,5]
+      @store.expects(:covered_lines_for_file).with('file2').returns [2]
+      @store.expects(:store_report).with( 'file2' => [1] )
+      @memory_store.store_report data
+    end
+
+  end
+
+end

data/test/unit/redis_store_test.rb

@@ -2,6 +2,22 @@ require File.expand_path('../test_helper', File.dirname(__FILE__))
 
 class RedisTest < Test::Unit::TestCase
 
+  def setup
+    @redis = Redis.new
+    @redis.flushdb
+    @store = Coverband::RedisStore.new(@redis)
+  end
+
+  def test_covered_lines_for_file
+    @redis.sadd('coverband.dog.rb', 1)
+    @redis.sadd('coverband.dog.rb', 2)
+    assert_equal @store.covered_lines_for_file('dog.rb').sort,  [1, 2]
+  end
+
+  def test_covered_lines_when_null
+    assert_equal @store.covered_lines_for_file('dog.rb'),  []
+  end
+
   private
 
   def test_data

data/test/unit/s3_report_writer_test.rb

@@ -0,0 +1,21 @@
+require File.expand_path('../test_helper', File.dirname(__FILE__))
+
+module Coverband
+
+  class S3ReportWriterTest < Test::Unit::TestCase
+
+    test 'it writes the coverage report to s3' do
+      s3 = mock('s3_resource')
+      bucket = mock('bucket')
+      object = mock('object')
+      s3.expects(:bucket).with('coverage-bucket').returns(bucket)
+      bucket.expects(:object).with('coverband/index.html').returns(object)
+      File.expects(:read).with("#{SimpleCov.coverage_dir}/index.html").returns("content ./assets/#{Gem::Specification.find_by_name('simplecov-html').version.version}/")
+      object.expects(:put).with(body: 'content ')
+      Aws::S3::Resource.expects(:new).returns(s3)
+      S3ReportWriter.new('coverage-bucket').persist!
+    end
+
+  end
+
+end

data/test/unit/s3_web_test.rb

@@ -0,0 +1,26 @@
+require File.expand_path('../test_helper', File.dirname(__FILE__))
+require File.expand_path('../../lib/coverband/s3_web', File.dirname(__FILE__))
+require 'rack/test'
+
+ENV['RACK_ENV'] = 'test'
+
+module Coverband
+  class S3WebTest < Test::Unit::TestCase
+
+    include Rack::Test::Methods
+
+    def app
+      Coverband::S3Web
+    end
+
+    test 'renders content from the coverband/index.html object' do
+      Coverband.configuration.s3_bucket = 'coverage-bucket'
+      s3 = mock('s3')
+      Aws::S3::Client.expects(:new).returns(s3)
+      s3.expects(:get_object).with(bucket: 'coverage-bucket', key: 'coverband/index.html').returns mock('response', body: mock('body', read: 'content'))
+      get '/'
+      assert last_response.ok?
+      assert_equal 'content', last_response.body
+    end
+  end
+end

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: coverband
 version: !ruby/object:Gem::Version
-  version: '1.1'
+  version: '1.2'
 platform: ruby
 authors:
 - Dan Mayer
 autorequire: 
 bindir: bin
 cert_chain: []
-date: 2016-01-24 00:00:00.000000000 Z
+date: 2016-11-12 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: bundler
@@ -66,6 +66,20 @@ dependencies:
     - - ">="
       - !ruby/object:Gem::Version
         version: '0'
+- !ruby/object:Gem::Dependency
+  name: rack-test
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
 - !ruby/object:Gem::Dependency
   name: test-unit
   requirement: !ruby/object:Gem::Requirement
@@ -122,6 +136,34 @@ dependencies:
     - - ">="
       - !ruby/object:Gem::Version
         version: '0'
+- !ruby/object:Gem::Dependency
+  name: aws-sdk
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '2'
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '2'
+- !ruby/object:Gem::Dependency
+  name: sinatra
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
 description: Rack middleware to help measure production code coverage
 email:
 - dan@mayerdan.com
@@ -142,21 +184,28 @@ files:
 - lib/coverband.rb
 - lib/coverband/base.rb
 - lib/coverband/configuration.rb
+- lib/coverband/memory_cache_store.rb
 - lib/coverband/middleware.rb
 - lib/coverband/redis_store.rb
 - lib/coverband/reporter.rb
+- lib/coverband/s3_report_writer.rb
+- lib/coverband/s3_web.rb
 - lib/coverband/tasks.rb
 - lib/coverband/version.rb
 - test/benchmarks/.gitignore
 - test/benchmarks/benchmark.rake
+- test/benchmarks/dog.rb
 - test/fake_app/basic_rack.rb
 - test/test_helper.rb
 - test/unit/base_test.rb
 - test/unit/configuration_test.rb
 - test/unit/dog.rb
+- test/unit/memory_cache_store_test.rb
 - test/unit/middleware_test.rb
 - test/unit/redis_store_test.rb
 - test/unit/reporter_test.rb
+- test/unit/s3_report_writer_test.rb
+- test/unit/s3_web_test.rb
 homepage: ''
 licenses:
 - MIT
@@ -177,18 +226,22 @@ required_rubygems_version: !ruby/object:Gem::Requirement
       version: '0'
 requirements: []
 rubyforge_project: 
-rubygems_version: 2.2.5
+rubygems_version: 2.5.1
 signing_key: 
 specification_version: 4
 summary: Rack middleware to help measure production code coverage
 test_files:
 - test/benchmarks/.gitignore
 - test/benchmarks/benchmark.rake
+- test/benchmarks/dog.rb
 - test/fake_app/basic_rack.rb
 - test/test_helper.rb
 - test/unit/base_test.rb
 - test/unit/configuration_test.rb
 - test/unit/dog.rb
+- test/unit/memory_cache_store_test.rb
 - test/unit/middleware_test.rb
 - test/unit/redis_store_test.rb
 - test/unit/reporter_test.rb
+- test/unit/s3_report_writer_test.rb
+- test/unit/s3_web_test.rb