coverband 6.0.3.rc.4 → 6.1.1

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: cf0b6bab823d8155a5e15d361c3acbbef271f84e79947b9818a650e640a61fc7
-  data.tar.gz: 25cd0843b3c1e852794bcc008be165bfcbf7e476f373ab681cbff7861f050904
+  metadata.gz: ed2b063179cbc773c9b3b8fab3403820f9855c32a9de007c347b58873eecf13a
+  data.tar.gz: 7968b26ac22e3209b0bf2a9baabdfe107ed9029a0d4557eea5b80c22439e3278
 SHA512:
-  metadata.gz: d2ae2023667369ac1e7be83ec41898c6e77662545f6226c02417ef9a43e75a4e2950ad0d775e18c96d800270d81da104162d4414b13768ac98c14bcff431acd1
-  data.tar.gz: 64ddff4e59042a6ee22cd4839e6e7ac57dff4cf289b02a8f6801c144aafea7edd417ed918c38309b61173176cb177ce548dbf6302963d4b53464d8901cb05d30
+  metadata.gz: 3bccf2dbc8e92d5980365a99b5ae2852fd8e04d224202a58e3a8490991b88eb88a881fbf620e4ea60d66cd4714a70f11a8015ebd96aa772a65a35e1fb642ea24
+  data.tar.gz: 784f5e0905989f865199261270072c397e520b2269ed387f0ecd1276ef5a7395fc5d4de7ec9990aceeda1cc883778bcfcf4c7c2260c792e81557914298778d87

data/.github/ISSUE_TEMPLATE/bug_report.md

@@ -28,11 +28,5 @@ If applicable, add screenshots to help explain your problem.
  - Browser [e.g. chrome, safari]
  - Version [e.g. 22]
 
-**Smartphone (please complete the following information):**
- - Device: [e.g. iPhone6]
- - OS: [e.g. iOS8.1]
- - Browser [e.g. stock browser, safari]
- - Version [e.g. 22]
-
 **Additional context**
 Add any other context about the problem here.

data/.github/workflows/main.yml

@@ -26,7 +26,7 @@ jobs:
     runs-on: ${{ matrix.os }}-latest
     steps:
       - uses: actions/checkout@v4
-      - uses: supercharge/redis-github-action@1.2.0
+      - uses: supercharge/redis-github-action@1.8.0
         with:
           redis-version: ${{ matrix.redis-version }}
       - uses: ruby/setup-ruby@v1

data/README.md

@@ -101,6 +101,22 @@ run ActionController::Dispatcher.new
 
   This triggers coverage collection on the current webserver process. Useful in development but confusing in production environments where many ruby processes are usually running.
 
+#### Interpreting results
+
+The columns in the web UI are as follows:
+
+- **% covered** - Percentage of total relevant lines covered
+- **% runtime** - Percentage of the runtime lines covered where runtime lines are lines that are hit after the application has been eagerly loaded
+- **Lines** - Total lines in the file including lines unreachable or uncover-able. An unreachable line would be an empty line with no code, comments, or `end` statements.
+- **Relevant lines** - Lines that are coverable, i.e. not empty
+- **Lines runtime** - Total lines minus uncoverable lines minus the lines that are only hit during eager loading of application
+- **Lines missed** - Relevant lines not covered
+- **Avg hits/line** - Total of coverage to the file divided by relevant lines.
+
+When viewing an individual file, a line of code such as a class or method definition may appear green because it is eager loaded by the application, but still not be hit at all in runtime by actual users.  
+
+![example of a file with lines not hit at runtime](https://user-images.githubusercontent.com/96786/63541229-aa98a580-c4eb-11e9-8eb8-c004fe1369db.png)
+
 ### Mounting as a Rack App
 
 Coverband comes with a mountable rack app for viewing reports. For Rails this can be done in `config/routes.rb` with:
@@ -191,7 +207,7 @@ Do you use figaro, mc-settings, dotenv or something else to inject environment v
 For example if you use dotenv, you need to do this, see https://github.com/bkeepers/dotenv#note-on-load-order
 
 ```
-gem 'dotenv-rails', require: 'dotenv/rails-now'
+gem 'dotenv-rails', require: 'dotenv/load'
 gem 'coverband'
 gem 'other-gem-that-requires-env-variables'
 ```
@@ -222,6 +238,14 @@ This feature is enabled by default. To stop this feature, disable the feature in
 
 ![image](https://raw.github.com/danmayer/coverband/master/docs/coverband_view_tracker.png)
 
+### Hiding settings
+
+Coverband provides a view of all of its current settings. Sometimes you might want to hide this view,
+such as when sharing coverband data with a large number of developers of varying trust levels.
+You can disable the settings view like so:
+
+`config.hide_settings = false`
+
 ### Fixing Coverage Only Shows Loading Hits
 
 If all your coverage is being counted as loading or eager_loading coverage, and nothing is showing as runtime Coverage the initialization hook failed for some reason. The most likely reason for this issue is manually calling `eager_load!` on some Plugin/Gem. If you or a plugin is altering the Rails initialization process, you can manually flip Coverband to runtime coverage by calling these two lines, in an `after_initialize` block, in `application.rb`.

data/changes.md

@@ -1,3 +1,25 @@
+### Coverband 6.1.1
+
+* Performance fix making paged report loading 10X faster
+
+### Coverband 6.1.0
+
+This release has a number of smaller fixes and improvements. It includes a sizable refactoring around the UI which should simplify improvements going forward. This release is mostly targetting large projects with 6K+ ruby files, use the new `config.paged_reporting = true` option with the HashRedisStore to enable paged reporting for large projects. The HashRedisStore now also includes the last time a line in a file was executed.
+
+* Thanks to @FeLvi-zzz for the last time accessed support for the Hash Redis Store
+* Thanks to @alpaca-tc for the improvements on the route tracker
+* Thanks to @ydah for typo fixes, doc updates, adding ruby 3.3 to build matrix, improvements on CI, standardrb fixes
+* Thanks to @trivett, @khaled-badenjki, @IsabelleLePivain  for improved docs
+* Thanks to @prastamaha for the memcached adapter
+* Thanks to @ursm for a yaml fix
+* Thanks to @Drowze for a layered cache approach for perf improvements
+* Thanks to @vs37559 for a sinatra pandrino fix
+* This release addresses large projects and adds in paged reporting
+  * to ensure even on projects with 10K+ files it can load on heroku under the 30s timeout
+  * only supports HashRedis store
+  * faster UI for web UI in general which should be noticable on non paged reports
+  * reduce redis calls
+
 ### Coverband 6.0.2
 
 * thanks makicamel for improved deferred eager loading
@@ -11,16 +33,12 @@
 
 ### Coverband 6.0.0
 
-__NOTE: I ended up having 5.2.6 in various RCs for a long time, mostly because I had some breaking changes that were related to dropping support for old versions of Ruby and Rails__
-
 * The 6.0.0 release is all that was the 6 different RC releases of 5.2.6
 * Added Rails test matrix to github actions to test on all the supported versions
   - Rails: 6.0.x, 6.1.x, 7.0.x, 7.1.x
 
 ### Coverband 5.2.6
 
-__NOTE: the current RCs include below, but this might turn into coverband 6.0__
-
 - add support for translation keys
 - refactor non Coverage.so based trackers
 - adds CSP report support (thanks @jwg2s)

data/coverband.gemspec

@@ -36,6 +36,7 @@ Gem::Specification.new do |spec|
   spec.add_development_dependency "memory_profiler"
   # breaking change in minitest and mocha...
   # note: we are also adding 'spy' as mocha doesn't want us to spy on redis calls...
+  spec.add_development_dependency "spy"
   # ^^^ probably need a large test cleanup refactor
   spec.add_development_dependency "minitest", "= 5.18.1"
   spec.add_development_dependency "minitest-fork_executor"
@@ -45,8 +46,9 @@ Gem::Specification.new do |spec|
   spec.add_development_dependency "rack-test"
   spec.add_development_dependency "rake"
   spec.add_development_dependency "resque"
-  spec.add_development_dependency "standard", "~> 1.34.0"
-  spec.add_development_dependency "standardrb"
+  spec.add_development_dependency "standard", "= 1.34.0"
+  # breaking changes in various rubocop versions
+  spec.add_development_dependency "rubocop", "= 1.60.0"
 
   spec.add_development_dependency "coveralls"
   # minitest-profile is not compatible with Rails 7.1.0 setup... dropping it for now

data/lib/coverband/adapters/hash_redis_store.rb

@@ -116,8 +116,8 @@ module Coverband
 
       def supported?
         Gem::Version.new(@redis.info["redis_version"]) >= Gem::Version.new("2.6.0")
-      rescue Redis::CannotConnectError => error
-        Coverband.configuration.logger.info "Redis is not available (#{error}), Coverband not configured"
+      rescue Redis::CannotConnectError => e
+        Coverband.configuration.logger.info "Redis is not available (#{e}), Coverband not configured"
         Coverband.configuration.logger.info "If this is a setup task like assets:precompile feel free to ignore"
       end
 
@@ -149,7 +149,7 @@ module Coverband
         updated_time = (type == Coverband::EAGER_TYPE) ? nil : report_time
         keys = []
         report.each_slice(@save_report_batch_size) do |slice|
-          files_data = slice.map { |(file, data)|
+          files_data = slice.map do |(file, data)|
             relative_file = @relative_file_converter.convert(file)
             file_hash = file_hash(relative_file)
             key = key(relative_file, file_hash: file_hash)
@@ -162,12 +162,12 @@ module Coverband
               report_time: report_time,
               updated_time: updated_time
             )
-          }
+          end
           next unless files_data.any?
 
           arguments_key = [@redis_namespace, SecureRandom.uuid].compact.join(".")
           @redis.set(arguments_key, {ttl: @ttl, files_data: files_data}.to_json, ex: JSON_PAYLOAD_EXPIRATION)
-          @redis.evalsha(hash_incr_script, [arguments_key])
+          @redis.evalsha(hash_incr_script, [arguments_key], [report_time])
         end
         @redis.sadd(files_key, keys) if keys.any?
       end
@@ -182,7 +182,9 @@ module Coverband
           elsif opts[:filename]
             type_key_prefix = key_prefix(local_type)
             # NOTE: a better way to extract filename from key would be better
-            files_set(local_type).select { |cache_key| cache_key.sub(type_key_prefix, "").match(short_name(opts[:filename])) } || {}
+            files_set(local_type).select do |cache_key|
+              cache_key.sub(type_key_prefix, "").match(short_name(opts[:filename]))
+            end || {}
           else
             files_set(local_type)
           end
@@ -213,13 +215,13 @@ module Coverband
         end
       end
 
-      def coverage_for_types(types, opts = {})
+      def coverage_for_types(_types, opts = {})
         page_size = opts[:page_size] || 250
-
-        local_type = Coverband::RUNTIME_TYPE
         hash_data = {}
 
-        runtime_file_set = files_set(local_type).each_slice(page_size).to_a[opts[:page] - 1] || []
+        runtime_file_set = files_set(Coverband::RUNTIME_TYPE)
+        @cached_file_count = runtime_file_set.length
+        runtime_file_set = runtime_file_set.each_slice(page_size).to_a[opts[:page] - 1] || []
 
         hash_data[Coverband::RUNTIME_TYPE] = runtime_file_set.each_slice(page_size).flat_map do |key_batch|
           @redis.pipelined do |pipeline|
@@ -229,14 +231,14 @@ module Coverband
           end
         end
 
+        # NOTE: This is kind of hacky, we find all the matching eager loading data
+        # for current page of runtime data.
         eager_key_pre = key_prefix(Coverband::EAGER_TYPE)
         runtime_key_pre = key_prefix(Coverband::RUNTIME_TYPE)
-        matched_file_set = files_set(Coverband::EAGER_TYPE)
-          .select { |eager_key, val|
-            runtime_file_set.any? { |runtime_key|
-              (eager_key.sub(eager_key_pre, "") == runtime_key.sub(runtime_key_pre, ""))
-            }
-          } || []
+        matched_file_set = runtime_file_set.map do |runtime_key|
+          runtime_key.sub(runtime_key_pre, eager_key_pre)
+        end
+
         hash_data[Coverband::EAGER_TYPE] = matched_file_set.each_slice(page_size).flat_map do |key_batch|
           @redis.pipelined do |pipeline|
             key_batch.each do |key|
@@ -244,6 +246,7 @@ module Coverband
             end
           end
         end
+
         hash_data[Coverband::RUNTIME_TYPE] = hash_data[Coverband::RUNTIME_TYPE].each_with_object({}) do |data_from_redis, hash|
           add_coverage_for_file(data_from_redis, hash)
         end
@@ -255,13 +258,17 @@ module Coverband
 
       def short_name(filename)
         filename.sub(/^#{Coverband.configuration.root}/, ".")
-          .gsub(%r{^\.\/}, "")
+          .gsub(%r{^\./}, "")
       end
 
       def file_count(local_type = nil)
         files_set(local_type).count { |filename| !Coverband.configuration.ignore.any? { |i| filename.match(i) } }
       end
 
+      def cached_file_count
+        @cached_file_count ||= file_count(Coverband::RUNTIME_TYPE)
+      end
+
       def raw_store
         @redis
       end
@@ -283,9 +290,14 @@ module Coverband
         return unless file_hash(file) == data_from_redis[FILE_HASH]
 
         data = coverage_data_from_redis(data_from_redis)
-        hash[file] = data_from_redis.select { |meta_data_key, _value| META_DATA_KEYS.include?(meta_data_key) }.merge!("data" => data)
-        hash[file][LAST_UPDATED_KEY] = (hash[file][LAST_UPDATED_KEY].nil? || hash[file][LAST_UPDATED_KEY] == "") ? nil : hash[file][LAST_UPDATED_KEY].to_i
-        hash[file].merge!(LAST_UPDATED_KEY => hash[file][LAST_UPDATED_KEY], FIRST_UPDATED_KEY => hash[file][FIRST_UPDATED_KEY].to_i)
+        timedata = coverage_time_data_from_redis(data_from_redis)
+        hash[file] = data_from_redis.select do |meta_data_key, _value|
+          META_DATA_KEYS.include?(meta_data_key)
+        end.merge!("data" => data, "timedata" => timedata)
+        hash[file][LAST_UPDATED_KEY] =
+          (hash[file][LAST_UPDATED_KEY].nil? || hash[file][LAST_UPDATED_KEY] == "") ? nil : hash[file][LAST_UPDATED_KEY].to_i
+        hash[file].merge!(LAST_UPDATED_KEY => hash[file][LAST_UPDATED_KEY],
+          FIRST_UPDATED_KEY => hash[file][FIRST_UPDATED_KEY].to_i)
       end
 
       def coverage_data_from_redis(data_from_redis)
@@ -296,10 +308,18 @@ module Coverband
         end
       end
 
+      def coverage_time_data_from_redis(data_from_redis)
+        max = data_from_redis[FILE_LENGTH_KEY].to_i - 1
+        Array.new(max + 1) do |index|
+          unixtime = data_from_redis["#{index}_last_posted"]
+          unixtime.nil? ? nil : Time.at(unixtime.to_i)
+        end
+      end
+
       def script_input(key:, file:, file_hash:, data:, report_time:, updated_time:)
-        coverage_data = data.each_with_index.each_with_object({}) { |(coverage, index), hash|
+        coverage_data = data.each_with_index.each_with_object({}) do |(coverage, index), hash|
           hash[index] = coverage if coverage
-        }
+        end
         meta = {
           first_updated_at: report_time,
           file: file,

data/lib/coverband/adapters/memcached_store.rb

@@ -0,0 +1,89 @@
+# frozen_string_literal: true
+
+module Coverband
+  module Adapters
+    class MemcachedStore < Base
+      STORAGE_FORMAT_VERSION = "coverband_3_2"
+
+      attr_reader :memcached_namespace
+
+      def initialize(memcached, opts = {})
+        super()
+        @memcached = memcached
+        @memcached_namespace = opts[:memcached_namespace]
+        @format_version = STORAGE_FORMAT_VERSION
+        @keys = {}
+        Coverband::TYPES.each do |type|
+          @keys[type] = [@format_version, @memcached_namespace, type].compact.join(".")
+        end
+      end
+
+      def clear!
+        Coverband::TYPES.each do |type|
+          @memcached.delete(type_base_key(type))
+        end
+      end
+
+      def clear_file!(filename)
+        Coverband::TYPES.each do |type|
+          data = coverage(type)
+          data.delete(filename)
+          save_coverage(data, type)
+        end
+      end
+
+      def size
+        @memcached.read(base_key) ? @memcached.read(base_key).bytesize : "N/A"
+      end
+
+      def migrate!
+        raise NotImplementedError, "MemcachedStore doesn't support migrations"
+      end
+
+      def type=(type)
+        super
+        reset_base_key
+      end
+
+      def coverage(local_type = nil, opts = {})
+        local_type ||= opts.key?(:override_type) ? opts[:override_type] : type
+        data = memcached.read(type_base_key(local_type))
+        data = data ? JSON.parse(data) : {}
+        data.delete_if { |file_path, file_data| file_hash(file_path) != file_data["file_hash"] } unless opts[:skip_hash_check]
+        data
+      end
+
+      def save_report(report)
+        data = report.dup
+        data = merge_reports(data, coverage(nil, skip_hash_check: true))
+        save_coverage(data)
+      end
+
+      def raw_store
+        raise NotImplementedError, "MemcachedStore doesn't support raw_store"
+      end
+
+      attr_reader :memcached
+
+      private
+
+      def reset_base_key
+        @base_key = nil
+      end
+
+      def base_key
+        @base_key ||= [@format_version, @memcached_namespace, type].compact.join(".")
+      end
+
+      def type_base_key(local_type)
+        @keys[local_type]
+      end
+
+      def save_coverage(data, local_type = nil)
+        local_type ||= type
+        key = type_base_key(local_type)
+        memcached.write(key, data.to_json)
+      end
+    end
+  end
+end

data/lib/coverband/collectors/route_tracker.rb

@@ -25,7 +25,10 @@ module Coverband
       # and ensure high performance
       ###
       def track_key(payload)
-        route = if payload[:request]
+        route = if payload.key?(:location)
+          # For redirect.action_dispatch
+          return unless Coverband.configuration.track_redirect_routes
+
           {
             controller: nil,
             action: nil,
@@ -33,6 +36,7 @@ module Coverband
             verb: payload[:request].method
           }
         else
+          # For start_processing.action_controller
           {
             controller: payload[:params]["controller"],
             action: payload[:action],
@@ -40,11 +44,10 @@ module Coverband
             verb: payload[:method]
           }
         end
-        if route
-          if newly_seen_key?(route)
-            @logged_keys << route
-            @keys_to_record << route if track_key?(route)
-          end
+
+        if newly_seen_key?(route)
+          @logged_keys << route
+          @keys_to_record << route if track_key?(route)
         end
       end
 

data/lib/coverband/configuration.rb

@@ -1,6 +1,9 @@
 # frozen_string_literal: true
 
 module Coverband
+  ###
+  # Configuration parsing and options for the coverband gem.
+  ###
   class Configuration
     attr_accessor :root_paths, :root,
       :verbose,
@@ -9,9 +12,9 @@ module Coverband
       :test_env, :web_enable_clear, :gem_details, :web_debug, :report_on_exit,
       :simulate_oneshot_lines_coverage,
       :view_tracker, :defer_eager_loading_data,
-      :track_routes, :route_tracker,
+      :track_routes, :track_redirect_routes, :route_tracker,
       :track_translations, :translations_tracker,
-      :trackers, :csp_policy
+      :trackers, :csp_policy, :hide_settings
 
     attr_writer :logger, :s3_region, :s3_bucket, :s3_access_key_id,
       :s3_secret_access_key, :password, :api_key, :service_url, :coverband_timeout, :service_dev_mode,
@@ -74,6 +77,7 @@ module Coverband
       @track_views = true
       @view_tracker = nil
       @track_routes = false
+      @track_redirect_routes = true
       @route_tracker = nil
       @track_translations = false
       @translations_tracker = nil
@@ -86,6 +90,7 @@ module Coverband
       @all_root_patterns = nil
       @password = nil
       @csp_policy = false
+      @hide_settings = false
 
       # coverband service settings
       @api_key = nil
@@ -131,8 +136,8 @@ module Coverband
         trackers << Coverband.configuration.view_tracker
       end
       trackers.each { |tracker| tracker.railtie! }
-    rescue Redis::CannotConnectError => error
-      Coverband.configuration.logger.info "Redis is not available (#{error}), Coverband not configured"
+    rescue Redis::CannotConnectError => e
+      Coverband.configuration.logger.info "Redis is not available (#{e}), Coverband not configured"
       Coverband.configuration.logger.info "If this is a setup task like assets:precompile feel free to ignore"
     end
 
@@ -169,7 +174,10 @@ module Coverband
 
     def store
       @store ||= if service?
-        raise "invalid configuration: unclear default store coverband expects either api_key or redis_url" if ENV["COVERBAND_REDIS_URL"]
+        if ENV["COVERBAND_REDIS_URL"]
+          raise "invalid configuration: unclear default store coverband expects either api_key or redis_url"
+        end
+
         require "coverband/adapters/web_service_store"
         Coverband::Adapters::WebServiceStore.new(service_url)
       else
@@ -179,8 +187,14 @@ module Coverband
 
     def store=(store)
       raise "Pass in an instance of Coverband::Adapters" unless store.is_a?(Coverband::Adapters::Base)
-      raise "invalid configuration: only coverband service expects an API Key" if api_key && store.class.to_s != "Coverband::Adapters::WebServiceStore"
-      raise "invalid configuration: coverband service shouldn't have redis url set" if ENV["COVERBAND_REDIS_URL"] && store.instance_of?(::Coverband::Adapters::WebServiceStore)
+      if api_key && store.class.to_s != "Coverband::Adapters::WebServiceStore"
+        raise "invalid configuration: only coverband service expects an API Key"
+      end
+      if ENV["COVERBAND_REDIS_URL"] &&
+          defined?(::Coverband::Adapters::WebServiceStore) &&
+          store.instance_of?(::Coverband::Adapters::WebServiceStore)
+        raise "invalid configuration: coverband service shouldn't have redis url set"
+      end
 
       @store = store
     end
@@ -240,7 +254,10 @@ module Coverband
     end
 
     def use_oneshot_lines_coverage=(value)
-      raise(StandardError, "One shot line coverage is only available in ruby >= 2.6") unless one_shot_coverage_implemented_in_ruby_version? || !value
+      unless one_shot_coverage_implemented_in_ruby_version? || !value
+        raise(StandardError,
+          "One shot line coverage is only available in ruby >= 2.6")
+      end
 
       @use_oneshot_lines_coverage = value
     end
@@ -321,7 +338,7 @@ module Coverband
     end
 
     def track_gems=(_value)
-      puts "gem tracking is deprecated, setting this will be ignored"
+      puts "gem tracking is deprecated, setting this will be ignored & eventually removed"
     end
 
     private

data/lib/coverband/reporters/json_report.rb

@@ -72,7 +72,7 @@ module Coverband
               data[:covered_strength].to_s
             ]
           end
-          filesreported = store.file_count(:runtime)
+          filesreported = store.cached_file_count
           data["iTotalRecords"] = filesreported
           data["iTotalDisplayRecords"] = filesreported
           data["aaData"] = row_data

data/lib/coverband/reporters/web.rb

@@ -141,6 +141,7 @@ module Coverband
       end
 
       def settings
+        return "" if Coverband.configuration.hide_settings
         Coverband::Utils::HTMLFormatter.new(nil, base_path: base_path).format_settings!
       end
 

data/lib/coverband/utils/html_formatter.rb

@@ -155,11 +155,11 @@ module Coverband
         Digest::SHA1.hexdigest(source_file.filename)
       end
 
-      def timeago(time)
-        if time
+      def timeago(time, err_msg = "Not Available")
+        if time.respond_to?(:iso8601)
           "<abbr class=\"timeago\" title=\"#{time.iso8601}\">#{time.iso8601}</abbr>"
         else
-          "Not Available"
+          err_msg
         end
       end
 

data/lib/coverband/utils/source_file.rb

@@ -25,13 +25,15 @@ module Coverband
         attr_reader :coverage
         # Whether this line was skipped
         attr_reader :skipped
+        # The coverage data posted time for this line: either nil (never), nil (missed) or Time instance (last posted)
+        attr_reader :coverage_posted
 
         # Lets grab some fancy aliases, shall we?
         alias source src
         alias line line_number
         alias number line_number
 
-        def initialize(src, line_number, coverage)
+        def initialize(src, line_number, coverage, coverage_posted = nil)
           raise ArgumentError, "Only String accepted for source" unless src.is_a?(String)
           raise ArgumentError, "Only Integer accepted for line_number" unless line_number.is_a?(Integer)
           raise ArgumentError, "Only Integer and nil accepted for coverage" unless coverage.is_a?(Integer) || coverage.nil?
@@ -40,6 +42,7 @@ module Coverband
           @line_number = line_number
           @coverage = coverage
           @skipped = false
+          @coverage_posted = coverage_posted
         end
 
         # Returns true if this is a line that should have been covered, but was not
@@ -82,6 +85,8 @@ module Coverband
       attr_reader :filename
       # The array of coverage data received from the Coverage.result
       attr_reader :coverage
+      # The array of coverage timedata received from the Coverage.result
+      attr_reader :coverage_posted
 
       # the date this version of the file first started to record coverage
       attr_reader :first_updated_at
@@ -96,6 +101,7 @@ module Coverband
         @runtime_relavant_lines = nil
         if file_data.is_a?(Hash)
           @coverage = file_data["data"]
+          @coverage_posted = file_data["timedata"] || [] # NOTE: only implement timedata for HashRedisStore
           @first_updated_at = @last_updated_at = NOT_AVAILABLE
           @first_updated_at = Time.at(file_data["first_updated_at"]) if file_data["first_updated_at"]
           @last_updated_at = Time.at(file_data["last_updated_at"]) if file_data["last_updated_at"]
@@ -139,7 +145,12 @@ module Coverband
         coverage_exceeding_source_warn if coverage.size > src.size
 
         lines = src.map.with_index(1) { |src, i|
-          Coverband::Utils::SourceFile::Line.new(src, i, never_loaded ? 0 : coverage[i - 1])
+          Coverband::Utils::SourceFile::Line.new(
+            src,
+            i,
+            never_loaded ? 0 : coverage[i - 1],
+            (never_loaded || !coverage_posted.is_a?(Array)) ? nil : coverage_posted[i - 1]
+          )
         }
 
         process_skipped_lines(lines)
@@ -200,6 +211,10 @@ module Coverband
         lines[index]&.coverage
       end
 
+      def line_coverage_posted(index)
+        lines[index]&.coverage_posted
+      end
+
       # Returns all lines that should have been, but were not covered
       # as instances of SimpleCov::SourceFile::Line
       def missed_lines

data/lib/coverband/version.rb

@@ -5,5 +5,5 @@
 # use format "4.2.1.rc.1" ~> 4.2.1.rc to prerelease versions like v4.2.1.rc.2 and v4.2.1.rc.3
 ###
 module Coverband
-  VERSION = "6.0.3.rc.4"
+  VERSION = "6.1.1"
 end

data/lua/lib/persist-coverage.lua

@@ -21,6 +21,9 @@ for _, file_data in ipairs(files_data) do
   redis.call('HSETNX', hash_key, 'first_updated_at', first_updated_at)
   for line, coverage in pairs(file_data.coverage) do
     redis.call("HINCRBY", hash_key, line, coverage)
+    if coverage > 0 then
+      redis.call("HSET", hash_key, line .. "_last_posted", ARGV[1])
+    end
   end
   if ttl and ttl ~= cjson.null then
     redis.call("EXPIRE", hash_key, ttl)

data/public/application.js

@@ -39,15 +39,16 @@ $(document).ready(function() {
   // best docs on our version of datatables 1.7 https://datatables.net/beta/1.7/examples/server_side/server_side.html
   if ($(".file_list.unsorted").length == 1) {
     $(".dataTables_empty").html("loading...");
-    var current_rows = 0;
     var total_rows = 0;
     var page = 1;
+    var all_data = [];
     
     // load and render page content before we start the loop
     // perhaps move this into a datatable ready event
+    $(".dataTables_empty").html("loading...");
     setTimeout(() => {
       get_page(page);
-    }, 1250);
+    }, 1200);
 
     function get_page(page) {
       $.ajax({
@@ -56,23 +57,23 @@ $(document).ready(function() {
         dataType: 'json',
         success: function(data) {
           total_rows = data["iTotalRecords"];
-          // NOTE: we request 250 at a time, but we seem to have some files that we have as a list but 0 coverage,
-          // so we don't get back 250 per page... to ensure we we need to account for filtered out and empty files
-          // this 250 at the moment is synced to the 250 in the hash redis store
-          current_rows += 250; //data["aaData"].length;
-          $(".file_list.unsorted").dataTable().fnAddData(data["aaData"]);
+          all_data = all_data.concat(data["aaData"]);
+          $(".dataTables_empty").html("loading... on " + all_data.length + " of " + total_rows + " files");
           page += 1;
-          // allow rendering to complete before we click the anchor
-          setTimeout(() => {
-            if (window.auto_click_anchor && $(window.auto_click_anchor).length > 0) {
-              $(window.auto_click_anchor).click();
-            }
-          }, 50);
+;
           // the page less than 100 is to stop infinite loop in case of folks never clearing out old coverage reports
-          if (page < 100 && current_rows < total_rows) {
+          if (page < 50 && all_data.length < total_rows) {
             setTimeout(() => {
               get_page(page);
-            }, 200);
+            }, 10);
+          } else {
+            $(".file_list.unsorted").dataTable().fnAddData(all_data);
+            // allow rendering to complete before we click the anchor
+            setTimeout(() => {
+              if (window.auto_click_anchor && $(window.auto_click_anchor).length > 0) {
+                $(window.auto_click_anchor).click();
+              }
+            }, 50)
           }
         }
       });

data/public/dependencies.js

@@ -88,7 +88,7 @@ var hljs=new function(){function l(o){return o.replace(/&/gm,"&amp;").replace(/<
             xhrError: "This content failed to load.",
             imgError: "This image failed to load.",
 
-            // accessbility
+            // accessibility
             returnFocus: true,
             trapFocus: true,
 
@@ -103,7 +103,7 @@ var hljs=new function(){function l(o){return o.replace(/&/gm,"&amp;").replace(/<
                 return this.rel;
             },
             href: function() {
-                // using this.href would give the absolute url, when the href may have been intedned as a selector (e.g. '#container')
+                // using this.href would give the absolute url, when the href may have been intended as a selector (e.g. '#container')
                 return $(this).attr('href');
             },
             title: function() {
@@ -145,7 +145,7 @@ var hljs=new function(){function l(o){return o.replace(/&/gm,"&amp;").replace(/<
         $prev,
         $close,
         $groupControls,
-        $events = $('<a/>'), // $({}) would be prefered, but there is an issue with jQuery 1.4.2
+        $events = $('<a/>'), // $({}) would be preferred, but there is an issue with jQuery 1.4.2
 
     // Variables for cached values or use across multiple functions
         settings,

data/test/coverband/adapters/hash_redis_store_test.rb

@@ -57,7 +57,8 @@ class HashRedisStoreTest < Minitest::Test
         "first_updated_at" => yesterday.to_i,
         "last_updated_at" => yesterday.to_i,
         "file_hash" => "abcd",
-        "data" => [0, 1, 2]
+        "data" => [0, 1, 2],
+        "timedata" => [nil, Time.at(yesterday.to_i), Time.at(yesterday.to_i)]
       },
       @store.coverage["./dog.rb"]
     )
@@ -65,15 +66,12 @@ class HashRedisStoreTest < Minitest::Test
     @store.save_report(
       "app_path/dog.rb" => [1, 1, 0]
     )
-    assert_equal(
-      {
-        "first_updated_at" => yesterday.to_i,
-        "last_updated_at" => today.to_i,
-        "file_hash" => "abcd",
-        "data" => [1, 2, 2]
-      },
-      @store.coverage["./dog.rb"]
-    )
+
+    assert_equal("abcd", @store.coverage["./dog.rb"]["file_hash"])
+    assert_equal(today.to_i, @store.coverage["./dog.rb"]["last_updated_at"])
+    assert_equal(yesterday.to_i, @store.coverage["./dog.rb"]["first_updated_at"])
+    assert_equal([1, 2, 2], @store.coverage["./dog.rb"]["data"])
+    assert_equal([Time.at(today.to_i), Time.at(today.to_i), Time.at(yesterday.to_i)], @store.coverage["./dog.rb"]["timedata"])
   end
 
   def test_ttl_set
@@ -109,7 +107,8 @@ class HashRedisStoreTest < Minitest::Test
         "first_updated_at" => current_time.to_i,
         "last_updated_at" => current_time.to_i,
         "file_hash" => "abcd",
-        "data" => [0, nil, 1, 2]
+        "data" => [0, nil, 1, 2],
+        "timedata" => [nil, nil, Time.at(current_time.to_i), Time.at(current_time.to_i)]
       }, @store.coverage["./dog.rb"]
     )
     assert_equal [1, 2, 0, 1, 5], @store.coverage["./cat.rb"]["data"]
@@ -213,7 +212,8 @@ class HashRedisStoreTest < Minitest::Test
         "first_updated_at" => yesterday.to_i,
         "last_updated_at" => yesterday.to_i,
         "file_hash" => "abcd",
-        "data" => [0, 1, 2]
+        "data" => [0, 1, 2],
+        "timedata" => [nil, Time.at(yesterday.to_i), Time.at(yesterday.to_i)]
       },
       @store.coverage["./dog.rb"]
     )
@@ -225,7 +225,8 @@ class HashRedisStoreTest < Minitest::Test
         "first_updated_at" => yesterday.to_i,
         "last_updated_at" => yesterday.to_i,
         "file_hash" => "abcd",
-        "data" => [0, 1, 2]
+        "data" => [0, 1, 2],
+        "timedata" => [nil, Time.at(yesterday.to_i), Time.at(yesterday.to_i)]
       },
       @store.coverage["./dog.rb"]
     )
@@ -235,9 +236,58 @@ class HashRedisStoreTest < Minitest::Test
         "first_updated_at" => yesterday.to_i,
         "last_updated_at" => yesterday.to_i,
         "file_hash" => "abcd",
-        "data" => [0, 2, 4]
+        "data" => [0, 2, 4],
+        "timedata" => [nil, Time.at(yesterday.to_i), Time.at(yesterday.to_i)]
       },
       @store.coverage["./dog.rb"]
     )
   end
+
+  def test_split_coverage
+    @store = Coverband::Adapters::HashRedisStore.new(
+      @redis,
+      redis_namespace: "coverband_test",
+      relative_file_converter: MockRelativeFileConverter
+    )
+
+    mock_file_hash
+    yesterday = DateTime.now.prev_day.to_time
+    mock_time(yesterday)
+
+    @store.type = :eager_loading
+    data = {
+      "app_path/dog.rb" => [0, nil, 1]
+    }
+    @store.save_report(data)
+
+    @store.type = :runtime
+    @store.save_report(
+      "app_path/dog.rb" => [0, 1, 2]
+    )
+    redis_pipelined = Spy.on(@redis, :pipelined).and_call_through
+    assert_equal(
+      {
+        runtime: {
+          "./dog.rb" => {
+            "first_updated_at" => yesterday.to_i,
+            "last_updated_at" => yesterday.to_i,
+            "file_hash" => "abcd",
+            "data" => [0, 1, 2],
+            "timedata" => [nil, Time.at(yesterday.to_i), Time.at(yesterday.to_i)]
+          }
+        },
+        eager_loading: {
+          "./dog.rb" => {
+            "first_updated_at" => yesterday.to_i,
+            "last_updated_at" => nil,
+            "file_hash" => "abcd",
+            "data" => [0, nil, 1],
+            "timedata" => [nil, nil, Time.at(yesterday.to_i)]
+          }
+        }
+      },
+      @store.split_coverage([Coverband::RUNTIME_TYPE, Coverband::EAGER_TYPE], {}, {page: 1})
+    )
+    assert_equal 2, redis_pipelined.calls.count
+  end
 end

data/test/coverband/collectors/route_tracker_test.rb

@@ -33,14 +33,32 @@ class RouterTrackerTest < Minitest::Test
     tracker = Coverband::Collectors::RouteTracker.new(store: store, roots: "dir")
 
     payload = {
-      request: Payload.new("path", "GET")
+      request: Payload.new("path", "GET"),
+      status: 302,
+      location: "https://coverband.dev/"
     }
     tracker.track_key(payload)
     tracker.save_report
     assert_equal [route_hash], tracker.logged_keys
   end
 
-  test "track controller routes" do
+  test "track redirect routes when track_redirect_routes is false" do
+    Coverband.configuration.track_redirect_routes = false
+
+    store = fake_store
+    tracker = Coverband::Collectors::RouteTracker.new(store: store, roots: "dir")
+
+    payload = {
+      request: Payload.new("path", "GET"),
+      status: 302,
+      location: "https://coverband.dev/"
+    }
+    tracker.track_key(payload)
+    tracker.save_report
+    assert_equal [], tracker.logged_keys
+  end
+
+  test "track controller routes in Rails < 6.1" do
     store = fake_store
     route_hash = {controller: "some/controller", action: "index", url_path: nil, verb: "GET"}
     store.raw_store.expects(:hset).with(tracker_key, route_hash.to_s, anything)
@@ -57,6 +75,27 @@ class RouterTrackerTest < Minitest::Test
     assert_equal [route_hash], tracker.logged_keys
   end
 
+  test "track controller routes in Rails >= 6.1" do
+    store = fake_store
+    route_hash = {controller: "some/controller", action: "index", url_path: nil, verb: "GET"}
+    store.raw_store.expects(:hset).with(tracker_key, route_hash.to_s, anything)
+    tracker = Coverband::Collectors::RouteTracker.new(store: store, roots: "dir")
+    payload = {
+      params: {
+        "controller" => "some/controller",
+        "action" => "index"
+      },
+      controller: "SomeController",
+      action: "index",
+      path: "path",
+      method: "GET",
+      request: Payload.new("path", "GET")
+    }
+    tracker.track_key(payload)
+    tracker.save_report
+    assert_equal [route_hash], tracker.logged_keys
+  end
+
   test "report used routes" do
     store = fake_store
     route_hash = {controller: "some/controller", action: "index", url_path: nil, verb: "GET"}

data/test/test_helper.rb

@@ -35,6 +35,7 @@ require "coverband/utils/results"
 require "coverband/reporters/html_report"
 require "coverband/reporters/json_report"
 require "webmock/minitest"
+require "spy/integration"
 
 require_relative "unique_files"
 $VERBOSE = original_verbosity

data/views/source_file.erb

@@ -45,6 +45,7 @@
             runtime:
             <%= result.file_with_type(source_file, Coverband::RUNTIME_TYPE)&.line_coverage(index) || 0 %>
             all: <%= line.coverage %>
+            last posted: <%= timeago(result.file_with_type(source_file, Coverband::RUNTIME_TYPE)&.line_coverage_posted(index), "-") %>
           </span><% end %>
           <% if line.skipped? %><span class="hits">skipped</span><% end %>
           <code class="ruby"><%= CGI.escapeHTML(line.src.chomp) %></code>

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: coverband
 version: !ruby/object:Gem::Version
-  version: 6.0.3.rc.4
+  version: 6.1.1
 platform: ruby
 authors:
 - Dan Mayer
@@ -9,7 +9,7 @@ authors:
 autorequire: 
 bindir: bin
 cert_chain: []
-date: 2024-04-09 00:00:00.000000000 Z
+date: 2024-04-23 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: benchmark-ips
@@ -67,6 +67,20 @@ dependencies:
     - - ">="
       - !ruby/object:Gem::Version
         version: '0'
+- !ruby/object:Gem::Dependency
+  name: spy
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
 - !ruby/object:Gem::Dependency
   name: minitest
   requirement: !ruby/object:Gem::Requirement
@@ -183,30 +197,30 @@ dependencies:
   name: standard
   requirement: !ruby/object:Gem::Requirement
     requirements:
-    - - "~>"
+    - - '='
       - !ruby/object:Gem::Version
         version: 1.34.0
   type: :development
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
-    - - "~>"
+    - - '='
       - !ruby/object:Gem::Version
         version: 1.34.0
 - !ruby/object:Gem::Dependency
-  name: standardrb
+  name: rubocop
   requirement: !ruby/object:Gem::Requirement
     requirements:
-    - - ">="
+    - - '='
       - !ruby/object:Gem::Version
-        version: '0'
+        version: 1.60.0
   type: :development
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
-    - - ">="
+    - - '='
       - !ruby/object:Gem::Version
-        version: '0'
+        version: 1.60.0
 - !ruby/object:Gem::Dependency
   name: coveralls
   requirement: !ruby/object:Gem::Requirement
@@ -284,6 +298,7 @@ files:
 - lib/coverband/adapters/base.rb
 - lib/coverband/adapters/file_store.rb
 - lib/coverband/adapters/hash_redis_store.rb
+- lib/coverband/adapters/memcached_store.rb
 - lib/coverband/adapters/null_store.rb
 - lib/coverband/adapters/redis_store.rb
 - lib/coverband/adapters/stdout_store.rb
@@ -509,9 +524,9 @@ required_ruby_version: !ruby/object:Gem::Requirement
       version: '2.7'
 required_rubygems_version: !ruby/object:Gem::Requirement
   requirements:
-  - - ">"
+  - - ">="
     - !ruby/object:Gem::Version
-      version: 1.3.1
+      version: '0'
 requirements: []
 rubygems_version: 3.4.10
 signing_key: