coverband 6.0.1 → 6.0.2

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 036f9cdceb426dd675d4e6d78af901a6618dee5062fc5572b47ce90518ffbb1f
-  data.tar.gz: d1d1d8f6c1cf4123d34ae630cd92767d673e1e04e4ef0f18660327c481b309f0
+  metadata.gz: 0e834afa9139c4fc6b0e17a151476a7e9769f082ff435d93a58f3d1a8c44ff1a
+  data.tar.gz: 935efdfe633f5ccafc3d43d4d5237558b8032545526f0a7320b11ba64a04c7c1
 SHA512:
-  metadata.gz: a10c5a56f2ef1cb15e42f5f52f4636432fa0bb779276e44dd719b06474d51da7529b03334d8d0a383fe25b38af932bc0d32c67e3c527aaa568bd9ab35466eb9e
-  data.tar.gz: 9be31a11af5c7d490e2f965a8e91f1fc916668efa3ec384afc88f78225a880056d3b3ae5dc4e1193be87e63a0d2cb07f35c72c4268018cf01e357c7d9835d3ae
+  metadata.gz: de37c02bb694b58b25dad7b31f8fd9bbe09f0c963d779f5094e1db8db6e971b4df4501722657e00a7772a3e0a1622d9538e75b8e4625996ce4e60e3faadf27a3
+  data.tar.gz: 8ce7d73882bbc4e501dedde77f58358165e16143e59794d779e7568ec2b15bbbe0dc5f286f995808283c1c8616b7b03704137ec22a28392cff0df95c0fc97d35

data/README.md

@@ -179,11 +179,6 @@ Coverband.configure do |config|
   # default false. button at the top of the web interface which clears all data
   config.web_enable_clear = true
 
-  # default false. Experimental support for tracking view layer tracking.
-  # Does not track line-level usage, only indicates if an entire file
-  # is used or not.
-  config.track_views = true
-
   # default false. Experimental support for routes usage tracking.
   config.track_routes = true
 end
@@ -221,9 +216,9 @@ config.ignore +=  ['config/application.rb',
 
 Coverband allows an optional feature to track all view files that are used by an application.
 
-To opt-in to this feature... enable the feature in your Coverband config.
+This feature is enabled by default. To stop this feature, disable the feature in your Coverband config.
 
-`config.track_views = true`
+`config.track_views = false`
 
 ![image](https://raw.github.com/danmayer/coverband/master/docs/coverband_view_tracker.png)
 
@@ -252,12 +247,25 @@ end
 
 ### Avoiding Cache Stampede
 
-If you have many servers and they all hit Redis at the same time you can see spikes in your Redis CPU, and memory. This is due to a concept called [cache stampede](https://en.wikipedia.org/wiki/Cache_stampede). It is better to spread out the reporting across your servers. A simple way to do this is to add a random wiggle on your background reporting. This configuration option allows a wiggle. The right amount of wiggle depends on the number of servers you have and how willing you are to have delays in your coverage reporting. I would recommend at least 1 second per server. Note, the default wiggle is set to 30 seconds.
+If you have many servers and they all hit Redis at the same time you can see spikes in your Redis CPU, and memory. This is due to a concept called [cache stampede](https://en.wikipedia.org/wiki/Cache_stampede).
+
+It is better to spread out the reporting across your servers. A simple way to do this is to add a random wiggle on your background reporting. This configuration option allows a wiggle. The right amount of wiggle depends on the number of servers you have and how willing you are to have delays in your coverage reporting. I would recommend at least 1 second per server. Note, the default wiggle is set to 30 seconds.
 
 Add a wiggle (in seconds) to the background thread to avoid all your servers reporting at the same time:
 
 `config.reporting_wiggle = 30`
 
+Another way to avoid cache stampede is to omit some reporting on starting servers. Coverband stores the results of eager_loading to Redis at server startup. The eager_loading results are the same for all servers, so there is no need to save all results. By configuring the eager_loading results of some servers to be stored in Redis, we can reduce the load on Redis during deployment.
+
+```ruby
+# To omit reporting on starting servers, need to defer saving eager_loading data
+config.defer_eager_loading_data = true
+# Store eager_loading data on 5% of servers
+config.send_deferred_eager_loading_data = rand(100) < 5
+# Store eager_loading data on servers with the environment variable
+config.send_deferred_eager_loading_data = ENV.fetch('ENABLE_EAGER_LOADING_COVERAGE', false)
+```
+
 ### Redis Hash Store
 
 Coverband on very high volume sites with many server processes reporting can have a race condition which can cause hit counts to be inaccurate. To resolve the race condition and reduce Ruby memory overhead we have introduced a new Redis storage option. This moves the some of the work from the Ruby processes to Redis. It is worth noting because of this, it has larger demands on the Redis server. So adjust your Redis instance accordingly. To help reduce the extra redis load you can also change the background reporting frequency.
@@ -267,6 +275,11 @@ Coverband on very high volume sites with many server processes reporting can hav
 
 See more discussion [here](https://github.com/danmayer/coverband/issues/384).
 
+Please note that with the Redis Hash Store, everytime you load the full report, Coverband will execute `HGETALL` queries in your Redis server twice for every file in the project (once for runtime coverage and once for eager loading coverage). This shouldn't have a big impact in small to medium projects, but can be quite a hassle if your project has a few thousand files.
+To help reduce the extra redis load when getting the coverage report, you can enable `get_coverage_cache` (but note that when doing that, you will always get a previous version of the report, while a cache is re-populated with a newer version).
+
+- Use Hash Redis Store with _get coverage cache_: `config.store = Coverband::Adapters::HashRedisStore.new(redis, get_coverage_cache: true)`
+
 ### Clear Coverage
 
 Now that Coverband uses MD5 hashes there should be no reason to manually clear coverage unless one is testing, changing versions, or possibly debugging Coverband itself.

data/changes.md

@@ -1,3 +1,14 @@
+### Coverband 6.0.2
+
+* thanks makicamel for improved deferred eager loading
+* thanks Drowze for two performance improvements to reporting coverage
+
+
+### Coverband 6.0.1
+
+* [fix on reload routes](https://github.com/danmayer/coverband/commit/f7a81c9499c01a7c027e5f8bc127815bf29a5cb7)
+
+
 ### Coverband 6.0.0
 
 __NOTE: I ended up having 5.2.6 in various RCs for a long time, mostly because I had some breaking changes that were related to dropping support for old versions of Ruby and Rails__

data/lib/coverband/adapters/hash_redis_store.rb

@@ -5,6 +5,84 @@ require "securerandom"
 module Coverband
   module Adapters
     class HashRedisStore < Base
+      class GetCoverageNullCacheStore
+        def self.clear!(*_local_types)
+        end
+
+        def self.fetch(_local_type)
+          yield(0)
+        end
+      end
+
+      class GetCoverageRedisCacheStore
+        LOCK_LIMIT = 60 * 30 # 30 minutes
+
+        def initialize(redis, key_prefix)
+          @redis = redis
+          @key_prefix = [key_prefix, "get-coverage"].join(".")
+        end
+
+        def fetch(local_type)
+          cached_result = get(local_type)
+
+          # if no cache available, block the call and populate the cache
+          # if cache is available, return it and start re-populating it (with a lock)
+          if cached_result.nil?
+            value = yield(0)
+            result = set(local_type, JSON.generate(value))
+            value
+          else
+            if lock!(local_type)
+              Thread.new do
+                begin
+                  result = yield(deferred_time)
+                  set(local_type, JSON.generate(result))
+                ensure
+                  unlock!(local_type)
+                end
+              end
+            end
+            JSON.parse(cached_result)
+          end
+        end
+
+        def clear!(local_types = Coverband::TYPES)
+          Array(local_types).each do |local_type|
+            del(local_type)
+            unlock!(local_type)
+          end
+        end
+
+        private
+
+        # sleep in between to avoid holding other redis commands..
+        # with a small random offset so runtime and eager types can be processed "at the same time"
+        def deferred_time
+          rand(3.0..4.0)
+        end
+
+        def del(local_type)
+          @redis.del("#{@key_prefix}.cache.#{local_type}")
+        end
+
+        def get(local_type)
+          @redis.get("#{@key_prefix}.cache.#{local_type}")
+        end
+
+        def set(local_type, value)
+          @redis.set("#{@key_prefix}.cache.#{local_type}", value)
+        end
+
+        # lock for at most 60 minutes
+        def lock!(local_type)
+          @redis.set("#{@key_prefix}.lock.#{local_type}", "1", nx: true, ex: LOCK_LIMIT)
+        end
+
+        def unlock!(local_type)
+          @redis.del("#{@key_prefix}.lock.#{local_type}")
+        end
+      end
+
       FILE_KEY = "file"
       FILE_LENGTH_KEY = "file_length"
       META_DATA_KEYS = [DATA_KEY, FIRST_UPDATED_KEY, LAST_UPDATED_KEY, FILE_HASH].freeze
@@ -17,7 +95,7 @@ module Coverband
 
       JSON_PAYLOAD_EXPIRATION = 5 * 60
 
-      attr_reader :redis_namespace
+      attr_reader :redis_namespace, :get_coverage_cache
 
       def initialize(redis, opts = {})
         super()
@@ -29,6 +107,13 @@ module Coverband
 
         @ttl = opts[:ttl]
         @relative_file_converter = opts[:relative_file_converter] || Utils::RelativeFileConverter
+
+        @get_coverage_cache = if opts[:get_coverage_cache]
+          key_prefix = [REDIS_STORAGE_FORMAT_VERSION, @redis_namespace].compact.join(".")
+          GetCoverageRedisCacheStore.new(redis, key_prefix)
+        else
+          GetCoverageNullCacheStore
+        end
       end
 
       def supported?
@@ -45,6 +130,7 @@ module Coverband
           file_keys = files_set
           @redis.del(*file_keys) if file_keys.any?
           @redis.del(files_key)
+          @get_coverage_cache.clear!(type)
         end
         self.type = old_type
       end
@@ -54,6 +140,7 @@ module Coverband
         relative_path_file = @relative_file_converter.convert(file)
         Coverband::TYPES.each do |type|
           @redis.del(key(relative_path_file, type, file_hash: file_hash))
+          @get_coverage_cache.clear!(type)
         end
         @redis.srem(files_key, relative_path_file)
       end
@@ -87,12 +174,21 @@ module Coverband
       end
 
       def coverage(local_type = nil)
-        files_set = files_set(local_type)
-        @redis.pipelined { |pipeline|
-          files_set.each do |key|
-            pipeline.hgetall(key)
+        cached_results = @get_coverage_cache.fetch(local_type || type) do |sleep_time|
+          files_set = files_set(local_type)
+
+          # use batches with a sleep in between to avoid overloading redis
+          files_set.each_slice(250).flat_map do |key_batch|
+            sleep sleep_time
+            @redis.pipelined do |pipeline|
+              key_batch.each do |key|
+                pipeline.hgetall(key)
+              end
+            end
           end
-        }.each_with_object({}) do |data_from_redis, hash|
+        end
+
+        cached_results.each_with_object({}) do |data_from_redis, hash|
           add_coverage_for_file(data_from_redis, hash)
         end
       end

data/lib/coverband/collectors/coverage.rb

@@ -64,7 +64,7 @@ module Coverband
           else
             if @deferred_eager_loading_data && Coverband.configuration.defer_eager_loading_data?
               toggle_eager_loading do
-                @store.save_report(@deferred_eager_loading_data)
+                @store.save_report(@deferred_eager_loading_data) if Coverband.configuration.send_deferred_eager_loading_data?
                 @deferred_eager_loading_data = nil
               end
             end

data/lib/coverband/configuration.rb

@@ -16,7 +16,8 @@ module Coverband
     attr_writer :logger, :s3_region, :s3_bucket, :s3_access_key_id,
       :s3_secret_access_key, :password, :api_key, :service_url, :coverband_timeout, :service_dev_mode,
       :service_test_mode, :process_type, :track_views, :redis_url,
-      :background_reporting_sleep_seconds, :reporting_wiggle
+      :background_reporting_sleep_seconds, :reporting_wiggle,
+      :send_deferred_eager_loading_data
 
     attr_reader :track_gems, :ignore, :use_oneshot_lines_coverage
 
@@ -67,6 +68,7 @@ module Coverband
       @background_reporting_enabled = true
       @background_reporting_sleep_seconds = nil
       @defer_eager_loading_data = false
+      @send_deferred_eager_loading_data = true
       @test_env = nil
       @web_enable_clear = false
       @track_views = true
@@ -287,6 +289,10 @@ module Coverband
       @defer_eager_loading_data
     end
 
+    def send_deferred_eager_loading_data?
+      @send_deferred_eager_loading_data
+    end
+
     def service_disabled_dev_test_env?
       return false unless service?
 

data/lib/coverband/utils/results.rb

@@ -17,7 +17,11 @@ module Coverband
       def file_with_type(source_file, results_type)
         return unless get_results(results_type)
 
-        get_results(results_type).source_files.find { |file| file.filename == source_file.filename }
+        @files_with_type ||= {}
+        @files_with_type[results_type] ||= get_results(results_type).source_files.map do |source_file|
+          [source_file.filename, source_file]
+        end.to_h
+        @files_with_type[results_type][source_file.filename]
       end
 
       def runtime_relevant_coverage(source_file)
@@ -48,7 +52,11 @@ module Coverband
       def file_from_path_with_type(full_path, results_type = :merged)
         return unless get_results(results_type)
 
-        get_results(results_type).source_files.find { |file| file.filename == full_path }
+        @files_from_path_with_type ||= {}
+        @files_from_path_with_type[results_type] ||= get_results(results_type).source_files.map do |source_file|
+          [source_file.filename, source_file]
+        end.to_h
+        @files_from_path_with_type[results_type][full_path]
       end
 
       def method_missing(method, *args)
@@ -70,11 +78,11 @@ module Coverband
       private
 
       def get_eager_file(source_file)
-        eager_loading_coverage.source_files.find { |file| file.filename == source_file.filename }
+        file_with_type(source_file, Coverband::EAGER_TYPE)
       end
 
       def get_runtime_file(source_file)
-        runtime_coverage.source_files.find { |file| file.filename == source_file.filename }
+        file_with_type(source_file, Coverband::RUNTIME_TYPE)
       end
 
       def eager_loading_coverage
@@ -93,11 +101,7 @@ module Coverband
       def get_results(type)
         return nil unless Coverband::ALL_TYPES.include?(type)
 
-        if @results.key?(type)
-          @results[type]
-        else
-          @results[type] = Coverband::Utils::Result.new(report[type])
-        end
+        @results[type] ||= Coverband::Utils::Result.new(report[type])
       end
     end
   end

data/lib/coverband/version.rb

@@ -5,5 +5,5 @@
 # use format "4.2.1.rc.1" ~> 4.2.1.rc to prerelease versions like v4.2.1.rc.2 and v4.2.1.rc.3
 ###
 module Coverband
-  VERSION = "6.0.1"
+  VERSION = "6.0.2"
 end

data/test/coverband/adapters/hash_redis_store_test.rb

@@ -192,4 +192,52 @@ class HashRedisStoreTest < Minitest::Test
     @store.clear_file!("app_path/dog.rb")
     assert_nil @store.get_coverage_report[:merged]["./dog.rb"]
   end
+
+  def test_get_coverage_cache
+    @store = Coverband::Adapters::HashRedisStore.new(
+      @redis,
+      redis_namespace: "coverband_test",
+      relative_file_converter: MockRelativeFileConverter,
+      get_coverage_cache: true
+    )
+    @store.get_coverage_cache.stubs(:deferred_time).returns(0)
+    @store.get_coverage_cache.clear!
+    mock_file_hash
+    yesterday = DateTime.now.prev_day.to_time
+    mock_time(yesterday)
+    @store.save_report(
+      "app_path/dog.rb" => [0, 1, 2]
+    )
+    assert_equal(
+      {
+        "first_updated_at" => yesterday.to_i,
+        "last_updated_at" => yesterday.to_i,
+        "file_hash" => "abcd",
+        "data" => [0, 1, 2]
+      },
+      @store.coverage["./dog.rb"]
+    )
+    @store.save_report(
+      "app_path/dog.rb" => [0, 1, 2]
+    )
+    assert_equal(
+      {
+        "first_updated_at" => yesterday.to_i,
+        "last_updated_at" => yesterday.to_i,
+        "file_hash" => "abcd",
+        "data" => [0, 1, 2]
+      },
+      @store.coverage["./dog.rb"]
+    )
+    sleep 0.1 # wait caching thread finish
+    assert_equal(
+      {
+        "first_updated_at" => yesterday.to_i,
+        "last_updated_at" => yesterday.to_i,
+        "file_hash" => "abcd",
+        "data" => [0, 2, 4]
+      },
+      @store.coverage["./dog.rb"]
+    )
+  end
 end

data/test/integration/full_stack_send_deferred_eager_test.rb

@@ -0,0 +1,52 @@
+# frozen_string_literal: true
+
+require File.expand_path("../test_helper", File.dirname(__FILE__))
+require "rack"
+
+class FullStackSendDeferredEagerTest < Minitest::Test
+  REDIS_STORAGE_FORMAT_VERSION = Coverband::Adapters::RedisStore::REDIS_STORAGE_FORMAT_VERSION
+  TEST_RACK_APP = "../fake_app/basic_rack.rb"
+
+  def setup
+    super
+    Coverband::Collectors::Coverage.instance.reset_instance
+    Coverband.configure do |config|
+      config.background_reporting_enabled = false
+      config.track_gems = true
+      config.defer_eager_loading_data = true
+      config.send_deferred_eager_loading_data = false
+    end
+    Coverband.start
+    Coverband::Collectors::Coverage.instance.eager_loading!
+    @rack_file = require_unique_file "fake_app/basic_rack.rb"
+    Coverband.report_coverage
+    Coverband::Collectors::Coverage.instance.runtime!
+  end
+
+  test "call app" do
+    # eager loaded class coverage starts empty
+    Coverband.eager_loading_coverage!
+    expected = {}
+    assert_equal expected, Coverband.configuration.store.coverage
+
+    Coverband::Collectors::Coverage.instance.runtime!
+    request = Rack::MockRequest.env_for("/anything.json")
+    middleware = Coverband::BackgroundMiddleware.new(fake_app_with_lines)
+    results = middleware.call(request)
+    assert_equal "Hello Rack!", results.last
+    Coverband.report_coverage
+    expected = [nil, nil, 0, nil, 0, 0, 1, nil, nil]
+    assert_equal expected, Coverband.configuration.store.coverage[@rack_file]["data"]
+
+    # eager loaded class coverage is skipped at first normal coverage report
+    Coverband.eager_loading_coverage!
+    expected = {}
+    assert_equal expected, Coverband.configuration.store.coverage
+  end
+
+  private
+
+  def fake_app_with_lines
+    @fake_app_with_lines ||= ::HelloWorld.new
+  end
+end

metadata

@@ -1,15 +1,15 @@
 --- !ruby/object:Gem::Specification
 name: coverband
 version: !ruby/object:Gem::Version
-  version: 6.0.1
+  version: 6.0.2
 platform: ruby
 authors:
 - Dan Mayer
 - Karl Baum
-autorequire:
+autorequire: 
 bindir: bin
 cert_chain: []
-date: 2023-11-14 00:00:00.000000000 Z
+date: 2024-01-24 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: benchmark-ips
@@ -418,6 +418,7 @@ files:
 - test/forked/rails_route_tracker_stack_test.rb
 - test/forked/rails_view_tracker_stack_test.rb
 - test/integration/full_stack_deferred_eager_test.rb
+- test/integration/full_stack_send_deferred_eager_test.rb
 - test/integration/full_stack_test.rb
 - test/jruby_check.rb
 - test/rails4_dummy/Rakefile
@@ -498,7 +499,7 @@ metadata:
   documentation_uri: https://github.com/danmayer/coverband
   changelog_uri: https://github.com/danmayer/coverband/blob/main/changes.md
   source_code_uri: https://github.com/danmayer/coverband
-post_install_message:
+post_install_message: 
 rdoc_options: []
 require_paths:
 - lib
@@ -513,8 +514,8 @@ required_rubygems_version: !ruby/object:Gem::Requirement
     - !ruby/object:Gem::Version
       version: '0'
 requirements: []
-rubygems_version: 3.2.32
-signing_key:
+rubygems_version: 3.4.10
+signing_key: 
 specification_version: 4
 summary: Rack middleware to measure production code usage (LOC runtime usage)
 test_files:
@@ -579,6 +580,7 @@ test_files:
 - test/forked/rails_route_tracker_stack_test.rb
 - test/forked/rails_view_tracker_stack_test.rb
 - test/integration/full_stack_deferred_eager_test.rb
+- test/integration/full_stack_send_deferred_eager_test.rb
 - test/integration/full_stack_test.rb
 - test/jruby_check.rb
 - test/rails4_dummy/Rakefile