simplecov 0.17.1 → 0.21.2

data/doc/alternate-formatters.md

@@ -1,7 +1,7 @@
 ## Alternate coverage report formatters
 
 The community around simplecov provides a whole bunch of alternate formatters beyond the official
-[simplecov-html](https://github.com/colszowka/simplecov-html) gem.
+[simplecov-html](https://github.com/simplecov-ruby/simplecov-html) gem.
 
 If you have built or found one that is missing here, please send a Pull Request for this document!
 
@@ -15,6 +15,11 @@ A formatter that generates a coverage badge for use in your project's readme usi
 
 A formatter that generates a small coverage badge for use in your project's readme using the SVG.
 
+#### [simplecov_badger](https://github.com/traels-it/simplecov_badger)
+*by traels.it*
+
+A formatter that uploads your coverage to a server, that will then host a SVG badge of the score. No need to have the badge stored in repository.
+
 #### [simplecov-cobertura](https://github.com/dashingrocket/simplecov-cobertura)
 *by Jesse Bowes*
 
@@ -54,3 +59,13 @@ A formatter that prints the coverage of the file under test when you run a singl
 *by [Yosuke Kabuto](https://github.com/ysksn)*
 
 t_wada AA formatter for SimpleCov
+
+#### [simplecov-tailwindcss](https://github.com/chiefpansancolt/simplecov-tailwindcss)
+*by [Chiefpansancolt](https://github.com/chiefpansancolt)*
+
+A TailwindCSS & TailwindUI Designed HTML formatter with clean and easy search of files with a tabular left Navigation.
+
+#### [simplecov-material](https://github.com/chiefpansancolt/simplecov-material)
+*by [Chiefpansancolt](https://github.com/chiefpansancolt)*
+
+A Material Designed HTML formatter with clean and easy search of files with a tabular left Navigation.

data/doc/commercial-services.md

@@ -4,6 +4,11 @@ There is a bunch of services that offer integration with your existing CI pipeli
 reports. Please note these are not associated with the SimpleCov project itself, so please report problems with
 these integrations with their respective owners.
 
+#### [Codacy](https://github.com/codacy/codacy-coverage-reporter)
+*by [Codacy](https://www.codacy.com/)*
+
+Upload coverage reports to [codacy.com](https://www.codacy.com/), a hosted (or self-hosted) software quality analysis platform that also includes coverage reporting.
+
 #### [codeclimate](https://github.com/codeclimate/ruby-test-reporter)
 *by [Code Climate](https://codeclimate.com/)*
 

data/lib/minitest/simplecov_plugin.rb

@@ -0,0 +1,15 @@
+# frozen_string_literal: true
+
+# How minitest plugins. See https://github.com/simplecov-ruby/simplecov/pull/756 for why we need this.
+# https://github.com/seattlerb/minitest#writing-extensions
+module Minitest
+  def self.plugin_simplecov_init(_options)
+    if defined?(SimpleCov)
+      SimpleCov.external_at_exit = true
+
+      Minitest.after_run do
+        SimpleCov.at_exit_behavior
+      end
+    end
+  end
+end

data/lib/simplecov.rb

@@ -2,29 +2,31 @@
 
 require "English"
 
-#
-# Code coverage for ruby 1.9. Please check out README for a full introduction.
-#
 # Coverage may be inaccurate under JRUBY.
-if defined?(JRUBY_VERSION) && defined?(JRuby)
+if defined?(JRUBY_VERSION) && defined?(JRuby) && !org.jruby.RubyInstanceConfig.FULL_TRACE_ENABLED
 
   # @see https://github.com/jruby/jruby/issues/1196
   # @see https://github.com/metricfu/metric_fu/pull/226
-  # @see https://github.com/colszowka/simplecov/issues/420
-  # @see https://github.com/colszowka/simplecov/issues/86
+  # @see https://github.com/simplecov-ruby/simplecov/issues/420
+  # @see https://github.com/simplecov-ruby/simplecov/issues/86
   # @see https://jira.codehaus.org/browse/JRUBY-6106
 
-  unless org.jruby.RubyInstanceConfig.FULL_TRACE_ENABLED
-    warn 'Coverage may be inaccurate; set the "--debug" command line option,' \
-      ' or do JRUBY_OPTS="--debug"' \
-      ' or set the "debug.fullTrace=true" option in your .jrubyrc'
-  end
+  warn 'Coverage may be inaccurate; set the "--debug" command line option,' \
+    ' or do JRUBY_OPTS="--debug"' \
+    ' or set the "debug.fullTrace=true" option in your .jrubyrc'
 end
+
+#
+# Code coverage for ruby. Please check out README for a full introduction.
+#
 module SimpleCov
   class << self
-    attr_accessor :running
-    attr_accessor :pid
-    attr_reader :exit_exception
+    attr_accessor :running, :pid
+
+    # Basically, should we take care of at_exit behavior or something else?
+    # Used by the minitest plugin. See lib/minitest/simplecov_plugin.rb
+    attr_accessor :external_at_exit
+    alias external_at_exit? external_at_exit
 
     #
     # Sets up SimpleCov to run against your project.
@@ -44,35 +46,52 @@ module SimpleCov
     # Please check out the RDoc for SimpleCov::Configuration to find about available config options
     #
     def start(profile = nil, &block)
-      if SimpleCov.usable?
-        load_profile(profile) if profile
-        configure(&block) if block_given?
-        @result = nil
-        self.running = true
-        self.pid = Process.pid
-        Coverage.start
-      else
-        warn "WARNING: SimpleCov is activated, but you're not running Ruby 1.9+ - no coverage analysis will happen"
-        warn "Starting with SimpleCov 1.0.0, even no-op compatibility with Ruby <= 1.8 will be entirely dropped."
-        false
-      end
+      require "coverage"
+      initial_setup(profile, &block)
+      require_relative "./simplecov/process" if SimpleCov.enabled_for_subprocesses? &&
+                                                ::Process.respond_to?(:fork)
+
+      make_parallel_tests_available
+
+      @result = nil
+      self.pid = Process.pid
+
+      start_coverage_measurement
     end
 
     #
-    # Finds files that were to be tracked but were not loaded and initializes
-    # the line-by-line coverage to zero (if relevant) or nil (comments / whitespace etc).
+    # Collate a series of SimpleCov result files into a single SimpleCov output.
     #
-    def add_not_loaded_files(result)
-      if tracked_files
-        result = result.dup
-        Dir[tracked_files].each do |file|
-          absolute = File.expand_path(file)
+    # You can optionally specify configuration with a block:
+    #   SimpleCov.collate Dir["simplecov-resultset-*/.resultset.json"]
+    #    OR
+    #   SimpleCov.collate Dir["simplecov-resultset-*/.resultset.json"], 'rails' # using rails profile
+    #    OR
+    #   SimpleCov.collate Dir["simplecov-resultset-*/.resultset.json"] do
+    #     add_filter 'test'
+    #   end
+    #    OR
+    #   SimpleCov.collate Dir["simplecov-resultset-*/.resultset.json"], 'rails' do
+    #     add_filter 'test'
+    #   end
+    #
+    # Please check out the RDoc for SimpleCov::Configuration to find about
+    # available config options, or checkout the README for more in-depth
+    # information about coverage collation
+    #
+    # By default `collate` ignores the merge_timeout so all results of all files specified will be
+    # merged together. If you want to honor the merge_timeout then provide the keyword argument
+    # `ignore_timeout: false`.
+    #
+    def collate(result_filenames, profile = nil, ignore_timeout: true, &block)
+      raise "There are no reports to be merged" if result_filenames.empty?
 
-          result[absolute] ||= LinesClassifier.new.classify(File.foreach(absolute))
-        end
-      end
+      initial_setup(profile, &block)
 
-      result
+      # Use the ResultMerger to produce a single, merged result, ready to use.
+      @result = ResultMerger.merge_and_store(*result_filenames, ignore_timeout: ignore_timeout)
+
+      run_exit_tasks!
     end
 
     #
@@ -83,9 +102,7 @@ module SimpleCov
       return @result if result?
 
       # Collect our coverage result
-      if running
-        @result = SimpleCov::Result.new add_not_loaded_files(Coverage.result)
-      end
+      process_coverage_result if running
 
       # If we're using merging of results, store the current result
       # first (if there is one), then merge the results and return those
@@ -147,22 +164,6 @@ module SimpleCov
       load_profile(name)
     end
 
-    #
-    # Checks whether we're on a proper version of Ruby (likely 1.9+) which
-    # provides coverage support
-    #
-    def usable?
-      return @usable if defined?(@usable) && !@usable.nil?
-
-      @usable = begin
-        require "coverage"
-        require "simplecov/jruby_fix"
-        true
-      rescue LoadError
-        false
-      end
-    end
-
     #
     # Clear out the previously cached .result. Primarily useful in testing
     #
@@ -170,44 +171,71 @@ module SimpleCov
       @result = nil
     end
 
+    def at_exit_behavior
+      # If we are in a different process than called start, don't interfere.
+      return if SimpleCov.pid != Process.pid
+
+      # If SimpleCov is no longer running then don't run exit tasks
+      SimpleCov.run_exit_tasks! if SimpleCov.running
+    end
+
+    # @api private
     #
-    # Capture the current exception if it exists
-    # This will get called inside the at_exit block
+    # Called from at_exit block
     #
-    def set_exit_exception
-      @exit_exception = $ERROR_INFO
+    def run_exit_tasks!
+      error_exit_status = exit_status_from_exception
+
+      at_exit.call
+
+      exit_and_report_previous_error(error_exit_status) if previous_error?(error_exit_status)
+      process_results_and_report_error if ready_to_process_results?
     end
 
+    #
+    # @api private
     #
     # Returns the exit status from the exit exception
     #
     def exit_status_from_exception
-      return SimpleCov::ExitCodes::SUCCESS unless exit_exception
+      # Capture the current exception if it exists
+      @exit_exception = $ERROR_INFO
+      return nil unless @exit_exception
 
-      if exit_exception.is_a?(SystemExit)
-        exit_exception.status
+      if @exit_exception.is_a?(SystemExit)
+        @exit_exception.status
       else
         SimpleCov::ExitCodes::EXCEPTION
       end
     end
 
     # @api private
+    def previous_error?(error_exit_status)
+      # Normally it'd be enough to check for previous error but when running test_unit
+      # status is 0
+      error_exit_status && error_exit_status != SimpleCov::ExitCodes::SUCCESS
+    end
+
     #
-    # Called from at_exit block
+    # @api private
     #
-    def run_exit_tasks!
-      exit_status = SimpleCov.exit_status_from_exception
+    # Thinking: Move this behavior earlier so if there was an error we do nothing?
+    def exit_and_report_previous_error(exit_status)
+      warn("Stopped processing SimpleCov as a previous error not related to SimpleCov has been detected") if print_error_status
+      Kernel.exit(exit_status)
+    end
 
-      SimpleCov.at_exit.call
+    # @api private
+    def ready_to_process_results?
+      final_result_process? && result?
+    end
 
-      # Don't modify the exit status unless the result has already been
-      # computed
-      exit_status = SimpleCov.process_result(SimpleCov.result, exit_status) if SimpleCov.result?
+    def process_results_and_report_error
+      exit_status = process_result(result)
 
       # Force exit with stored status (see github issue #5)
-      # unless it's nil or 0 (see github issue #281)
-      if exit_status && exit_status > 0
-        $stderr.printf("SimpleCov failed with exit %d", exit_status)
+      if exit_status.positive?
+        warn("SimpleCov failed with exit #{exit_status} due to a coverage related error") if print_error_status
         Kernel.exit exit_status
       end
     end
@@ -217,48 +245,29 @@ module SimpleCov
     # Usage:
     #   exit_status = SimpleCov.process_result(SimpleCov.result, exit_status)
     #
-    def process_result(result, exit_status)
-      return exit_status if exit_status != SimpleCov::ExitCodes::SUCCESS # Existing errors
-
-      covered_percent = result.covered_percent.round(2)
-      result_exit_status = result_exit_status(result, covered_percent)
-      if result_exit_status == SimpleCov::ExitCodes::SUCCESS # No result errors
-        write_last_run(covered_percent)
-      end
-      final_result_process? ? result_exit_status : SimpleCov::ExitCodes::SUCCESS
+    def process_result(result)
+      result_exit_status = result_exit_status(result)
+      write_last_run(result) if result_exit_status == SimpleCov::ExitCodes::SUCCESS
+      result_exit_status
     end
 
     # @api private
-    #
-    # rubocop:disable Metrics/MethodLength
-    def result_exit_status(result, covered_percent)
-      covered_percentages = result.covered_percentages.map { |percentage| percentage.round(2) }
-      if covered_percent < SimpleCov.minimum_coverage
-        $stderr.printf("Coverage (%.2f%%) is below the expected minimum coverage (%.2f%%).\n", covered_percent, SimpleCov.minimum_coverage)
-        SimpleCov::ExitCodes::MINIMUM_COVERAGE
-      elsif covered_percentages.any? { |p| p < SimpleCov.minimum_coverage_by_file }
-        $stderr.printf("File (%s) is only (%.2f%%) covered. This is below the expected minimum coverage per file of (%.2f%%).\n", result.least_covered_file, covered_percentages.min, SimpleCov.minimum_coverage_by_file)
-        SimpleCov::ExitCodes::MINIMUM_COVERAGE
-      elsif (last_run = SimpleCov::LastRun.read)
-        coverage_diff = last_run["result"]["covered_percent"] - covered_percent
-        if coverage_diff > SimpleCov.maximum_coverage_drop
-          $stderr.printf("Coverage has dropped by %.2f%% since the last time (maximum allowed: %.2f%%).\n", coverage_diff, SimpleCov.maximum_coverage_drop)
-          SimpleCov::ExitCodes::MAXIMUM_COVERAGE_DROP
-        else
-          SimpleCov::ExitCodes::SUCCESS
-        end
-      else
-        SimpleCov::ExitCodes::SUCCESS
-      end
+    CoverageLimits = Struct.new(:minimum_coverage, :minimum_coverage_by_file, :maximum_coverage_drop, keyword_init: true)
+    def result_exit_status(result)
+      coverage_limits = CoverageLimits.new(
+        minimum_coverage: minimum_coverage, minimum_coverage_by_file: minimum_coverage_by_file,
+        maximum_coverage_drop: maximum_coverage_drop
+      )
+
+      ExitCodes::ExitCodeHandling.call(result, coverage_limits: coverage_limits)
     end
-    # rubocop:enable Metrics/MethodLength
 
     #
     # @api private
     #
     def final_result_process?
-      # checking for ENV["TEST_ENV_NUMBER"] to determine if the tess are being run in parallel
-      !defined?(ParallelTests) || !ENV["TEST_ENV_NUMBER"] || ParallelTests.number_of_running_processes <= 1
+      # checking for ENV["TEST_ENV_NUMBER"] to determine if the tests are being run in parallel
+      !defined?(ParallelTests) || !ENV["TEST_ENV_NUMBER"] || ParallelTests.last_process?
     end
 
     #
@@ -266,37 +275,194 @@ module SimpleCov
     #
     def wait_for_other_processes
       return unless defined?(ParallelTests) && final_result_process?
+
       ParallelTests.wait_for_other_processes_to_finish
     end
 
     #
     # @api private
     #
-    def write_last_run(covered_percent)
-      SimpleCov::LastRun.write(:result => {:covered_percent => covered_percent})
+    def write_last_run(result)
+      SimpleCov::LastRun.write(result:
+        result.coverage_statistics.transform_values do |stats|
+          round_coverage(stats.percent)
+        end)
+    end
+
+    #
+    # @api private
+    #
+    # Rounding down to be extra strict, see #679
+    def round_coverage(coverage)
+      coverage.floor(2)
+    end
+
+  private
+
+    def initial_setup(profile, &block)
+      load_profile(profile) if profile
+      configure(&block) if block_given?
+      self.running = true
+    end
+
+    #
+    # Trigger Coverage.start depends on given config coverage_criterion
+    #
+    # With Positive branch it supports all coverage measurement types
+    # With Negative branch it supports only line coverage measurement type
+    #
+    def start_coverage_measurement
+      # This blog post gives a good run down of the coverage criterias introduced
+      # in Ruby 2.5: https://blog.bigbinary.com/2018/04/11/ruby-2-5-supports-measuring-branch-and-method-coverages.html
+      # There is also a nice writeup of the different coverage criteria made in this
+      # comment  https://github.com/simplecov-ruby/simplecov/pull/692#discussion_r281836176 :
+      # Ruby < 2.5:
+      # https://github.com/ruby/ruby/blob/v1_9_3_374/ext/coverage/coverage.c
+      # traditional mode (Array)
+      #
+      # Ruby 2.5:
+      # https://bugs.ruby-lang.org/issues/13901
+      # https://github.com/ruby/ruby/blob/v2_5_3/ext/coverage/coverage.c
+      # default: traditional/compatible mode (Array)
+      # :lines - like traditional mode but using Hash
+      # :branches
+      # :methods
+      # :all - same as lines + branches + methods
+      #
+      # Ruby >= 2.6:
+      # https://bugs.ruby-lang.org/issues/15022
+      # https://github.com/ruby/ruby/blob/v2_6_3/ext/coverage/coverage.c
+      # default: traditional/compatible mode (Array)
+      # :lines - like traditional mode but using Hash
+      # :branches
+      # :methods
+      # :oneshot_lines - can not be combined with lines
+      # :all - same as lines + branches + methods
+      #
+      if coverage_start_arguments_supported?
+        start_coverage_with_criteria
+      else
+        Coverage.start
+      end
+    end
+
+    def start_coverage_with_criteria
+      start_arguments = coverage_criteria.map do |criterion|
+        [lookup_corresponding_ruby_coverage_name(criterion), true]
+      end.to_h
+
+      Coverage.start(start_arguments)
+    end
+
+    CRITERION_TO_RUBY_COVERAGE = {
+      branch: :branches,
+      line: :lines
+    }.freeze
+    def lookup_corresponding_ruby_coverage_name(criterion)
+      CRITERION_TO_RUBY_COVERAGE.fetch(criterion)
+    end
+
+    #
+    # Finds files that were to be tracked but were not loaded and initializes
+    # the line-by-line coverage to zero (if relevant) or nil (comments / whitespace etc).
+    #
+    def add_not_loaded_files(result)
+      if tracked_files
+        result = result.dup
+        Dir[tracked_files].each do |file|
+          absolute_path = File.expand_path(file)
+          result[absolute_path] ||= SimulateCoverage.call(absolute_path)
+        end
+      end
+
+      result
+    end
+
+    #
+    # Call steps that handle process coverage result
+    #
+    # @return [Hash]
+    #
+    def process_coverage_result
+      adapt_coverage_result
+      remove_useless_results
+      result_with_not_loaded_files
+    end
+
+    #
+    # Unite the result so it wouldn't matter what coverage type was called
+    #
+    # @return [Hash]
+    #
+    def adapt_coverage_result
+      @result = SimpleCov::ResultAdapter.call(Coverage.result)
+    end
+
+    #
+    # Filter coverage result
+    # The result before filter also has result of coverage for files
+    # are not related to the project like loaded gems coverage.
+    #
+    # @return [Hash]
+    #
+    def remove_useless_results
+      @result = SimpleCov::UselessResultsRemover.call(@result)
+    end
+
+    #
+    # Initialize result with files that are not included by coverage
+    # and added inside the config block
+    #
+    # @return [Hash]
+    #
+    def result_with_not_loaded_files
+      @result = SimpleCov::Result.new(add_not_loaded_files(@result))
+    end
+
+    # parallel_tests isn't always available, see: https://github.com/grosser/parallel_tests/issues/772
+    def make_parallel_tests_available
+      return if defined?(ParallelTests)
+      return unless probably_running_parallel_tests?
+
+      require "parallel_tests"
+    rescue LoadError
+      warn("SimpleCov guessed you were running inside parallel tests but couldn't load it. Please file a bug report with us!")
+    end
+
+    def probably_running_parallel_tests?
+      ENV["TEST_ENV_NUMBER"] && ENV["PARALLEL_TEST_GROUPS"]
     end
   end
 end
 
-$LOAD_PATH.unshift(File.join(File.dirname(__FILE__)))
-require "simplecov/configuration"
-SimpleCov.send :extend, SimpleCov::Configuration
-require "simplecov/exit_codes"
-require "simplecov/profiles"
-require "simplecov/source_file"
-require "simplecov/file_list"
-require "simplecov/result"
-require "simplecov/filter"
-require "simplecov/formatter"
-require "simplecov/last_run"
-require "simplecov/lines_classifier"
-require "simplecov/raw_coverage"
-require "simplecov/result_merger"
-require "simplecov/command_guesser"
-require "simplecov/version"
+# requires are down here here for a load order reason I'm not sure what it is about
+require "set"
+require "forwardable"
+require_relative "simplecov/configuration"
+SimpleCov.extend SimpleCov::Configuration
+require_relative "simplecov/coverage_statistics"
+require_relative "simplecov/exit_codes"
+require_relative "simplecov/profiles"
+require_relative "simplecov/source_file/line"
+require_relative "simplecov/source_file/branch"
+require_relative "simplecov/source_file"
+require_relative "simplecov/file_list"
+require_relative "simplecov/result"
+require_relative "simplecov/filter"
+require_relative "simplecov/formatter"
+require_relative "simplecov/last_run"
+require_relative "simplecov/lines_classifier"
+require_relative "simplecov/result_merger"
+require_relative "simplecov/command_guesser"
+require_relative "simplecov/version"
+require_relative "simplecov/result_adapter"
+require_relative "simplecov/combine"
+require_relative "simplecov/combine/branches_combiner"
+require_relative "simplecov/combine/files_combiner"
+require_relative "simplecov/combine/lines_combiner"
+require_relative "simplecov/combine/results_combiner"
+require_relative "simplecov/useless_results_remover"
+require_relative "simplecov/simulate_coverage"
 
 # Load default config
-require "simplecov/defaults" unless ENV["SIMPLECOV_NO_DEFAULTS"]
-
-# Load Rails integration (only for Rails 3, see #113)
-require "simplecov/railtie" if defined? Rails::Railtie
+require_relative "simplecov/defaults" unless ENV["SIMPLECOV_NO_DEFAULTS"]