simplecov 0.6.4 → 0.17.1

data/doc/alternate-formatters.md

@@ -0,0 +1,56 @@
+## Alternate coverage report formatters
+
+The community around simplecov provides a whole bunch of alternate formatters beyond the official
+[simplecov-html](https://github.com/colszowka/simplecov-html) gem.
+
+If you have built or found one that is missing here, please send a Pull Request for this document!
+
+#### [simplecov-badge](https://github.com/matthew342/simplecov-badge)
+*by Matt Hale*
+
+A formatter that generates a coverage badge for use in your project's readme using ImageMagick.
+
+#### [simplecov-small-badge](https://github.com/marcgrimme/simplecov-small-badge)
+*by Marc Grimme*
+
+A formatter that generates a small coverage badge for use in your project's readme using the SVG.
+
+#### [simplecov-cobertura](https://github.com/dashingrocket/simplecov-cobertura)
+*by Jesse Bowes*
+
+A formatter that generates Cobertura XML.
+
+#### [simplecov-csv](https://github.com/fguillen/simplecov-csv)
+*by Fernando Guillen*
+
+CSV formatter for SimpleCov
+
+#### [simplecov-erb](https://github.com/kpaulisse/simplecov-erb)
+*by [Kevin Paulisse](https://github.com/kpaulisse)*
+
+Flexible formatter that generates the output from an ERB template.
+
+#### [simplecov-json](https://github.com/vicentllongo/simplecov-json)
+*by Vicent Llongo*
+
+JSON formatter for SimpleCov
+
+#### [simplecov-lcov](https://github.com/fortissimo1997/simplecov-lcov)
+*by fortissimo1997*
+
+lcov formatter for SimpleCov
+
+#### [simplecov-rcov](https://github.com/fguillen/simplecov-rcov)
+*by Fernando Guillen*
+
+"The target of this formatter is to cheat on Hudson so I can use the Ruby metrics plugin with SimpleCov."
+
+#### [simplecov-single_file_reporter](https://github.com/grosser/simplecov-single_file_reporter)
+*by [Michael Grosser](http://grosser.it)*
+
+A formatter that prints the coverage of the file under test when you run a single test file.
+
+#### [simplecov-t_wada](https://github.com/ysksn/simplecov-t_wada)
+*by [Yosuke Kabuto](https://github.com/ysksn)*
+
+t_wada AA formatter for SimpleCov

data/doc/commercial-services.md

@@ -0,0 +1,20 @@
+## Commercial Services with SimpleCov integration
+
+There is a bunch of services that offer integration with your existing CI pipeline and SimpleCov coverage
+reports. Please note these are not associated with the SimpleCov project itself, so please report problems with
+these integrations with their respective owners.
+
+#### [codeclimate](https://github.com/codeclimate/ruby-test-reporter)
+*by [Code Climate](https://codeclimate.com/)*
+
+Upload coverage reports to [codeclimate.com](https://codeclimate.com/), a hosted software quality analysis and that also includes coverage reporting.
+
+#### [codecov](https://github.com/codecov/codecov-ruby)
+*by [Codecov](https://codecov.io/)*
+
+Upload coverage reports to [codecov.io](https://codecov.io/), a hosted coverage reporting solution.
+
+#### [coveralls](https://github.com/lemurheavy/coveralls-ruby)
+*by [Coveralls](https://coveralls.io/)*
+
+Upload coverage reports to [coveralls.io](https://coveralls.io/), a hosted coverage reporting solution.

data/doc/editor-integration.md

@@ -0,0 +1,18 @@
+## Editor integration
+
+Some editors have a graphical integration for the simplecov gem.
+
+#### [Atom Editor: coverage](https://atom.io/packages/coverage)
+*by Philip Giuliani*
+
+Adds an overview of your current test coverage to Atom.
+
+#### [Sublime Editor: Simple​Cov](https://packagecontrol.io/packages/SimpleCov)
+*by sentience*
+
+Adds in editor live coverage highlighting, status bar coverage information, and summary coverage information.
+
+#### [cadre](https://github.com/nyarly/cadre)
+*by Judson Lester*
+
+Includes a formatter for Simplecov that emits a Vim script to mark up code files with coverage information.

data/lib/simplecov.rb

@@ -1,19 +1,37 @@
+# frozen_string_literal: true
+
+require "English"
+
 #
 # Code coverage for ruby 1.9. Please check out README for a full introduction.
 #
-module SimpleCov
-  # Indicates invalid coverage data
-  class CoverageDataError < StandardError; end;
+# Coverage may be inaccurate under JRUBY.
+if defined?(JRUBY_VERSION) && defined?(JRuby)
 
+  # @see https://github.com/jruby/jruby/issues/1196
+  # @see https://github.com/metricfu/metric_fu/pull/226
+  # @see https://github.com/colszowka/simplecov/issues/420
+  # @see https://github.com/colszowka/simplecov/issues/86
+  # @see https://jira.codehaus.org/browse/JRUBY-6106
+
+  unless org.jruby.RubyInstanceConfig.FULL_TRACE_ENABLED
+    warn 'Coverage may be inaccurate; set the "--debug" command line option,' \
+      ' or do JRUBY_OPTS="--debug"' \
+      ' or set the "debug.fullTrace=true" option in your .jrubyrc'
+  end
+end
+module SimpleCov
   class << self
-    attr_accessor :running#, :result # TODO: Remove result?
+    attr_accessor :running
+    attr_accessor :pid
+    attr_reader :exit_exception
 
     #
     # Sets up SimpleCov to run against your project.
-    # You can optionally specify an adapter to use as well as configuration with a block:
+    # You can optionally specify a profile to use as well as configuration with a block:
     #   SimpleCov.start
     #    OR
-    #   SimpleCov.start 'rails' # using rails adapter
+    #   SimpleCov.start 'rails' # using rails profile
     #    OR
     #   SimpleCov.start do
     #     add_filter 'test'
@@ -25,15 +43,36 @@ module SimpleCov
     #
     # Please check out the RDoc for SimpleCov::Configuration to find about available config options
     #
-    def start(adapter=nil, &block)
-      return false unless SimpleCov.usable?
+    def start(profile = nil, &block)
+      if SimpleCov.usable?
+        load_profile(profile) if profile
+        configure(&block) if block_given?
+        @result = nil
+        self.running = true
+        self.pid = Process.pid
+        Coverage.start
+      else
+        warn "WARNING: SimpleCov is activated, but you're not running Ruby 1.9+ - no coverage analysis will happen"
+        warn "Starting with SimpleCov 1.0.0, even no-op compatibility with Ruby <= 1.8 will be entirely dropped."
+        false
+      end
+    end
+
+    #
+    # Finds files that were to be tracked but were not loaded and initializes
+    # the line-by-line coverage to zero (if relevant) or nil (comments / whitespace etc).
+    #
+    def add_not_loaded_files(result)
+      if tracked_files
+        result = result.dup
+        Dir[tracked_files].each do |file|
+          absolute = File.expand_path(file)
 
-      require 'coverage'
-      load_adapter(adapter) unless adapter.nil?
-      Coverage.start
-      configure(&block) if block_given?
-      @result = nil
-      self.running = true
+          result[absolute] ||= LinesClassifier.new.classify(File.foreach(absolute))
+        end
+      end
+
+      result
     end
 
     #
@@ -41,26 +80,41 @@ module SimpleCov
     # from cache using SimpleCov::ResultMerger if use_merging is activated (default)
     #
     def result
-      @result ||= SimpleCov::Result.new(Coverage.result) if running
+      return @result if result?
+
+      # Collect our coverage result
+      if running
+        @result = SimpleCov::Result.new add_not_loaded_files(Coverage.result)
+      end
+
       # If we're using merging of results, store the current result
-      # first, then merge the results and return those
+      # first (if there is one), then merge the results and return those
       if use_merging
-        SimpleCov::ResultMerger.store_result(@result) if @result
-        return SimpleCov::ResultMerger.merged_result
-      else
-        return @result if defined? @result
+        wait_for_other_processes
+        SimpleCov::ResultMerger.store_result(@result) if result?
+        @result = SimpleCov::ResultMerger.merged_result
       end
+
+      @result
     ensure
       self.running = false
     end
 
+    #
+    # Returns nil if the result has not been computed
+    # Otherwise, returns the result
+    #
+    def result?
+      defined?(@result) && @result
+    end
+
     #
     # Applies the configured filters to the given array of SimpleCov::SourceFile items
     #
     def filtered(files)
       result = files.clone
       filters.each do |filter|
-        result = result.reject {|source_file| filter.matches?(source_file) }
+        result = result.reject { |source_file| filter.matches?(source_file) }
       end
       SimpleCov::FileList.new result
     end
@@ -72,52 +126,177 @@ module SimpleCov
       grouped = {}
       grouped_files = []
       groups.each do |name, filter|
-        grouped[name] = SimpleCov::FileList.new(files.select {|source_file| filter.matches?(source_file)})
+        grouped[name] = SimpleCov::FileList.new(files.select { |source_file| filter.matches?(source_file) })
         grouped_files += grouped[name]
       end
-      if groups.length > 0 and (other_files = files.reject {|source_file| grouped_files.include?(source_file)}).length > 0
+      if !groups.empty? && !(other_files = files.reject { |source_file| grouped_files.include?(source_file) }).empty?
         grouped["Ungrouped"] = SimpleCov::FileList.new(other_files)
       end
       grouped
     end
 
     #
-    # Applies the adapter of given name on SimpleCov configuration
+    # Applies the profile of given name on SimpleCov configuration
     #
+    def load_profile(name)
+      profiles.load(name)
+    end
+
     def load_adapter(name)
-      adapters.load(name)
+      warn "#{Kernel.caller.first}: [DEPRECATION] #load_adapter is deprecated. Use #load_profile instead."
+      load_profile(name)
     end
 
     #
-    # Checks whether we're on a proper version of ruby (1.9+) and returns false if this is not the case,
-    # also printing an appropriate warning
+    # Checks whether we're on a proper version of Ruby (likely 1.9+) which
+    # provides coverage support
     #
     def usable?
-      unless "1.9".respond_to?(:encoding)
-        warn "WARNING: SimpleCov is activated, but you're not running Ruby 1.9+ - no coverage analysis will happen"
-        return false
+      return @usable if defined?(@usable) && !@usable.nil?
+
+      @usable = begin
+        require "coverage"
+        require "simplecov/jruby_fix"
+        true
+      rescue LoadError
+        false
       end
-      true
+    end
+
+    #
+    # Clear out the previously cached .result. Primarily useful in testing
+    #
+    def clear_result
+      @result = nil
+    end
+
+    #
+    # Capture the current exception if it exists
+    # This will get called inside the at_exit block
+    #
+    def set_exit_exception
+      @exit_exception = $ERROR_INFO
+    end
+
+    #
+    # Returns the exit status from the exit exception
+    #
+    def exit_status_from_exception
+      return SimpleCov::ExitCodes::SUCCESS unless exit_exception
+
+      if exit_exception.is_a?(SystemExit)
+        exit_exception.status
+      else
+        SimpleCov::ExitCodes::EXCEPTION
+      end
+    end
+
+    # @api private
+    #
+    # Called from at_exit block
+    #
+    def run_exit_tasks!
+      exit_status = SimpleCov.exit_status_from_exception
+
+      SimpleCov.at_exit.call
+
+      # Don't modify the exit status unless the result has already been
+      # computed
+      exit_status = SimpleCov.process_result(SimpleCov.result, exit_status) if SimpleCov.result?
+
+      # Force exit with stored status (see github issue #5)
+      # unless it's nil or 0 (see github issue #281)
+      if exit_status && exit_status > 0
+        $stderr.printf("SimpleCov failed with exit %d", exit_status)
+        Kernel.exit exit_status
+      end
+    end
+
+    # @api private
+    #
+    # Usage:
+    #   exit_status = SimpleCov.process_result(SimpleCov.result, exit_status)
+    #
+    def process_result(result, exit_status)
+      return exit_status if exit_status != SimpleCov::ExitCodes::SUCCESS # Existing errors
+
+      covered_percent = result.covered_percent.round(2)
+      result_exit_status = result_exit_status(result, covered_percent)
+      if result_exit_status == SimpleCov::ExitCodes::SUCCESS # No result errors
+        write_last_run(covered_percent)
+      end
+      final_result_process? ? result_exit_status : SimpleCov::ExitCodes::SUCCESS
+    end
+
+    # @api private
+    #
+    # rubocop:disable Metrics/MethodLength
+    def result_exit_status(result, covered_percent)
+      covered_percentages = result.covered_percentages.map { |percentage| percentage.round(2) }
+      if covered_percent < SimpleCov.minimum_coverage
+        $stderr.printf("Coverage (%.2f%%) is below the expected minimum coverage (%.2f%%).\n", covered_percent, SimpleCov.minimum_coverage)
+        SimpleCov::ExitCodes::MINIMUM_COVERAGE
+      elsif covered_percentages.any? { |p| p < SimpleCov.minimum_coverage_by_file }
+        $stderr.printf("File (%s) is only (%.2f%%) covered. This is below the expected minimum coverage per file of (%.2f%%).\n", result.least_covered_file, covered_percentages.min, SimpleCov.minimum_coverage_by_file)
+        SimpleCov::ExitCodes::MINIMUM_COVERAGE
+      elsif (last_run = SimpleCov::LastRun.read)
+        coverage_diff = last_run["result"]["covered_percent"] - covered_percent
+        if coverage_diff > SimpleCov.maximum_coverage_drop
+          $stderr.printf("Coverage has dropped by %.2f%% since the last time (maximum allowed: %.2f%%).\n", coverage_diff, SimpleCov.maximum_coverage_drop)
+          SimpleCov::ExitCodes::MAXIMUM_COVERAGE_DROP
+        else
+          SimpleCov::ExitCodes::SUCCESS
+        end
+      else
+        SimpleCov::ExitCodes::SUCCESS
+      end
+    end
+    # rubocop:enable Metrics/MethodLength
+
+    #
+    # @api private
+    #
+    def final_result_process?
+      # checking for ENV["TEST_ENV_NUMBER"] to determine if the tess are being run in parallel
+      !defined?(ParallelTests) || !ENV["TEST_ENV_NUMBER"] || ParallelTests.number_of_running_processes <= 1
+    end
+
+    #
+    # @api private
+    #
+    def wait_for_other_processes
+      return unless defined?(ParallelTests) && final_result_process?
+      ParallelTests.wait_for_other_processes_to_finish
+    end
+
+    #
+    # @api private
+    #
+    def write_last_run(covered_percent)
+      SimpleCov::LastRun.write(:result => {:covered_percent => covered_percent})
     end
   end
 end
 
 $LOAD_PATH.unshift(File.join(File.dirname(__FILE__)))
-require 'simplecov/configuration'
+require "simplecov/configuration"
 SimpleCov.send :extend, SimpleCov::Configuration
-require 'simplecov/adapters'
-require 'simplecov/source_file'
-require 'simplecov/file_list'
-require 'simplecov/result'
-require 'simplecov/filter'
-require 'simplecov/formatter'
-require 'simplecov/merge_helpers'
-require 'simplecov/result_merger'
-require 'simplecov/command_guesser'
-require 'simplecov/version'
+require "simplecov/exit_codes"
+require "simplecov/profiles"
+require "simplecov/source_file"
+require "simplecov/file_list"
+require "simplecov/result"
+require "simplecov/filter"
+require "simplecov/formatter"
+require "simplecov/last_run"
+require "simplecov/lines_classifier"
+require "simplecov/raw_coverage"
+require "simplecov/result_merger"
+require "simplecov/command_guesser"
+require "simplecov/version"
 
 # Load default config
-require 'simplecov/defaults'
+require "simplecov/defaults" unless ENV["SIMPLECOV_NO_DEFAULTS"]
 
 # Load Rails integration (only for Rails 3, see #113)
-require 'simplecov/railtie' if defined? Rails::Railtie
+require "simplecov/railtie" if defined? Rails::Railtie

data/lib/simplecov/command_guesser.rb

@@ -1,49 +1,61 @@
+# frozen_string_literal: true
+
 #
 # Helper that tries to find out what test suite is running (for SimpleCov.command_name)
 #
-module SimpleCov::CommandGuesser
-  class << self
-    # Storage for the original command line call that invoked the test suite.
-    # This has got to be stored as early as possible because i.e. rake and test/unit 2
-    # have a habit of tampering with ARGV, which makes i.e. the automatic distinction
-    # between rails unit/functional/integration tests impossible without this cached
-    # item.
-    attr_accessor :original_run_command
-    
-    def guess
-      from_command_line_options || from_defined_constants
-    end
-    
+module SimpleCov
+  module CommandGuesser
+    class << self
+      # Storage for the original command line call that invoked the test suite.
+      # This has got to be stored as early as possible because i.e. rake and test/unit 2
+      # have a habit of tampering with ARGV, which makes i.e. the automatic distinction
+      # between rails unit/functional/integration tests impossible without this cached
+      # item.
+      attr_accessor :original_run_command
+
+      def guess
+        from_env || from_command_line_options || from_defined_constants
+      end
+
     private
-    
-    def from_command_line_options
-      case original_run_command
-        when /#{'test/functional/'}/
+
+      def from_env
+        # If being run from inside parallel_tests set the command name according to the process number
+        return unless ENV["PARALLEL_TEST_GROUPS"] && ENV["TEST_ENV_NUMBER"]
+        number = ENV["TEST_ENV_NUMBER"]
+        number = "1" if number.empty?
+        "(#{number}/#{ENV['PARALLEL_TEST_GROUPS']})"
+      end
+
+      def from_command_line_options
+        case original_run_command
+        when /test\/functional\//, /test\/\{.*functional.*\}\//
           "Functional Tests"
-        when /#{'test/integration/'}/
+        when /test\/integration\//
           "Integration Tests"
-        when /#{'test/'}/
+        when /test\//
           "Unit Tests"
+        when /spec/
+          "RSpec"
         when /cucumber/, /features/
           "Cucumber Features"
-        when /spec/
+        end
+      end
+
+      def from_defined_constants
+        # If the command regexps fail, let's try checking defined constants.
+        if defined?(RSpec)
           "RSpec"
+        elsif defined?(Test::Unit)
+          "Unit Tests"
+        elsif defined?(MiniTest)
+          "MiniTest"
         else
-          nil
-      end
-    end
-  
-    def from_defined_constants
-      # If the command regexps fail, let's try checking defined constants.
-      if defined?(RSpec)
-        "RSpec"
-      elsif defined?(Test::Unit)
-        "Unit Tests"
-      else
-        # TODO: Provide link to docs/wiki article
-        warn "SimpleCov failed to recognize the test framework and/or suite used. Please specify manually using SimpleCov.command_name 'Unit Tests'."
-        'Unknown Test Framework'
+          # TODO: Provide link to docs/wiki article
+          warn "SimpleCov failed to recognize the test framework and/or suite used. Please specify manually using SimpleCov.command_name 'Unit Tests'."
+          "Unknown Test Framework"
+        end
       end
     end
   end
-end
+end