rspec-tracer 0.9.0 → 1.0.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 1df1d0c90323ae123f24629f74f654bce0fff93bba8e8b052388d582dfb64d8f
-  data.tar.gz: 5caabede1fb71ffcbbd70f0e9bfe8ca355303e843dbb99cacf14f146149afe5a
+  metadata.gz: 9aaeb8a00f4cebc311367e6a61d59526014849df9f7fe890455d64cef16d435f
+  data.tar.gz: b803d5104a0a35f783bdd15b7a43252a1cfddcdd56d9f0a26c5a07bb8c36ef5e
 SHA512:
-  metadata.gz: 5ad9ec360d867ed745039fc2cf2a4aef314912a5b6a596c8c79e9d6615b00f2b578f5d3a164c2725ed710d9b0aa055855c607eebe52f7b032d1b959d483cb079
-  data.tar.gz: 347236ad3e59372ff1cf33aa3c812ae5a22911ff6b46bfdfced44b96ef80161c4da0798a9afecf96846bc44963713bac4aa625c714266431b975c3152ebd92b1
+  metadata.gz: 56acd78d6d4bf7b6270e8517a82d0bb1237dd78518a22d1a84a96de29e00c59379092cd725cc276d3f59ca2916306835c38426917747dcaf278e45d611c99389
+  data.tar.gz: a4e026b2c6fdaad653bd48063c9d0b771e7e7bc8876f653cc2e32279930d6a043b546ab1c7a713e12aac934f1f069497decdb5856246d4ee8033e905577be1db

data/CHANGELOG.md

@@ -1,3 +1,31 @@
+## [1.0.0] - 2021-10-21
+
+### Added
+
+- [JRuby](https://github.com/jruby/jruby) support
+- [Parallel Tests](https://github.com/grosser/parallel_tests) support
+
+### Breaking Changes
+
+The first run on this version will not use any cache on the CI because the number
+of files changed from eight to eleven, so there will be no appropriate cache to use.
+
+## [0.9.3] - 2021-10-03
+
+Generate reports ignoring duplicate examples (#42)
+
+## [0.9.2] - 2021-09-30
+
+### Fixed
+
+Caches getting corrupted on interrupts (#39)
+
+## [0.9.1] - 2021-09-23
+
+### Fixed
+
+Flaky and failed examples dependency check (#38)
+
 ## [0.9.0] - 2021-09-15
 
 ### Added

data/README.md

@@ -1,5 +1,6 @@
 ![](./readme_files/rspec_tracer.png)
 
+[![Discord](https://badgen.net/badge/icon/discord?icon=discord&label)](https://discord.gg/H2G9yWeuRZ)
 [![Maintainability](https://api.codeclimate.com/v1/badges/eabce2757839c08d8f8d/maintainability)](https://codeclimate.com/github/avmnu-sng/rspec-tracer/maintainability)
 [![Test Coverage](https://api.codeclimate.com/v1/badges/eabce2757839c08d8f8d/test_coverage)](https://codeclimate.com/github/avmnu-sng/rspec-tracer/test_coverage)
 [![Gem Version](https://badge.fury.io/rb/rspec-tracer.svg)](https://badge.fury.io/rb/rspec-tracer)
@@ -28,19 +29,19 @@ recommended to use **simplecov >= 0.12.0**. To use RSpec Tracer **cache on CI**,
 need to have an **S3 bucket** and **[AWS CLI](https://aws.amazon.com/cli/)**
 installed.
 
-You should take some time and go through the **[document](./RSPEC_TRACER.md)**
-describing the **intention** and implementation details of **managing dependency**,
-**managing flaky tests**, **skipping tests**, and **caching on CI**.
+> You should take some time and go through the **[document](./RSPEC_TRACER.md)** describing the **intention** and implementation details of **managing dependency**, **managing flaky tests**, **skipping tests**, and **caching on CI**.
 
 ## Table of Contents
 
 * [Demo](#demo)
 * [Getting Started](#getting-started)
+  * [Working with JRuby](#working-with-jruby)
+  * [Working with Parallel Tests](#working-with-parallel-tests)
 * [Configuring CI Caching](#configuring-ci-caching)
 * [Advanced Configuration](#advanced-configuration)
 * [Filters](#filters)
 * [Environment Variables](#environment-variables)
-* [When Should You Not Use RSpec Tracer](#when-should-you-not-use-rspec-tracer)
+* [Duplicate Examples](#duplicate-examples)
 
 ## Demo
 
@@ -64,6 +65,12 @@ These reports provide basic test information:
 
 ![](./readme_files/examples_report_next_run.png)
 
+### Duplicate Examples Report
+
+These reports provide duplicate tests information.
+
+![](./readme_files/duplicate_examples_report.png)
+
 ### Flaky Examples Report
 
 These reports provide flaky tests information. Assuming **the following two tests
@@ -131,6 +138,28 @@ any of the application code.**
 3. After running your tests, open `rspec_tracer_report/index.html` in the browser
 of your choice.
 
+### Working with JRuby
+
+It is recommend to use **JRuby 9.2.10.0+**. Also, configure it with **`JRUBY_OPTS="--debug -X+O"`**
+or have the `.jrubyrc` file:
+
+```ruby
+debug.fullTrace=true
+objectspace.enabled=true
+```
+
+### Working with Parallel Tests
+
+The Rspec tracer, by default, supports working with [parallel_tests](https://github.com/grosser/parallel_tests/)
+gem. It maintains a lock file `/tmp/parallel_tests.lock` to identify the last
+running process. Usually, you are not required to do anything special unless you
+interrupt the execution in between and the process did not complete correctly.
+In such a case, you must delete the lock file before the next run.
+
+```sh
+rm -f /tmp/parallel_tests.lock && bundle exec parallel_rspec
+```
+
 ## Configuring CI Caching
 
 To enable RSpec Tracer to share cache between different builds on CI, update the
@@ -224,11 +253,11 @@ variables:
   ```
 - **`RSPEC_TRACER_COVERAGE_DIR`** to update the default coverage directory (`rspec_tracer_coverage`).
   ```sh
-  export RSPEC_TRACER_CACHE_DIR=/tmp/rspec_tracer_coverage
+  export RSPEC_TRACER_COVERAGE_DIR=/tmp/rspec_tracer_coverage
   ```
 - **`RSPEC_TRACER_REPORT_DIR`** to update the default html reports directory (`rspec_tracer_report`).
   ```sh
-  export RSPEC_TRACER_CACHE_DIR=/tmp/rspec_tracer_report
+  export RSPEC_TRACER_REPORT_DIR=/tmp/rspec_tracer_report
   ```
 
 These settings are available through environment variables because the rake tasks
@@ -309,6 +338,9 @@ development environment. You can install [localstack](https://github.com/localst
 and [awscli-local](https://github.com/localstack/awscli-local) and then invoke the
 rake tasks with `LOCAL_AWS=true`.
 
+- **`RSPEC_TRACER_FAIL_ON_DUPLICATES (default: true)`:** By default, RSpec Tracer
+exits with one if there are [duplicate examples](#duplicate-examples).
+
 - **`RSPEC_TRACER_NO_SKIP (default: false)`:** Use this environment variables to
 not skip any tests. Note that it will continue to maintain cache files and generate
 reports.
@@ -335,7 +367,7 @@ specific test suites and not merge them.
   TEST_SUITE_ID=2 bundle exec rspec spec/helpers
   ```
 
-## When Should You Not Use RSpec Tracer
+## Duplicate Examples
 
 To uniquely identify the examples is one of the requirements for the correctness
 of the RSpec Tracer. Sometimes, it would not be possible to do so depending upon
@@ -425,17 +457,12 @@ Calculator
 ```
 
 In this scenario, RSpec Tracer cannot determine the `Calculator#add` and
-`Calculator#sub` group examples. Also, it will ask you not to use the gem unless
-you have made some changes to your spec files.
+`Calculator#sub` group examples.
 
 ```
 ================================================================================
-                IMPORTANT NOTICE -- DO NOT USE RSPEC TRACER
+   IMPORTANT NOTICE -- RSPEC TRACER COULD NOT IDENTIFY SOME EXAMPLES UNIQUELY
 ================================================================================
-    It would be best to make changes so that the RSpec tracer can uniquely
-    identify all the examples, and then you can enable the RSpec tracer back.
-================================================================================
-
 RSpec tracer could not uniquely identify the following 10 examples:
   - Example ID: eabd51a899db4f64d5839afe96004f03 (5 examples)
       * Calculator#add (spec/calculator_spec.rb:13)

data/lib/rspec_tracer/cache.rb

@@ -2,16 +2,16 @@
 
 module RSpecTracer
   class Cache
-    attr_reader :all_examples, :flaky_examples, :failed_examples, :pending_examples,
-                :all_files, :dependency, :run_id
+    attr_reader :all_examples, :duplicate_examples, :interrupted_examples,
+                :flaky_examples, :failed_examples, :pending_examples, :skipped_examples,
+                :all_files, :dependency, :examples_coverage, :run_id
 
     def initialize
-      @run_id = last_run_id
-      @cache_dir = File.join(RSpecTracer.cache_path, @run_id) if @run_id
-
       @cached = false
 
       @all_examples = {}
+      @duplicate_examples = {}
+      @interrupted_examples = Set.new
       @flaky_examples = Set.new
       @failed_examples = Set.new
       @pending_examples = Set.new
@@ -20,52 +20,67 @@ module RSpecTracer
     end
 
     def load_cache_for_run
-      return if @run_id.nil? || @cached
+      return if @cached
+
+      cache_path = RSpecTracer.cache_path
+      cache_path = File.dirname(cache_path) if RSpecTracer.parallel_tests?
+      run_id = last_run_id(cache_path)
+
+      return if run_id.nil?
 
       starting = Process.clock_gettime(Process::CLOCK_MONOTONIC)
+      cache_dir = File.join(cache_path, run_id)
 
-      load_all_examples_cache
-      load_flaky_examples_cache
-      load_failed_examples_cache
-      load_pending_examples_cache
-      load_all_files_cache
-      load_dependency_cache
+      load_all_examples_cache(cache_dir)
+      load_duplicate_examples_cache(cache_dir)
+      load_interrupted_examples_cache(cache_dir)
+      load_flaky_examples_cache(cache_dir)
+      load_failed_examples_cache(cache_dir)
+      load_pending_examples_cache(cache_dir)
+      load_all_files_cache(cache_dir)
+      load_dependency_cache(cache_dir)
 
       ending = Process.clock_gettime(Process::CLOCK_MONOTONIC)
 
       @cached = true
 
-      elpased = RSpecTracer::TimeFormatter.format_time(ending - starting)
+      elapsed = RSpecTracer::TimeFormatter.format_time(ending - starting)
 
-      puts "RSpec tracer loaded cache from #{@cache_dir} (took #{elpased})"
+      puts "RSpec tracer loaded cache from #{cache_dir} (took #{elapsed})"
     end
 
     def cached_examples_coverage
       return @examples_coverage if defined?(@examples_coverage)
-      return @examples_coverage = {} if @run_id.nil?
+
+      cache_path = RSpecTracer.cache_path
+      cache_path = File.dirname(cache_path) if RSpecTracer.parallel_tests?
+      run_id = last_run_id(cache_path)
+
+      return @examples_coverage = {} if run_id.nil?
 
       starting = Process.clock_gettime(Process::CLOCK_MONOTONIC)
-      coverage = load_examples_coverage_cache
+      cache_dir = File.join(cache_path, run_id)
+      coverage = load_examples_coverage_cache(cache_dir)
       ending = Process.clock_gettime(Process::CLOCK_MONOTONIC)
-      elpased = RSpecTracer::TimeFormatter.format_time(ending - starting)
+      elapsed = RSpecTracer::TimeFormatter.format_time(ending - starting)
 
-      puts "RSpec tracer loaded cached examples coverage (took #{elpased})" if RSpecTracer.verbose?
+      puts "RSpec tracer loaded cached examples coverage (took #{elapsed})" if RSpecTracer.verbose?
 
       coverage
     end
 
     private
 
-    def last_run_id
-      file_name = File.join(RSpecTracer.cache_path, 'last_run.json')
+    def last_run_id(cache_dir)
+      file_name = File.join(cache_dir, 'last_run.json')
 
       return unless File.file?(file_name)
 
       JSON.parse(File.read(file_name))['run_id']
     end
 
-    def load_all_examples_cache
-      file_name = File.join(@cache_dir, 'all_examples.json')
+    def load_all_examples_cache(cache_dir, discard_run_reason: true)
+      file_name = File.join(cache_dir, 'all_examples.json')
 
       return unless File.file?(file_name)
 
@@ -74,38 +89,63 @@ module RSpecTracer
       end
 
       @all_examples.each_value do |example|
-        example[:execution_result].transform_keys!(&:to_sym)
+        example[:execution_result].transform_keys!(&:to_sym) if example.key?(:execution_result)
+        example[:run_reason] = nil if discard_run_reason
+      end
+    end
+
+    def load_duplicate_examples_cache(cache_dir)
+      file_name = File.join(cache_dir, 'duplicate_examples.json')
+
+      return unless File.file?(file_name)
 
-        example[:run_reason] = nil
+      @duplicate_examples = JSON.parse(File.read(file_name)).transform_values do |examples|
+        examples.map { |example| example.transform_keys(&:to_sym) }
       end
     end
 
-    def load_flaky_examples_cache
-      file_name = File.join(@cache_dir, 'flaky_examples.json')
+    def load_interrupted_examples_cache(cache_dir)
+      file_name = File.join(cache_dir, 'interrupted_examples.json')
+
+      return unless File.file?(file_name)
+
+      @interrupted_examples = JSON.parse(File.read(file_name)).to_set
+    end
+
+    def load_flaky_examples_cache(cache_dir)
+      file_name = File.join(cache_dir, 'flaky_examples.json')
 
       return unless File.file?(file_name)
 
       @flaky_examples = JSON.parse(File.read(file_name)).to_set
     end
 
-    def load_failed_examples_cache
-      file_name = File.join(@cache_dir, 'failed_examples.json')
+    def load_failed_examples_cache(cache_dir)
+      file_name = File.join(cache_dir, 'failed_examples.json')
 
       return unless File.file?(file_name)
 
       @failed_examples = JSON.parse(File.read(file_name)).to_set
     end
 
-    def load_pending_examples_cache
-      file_name = File.join(@cache_dir, 'pending_examples.json')
+    def load_pending_examples_cache(cache_dir)
+      file_name = File.join(cache_dir, 'pending_examples.json')
 
       return unless File.file?(file_name)
 
       @pending_examples = JSON.parse(File.read(file_name)).to_set
     end
 
-    def load_all_files_cache
-      file_name = File.join(@cache_dir, 'all_files.json')
+    def load_skipped_examples_cache(cache_dir)
+      file_name = File.join(cache_dir, 'skipped_examples.json')
+
+      return unless File.file?(file_name)
+
+      @skipped_examples = JSON.parse(File.read(file_name)).to_set
+    end
+
+    def load_all_files_cache(cache_dir)
+      file_name = File.join(cache_dir, 'all_files.json')
 
       return unless File.file?(file_name)
 
@@ -114,16 +154,16 @@ module RSpecTracer
       end
     end
 
-    def load_dependency_cache
-      file_name = File.join(@cache_dir, 'dependency.json')
+    def load_dependency_cache(cache_dir)
+      file_name = File.join(cache_dir, 'dependency.json')
 
       return unless File.file?(file_name)
 
       @dependency = JSON.parse(File.read(file_name)).transform_values(&:to_set)
     end
 
-    def load_examples_coverage_cache
-      file_name = File.join(@cache_dir, 'examples_coverage.json')
+    def load_examples_coverage_cache(cache_dir)
+      file_name = File.join(cache_dir, 'examples_coverage.json')
 
       return unless File.file?(file_name)
 

data/lib/rspec_tracer/configuration.rb

@@ -31,7 +31,8 @@ module RSpecTracer
     def cache_path
       @cache_path ||= begin
         cache_path = File.expand_path(cache_dir, root)
-        cache_path = File.join(cache_path, ENV['TEST_SUITE_ID'].to_s)
+        cache_path = File.join(cache_path, ENV['TEST_SUITE_ID'].to_s) if ENV['TEST_SUITE_ID']
+        cache_path = File.join(cache_path, parallel_tests_id) if RSpecTracer.parallel_tests?
 
         FileUtils.mkdir_p(cache_path)
 
@@ -46,7 +47,8 @@ module RSpecTracer
     def report_path
       @report_path ||= begin
         report_path = File.expand_path(report_dir, root)
-        report_path = File.join(report_path, ENV['TEST_SUITE_ID'].to_s)
+        report_path = File.join(report_path, ENV['TEST_SUITE_ID'].to_s) if ENV['TEST_SUITE_ID']
+        report_path = File.join(report_path, parallel_tests_id) if RSpecTracer.parallel_tests?
 
         FileUtils.mkdir_p(report_path)
 
@@ -61,7 +63,8 @@ module RSpecTracer
     def coverage_path
       @coverage_path ||= begin
         coverage_path = File.expand_path(coverage_dir, root)
-        coverage_path = File.join(coverage_path, ENV['TEST_SUITE_ID'].to_s)
+        coverage_path = File.join(coverage_path, ENV['TEST_SUITE_ID'].to_s) if ENV['TEST_SUITE_ID']
+        coverage_path = File.join(coverage_path, parallel_tests_id) if RSpecTracer.parallel_tests?
 
         FileUtils.mkdir_p(coverage_path)
 
@@ -93,6 +96,10 @@ module RSpecTracer
       @coverage_filters ||= []
     end
 
+    def parallel_tests_lock_file
+      '/tmp/parallel_tests.lock'
+    end
+
     def verbose?
       @verbose ||= (ENV.fetch('RSPEC_TRACER_VERBOSE', 'false') == 'true')
     end
@@ -103,12 +110,12 @@ module RSpecTracer
 
     private
 
-    def test_suite_id
-      suite_id = ENV.fetch('TEST_SUITE_ID', '')
-
-      return if suite_id.empty?
-
-      suite_id
+    def parallel_tests_id
+      if ParallelTests.first_process?
+        'parallel_tests_1'
+      else
+        "parallel_tests_#{ENV['TEST_ENV_NUMBER']}"
+      end
     end
 
     def at_exit(&block)

data/lib/rspec_tracer/coverage_merger.rb

@@ -0,0 +1,41 @@
+# frozen_string_literal: true
+
+module RSpecTracer
+  class CoverageMerger
+    attr_reader :coverage
+
+    def initialize
+      @coverage = {}
+    end
+
+    def merge(reports_dir)
+      return if RSpecTracer.simplecov?
+
+      reports_dir.each do |report_dir|
+        next unless File.directory?(report_dir)
+
+        cache_coverage = JSON.parse(File.read("#{report_dir}/coverage.json"))['RSpecTracer']['coverage']
+
+        cache_coverage.each_pair do |file_name, line_coverage|
+          unless @coverage.key?(file_name)
+            @coverage[file_name] = line_coverage
+
+            next
+          end
+
+          merge_line_coverage(file_name, line_coverage)
+        end
+      end
+    end
+
+    private
+
+    def merge_line_coverage(file_name, line_coverage)
+      line_coverage.each_with_index do |strength, line_number|
+        next unless strength && @coverage[file_name][line_number]
+
+        @coverage[file_name][line_number] += strength
+      end
+    end
+  end
+end

data/lib/rspec_tracer/coverage_reporter.rb

@@ -88,8 +88,6 @@ module RSpecTracer
       all_files.each do |file_path|
         @coverage[file_path] ||= line_stub(file_path).freeze
       end
-
-      generate_final_coverage_stat
     end
 
     private
@@ -131,31 +129,6 @@ module RSpecTracer
       all_files.sort
     end
 
-    def generate_final_coverage_stat
-      total_loc = 0
-      covered_loc = 0
-
-      @coverage.each_pair do |_file_path, line_coverage|
-        line_coverage.each do |strength|
-          next if strength.nil?
-
-          total_loc += 1
-          covered_loc += 1 if strength.positive?
-        end
-      end
-
-      @coverage_stat = {
-        total_lines: total_loc,
-        covered_lines: covered_loc,
-        missed_lines: total_loc - covered_loc,
-        covered_percent: 0.0
-      }
-
-      return if total_loc.zero?
-
-      @coverage_stat[:covered_percent] = (100.0 * covered_loc / total_loc).round(2)
-    end
-
     def peek_coverage
       data = ::Coverage.peek_result.select do |file_path, _|
         file_path.start_with?(RSpecTracer.root)
@@ -167,8 +140,17 @@ module RSpecTracer
     end
 
     def line_stub(file_path)
+      case RUBY_ENGINE
+      when 'ruby'
+        ruby_line_stub(file_path)
+      when 'jruby'
+        jruby_line_stub(file_path)
+      end
+    end
+
+    def ruby_line_stub(file_path)
       lines = File.foreach(file_path).map { nil }
-      iseqs = [RubyVM::InstructionSequence.compile_file(file_path)]
+      iseqs = [::RubyVM::InstructionSequence.compile_file(file_path)]
 
       until iseqs.empty?
         iseq = iseqs.pop
@@ -179,5 +161,26 @@ module RSpecTracer
 
       lines
     end
+
+    def jruby_line_stub(file_path)
+      lines = File.foreach(file_path).map { nil }
+      root_node = ::JRuby.parse(File.read(file_path))
+
+      visitor = org.jruby.ast.visitor.NodeVisitor.impl do |_name, node|
+        if node.newline?
+          if node.respond_to?(:position)
+            lines[node.position.line] = 0
+          else
+            lines[node.line] = 0
+          end
+        end
+
+        node.child_nodes.each { |child| child&.accept(visitor) }
+      end
+
+      root_node.accept(visitor)
+
+      lines
+    end
   end
 end

data/lib/rspec_tracer/coverage_writer.rb

@@ -0,0 +1,58 @@
+# frozen_string_literal: true
+
+module RSpecTracer
+  class CoverageWriter
+    def initialize(file_name, reporter)
+      @file_name = file_name
+      @reporter = reporter
+    end
+
+    def write_report
+      report = {
+        RSpecTracer: {
+          coverage: @reporter.coverage,
+          timestamp: Time.now.utc.to_i
+        }
+      }
+
+      File.write(@file_name, JSON.pretty_generate(report))
+    end
+
+    def print_stats(elapsed_time)
+      starting = Process.clock_gettime(Process::CLOCK_MONOTONIC)
+
+      total, covered, percent = coverage_stats
+
+      ending = Process.clock_gettime(Process::CLOCK_MONOTONIC)
+      elapsed = RSpecTracer::TimeFormatter.format_time((ending - starting) + elapsed_time)
+
+      puts <<-STATS.strip.gsub(/\s+/, ' ')
+        Coverage report generated for RSpecTracer to #{@file_name}.
+        #{covered} / #{total} LOC (#{percent}%) covered (took #{elapsed})
+      STATS
+    end
+
+    private
+
+    def coverage_stats
+      total_loc = 0
+      covered_loc = 0
+      covered_percent = 0.0
+
+      @reporter.coverage.each_pair do |_file_path, line_coverage|
+        line_coverage.each do |strength|
+          next if strength.nil?
+
+          total_loc += 1
+          covered_loc += 1 if strength.positive?
+        end
+      end
+
+      return [total_loc, covered_loc, covered_percent] if total_loc.zero?
+
+      covered_percent = (100.0 * covered_loc / total_loc).round(2)
+
+      [total_loc, covered_loc, covered_percent]
+    end
+  end
+end