rspec-tracer 0.5.0 → 0.6.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 2817cccf4825e28177bb8794f8d261635f81ec5c1bd6ca0242ba171e702850ca
-  data.tar.gz: 9c5f6af579eec43a994c28b5e8132fec5d71f7a2ddddfb531fafd5c51943d283
+  metadata.gz: 885e868cf847ca1bab4ba2243cd4cd6a4bc3f936a3c8c10f677146433acc2068
+  data.tar.gz: d60fcdeef0067a37faec3ea876ffa519bdd7039a1765ad59e551427bcf2c8e29
 SHA512:
-  metadata.gz: 2d3af598e09b8ac043b5f605acf39b588701c4f0e7b4d74646eb0da74c1c85746db60f215e02cd5fba1880bb75288d0d35df3986b08cc71cfbad4d45ad722d2f
-  data.tar.gz: 7ce36336213d4d37337afa5b50fd774bd6ef7b235539f3a289f1a804a878fba3fa1e7972e9bd09bb8083c80929c0fc3b6bf0b30f257542fdb5e5c466c1247360
+  metadata.gz: cffa43507448703f4471ed2a463e7ef486f105b06560663d307b21c3303368b0ae0260bb36bbc392f900bfb0e54482fdc8055d231ab75fb33e5a93360be46ac9
+  data.tar.gz: d59157d318beda7c81a65cdd1a38ff3d849521a502e31c6c805916ace8f6f3568a5114f201bc03cdb33e63cae3c0a86ae5e81dd148c1c32f735d7d5d3cea5750

data/CHANGELOG.md

@@ -1,3 +1,17 @@
+## [0.6.0] - 2021-09-05
+
+### Added
+
+- Improved dependency change detection (#18)
+- Flaky tests detection (#19)
+- Exclude vendor files from analysis (#21)
+- Report elapsed time at various stages (#23)
+
+### Note
+
+The first run on this version will not use any cache on the CI because the number
+of files changed from eight to nine, so there will be no appropriate cache to use.
+
 ## [0.5.0] - 2021-09-03
 
 ### Fixed

data/README.md

@@ -8,7 +8,11 @@ It uses [Ruby's built-in coverage library](https://ruby-doc.org/stdlib/libdoc/co
 to keep track of the coverage for each test. For each test executed, the coverage
 diff provides the desired file list. RSpec Tracer takes care of reporting the
 **correct code coverage when skipping tests** by using the cached reports. Also,
-note that it will **never skip any tests which failed or were pending** in the last runs.
+note that it will **never skip**:
+
+- **Flaky examples**
+- **Failed examples**
+- **Pending examples**
 
 Knowing the examples and files dependency gives us a better insight into the codebase,
 and we have **a clear idea of what to test for when making any changes**. With this data,
@@ -16,9 +20,10 @@ we can also analyze the coupling between different components and much more.
 
 ## Note
 
-You should take some time and go through the [document](./RSPEC_TRACER.md) describing
-the **intention** and implementation details of **skipping tests**, **managing coverage**,
-and **caching on CI**, etc.
+You should take some time and go through the **[document](./RSPEC_TRACER.md)** describing
+the **intention** and implementation details of **managing dependency**, **managing flaky tests**,
+**skipping tests**, and **caching on CI**. You must go through the README file before
+integrating the gem into your project to better understand what is happening.
 
 ## Table of Contents
 
@@ -28,6 +33,7 @@ and **caching on CI**, etc.
   * [Additional Tools](#additional-tools)
 * [Getting Started](#getting-started)
 * [Environment Variables](#environment-variables)
+  * [BUNDLE_PATH](#bundle_path)
   * [CI](#ci)
   * [LOCAL_AWS](#local_aws)
   * [RSPEC_TRACER_NO_SKIP](#rspec_tracer_no_skip)
@@ -37,6 +43,7 @@ and **caching on CI**, etc.
   * [TEST_SUITE_ID](#test_suite_id)
 * [Sample Reports](#sample-reports)
     * [Examples](#examples)
+    * [Flaky Examples](#flaky-examples)
     * [Examples Dependency](#examples-dependency)
     * [Files Dependency](#files-dependency)
 * [Configuring RSpec Tracer](#configuring-rspec-tracer)
@@ -142,7 +149,15 @@ browser of your choice.
 
 ## Environment Variables
 
-To get better control on execution, you can use the following two environment variables:
+To get better control on execution, you can use the following environment variables
+whenever required.
+
+### BUNDLE_PATH
+
+Since the bundler uses a vendor directory inside the project, it might cause slowness
+depending on the vendor size. You can configure the bundle path outside of the project
+using `BUNDLE_PATH` environment variable, for example, `BUNDLE_PATH=$HOME/vendor/bundle`.
+Make sure to cache this directory in the CI configuration.
 
 ### CI
 
@@ -190,7 +205,7 @@ export TEST_SUITES=8
 
 If you have a large set of tests to run, it is recommended to run them in
 separate groups. This way, RSpec Tracer is not overwhelmed with loading massive
-cached data in the memory. Also, it generate and use cache for specific test suites
+cached data in the memory. Also, it generates and uses cache for specific test suites
 and not merge them.
 
 ```ruby
@@ -229,6 +244,19 @@ These reports provide basic test information:
 
 ![](./readme_files/examples_report_next_run.png)
 
+### Flaky Examples
+
+These reports provide flaky tests information. Assuming **the following two tests
+failed in the first run.**
+
+**Next Run**
+
+![](./readme_files/flaky_examples_report_first_run.png)
+
+**Another Run**
+
+![](./readme_files/flaky_examples_report_next_run.png)
+
 ### Examples Dependency
 
 These reports show a list of dependent files for each test.

data/lib/rspec_tracer/cache.rb

@@ -2,7 +2,8 @@
 
 module RSpecTracer
   class Cache
-    attr_reader :all_examples, :failed_examples, :pending_examples, :all_files, :dependency
+    attr_reader :all_examples, :flaky_examples, :failed_examples, :pending_examples,
+                :all_files, :dependency, :run_id
 
     def initialize
       @run_id = last_run_id
@@ -11,6 +12,7 @@ module RSpecTracer
       @cached = false
 
       @all_examples = {}
+      @flaky_examples = Set.new
       @failed_examples = Set.new
       @pending_examples = Set.new
       @all_files = {}
@@ -20,22 +22,36 @@ module RSpecTracer
     def load_cache_for_run
       return if @run_id.nil? || @cached
 
+      starting = Process.clock_gettime(Process::CLOCK_MONOTONIC)
+
       load_all_examples_cache
+      load_flaky_examples_cache
       load_failed_examples_cache
       load_pending_examples_cache
       load_all_files_cache
       load_dependency_cache
 
+      ending = Process.clock_gettime(Process::CLOCK_MONOTONIC)
+
       @cached = true
 
-      puts "RSpec tracer loaded cache from #{@cache_dir}" if @run_id
+      elpased = RSpecTracer::TimeFormatter.format_time(ending - starting)
+
+      puts "RSpec tracer loaded cache from #{@cache_dir} (took #{elpased})"
     end
 
     def cached_examples_coverage
       return @examples_coverage if defined?(@examples_coverage)
       return @examples_coverage = {} if @run_id.nil?
 
-      load_examples_coverage_cache
+      starting = Process.clock_gettime(Process::CLOCK_MONOTONIC)
+      coverage = load_examples_coverage_cache
+      ending = Process.clock_gettime(Process::CLOCK_MONOTONIC)
+      elpased = RSpecTracer::TimeFormatter.format_time(ending - starting)
+
+      puts "RSpec tracer loaded cached examples coverage (took #{elpased})"
+
+      coverage
     end
 
     private
@@ -64,6 +80,14 @@ module RSpecTracer
       end
     end
 
+    def load_flaky_examples_cache
+      file_name = File.join(@cache_dir, 'flaky_examples.json')
+
+      return unless File.file?(file_name)
+
+      @flaky_examples = JSON.parse(File.read(file_name)).to_set
+    end
+
     def load_failed_examples_cache
       file_name = File.join(@cache_dir, 'failed_examples.json')
 

data/lib/rspec_tracer/defaults.rb

@@ -2,7 +2,13 @@
 
 RSpecTracer.configure do
   add_filter '/vendor/bundle/'
-  add_coverage_filter %w[/autotest/ /features/ /spec/ /test/].freeze
+  add_coverage_filter %w[
+    /autotest/
+    /features/
+    /spec/
+    /test/
+    /vendor/bundle/
+  ].freeze
 end
 
 at_exit do

data/lib/rspec_tracer/html_reporter/reporter.rb

@@ -6,21 +6,22 @@ require 'time'
 module RSpecTracer
   module HTMLReporter
     class Reporter
-      attr_reader :last_run, :examples, :examples_dependency, :files_dependency
+      attr_reader :last_run, :examples, :flaky_examples, :examples_dependency, :files_dependency
 
       def initialize
         @reporter = RSpecTracer.runner.reporter
 
         format_last_run
         format_examples
+        format_flaky_examples
         format_examples_dependency
         format_files_dependency
       end
 
       def generate_report
-        Dir[File.join(File.dirname(__FILE__), 'public/*')].each do |path|
-          FileUtils.cp_r(path, asset_output_path)
-        end
+        starting = Process.clock_gettime(Process::CLOCK_MONOTONIC)
+
+        copy_assets
 
         file_name = File.join(RSpecTracer.report_path, 'index.html')
 
@@ -28,11 +29,20 @@ module RSpecTracer
           file.puts(template('layout').result(binding))
         end
 
-        puts "RSpecTracer generated HTML report to #{file_name}"
+        ending = Process.clock_gettime(Process::CLOCK_MONOTONIC)
+        elpased = RSpecTracer::TimeFormatter.format_time(ending - starting)
+
+        puts "RSpecTracer generated HTML report to #{file_name} (took #{elpased})"
       end
 
       private
 
+      def copy_assets
+        Dir[File.join(File.dirname(__FILE__), 'public/*')].each do |path|
+          FileUtils.cp_r(path, asset_output_path)
+        end
+      end
+
       def format_last_run
         @last_run = @reporter.last_run.slice(
           :actual_count,
@@ -57,6 +67,10 @@ module RSpecTracer
         end
       end
 
+      def format_flaky_examples
+        @flaky_examples = @examples.slice(*@reporter.flaky_examples).values
+      end
+
       def example_run_local_time(utc_time)
         case utc_time
         when Time
@@ -128,6 +142,14 @@ module RSpecTracer
         template(title_id).result(current_binding)
       end
 
+      def formatted_flaky_examples(title, flaky_examples)
+        title_id = report_container_id(title)
+        current_binding = binding
+
+        current_binding.local_variable_set(:title_id, title_id)
+        template(title_id).result(current_binding)
+      end
+
       def formatted_examples_dependency(title, examples_dependency)
         title_id = report_container_id(title)
         current_binding = binding
@@ -154,7 +176,7 @@ module RSpecTracer
 
       def example_status_css_class(example_status)
         case example_status.split.first
-        when 'Failed'
+        when 'Failed', 'Flaky'
           'red'
         when 'Pending'
           'yellow'

data/lib/rspec_tracer/html_reporter/views/flaky_examples.erb

@@ -0,0 +1,38 @@
+<div class="report_container" id="<%= title_id %>">
+  <h2>
+    <span class="group_name"><%= title %></span>
+    (
+      <span class="blue">
+        <strong><%= flaky_examples.count %></strong>
+      </span> examples
+    )
+  </h2>
+
+  <a name="<%= title_id %>"></a>
+
+  <div class="report-table--responsive">
+    <table class="report-table">
+      <thead>
+        <tr>
+          <th>ID</th>
+          <th>Description</th>
+          <th>Location</th>
+          <th>Result</th>
+          <th>Run At</th>
+        </tr>
+      </thead>
+
+      <tbody>
+        <% flaky_examples.each do |example| %>
+          <tr>
+            <td><%= example[:id] %></td>
+            <td><%= example[:description] %></td>
+            <td><%= example[:location] %></td>
+            <td><strong class="<%= example_result_css_class(example[:result]) %>"><%= example[:result] %></strong></td>
+            <td width="8%"><%= example[:last_run] %></td>
+          </tr>
+        <% end %>
+      </tbody>
+    </table>
+  </div>
+</div>

data/lib/rspec_tracer/html_reporter/views/layout.erb

@@ -19,6 +19,9 @@
 
       <div id="content">
         <%= formatted_examples('Examples', examples.values) %>
+        <% unless flaky_examples.empty? %>
+          <%= formatted_flaky_examples('Flaky Examples', flaky_examples) %>
+        <% end %>
         <%= formatted_examples_dependency('Examples Dependency', examples_dependency) %>
         <%= formatted_files_dependency("Files Dependency", files_dependency) %>
       </div>

data/lib/rspec_tracer/remote_cache/cache.rb

@@ -11,7 +11,7 @@ module RSpecTracer
 
       class LocalCacheNotFoundError < StandardError; end
 
-      CACHE_FILES_PER_TEST_SUITE = 7
+      CACHE_FILES_PER_TEST_SUITE = 8
 
       def initialize
         @s3_uri = ENV['RSPEC_TRACER_S3_URI']

data/lib/rspec_tracer/reporter.rb

@@ -2,8 +2,9 @@
 
 module RSpecTracer
   class Reporter
-    attr_reader :all_examples, :pending_examples, :all_files, :dependency,
-                :reverse_dependency, :examples_coverage, :last_run
+    attr_reader :all_examples, :possibly_flaky_examples, :flaky_examples, :pending_examples,
+                :all_files, :modified_files, :deleted_files, :dependency, :reverse_dependency,
+                :examples_coverage, :last_run
 
     def initialize
       initialize_examples
@@ -21,6 +22,7 @@ module RSpecTracer
     end
 
     def on_example_passed(example_id, result)
+      @passed_examples << example_id
       @all_examples[example_id][:execution_result] = formatted_execution_result(result)
     end
 
@@ -44,6 +46,14 @@ module RSpecTracer
       end
     end
 
+    def register_possibly_flaky_example(example_id)
+      @possibly_flaky_examples << example_id
+    end
+
+    def register_flaky_example(example_id)
+      @flaky_examples << example_id
+    end
+
     def register_failed_example(example_id)
       @failed_examples << example_id
     end
@@ -52,6 +62,10 @@ module RSpecTracer
       @pending_examples << example_id
     end
 
+    def example_passed?(example_id)
+      @passed_examples.include?(example_id)
+    end
+
     def example_skipped?(example_id)
       @skipped_examples.include?(example_id)
     end
@@ -126,6 +140,8 @@ module RSpecTracer
     end
 
     def write_reports
+      starting = Process.clock_gettime(Process::CLOCK_MONOTONIC)
+
       @run_id = Digest::MD5.hexdigest(@all_examples.keys.sort.to_json)
       @cache_dir = File.join(RSpecTracer.cache_path, @run_id)
 
@@ -133,6 +149,7 @@ module RSpecTracer
 
       %i[
         all_examples
+        flaky_examples
         failed_examples
         pending_examples
         all_files
@@ -142,13 +159,19 @@ module RSpecTracer
         last_run
       ].each { |report_type| send("write_#{report_type}_report") }
 
-      puts "RSpec tracer reports generated to #{@cache_dir}"
+      ending = Process.clock_gettime(Process::CLOCK_MONOTONIC)
+      elpased = RSpecTracer::TimeFormatter.format_time(ending - starting)
+
+      puts "RSpec tracer reports written to #{@cache_dir} (took #{elpased})"
     end
 
     private
 
     def initialize_examples
       @all_examples = {}
+      @passed_examples = Set.new
+      @possibly_flaky_examples = Set.new
+      @flaky_examples = Set.new
       @failed_examples = Set.new
       @skipped_examples = Set.new
       @pending_examples = Set.new
@@ -206,50 +229,56 @@ module RSpecTracer
     def write_all_examples_report
       file_name = File.join(@cache_dir, 'all_examples.json')
 
-      File.write(file_name, JSON.pretty_generate(@all_examples))
+      File.write(file_name, JSON.generate(@all_examples))
+    end
+
+    def write_flaky_examples_report
+      file_name = File.join(@cache_dir, 'flaky_examples.json')
+
+      File.write(file_name, JSON.generate(@flaky_examples.to_a))
     end
 
     def write_failed_examples_report
       file_name = File.join(@cache_dir, 'failed_examples.json')
 
-      File.write(file_name, JSON.pretty_generate(@failed_examples.to_a))
+      File.write(file_name, JSON.generate(@failed_examples.to_a))
     end
 
     def write_pending_examples_report
       file_name = File.join(@cache_dir, 'pending_examples.json')
 
-      File.write(file_name, JSON.pretty_generate(@pending_examples.to_a))
+      File.write(file_name, JSON.generate(@pending_examples.to_a))
     end
 
     def write_all_files_report
       file_name = File.join(@cache_dir, 'all_files.json')
 
-      File.write(file_name, JSON.pretty_generate(@all_files))
+      File.write(file_name, JSON.generate(@all_files))
     end
 
     def write_dependency_report
       file_name = File.join(@cache_dir, 'dependency.json')
 
-      File.write(file_name, JSON.pretty_generate(@dependency))
+      File.write(file_name, JSON.generate(@dependency))
     end
 
     def write_reverse_dependency_report
       file_name = File.join(@cache_dir, 'reverse_dependency.json')
 
-      File.write(file_name, JSON.pretty_generate(@reverse_dependency))
+      File.write(file_name, JSON.generate(@reverse_dependency))
     end
 
     def write_examples_coverage_report
       file_name = File.join(@cache_dir, 'examples_coverage.json')
 
-      File.write(file_name, JSON.pretty_generate(@examples_coverage))
+      File.write(file_name, JSON.generate(@examples_coverage))
     end
 
     def write_last_run_report
       file_name = File.join(RSpecTracer.cache_path, 'last_run.json')
       last_run_data = @last_run.merge(run_id: @run_id, timestamp: Time.now.utc)
 
-      File.write(file_name, JSON.pretty_generate(last_run_data))
+      File.write(file_name, JSON.generate(last_run_data))
     end
   end
 end

data/lib/rspec_tracer/rspec_runner.rb

@@ -2,24 +2,29 @@
 
 module RSpecTracer
   module RSpecRunner
+    # rubocop:disable Metrics/AbcSize
     def run_specs(_example_groups)
       actual_count = RSpec.world.example_count
+      starting = Process.clock_gettime(Process::CLOCK_MONOTONIC)
       filtered_examples, example_groups = RSpecTracer.filter_examples
 
       RSpec.world.instance_variable_set(:@filtered_examples, filtered_examples)
       RSpec.world.instance_variable_set(:@example_groups, example_groups)
 
       current_count = RSpec.world.example_count
+      ending = Process.clock_gettime(Process::CLOCK_MONOTONIC)
+      elpased = RSpecTracer::TimeFormatter.format_time(ending - starting)
 
       puts
       puts <<-EXAMPLES.strip.gsub(/\s+/, ' ')
         RSpec tracer is running #{current_count} examples (actual: #{actual_count},
-        skipped: #{actual_count - current_count})
+        skipped: #{actual_count - current_count}) (took #{elpased})
       EXAMPLES
 
       RSpecTracer.running = true
 
       super(example_groups)
     end
+    # rubocop:enable Metrics/AbcSize
   end
 end

data/lib/rspec_tracer/runner.rb

@@ -8,6 +8,7 @@ module RSpecTracer
     EXAMPLE_RUN_REASON = {
       explicit_run: 'Explicit run',
       no_cache: 'No cache',
+      flaky_example: 'Flaky example',
       failed_example: 'Failed previously',
       pending_example: 'Pending previously',
       files_changed: 'Files changed'
@@ -20,6 +21,8 @@ module RSpecTracer
       @reporter = RSpecTracer::Reporter.new
       @filtered_examples = {}
 
+      return if @cache.run_id.nil?
+
       @cache.load_cache_for_run
       filter_examples_to_run
     end
@@ -127,15 +130,19 @@ module RSpecTracer
 
     def generate_report
       @reporter.generate_last_run_report
+      generate_examples_status_report
 
-      generate_failed_examples_report
-      generate_pending_examples_report
+      %i[all_files all_examples dependency examples_coverage reverse_dependency].each do |report_type|
+        starting = Process.clock_gettime(Process::CLOCK_MONOTONIC)
 
-      %i[all_files all_examples dependency examples_coverage].each do |report_type|
         send("generate_#{report_type}_report")
+
+        ending = Process.clock_gettime(Process::CLOCK_MONOTONIC)
+        elpased = RSpecTracer::TimeFormatter.format_time(ending - starting)
+
+        puts "RSpec tracer generated #{report_type.to_s.tr('_', ' ')} report (took #{elpased})"
       end
 
-      @reporter.generate_reverse_dependency_report
       @reporter.write_reports
     end
 
@@ -146,54 +153,74 @@ module RSpecTracer
     end
 
     def filter_examples_to_run
-      add_previously_failed_examples
-      add_previously_pending_examples
+      starting = Process.clock_gettime(Process::CLOCK_MONOTONIC)
+      @changed_files = fetch_changed_files
+
+      filter_by_example_status
       filter_by_files_changed
-    end
 
-    def add_previously_failed_examples
-      @cache.failed_examples.each do |example_id|
-        @filtered_examples[example_id] = EXAMPLE_RUN_REASON[:failed_example]
-      end
+      ending = Process.clock_gettime(Process::CLOCK_MONOTONIC)
+      elpased = RSpecTracer::TimeFormatter.format_time(ending - starting)
+
+      puts "RSpec tracer processed cache (took #{elpased})"
     end
 
-    def add_previously_pending_examples
-      @cache.pending_examples.each do |example_id|
-        @filtered_examples[example_id] = EXAMPLE_RUN_REASON[:pending_example]
-      end
+    def filter_by_example_status
+      add_previously_flaky_examples
+      add_previously_failed_examples
+      add_previously_pending_examples
     end
 
     def filter_by_files_changed
       @cache.dependency.each_pair do |example_id, files|
         next if @filtered_examples.key?(example_id)
+        next if (@changed_files & files).empty?
 
-        files.each do |file_name|
-          break if filtered_by_file_changed?(example_id, file_name)
-        end
+        @filtered_examples[example_id] = EXAMPLE_RUN_REASON[:files_changed]
       end
     end
 
-    def filtered_by_file_changed?(example_id, file_name)
-      if @reporter.file_changed?(file_name)
-        @filtered_examples[example_id] = EXAMPLE_RUN_REASON[:files_changed]
+    def add_previously_flaky_examples
+      @cache.flaky_examples.each do |example_id|
+        @filtered_examples[example_id] = EXAMPLE_RUN_REASON[:flaky_example]
 
-        return true
+        next unless (@changed_files & @cache.dependency[example_id]).empty?
+
+        @reporter.register_possibly_flaky_example(example_id)
       end
+    end
 
-      source_file = registered_source_file(file_name)
+    def add_previously_failed_examples
+      @cache.failed_examples.each do |example_id|
+        next if @filtered_examples.key?(example_id)
+
+        @filtered_examples[example_id] = EXAMPLE_RUN_REASON[:failed_example]
 
-      return false if source_file &&
-        @cache.all_files[file_name][:digest] == source_file[:digest]
+        next unless (@changed_files & @cache.dependency[example_id]).empty?
+
+        @reporter.register_possibly_flaky_example(example_id)
+      end
+    end
+
+    def add_previously_pending_examples
+      @cache.pending_examples.each do |example_id|
+        @filtered_examples[example_id] = EXAMPLE_RUN_REASON[:pending_example]
+      end
+    end
 
-      @filtered_examples[example_id] = EXAMPLE_RUN_REASON[:files_changed]
+    def fetch_changed_files
+      @cache.all_files.each_value do |cached_file|
+        file_name = cached_file[:file_name]
+        source_file = RSpecTracer::SourceFile.from_name(file_name)
 
-      if source_file.nil?
-        @reporter.on_file_deleted(file_name)
-      else
-        @reporter.on_file_modified(file_name)
+        if source_file.nil?
+          @reporter.on_file_deleted(file_name)
+        elsif cached_file[:digest] != source_file[:digest]
+          @reporter.on_file_modified(file_name)
+        end
       end
 
-      true
+      @reporter.modified_files | @reporter.deleted_files
     end
 
     def generate_untraced_files(trace_point_files)
@@ -227,14 +254,23 @@ module RSpecTracer
     end
 
     def register_example_file_dependency(example_id, file_name)
-      source_file = registered_source_file(file_name)
+      source_file = RSpecTracer::SourceFile.from_name(file_name)
 
       @reporter.register_source_file(source_file)
       @reporter.register_dependency(example_id, file_name)
     end
 
-    def registered_source_file(file_name)
-      @reporter.all_files[file_name] || RSpecTracer::SourceFile.from_name(file_name)
+    def generate_examples_status_report
+      starting = Process.clock_gettime(Process::CLOCK_MONOTONIC)
+
+      generate_flaky_examples_report
+      generate_failed_examples_report
+      generate_pending_examples_report
+
+      ending = Process.clock_gettime(Process::CLOCK_MONOTONIC)
+      elpased = RSpecTracer::TimeFormatter.format_time(ending - starting)
+
+      puts "RSpec tracer generated flaky, failed, and pending examples report (took #{elpased})"
     end
 
     def generate_all_files_report
@@ -255,6 +291,16 @@ module RSpecTracer
       end
     end
 
+    def generate_flaky_examples_report
+      @reporter.possibly_flaky_examples.each do |example_id|
+        next if @reporter.example_deleted?(example_id)
+        next unless @cache.flaky_examples.include?(example_id) ||
+          @reporter.example_passed?(example_id)
+
+        @reporter.register_flaky_example(example_id)
+      end
+    end
+
     def generate_failed_examples_report
       @cache.failed_examples.each do |example_id|
         next if @reporter.example_deleted?(example_id) ||
@@ -296,5 +342,9 @@ module RSpecTracer
         end
       end
     end
+
+    def generate_reverse_dependency_report
+      @reporter.generate_reverse_dependency_report
+    end
   end
 end

data/lib/rspec_tracer/time_formatter.rb

@@ -0,0 +1,52 @@
+# frozen_string_literal: true
+
+module RSpecTracer
+  module TimeFormatter
+    DEFAULT_PRECISION = 2
+    SECONDS_PRECISION = 5
+
+    UNITS = {
+      second: 60,
+      minute: 60,
+      hour: 24,
+      day: Float::INFINITY
+    }.freeze
+
+    module_function
+
+    def format_time(seconds)
+      return pluralize(format_duration(seconds), 'second') if seconds < 60
+
+      formatted_duration = UNITS.each_pair.with_object([]) do |(unit, count), duration|
+        next unless seconds.positive?
+
+        seconds, remainder = seconds.divmod(count)
+        remainder = format_duration(remainder)
+
+        next if remainder.zero?
+
+        duration << pluralize(remainder, unit)
+      end
+
+      formatted_duration.reverse.join(' ')
+    end
+
+    def format_duration(duration)
+      return 0 if duration.negative?
+
+      precision = duration < 1 ? SECONDS_PRECISION : DEFAULT_PRECISION
+
+      format("%<duration>0.#{precision}f", duration: duration)
+    end
+
+    def pluralize(duration, unit)
+      if duration == 1
+        "#{duration} #{unit}"
+      else
+        "#{duration} #{unit}s"
+      end
+    end
+
+    private_class_method :format_duration, :pluralize
+  end
+end

data/lib/rspec_tracer/version.rb

@@ -1,5 +1,5 @@
 # frozen_string_literal: true
 
 module RSpecTracer
-  VERSION = '0.5.0'
+  VERSION = '0.6.0'
 end

data/lib/rspec_tracer.rb

@@ -23,6 +23,7 @@ require_relative 'rspec_tracer/rspec_runner'
 require_relative 'rspec_tracer/ruby_coverage'
 require_relative 'rspec_tracer/runner'
 require_relative 'rspec_tracer/source_file'
+require_relative 'rspec_tracer/time_formatter'
 require_relative 'rspec_tracer/version'
 
 module RSpecTracer
@@ -184,22 +185,36 @@ module RSpecTracer
     def generate_reports
       puts 'RSpec tracer is generating reports'
 
-      generate_tracer_reports
-      generate_coverage_reports
+      process_dependency
+      process_coverage
       runner.generate_report
       RSpecTracer::HTMLReporter::Reporter.new.generate_report
     end
 
-    def generate_tracer_reports
+    def process_dependency
+      starting = Process.clock_gettime(Process::CLOCK_MONOTONIC)
+
       runner.register_deleted_examples
       runner.register_dependency(coverage_reporter.examples_coverage)
       runner.register_untraced_dependency(@traced_files)
+
+      ending = Process.clock_gettime(Process::CLOCK_MONOTONIC)
+      elpased = RSpecTracer::TimeFormatter.format_time(ending - starting)
+
+      puts "RSpec tracer processed dependency (took #{elpased})"
     end
 
-    def generate_coverage_reports
+    def process_coverage
+      starting = Process.clock_gettime(Process::CLOCK_MONOTONIC)
+
       coverage_reporter.generate_final_examples_coverage
       coverage_reporter.merge_coverage(runner.generate_missed_coverage)
       runner.register_examples_coverage(coverage_reporter.examples_coverage)
+
+      ending = Process.clock_gettime(Process::CLOCK_MONOTONIC)
+      elpased = RSpecTracer::TimeFormatter.format_time(ending - starting)
+
+      puts "RSpec tracer processed coverage (took #{elpased})"
     end
 
     def run_simplecov_exit_task
@@ -213,12 +228,18 @@ module RSpecTracer
     end
 
     def run_coverage_exit_task
+      starting = Process.clock_gettime(Process::CLOCK_MONOTONIC)
+
       coverage_reporter.generate_final_coverage
 
       file_name = File.join(RSpecTracer.coverage_path, 'coverage.json')
 
       write_coverage_report(file_name)
-      print_coverage_stats(file_name)
+
+      ending = Process.clock_gettime(Process::CLOCK_MONOTONIC)
+      elpased = RSpecTracer::TimeFormatter.format_time(ending - starting)
+
+      print_coverage_stats(file_name, elpased)
     end
 
     def write_coverage_report(file_name)
@@ -229,15 +250,16 @@ module RSpecTracer
         }
       }
 
-      File.write(file_name, JSON.pretty_generate(report))
+      File.write(file_name, JSON.generate(report))
     end
 
-    def print_coverage_stats(file_name)
+    def print_coverage_stats(file_name, elpased)
       stat = coverage_reporter.coverage_stat
 
       puts <<-REPORT.strip.gsub(/\s+/, ' ')
         Coverage report generated for RSpecTracer to #{file_name}. #{stat[:covered_lines]}
         / #{stat[:total_lines]} LOC (#{stat[:covered_percent]}%) covered
+        (took #{elpased})
       REPORT
     end
   end

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: rspec-tracer
 version: !ruby/object:Gem::Version
-  version: 0.5.0
+  version: 0.6.0
 platform: ruby
 authors:
 - Abhimanyu Singh
 autorequire:
 bindir: bin
 cert_chain: []
-date: 2021-09-03 00:00:00.000000000 Z
+date: 2021-09-05 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: docile
@@ -90,6 +90,7 @@ files:
 - lib/rspec_tracer/html_reporter/views/examples.erb
 - lib/rspec_tracer/html_reporter/views/examples_dependency.erb
 - lib/rspec_tracer/html_reporter/views/files_dependency.erb
+- lib/rspec_tracer/html_reporter/views/flaky_examples.erb
 - lib/rspec_tracer/html_reporter/views/layout.erb
 - lib/rspec_tracer/remote_cache/cache.rb
 - lib/rspec_tracer/remote_cache/git.rb
@@ -99,13 +100,14 @@ files:
 - lib/rspec_tracer/ruby_coverage.rb
 - lib/rspec_tracer/runner.rb
 - lib/rspec_tracer/source_file.rb
+- lib/rspec_tracer/time_formatter.rb
 - lib/rspec_tracer/version.rb
 homepage: https://github.com/avmnu-sng/rspec-tracer
 licenses:
 - MIT
 metadata:
   homepage_uri: https://github.com/avmnu-sng/rspec-tracer
-  source_code_uri: https://github.com/avmnu-sng/rspec-tracer/tree/v0.5.0
+  source_code_uri: https://github.com/avmnu-sng/rspec-tracer/tree/v0.6.0
   changelog_uri: https://github.com/avmnu-sng/rspec-tracer/blob/main/CHANGELOG.md
   bug_tracker_uri: https://github.com/avmnu-sng/rspec-tracer/issues
 post_install_message: