rspec-tracer 0.5.0 → 0.7.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 2817cccf4825e28177bb8794f8d261635f81ec5c1bd6ca0242ba171e702850ca
-  data.tar.gz: 9c5f6af579eec43a994c28b5e8132fec5d71f7a2ddddfb531fafd5c51943d283
+  metadata.gz: 4825c225fdac05b09d4c56c5b6b191d0b32ad4198fc39b72a215bef3d066f7bd
+  data.tar.gz: 4bdcb243ed2cf200f6dae4b00b4dfd50fbc8bb70b9f59552056059ec3be643b1
 SHA512:
-  metadata.gz: 2d3af598e09b8ac043b5f605acf39b588701c4f0e7b4d74646eb0da74c1c85746db60f215e02cd5fba1880bb75288d0d35df3986b08cc71cfbad4d45ad722d2f
-  data.tar.gz: 7ce36336213d4d37337afa5b50fd774bd6ef7b235539f3a289f1a804a878fba3fa1e7972e9bd09bb8083c80929c0fc3b6bf0b30f257542fdb5e5c466c1247360
+  metadata.gz: 80dff35e9e15e2b28bd5c0658fdc027b659a9a1cf4265e17d7207ce0327bc975b05890d9e23fd9b22faa81b6a471d9a3d68d5bec0e501314907534796ff6088c
+  data.tar.gz: 77d1e9a4a9c4fe26713da06d594e7c6088655a2bc84c7e38ccd145c27f60247a9554043631c526c4a262637304c730c59ae767b3d4e8462ebc4e7e397e330055

data/CHANGELOG.md

@@ -1,3 +1,39 @@
+## [0.7.0] - 2021-09-10
+
+### Fixed
+
+Missing spec files for the gem
+
+## [0.6.2] - 2021-09-07
+
+### Added
+
+Improvements towards reducing dependency and coverage processing time (#26)
+
+## [0.6.1] - 2021-09-06
+
+### Fixed
+
+Bug in time formatter (#24)
+
+### Added
+
+Environment variable to control verbose output (#25)
+
+## [0.6.0] - 2021-09-05
+
+### Added
+
+- Improved dependency change detection (#18)
+- Flaky tests detection (#19)
+- Exclude vendor files from analysis (#21)
+- Report elapsed time at various stages (#23)
+
+### Note
+
+The first run on this version will not use any cache on the CI because the number
+of files changed from eight to nine, so there will be no appropriate cache to use.
+
 ## [0.5.0] - 2021-09-03
 
 ### Fixed

data/README.md

@@ -8,7 +8,11 @@ It uses [Ruby's built-in coverage library](https://ruby-doc.org/stdlib/libdoc/co
 to keep track of the coverage for each test. For each test executed, the coverage
 diff provides the desired file list. RSpec Tracer takes care of reporting the
 **correct code coverage when skipping tests** by using the cached reports. Also,
-note that it will **never skip any tests which failed or were pending** in the last runs.
+note that it will **never skip**:
+
+- **Flaky examples**
+- **Failed examples**
+- **Pending examples**
 
 Knowing the examples and files dependency gives us a better insight into the codebase,
 and we have **a clear idea of what to test for when making any changes**. With this data,
@@ -16,9 +20,10 @@ we can also analyze the coupling between different components and much more.
 
 ## Note
 
-You should take some time and go through the [document](./RSPEC_TRACER.md) describing
-the **intention** and implementation details of **skipping tests**, **managing coverage**,
-and **caching on CI**, etc.
+You should take some time and go through the **[document](./RSPEC_TRACER.md)** describing
+the **intention** and implementation details of **managing dependency**, **managing flaky tests**,
+**skipping tests**, and **caching on CI**. You must go through the README file before
+integrating the gem into your project to better understand what is happening.
 
 ## Table of Contents
 
@@ -28,15 +33,18 @@ and **caching on CI**, etc.
   * [Additional Tools](#additional-tools)
 * [Getting Started](#getting-started)
 * [Environment Variables](#environment-variables)
+  * [BUNDLE_PATH](#bundle_path)
   * [CI](#ci)
   * [LOCAL_AWS](#local_aws)
   * [RSPEC_TRACER_NO_SKIP](#rspec_tracer_no_skip)
   * [RSPEC_TRACER_S3_URI](#rspec_tracer_s3_uri)
   * [RSPEC_TRACER_UPLOAD_LOCAL_CACHE](#rspec_tracer_upload_local_cache)
+  * [RSPEC_TRACER_VERBOSE](#rspec_tracer_verbose)
   * [TEST_SUITES](#test_suites)
   * [TEST_SUITE_ID](#test_suite_id)
 * [Sample Reports](#sample-reports)
     * [Examples](#examples)
+    * [Flaky Examples](#flaky-examples)
     * [Examples Dependency](#examples-dependency)
     * [Files Dependency](#files-dependency)
 * [Configuring RSpec Tracer](#configuring-rspec-tracer)
@@ -63,7 +71,7 @@ and **caching on CI**, etc.
 
 Add this line to your `Gemfile` and `bundle install`:
 ```ruby
-gem 'rspec-tracer', group: :test, require: false
+gem 'rspec-tracer', '~> 0.6', group: :test, require: false
 ```
 
 And, add the followings to your `.gitignore`:
@@ -142,7 +150,15 @@ browser of your choice.
 
 ## Environment Variables
 
-To get better control on execution, you can use the following two environment variables:
+To get better control on execution, you can use the following environment variables
+whenever required.
+
+### BUNDLE_PATH
+
+Since the bundler uses a vendor directory inside the project, it might cause slowness
+depending on the vendor size. You can configure the bundle path outside of the project
+using `BUNDLE_PATH` environment variable, for example, `BUNDLE_PATH=$HOME/vendor/bundle`.
+Make sure to cache this directory in the CI configuration.
 
 ### CI
 
@@ -177,6 +193,14 @@ export RSPEC_TRACER_S3_URI=s3://ci-artifacts-bucket/rspec-tracer-cache
 By default, RSpec Tracer does not upload local cache files. You can set this
 environment variable to `true` to upload the local cache to S3.
 
+### RSPEC_TRACER_VERBOSE
+
+To print the intermediate steps and time taken, use this environment variable:
+
+```sh
+export RSPEC_TRACER_VERBOSE=true
+```
+
 ### TEST_SUITES
 
 Set this environment variable when using test suite id. It determines the total
@@ -190,7 +214,7 @@ export TEST_SUITES=8
 
 If you have a large set of tests to run, it is recommended to run them in
 separate groups. This way, RSpec Tracer is not overwhelmed with loading massive
-cached data in the memory. Also, it generate and use cache for specific test suites
+cached data in the memory. Also, it generates and uses cache for specific test suites
 and not merge them.
 
 ```ruby
@@ -229,6 +253,19 @@ These reports provide basic test information:
 
 ![](./readme_files/examples_report_next_run.png)
 
+### Flaky Examples
+
+These reports provide flaky tests information. Assuming **the following two tests
+failed in the first run.**
+
+**Next Run**
+
+![](./readme_files/flaky_examples_report_first_run.png)
+
+**Another Run**
+
+![](./readme_files/flaky_examples_report_next_run.png)
+
 ### Examples Dependency
 
 These reports show a list of dependent files for each test.

data/lib/rspec_tracer/cache.rb

@@ -2,7 +2,8 @@
 
 module RSpecTracer
   class Cache
-    attr_reader :all_examples, :failed_examples, :pending_examples, :all_files, :dependency
+    attr_reader :all_examples, :flaky_examples, :failed_examples, :pending_examples,
+                :all_files, :dependency, :run_id
 
     def initialize
       @run_id = last_run_id
@@ -11,6 +12,7 @@ module RSpecTracer
       @cached = false
 
       @all_examples = {}
+      @flaky_examples = Set.new
       @failed_examples = Set.new
       @pending_examples = Set.new
       @all_files = {}
@@ -20,22 +22,36 @@ module RSpecTracer
     def load_cache_for_run
       return if @run_id.nil? || @cached
 
+      starting = Process.clock_gettime(Process::CLOCK_MONOTONIC)
+
       load_all_examples_cache
+      load_flaky_examples_cache
       load_failed_examples_cache
       load_pending_examples_cache
       load_all_files_cache
       load_dependency_cache
 
+      ending = Process.clock_gettime(Process::CLOCK_MONOTONIC)
+
       @cached = true
 
-      puts "RSpec tracer loaded cache from #{@cache_dir}" if @run_id
+      elpased = RSpecTracer::TimeFormatter.format_time(ending - starting)
+
+      puts "RSpec tracer loaded cache from #{@cache_dir} (took #{elpased})"
     end
 
     def cached_examples_coverage
       return @examples_coverage if defined?(@examples_coverage)
       return @examples_coverage = {} if @run_id.nil?
 
-      load_examples_coverage_cache
+      starting = Process.clock_gettime(Process::CLOCK_MONOTONIC)
+      coverage = load_examples_coverage_cache
+      ending = Process.clock_gettime(Process::CLOCK_MONOTONIC)
+      elpased = RSpecTracer::TimeFormatter.format_time(ending - starting)
+
+      puts "RSpec tracer loaded cached examples coverage (took #{elpased})" if RSpecTracer.verbose?
+
+      coverage
     end
 
     private
@@ -64,6 +80,14 @@ module RSpecTracer
       end
     end
 
+    def load_flaky_examples_cache
+      file_name = File.join(@cache_dir, 'flaky_examples.json')
+
+      return unless File.file?(file_name)
+
+      @flaky_examples = JSON.parse(File.read(file_name)).to_set
+    end
+
     def load_failed_examples_cache
       file_name = File.join(@cache_dir, 'failed_examples.json')
 

data/lib/rspec_tracer/configuration.rb

@@ -102,6 +102,12 @@ module RSpecTracer
       @coverage_filters ||= []
     end
 
+    def verbose?
+      return @verbose if defined?(@verbose)
+
+      @verbose = ENV.fetch('RSPEC_TRACER_VERBOSE', 'false') == 'true'
+    end
+
     def configure(&block)
       Docile.dsl_eval(self, &block)
     end

data/lib/rspec_tracer/coverage_reporter.rb

@@ -29,11 +29,15 @@ module RSpecTracer
 
     def compute_diff(example_id)
       peek_coverage.each_pair do |file_path, current_stats|
-        if @coverage.key?(file_path)
-          existing_file_diff_coverage(example_id, file_path, current_stats)
-        else
+        unless @coverage.key?(file_path)
           missing_file_diff_coverage(example_id, file_path, current_stats)
+
+          next
         end
+
+        next if current_stats == @coverage[file_path]
+
+        existing_file_diff_coverage(example_id, file_path, current_stats)
       end
     end
 

data/lib/rspec_tracer/defaults.rb

@@ -2,7 +2,13 @@
 
 RSpecTracer.configure do
   add_filter '/vendor/bundle/'
-  add_coverage_filter %w[/autotest/ /features/ /spec/ /test/].freeze
+  add_coverage_filter %w[
+    /autotest/
+    /features/
+    /spec/
+    /test/
+    /vendor/bundle/
+  ].freeze
 end
 
 at_exit do

data/lib/rspec_tracer/html_reporter/Rakefile

@@ -0,0 +1,18 @@
+# frozen_string_literal: true
+
+namespace :assets do
+  desc 'Compiles all assets'
+  task :precompile do
+    require 'sprockets'
+
+    assets = Sprockets::Environment.new do |env|
+      env.append_path 'assets/javascripts'
+      env.append_path 'assets/stylesheets'
+      env.js_compressor = :uglifier
+      env.css_compressor = :yui
+    end
+
+    assets['application.js'].write_to('public/application.js')
+    assets['application.css'].write_to('public/application.css')
+  end
+end

data/lib/rspec_tracer/html_reporter/reporter.rb

@@ -6,21 +6,22 @@ require 'time'
 module RSpecTracer
   module HTMLReporter
     class Reporter
-      attr_reader :last_run, :examples, :examples_dependency, :files_dependency
+      attr_reader :last_run, :examples, :flaky_examples, :examples_dependency, :files_dependency
 
       def initialize
         @reporter = RSpecTracer.runner.reporter
 
         format_last_run
         format_examples
+        format_flaky_examples
         format_examples_dependency
         format_files_dependency
       end
 
       def generate_report
-        Dir[File.join(File.dirname(__FILE__), 'public/*')].each do |path|
-          FileUtils.cp_r(path, asset_output_path)
-        end
+        starting = Process.clock_gettime(Process::CLOCK_MONOTONIC)
+
+        copy_assets
 
         file_name = File.join(RSpecTracer.report_path, 'index.html')
 
@@ -28,11 +29,20 @@ module RSpecTracer
           file.puts(template('layout').result(binding))
         end
 
-        puts "RSpecTracer generated HTML report to #{file_name}"
+        ending = Process.clock_gettime(Process::CLOCK_MONOTONIC)
+        elpased = RSpecTracer::TimeFormatter.format_time(ending - starting)
+
+        puts "RSpecTracer generated HTML report to #{file_name} (took #{elpased})"
       end
 
       private
 
+      def copy_assets
+        Dir[File.join(File.dirname(__FILE__), 'public/*')].each do |path|
+          FileUtils.cp_r(path, asset_output_path)
+        end
+      end
+
       def format_last_run
         @last_run = @reporter.last_run.slice(
           :actual_count,
@@ -57,6 +67,10 @@ module RSpecTracer
         end
       end
 
+      def format_flaky_examples
+        @flaky_examples = @examples.slice(*@reporter.flaky_examples).values
+      end
+
       def example_run_local_time(utc_time)
         case utc_time
         when Time
@@ -128,6 +142,14 @@ module RSpecTracer
         template(title_id).result(current_binding)
       end
 
+      def formatted_flaky_examples(title, flaky_examples)
+        title_id = report_container_id(title)
+        current_binding = binding
+
+        current_binding.local_variable_set(:title_id, title_id)
+        template(title_id).result(current_binding)
+      end
+
       def formatted_examples_dependency(title, examples_dependency)
         title_id = report_container_id(title)
         current_binding = binding
@@ -154,7 +176,7 @@ module RSpecTracer
 
       def example_status_css_class(example_status)
         case example_status.split.first
-        when 'Failed'
+        when 'Failed', 'Flaky'
           'red'
         when 'Pending'
           'yellow'

data/lib/rspec_tracer/html_reporter/views/flaky_examples.erb

@@ -0,0 +1,38 @@
+<div class="report_container" id="<%= title_id %>">
+  <h2>
+    <span class="group_name"><%= title %></span>
+    (
+      <span class="blue">
+        <strong><%= flaky_examples.count %></strong>
+      </span> examples
+    )
+  </h2>
+
+  <a name="<%= title_id %>"></a>
+
+  <div class="report-table--responsive">
+    <table class="report-table">
+      <thead>
+        <tr>
+          <th>ID</th>
+          <th>Description</th>
+          <th>Location</th>
+          <th>Result</th>
+          <th>Run At</th>
+        </tr>
+      </thead>
+
+      <tbody>
+        <% flaky_examples.each do |example| %>
+          <tr>
+            <td><%= example[:id] %></td>
+            <td><%= example[:description] %></td>
+            <td><%= example[:location] %></td>
+            <td><strong class="<%= example_result_css_class(example[:result]) %>"><%= example[:result] %></strong></td>
+            <td width="8%"><%= example[:last_run] %></td>
+          </tr>
+        <% end %>
+      </tbody>
+    </table>
+  </div>
+</div>

data/lib/rspec_tracer/html_reporter/views/layout.erb

@@ -19,6 +19,9 @@
 
       <div id="content">
         <%= formatted_examples('Examples', examples.values) %>
+        <% unless flaky_examples.empty? %>
+          <%= formatted_flaky_examples('Flaky Examples', flaky_examples) %>
+        <% end %>
         <%= formatted_examples_dependency('Examples Dependency', examples_dependency) %>
         <%= formatted_files_dependency("Files Dependency", files_dependency) %>
       </div>

data/lib/rspec_tracer/remote_cache/Rakefile

@@ -0,0 +1,38 @@
+# frozen_string_literal: true
+
+namespace :rspec_tracer do
+  namespace :remote_cache do
+    desc 'Download cache'
+    task :download do
+      unless system('git', 'rev-parse', 'HEAD', out: File::NULL, err: File::NULL)
+        puts 'Not a git repository'
+
+        exit
+      end
+
+      require 'rspec_tracer'
+
+      RSpecTracer::RemoteCache::Cache.new.download
+    end
+
+    desc 'Upload cache'
+    task :upload do
+      unless system('git', 'rev-parse', 'HEAD', out: File::NULL, err: File::NULL)
+        puts 'Not a git repository'
+
+        exit
+      end
+
+      unless ENV.fetch('CI', 'false') == 'true' || ENV.fetch('RSPEC_TRACER_UPLOAD_LOCAL_CACHE', 'false') == 'true'
+        puts 'Skipping upload from local development environment'
+        puts 'Use RSPEC_TRACER_UPLOAD_LOCAL_CACHE=true to upload local cache'
+
+        exit
+      end
+
+      require 'rspec_tracer'
+
+      RSpecTracer::RemoteCache::Cache.new.upload
+    end
+  end
+end

data/lib/rspec_tracer/remote_cache/cache.rb

@@ -11,7 +11,7 @@ module RSpecTracer
 
       class LocalCacheNotFoundError < StandardError; end
 
-      CACHE_FILES_PER_TEST_SUITE = 7
+      CACHE_FILES_PER_TEST_SUITE = 8
 
       def initialize
         @s3_uri = ENV['RSPEC_TRACER_S3_URI']

data/lib/rspec_tracer/reporter.rb

@@ -2,8 +2,9 @@
 
 module RSpecTracer
   class Reporter
-    attr_reader :all_examples, :pending_examples, :all_files, :dependency,
-                :reverse_dependency, :examples_coverage, :last_run
+    attr_reader :all_examples, :possibly_flaky_examples, :flaky_examples, :pending_examples,
+                :all_files, :modified_files, :deleted_files, :dependency, :reverse_dependency,
+                :examples_coverage, :last_run
 
     def initialize
       initialize_examples
@@ -21,6 +22,7 @@ module RSpecTracer
     end
 
     def on_example_passed(example_id, result)
+      @passed_examples << example_id
       @all_examples[example_id][:execution_result] = formatted_execution_result(result)
     end
 
@@ -44,6 +46,14 @@ module RSpecTracer
       end
     end
 
+    def register_possibly_flaky_example(example_id)
+      @possibly_flaky_examples << example_id
+    end
+
+    def register_flaky_example(example_id)
+      @flaky_examples << example_id
+    end
+
     def register_failed_example(example_id)
       @failed_examples << example_id
     end
@@ -52,6 +62,10 @@ module RSpecTracer
       @pending_examples << example_id
     end
 
+    def example_passed?(example_id)
+      @passed_examples.include?(example_id)
+    end
+
     def example_skipped?(example_id)
       @skipped_examples.include?(example_id)
     end
@@ -126,6 +140,8 @@ module RSpecTracer
     end
 
     def write_reports
+      starting = Process.clock_gettime(Process::CLOCK_MONOTONIC)
+
       @run_id = Digest::MD5.hexdigest(@all_examples.keys.sort.to_json)
       @cache_dir = File.join(RSpecTracer.cache_path, @run_id)
 
@@ -133,6 +149,7 @@ module RSpecTracer
 
       %i[
         all_examples
+        flaky_examples
         failed_examples
         pending_examples
         all_files
@@ -142,13 +159,19 @@ module RSpecTracer
         last_run
       ].each { |report_type| send("write_#{report_type}_report") }
 
-      puts "RSpec tracer reports generated to #{@cache_dir}"
+      ending = Process.clock_gettime(Process::CLOCK_MONOTONIC)
+      elpased = RSpecTracer::TimeFormatter.format_time(ending - starting)
+
+      puts "RSpec tracer reports written to #{@cache_dir} (took #{elpased})"
     end
 
     private
 
     def initialize_examples
       @all_examples = {}
+      @passed_examples = Set.new
+      @possibly_flaky_examples = Set.new
+      @flaky_examples = Set.new
       @failed_examples = Set.new
       @skipped_examples = Set.new
       @pending_examples = Set.new
@@ -209,6 +232,12 @@ module RSpecTracer
       File.write(file_name, JSON.pretty_generate(@all_examples))
     end
 
+    def write_flaky_examples_report
+      file_name = File.join(@cache_dir, 'flaky_examples.json')
+
+      File.write(file_name, JSON.pretty_generate(@flaky_examples.to_a))
+    end
+
     def write_failed_examples_report
       file_name = File.join(@cache_dir, 'failed_examples.json')
 

data/lib/rspec_tracer/rspec_runner.rb

@@ -2,24 +2,29 @@
 
 module RSpecTracer
   module RSpecRunner
+    # rubocop:disable Metrics/AbcSize
     def run_specs(_example_groups)
       actual_count = RSpec.world.example_count
+      starting = Process.clock_gettime(Process::CLOCK_MONOTONIC)
       filtered_examples, example_groups = RSpecTracer.filter_examples
 
       RSpec.world.instance_variable_set(:@filtered_examples, filtered_examples)
       RSpec.world.instance_variable_set(:@example_groups, example_groups)
 
       current_count = RSpec.world.example_count
+      ending = Process.clock_gettime(Process::CLOCK_MONOTONIC)
+      elpased = RSpecTracer::TimeFormatter.format_time(ending - starting)
 
       puts
       puts <<-EXAMPLES.strip.gsub(/\s+/, ' ')
         RSpec tracer is running #{current_count} examples (actual: #{actual_count},
-        skipped: #{actual_count - current_count})
+        skipped: #{actual_count - current_count}) (took #{elpased})
       EXAMPLES
 
       RSpecTracer.running = true
 
       super(example_groups)
     end
+    # rubocop:enable Metrics/AbcSize
   end
 end

data/lib/rspec_tracer/runner.rb

@@ -8,6 +8,7 @@ module RSpecTracer
     EXAMPLE_RUN_REASON = {
       explicit_run: 'Explicit run',
       no_cache: 'No cache',
+      flaky_example: 'Flaky example',
       failed_example: 'Failed previously',
       pending_example: 'Pending previously',
       files_changed: 'Files changed'
@@ -20,6 +21,8 @@ module RSpecTracer
       @reporter = RSpecTracer::Reporter.new
       @filtered_examples = {}
 
+      return if @cache.run_id.nil?
+
       @cache.load_cache_for_run
       filter_examples_to_run
     end
@@ -87,16 +90,15 @@ module RSpecTracer
     # rubocop:enable Metrics/AbcSize
 
     def register_dependency(examples_coverage)
+      filtered_files = Set.new
+
       examples_coverage.each_pair do |example_id, example_coverage|
         register_example_files_dependency(example_id)
 
         example_coverage.each_key do |file_path|
-          source_file = RSpecTracer::SourceFile.from_path(file_path)
+          next if filtered_files.include?(file_path)
 
-          next if RSpecTracer.filters.any? { |filter| filter.match?(source_file) }
-
-          @reporter.register_source_file(source_file)
-          @reporter.register_dependency(example_id, source_file[:file_name])
+          filtered_files << file_path unless register_file_dependency(example_id, file_path)
         end
       end
 
@@ -127,15 +129,19 @@ module RSpecTracer
 
     def generate_report
       @reporter.generate_last_run_report
+      generate_examples_status_report
 
-      generate_failed_examples_report
-      generate_pending_examples_report
+      %i[all_files all_examples dependency examples_coverage reverse_dependency].each do |report_type|
+        starting = Process.clock_gettime(Process::CLOCK_MONOTONIC)
 
-      %i[all_files all_examples dependency examples_coverage].each do |report_type|
         send("generate_#{report_type}_report")
+
+        ending = Process.clock_gettime(Process::CLOCK_MONOTONIC)
+        elpased = RSpecTracer::TimeFormatter.format_time(ending - starting)
+
+        puts "RSpec tracer generated #{report_type.to_s.tr('_', ' ')} report (took #{elpased})" if RSpecTracer.verbose?
       end
 
-      @reporter.generate_reverse_dependency_report
       @reporter.write_reports
     end
 
@@ -146,54 +152,74 @@ module RSpecTracer
     end
 
     def filter_examples_to_run
-      add_previously_failed_examples
-      add_previously_pending_examples
+      starting = Process.clock_gettime(Process::CLOCK_MONOTONIC)
+      @changed_files = fetch_changed_files
+
+      filter_by_example_status
       filter_by_files_changed
-    end
 
-    def add_previously_failed_examples
-      @cache.failed_examples.each do |example_id|
-        @filtered_examples[example_id] = EXAMPLE_RUN_REASON[:failed_example]
-      end
+      ending = Process.clock_gettime(Process::CLOCK_MONOTONIC)
+      elpased = RSpecTracer::TimeFormatter.format_time(ending - starting)
+
+      puts "RSpec tracer processed cache (took #{elpased})" if RSpecTracer.verbose?
     end
 
-    def add_previously_pending_examples
-      @cache.pending_examples.each do |example_id|
-        @filtered_examples[example_id] = EXAMPLE_RUN_REASON[:pending_example]
-      end
+    def filter_by_example_status
+      add_previously_flaky_examples
+      add_previously_failed_examples
+      add_previously_pending_examples
     end
 
     def filter_by_files_changed
       @cache.dependency.each_pair do |example_id, files|
         next if @filtered_examples.key?(example_id)
+        next if (@changed_files & files).empty?
 
-        files.each do |file_name|
-          break if filtered_by_file_changed?(example_id, file_name)
-        end
+        @filtered_examples[example_id] = EXAMPLE_RUN_REASON[:files_changed]
       end
     end
 
-    def filtered_by_file_changed?(example_id, file_name)
-      if @reporter.file_changed?(file_name)
-        @filtered_examples[example_id] = EXAMPLE_RUN_REASON[:files_changed]
+    def add_previously_flaky_examples
+      @cache.flaky_examples.each do |example_id|
+        @filtered_examples[example_id] = EXAMPLE_RUN_REASON[:flaky_example]
+
+        next unless (@changed_files & @cache.dependency[example_id]).empty?
 
-        return true
+        @reporter.register_possibly_flaky_example(example_id)
       end
+    end
+
+    def add_previously_failed_examples
+      @cache.failed_examples.each do |example_id|
+        next if @filtered_examples.key?(example_id)
+
+        @filtered_examples[example_id] = EXAMPLE_RUN_REASON[:failed_example]
+
+        next unless (@changed_files & @cache.dependency[example_id]).empty?
 
-      source_file = registered_source_file(file_name)
+        @reporter.register_possibly_flaky_example(example_id)
+      end
+    end
 
-      return false if source_file &&
-        @cache.all_files[file_name][:digest] == source_file[:digest]
+    def add_previously_pending_examples
+      @cache.pending_examples.each do |example_id|
+        @filtered_examples[example_id] = EXAMPLE_RUN_REASON[:pending_example]
+      end
+    end
 
-      @filtered_examples[example_id] = EXAMPLE_RUN_REASON[:files_changed]
+    def fetch_changed_files
+      @cache.all_files.each_value do |cached_file|
+        file_name = cached_file[:file_name]
+        source_file = RSpecTracer::SourceFile.from_name(file_name)
 
-      if source_file.nil?
-        @reporter.on_file_deleted(file_name)
-      else
-        @reporter.on_file_modified(file_name)
+        if source_file.nil?
+          @reporter.on_file_deleted(file_name)
+        elsif cached_file[:digest] != source_file[:digest]
+          @reporter.on_file_modified(file_name)
+        end
       end
 
-      true
+      @reporter.modified_files | @reporter.deleted_files
     end
 
     def generate_untraced_files(trace_point_files)
@@ -227,14 +253,34 @@ module RSpecTracer
     end
 
     def register_example_file_dependency(example_id, file_name)
-      source_file = registered_source_file(file_name)
+      source_file = RSpecTracer::SourceFile.from_name(file_name)
 
       @reporter.register_source_file(source_file)
       @reporter.register_dependency(example_id, file_name)
     end
 
-    def registered_source_file(file_name)
-      @reporter.all_files[file_name] || RSpecTracer::SourceFile.from_name(file_name)
+    def register_file_dependency(example_id, file_path)
+      source_file = RSpecTracer::SourceFile.from_path(file_path)
+
+      return false if RSpecTracer.filters.any? { |filter| filter.match?(source_file) }
+
+      @reporter.register_source_file(source_file)
+      @reporter.register_dependency(example_id, source_file[:file_name])
+
+      true
+    end
+
+    def generate_examples_status_report
+      starting = Process.clock_gettime(Process::CLOCK_MONOTONIC)
+
+      generate_flaky_examples_report
+      generate_failed_examples_report
+      generate_pending_examples_report
+
+      ending = Process.clock_gettime(Process::CLOCK_MONOTONIC)
+      elpased = RSpecTracer::TimeFormatter.format_time(ending - starting)
+
+      puts "RSpec tracer generated flaky, failed, and pending examples report (took #{elpased})" if RSpecTracer.verbose?
     end
 
     def generate_all_files_report
@@ -255,6 +301,16 @@ module RSpecTracer
       end
     end
 
+    def generate_flaky_examples_report
+      @reporter.possibly_flaky_examples.each do |example_id|
+        next if @reporter.example_deleted?(example_id)
+        next unless @cache.flaky_examples.include?(example_id) ||
+          @reporter.example_passed?(example_id)
+
+        @reporter.register_flaky_example(example_id)
+      end
+    end
+
     def generate_failed_examples_report
       @cache.failed_examples.each do |example_id|
         next if @reporter.example_deleted?(example_id) ||
@@ -296,5 +352,9 @@ module RSpecTracer
         end
       end
     end
+
+    def generate_reverse_dependency_report
+      @reporter.generate_reverse_dependency_report
+    end
   end
 end

data/lib/rspec_tracer/time_formatter.rb

@@ -0,0 +1,55 @@
+# frozen_string_literal: true
+
+module RSpecTracer
+  module TimeFormatter
+    DEFAULT_PRECISION = 2
+    SECONDS_PRECISION = 5
+
+    UNITS = {
+      second: 60,
+      minute: 60,
+      hour: 24,
+      day: Float::INFINITY
+    }.freeze
+
+    module_function
+
+    def format_time(seconds)
+      return pluralize(format_duration(seconds), 'second') if seconds < 60
+
+      formatted_duration = UNITS.each_pair.with_object([]) do |(unit, count), duration|
+        next unless seconds.positive?
+
+        seconds, remainder = seconds.divmod(count)
+
+        next if remainder.zero?
+
+        duration << pluralize(format_duration(remainder), unit)
+      end
+
+      formatted_duration.reverse.join(' ')
+    end
+
+    def format_duration(duration)
+      return 0 if duration.negative?
+
+      precision = duration < 1 ? SECONDS_PRECISION : DEFAULT_PRECISION
+
+      strip_trailing_zeroes(format("%<duration>0.#{precision}f", duration: duration))
+    end
+
+    def strip_trailing_zeroes(formatted_duration)
+      formatted_duration.sub(/(?:(\..*[^0])0+|\.0+)$/, '\1')
+    end
+
+    def pluralize(duration, unit)
+      if (duration.to_f - 1).abs < Float::EPSILON
+        "#{duration} #{unit}"
+      else
+        "#{duration} #{unit}s"
+      end
+    end
+
+    private_class_method :format_duration, :strip_trailing_zeroes, :pluralize
+  end
+end

data/lib/rspec_tracer/version.rb

@@ -1,5 +1,5 @@
 # frozen_string_literal: true
 
 module RSpecTracer
-  VERSION = '0.5.0'
+  VERSION = '0.7.0'
 end

data/lib/rspec_tracer.rb

@@ -23,6 +23,7 @@ require_relative 'rspec_tracer/rspec_runner'
 require_relative 'rspec_tracer/ruby_coverage'
 require_relative 'rspec_tracer/runner'
 require_relative 'rspec_tracer/source_file'
+require_relative 'rspec_tracer/time_formatter'
 require_relative 'rspec_tracer/version'
 
 module RSpecTracer
@@ -184,22 +185,36 @@ module RSpecTracer
     def generate_reports
       puts 'RSpec tracer is generating reports'
 
-      generate_tracer_reports
-      generate_coverage_reports
+      process_dependency
+      process_coverage
       runner.generate_report
       RSpecTracer::HTMLReporter::Reporter.new.generate_report
     end
 
-    def generate_tracer_reports
+    def process_dependency
+      starting = Process.clock_gettime(Process::CLOCK_MONOTONIC)
+
       runner.register_deleted_examples
       runner.register_dependency(coverage_reporter.examples_coverage)
       runner.register_untraced_dependency(@traced_files)
+
+      ending = Process.clock_gettime(Process::CLOCK_MONOTONIC)
+      elpased = RSpecTracer::TimeFormatter.format_time(ending - starting)
+
+      puts "RSpec tracer processed dependency (took #{elpased})" if RSpecTracer.verbose?
     end
 
-    def generate_coverage_reports
+    def process_coverage
+      starting = Process.clock_gettime(Process::CLOCK_MONOTONIC)
+
       coverage_reporter.generate_final_examples_coverage
       coverage_reporter.merge_coverage(runner.generate_missed_coverage)
       runner.register_examples_coverage(coverage_reporter.examples_coverage)
+
+      ending = Process.clock_gettime(Process::CLOCK_MONOTONIC)
+      elpased = RSpecTracer::TimeFormatter.format_time(ending - starting)
+
+      puts "RSpec tracer processed coverage (took #{elpased})" if RSpecTracer.verbose?
     end
 
     def run_simplecov_exit_task
@@ -213,12 +228,18 @@ module RSpecTracer
     end
 
     def run_coverage_exit_task
+      starting = Process.clock_gettime(Process::CLOCK_MONOTONIC)
+
       coverage_reporter.generate_final_coverage
 
       file_name = File.join(RSpecTracer.coverage_path, 'coverage.json')
 
       write_coverage_report(file_name)
-      print_coverage_stats(file_name)
+
+      ending = Process.clock_gettime(Process::CLOCK_MONOTONIC)
+      elpased = RSpecTracer::TimeFormatter.format_time(ending - starting)
+
+      print_coverage_stats(file_name, elpased)
     end
 
     def write_coverage_report(file_name)
@@ -232,12 +253,13 @@ module RSpecTracer
       File.write(file_name, JSON.pretty_generate(report))
     end
 
-    def print_coverage_stats(file_name)
+    def print_coverage_stats(file_name, elpased)
       stat = coverage_reporter.coverage_stat
 
       puts <<-REPORT.strip.gsub(/\s+/, ' ')
         Coverage report generated for RSpecTracer to #{file_name}. #{stat[:covered_lines]}
         / #{stat[:total_lines]} LOC (#{stat[:covered_percent]}%) covered
+        (took #{elpased})
       REPORT
     end
   end

metadata

@@ -1,55 +1,55 @@
 --- !ruby/object:Gem::Specification
 name: rspec-tracer
 version: !ruby/object:Gem::Version
-  version: 0.5.0
+  version: 0.7.0
 platform: ruby
 authors:
 - Abhimanyu Singh
 autorequire:
 bindir: bin
 cert_chain: []
-date: 2021-09-03 00:00:00.000000000 Z
+date: 2021-09-10 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: docile
   requirement: !ruby/object:Gem::Requirement
     requirements:
-    - - ">="
-      - !ruby/object:Gem::Version
-        version: 1.1.0
     - - "~>"
       - !ruby/object:Gem::Version
         version: '1.1'
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: 1.1.0
   type: :runtime
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
-    - - ">="
-      - !ruby/object:Gem::Version
-        version: 1.1.0
     - - "~>"
       - !ruby/object:Gem::Version
         version: '1.1'
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: 1.1.0
 - !ruby/object:Gem::Dependency
   name: rspec-core
   requirement: !ruby/object:Gem::Requirement
     requirements:
-    - - ">="
-      - !ruby/object:Gem::Version
-        version: 3.6.0
     - - "~>"
       - !ruby/object:Gem::Version
         version: '3.6'
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: 3.6.0
   type: :runtime
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
-    - - ">="
-      - !ruby/object:Gem::Version
-        version: 3.6.0
     - - "~>"
       - !ruby/object:Gem::Version
         version: '3.6'
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: 3.6.0
 description: RSpec Tracer is a specs dependency analysis tool and a test skipper for
   RSpec. It maintains a list of files for each test, enabling itself to skip tests
   in the subsequent runs if none of the dependent files are changed.
@@ -69,6 +69,7 @@ files:
 - lib/rspec_tracer/defaults.rb
 - lib/rspec_tracer/example.rb
 - lib/rspec_tracer/filter.rb
+- lib/rspec_tracer/html_reporter/Rakefile
 - lib/rspec_tracer/html_reporter/assets/javascripts/application.js
 - lib/rspec_tracer/html_reporter/assets/javascripts/libraries/jquery.js
 - lib/rspec_tracer/html_reporter/assets/javascripts/plugins/datatables.js
@@ -90,7 +91,9 @@ files:
 - lib/rspec_tracer/html_reporter/views/examples.erb
 - lib/rspec_tracer/html_reporter/views/examples_dependency.erb
 - lib/rspec_tracer/html_reporter/views/files_dependency.erb
+- lib/rspec_tracer/html_reporter/views/flaky_examples.erb
 - lib/rspec_tracer/html_reporter/views/layout.erb
+- lib/rspec_tracer/remote_cache/Rakefile
 - lib/rspec_tracer/remote_cache/cache.rb
 - lib/rspec_tracer/remote_cache/git.rb
 - lib/rspec_tracer/reporter.rb
@@ -99,13 +102,14 @@ files:
 - lib/rspec_tracer/ruby_coverage.rb
 - lib/rspec_tracer/runner.rb
 - lib/rspec_tracer/source_file.rb
+- lib/rspec_tracer/time_formatter.rb
 - lib/rspec_tracer/version.rb
 homepage: https://github.com/avmnu-sng/rspec-tracer
 licenses:
 - MIT
 metadata:
   homepage_uri: https://github.com/avmnu-sng/rspec-tracer
-  source_code_uri: https://github.com/avmnu-sng/rspec-tracer/tree/v0.5.0
+  source_code_uri: https://github.com/avmnu-sng/rspec-tracer/tree/v0.7.0
   changelog_uri: https://github.com/avmnu-sng/rspec-tracer/blob/main/CHANGELOG.md
   bug_tracker_uri: https://github.com/avmnu-sng/rspec-tracer/issues
 post_install_message:
@@ -123,7 +127,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
     - !ruby/object:Gem::Version
       version: '0'
 requirements: []
-rubygems_version: 3.0.9
+rubygems_version: 3.2.26
 signing_key:
 specification_version: 4
 summary: RSpec Tracer is a specs dependency analysis tool and a test skipper for RSpec