rspec-tracer 0.1.0 → 0.5.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: fd68858a56e3f240a0710c3dc4d7af601268bc2f4029c3a67f5e93ccd301dc48
-  data.tar.gz: b89cb78a0fef102e99dd60ccb53b45238d0785d912750c13b9546e0e2fcbe6f3
+  metadata.gz: 2817cccf4825e28177bb8794f8d261635f81ec5c1bd6ca0242ba171e702850ca
+  data.tar.gz: 9c5f6af579eec43a994c28b5e8132fec5d71f7a2ddddfb531fafd5c51943d283
 SHA512:
-  metadata.gz: 6e203ace8d29199999af09cd82418e0160160dae0077b5f4cb56c9c4de7bf40c25c5506e5fbc2d9081af4e7e69c47a80b62ce0a5de94aaee473f9506c62b4d83
-  data.tar.gz: f955b25c90b5fad1a0f0d4dd1c46f6c3b8c7e1c7d9d5a330a2cb4a13ba543d5264cd0fcdf7fcd545f44fa8d5c636dc6194341cb1ea484a22fd37c925b254d05c
+  metadata.gz: 2d3af598e09b8ac043b5f605acf39b588701c4f0e7b4d74646eb0da74c1c85746db60f215e02cd5fba1880bb75288d0d35df3986b08cc71cfbad4d45ad722d2f
+  data.tar.gz: 7ce36336213d4d37337afa5b50fd774bd6ef7b235539f3a289f1a804a878fba3fa1e7972e9bd09bb8083c80929c0fc3b6bf0b30f257542fdb5e5c466c1247360

data/CHANGELOG.md

@@ -1,4 +1,26 @@
-## [Unreleased]
+## [0.5.0] - 2021-09-03
+
+### Fixed
+
+- Limit number of cached files download (#16)
+
+## [0.4.0] - 2021-09-03
+
+### Added
+
+- Support for CI
+
+## [0.3.0] - 2021-08-30
+
+### Fixed
+
+- `docile` version compatability with `simplecov`
+
+## [0.2.0] - 2021-08-28
+
+### Fixed
+
+- Failures when RSpec required files are outside of project
 
 ## [0.1.0] - 2021-08-27
 

data/README.md

@@ -14,16 +14,50 @@ Knowing the examples and files dependency gives us a better insight into the cod
 and we have **a clear idea of what to test for when making any changes**. With this data,
 we can also analyze the coupling between different components and much more.
 
+## Note
+
+You should take some time and go through the [document](./RSPEC_TRACER.md) describing
+the **intention** and implementation details of **skipping tests**, **managing coverage**,
+and **caching on CI**, etc.
+
+## Table of Contents
+
+* [Demo](#demo)
+* [Installation](#installation)
+  * [Compatibility](#compatibility)
+  * [Additional Tools](#additional-tools)
+* [Getting Started](#getting-started)
+* [Environment Variables](#environment-variables)
+  * [CI](#ci)
+  * [LOCAL_AWS](#local_aws)
+  * [RSPEC_TRACER_NO_SKIP](#rspec_tracer_no_skip)
+  * [RSPEC_TRACER_S3_URI](#rspec_tracer_s3_uri)
+  * [RSPEC_TRACER_UPLOAD_LOCAL_CACHE](#rspec_tracer_upload_local_cache)
+  * [TEST_SUITES](#test_suites)
+  * [TEST_SUITE_ID](#test_suite_id)
+* [Sample Reports](#sample-reports)
+    * [Examples](#examples)
+    * [Examples Dependency](#examples-dependency)
+    * [Files Dependency](#files-dependency)
+* [Configuring RSpec Tracer](#configuring-rspec-tracer)
+* [Filters](#filters)
+  * [Defining Custom Filteres](#defining-custom-filteres)
+    * [String Filter](#string-filter)
+    * [Regex Filter](#regex-filter)
+    * [Block Filter](#block-filter)
+    * [Array Filter](#array-filter)
+* [Contributing](#contributing)
+* [License](#license)
+* [Code of Conduct](#code-of-conduct)
+
+## Demo
+
 **First Run**
 ![](./readme_files/first_run.gif)
 
 **Next Run**
 ![](./readme_files/next_run.gif)
 
-## Note
-
-**RSpec Tracer is currently available for use in the local development
-environment only.**
 
 ## Installation
 
@@ -41,8 +75,14 @@ And, add the followings to your `.gitignore`:
 
 ### Compatibility
 
-RSpec Tracer requires **Ruby 2.5+** and **rspec-core >= 3.6.0**. If you are using
-SimpleCov, it is recommended to use **simplecov >= 0.12.0**.
+RSpec Tracer requires **Ruby 2.5+** and **rspec-core >= 3.6.0**. To use with **Rails 5+**,
+make sure to use **rspec-rails >= 4.0.0**. If you are using SimpleCov, it is
+recommended to use **simplecov >= 0.12.0**.
+
+### Additional Tools
+
+To use RSpec Tracer on CI, you need to have an **S3 bucket** and
+**[AWS CLI](https://aws.amazon.com/cli/)** installed.
 
 ## Getting Started
 
@@ -58,7 +98,7 @@ SimpleCov, it is recommended to use **simplecov >= 0.12.0**.
         SimpleCov.start
 
         # Load RSpec Tracer
-        require 'rspec-tracer'
+        require 'rspec_tracer'
         RSpecTracer.start
         ```
 
@@ -74,18 +114,47 @@ SimpleCov, it is recommended to use **simplecov >= 0.12.0**.
 
         ```ruby
         # Load RSpec Tracer
-        require 'rspec-tracer'
+        require 'rspec_tracer'
         RSpecTracer.start
         ```
 
-2. Run the tests with RSpec using `bundle exec rspec`.
-3. After running your tests, open `rspec_tracer_report/index.html` in the
+2. To enable RSpec Tracer to share cache between different builds on CI, update the
+Rakefile in your project to have the following:
+
+    ```ruby
+    spec = Gem::Specification.find_by_name('rspec-tracer')
+
+    load "#{spec.gem_dir}/lib/rspec_tracer/remote_cache/Rakefile"
+    ```
+3. Before running tests, download the remote cache using the following rake task:
+
+    ```sh
+    bundle exec rake rspec_tracer:remote_cache:download
+    ```
+4. Run the tests with RSpec using `bundle exec rspec`.
+5. After running tests, upload the local cache using the following rake task:
+
+    ```sh
+    bundle exec rake rspec_tracer:remote_cache:upload
+    ```
+6. After running your tests, open `rspec_tracer_report/index.html` in the
 browser of your choice.
 
 ## Environment Variables
 
 To get better control on execution, you can use the following two environment variables:
 
+### CI
+
+Mostly all the CI have `CI=true`. If not, you should explicitly set it to `true`.
+
+### LOCAL_AWS
+
+In case you want to test out the caching feature in the local development environment.
+You can install [localstack](https://github.com/localstack/localstack) and
+[awscli-local](https://github.com/localstack/awscli-local) and then invoke the
+rake tasks with `LOCAL_AWS=true`.
+
 ### RSPEC_TRACER_NO_SKIP
 
 The default value is `false.` If set to `true`, the RSpec Tracer will not skip
@@ -95,6 +164,28 @@ any tests. Note that it will continue to maintain cache files and generate repor
 RSPEC_TRACER_NO_SKIP=true bundle exec rspec
 ```
 
+### RSPEC_TRACER_S3_URI
+
+You should provide the S3 bucket path to store the cache files.
+
+```ruby
+export RSPEC_TRACER_S3_URI=s3://ci-artifacts-bucket/rspec-tracer-cache
+```
+
+### RSPEC_TRACER_UPLOAD_LOCAL_CACHE
+
+By default, RSpec Tracer does not upload local cache files. You can set this
+environment variable to `true` to upload the local cache to S3.
+
+### TEST_SUITES
+
+Set this environment variable when using test suite id. It determines the total
+number of different test suites you are running.
+
+```ruby
+export TEST_SUITES=8
+```
+
 ### TEST_SUITE_ID
 
 If you have a large set of tests to run, it is recommended to run them in
@@ -107,6 +198,21 @@ TEST_SUITE_ID=1 bundle exec rspec spec/models
 TEST_SUITE_ID=2 bundle exec rspec spec/helpers
 ```
 
+If you run parallel builds on the CI, you should specify the test suite ID and
+the total number of test suites when downloading the cache files.
+
+```sh
+$ TEST_SUITES=5 TEST_SUITE_ID=1 bundle exec rake rspec_tracer:remote_cache:download
+```
+
+In this case, the appropriate cache should have all the cache files available on
+the S3 for each test suite, not just for the current one. Also, while uploading,
+make sure to provide the test suite id.
+
+```sh
+$ TEST_SUITE_ID=1 bundle exec rake rspec_tracer:remote_cache:upload
+```
+
 ## Sample Reports
 
 You get the following three reports:

data/lib/rspec_tracer/remote_cache/cache.rb

@@ -0,0 +1,187 @@
+# frozen_string_literal: true
+
+require_relative 'git'
+
+module RSpecTracer
+  module RemoteCache
+    class Cache
+      class CacheDownloadError < StandardError; end
+
+      class CacheUploadError < StandardError; end
+
+      class LocalCacheNotFoundError < StandardError; end
+
+      CACHE_FILES_PER_TEST_SUITE = 7
+
+      def initialize
+        @s3_uri = ENV['RSPEC_TRACER_S3_URI']
+        @aws_s3 = if ENV.fetch('LOCAL_AWS', 'false') == 'true'
+                    'awslocal'
+                  else
+                    'aws'
+                  end
+      end
+
+      def download
+        if @s3_uri.nil?
+          puts 'S3 URI is not configured'
+
+          return
+        end
+
+        prepare_for_download
+
+        if @cache_sha.nil?
+          puts 'Could not find a suitable cache sha to download'
+
+          return
+        end
+
+        download_files
+
+        puts "Downloaded cache from #{@download_prefix} to #{@download_path}"
+      rescue StandardError => e
+        puts "Errored: #{e.message}"
+      end
+
+      def upload
+        if @s3_uri.nil?
+          puts 'S3 URI is not configured'
+
+          return
+        end
+
+        prepare_for_upload
+        upload_files
+
+        puts "Uploaded cache from #{@upload_path} to #{@upload_prefix}"
+      rescue CacheUploadError => e
+        puts "Errored: #{e.message}"
+      end
+
+      private
+
+      def prepare_for_download
+        @test_suite_id = ENV['TEST_SUITE_ID']
+        @test_suites = ENV['TEST_SUITES']
+
+        if @test_suite_id.nil? ^ @test_suites.nil?
+          raise(
+            CacheDownloadError,
+            'Both the enviornment variables TEST_SUITE_ID and TEST_SUITES are not set'
+          )
+        end
+
+        @git = RSpecTracer::RemoteCache::Git.new
+        @git.prepare_for_download
+
+        generate_cached_files_count_and_regex
+
+        @cache_sha = nearest_cache_sha
+      end
+
+      def generate_cached_files_count_and_regex
+        if @test_suites.nil?
+          @last_run_files_count = 1
+          @last_run_files_regex = '/%<ref>s/last_run.json$'
+          @cached_files_count = CACHE_FILES_PER_TEST_SUITE
+          @cached_files_regex = '/%<ref>s/[0-9a-f]{32}/.+.json'
+        else
+          @test_suites = @test_suites.to_i
+          @test_suites_regex = (1..@test_suites).to_a.join('|')
+
+          @last_run_files_count = @test_suites
+          @last_run_files_regex = "/%<ref>s/(#{@test_suites_regex})/last_run.json$"
+          @cached_files_count = CACHE_FILES_PER_TEST_SUITE * @test_suites.to_i
+          @cached_files_regex = "/%<ref>s/(#{@test_suites_regex})/[0-9a-f]{32}/.+.json$"
+        end
+      end
+
+      def nearest_cache_sha
+        @git.ref_list.detect do |ref|
+          prefix = "#{@s3_uri}/#{ref}/"
+
+          puts "Testing prefix #{prefix}"
+
+          objects = `#{@aws_s3} s3 ls #{prefix} --recursive`.chomp.split("\n")
+
+          last_run_regex = Regexp.new(format(@last_run_files_regex, ref: ref))
+
+          next if objects.count { |object| object.match?(last_run_regex) } != @last_run_files_count
+
+          cache_regex = Regexp.new(format(@cached_files_regex, ref: ref))
+
+          objects.count { |object| object.match?(cache_regex) } == @cached_files_count
+        end
+      end
+
+      def download_files
+        @download_prefix = "#{@s3_uri}/#{@cache_sha}/#{@test_suite_id}/".sub(%r{/+$}, '/')
+        @download_path = RSpecTracer.cache_path
+
+        raise CacheDownloadError, 'Failed to download cache files' unless system(
+          @aws_s3, 's3', 'cp',
+          File.join(@download_prefix, 'last_run.json'),
+          @download_path,
+          out: File::NULL, err: File::NULL
+        )
+
+        @run_id = last_run_id
+
+        return if system(
+          @aws_s3, 's3', 'cp',
+          File.join(@download_prefix, @run_id),
+          File.join(@download_path, @run_id),
+          '--recursive',
+          out: File::NULL, err: File::NULL
+        )
+
+        FileUtils.rm_rf(@download_path)
+
+        raise CacheDownloadError, 'Failed to download cache files'
+      end
+
+      def prepare_for_upload
+        @git = RSpecTracer::RemoteCache::Git.new
+        @test_suite_id = ENV['TEST_SUITE_ID']
+        @upload_prefix = if @test_suite_id.nil?
+                           "#{@s3_uri}/#{@git.branch_ref}/"
+                         else
+                           "#{@s3_uri}/#{@git.branch_ref}/#{@test_suite_id}/"
+                         end
+
+        @upload_path = RSpecTracer.cache_path
+        @run_id = last_run_id
+      end
+
+      def upload_files
+        return if system(
+          @aws_s3, 's3', 'cp',
+          File.join(@upload_path, 'last_run.json'),
+          @upload_prefix,
+          out: File::NULL, err: File::NULL
+        ) && system(
+          @aws_s3, 's3', 'cp',
+          File.join(@upload_path, @run_id),
+          File.join(@upload_prefix, @run_id),
+          '--recursive',
+          out: File::NULL, err: File::NULL
+        )
+
+        raise CacheUploadError, 'Failed to upload cache files'
+      end
+
+      def last_run_id
+        file_name = File.join(RSpecTracer.cache_path, 'last_run.json')
+
+        return unless File.file?(file_name)
+
+        run_id = JSON.parse(File.read(file_name))['run_id']
+
+        raise LocalCacheNotFoundError, 'Could not find any local cache to upload' if run_id.nil?
+
+        run_id
+      end
+    end
+  end
+end

data/lib/rspec_tracer/remote_cache/git.rb

@@ -0,0 +1,113 @@
+# frozen_string_literal: true
+
+module RSpecTracer
+  module RemoteCache
+    class Git
+      class GitOperationError < StandardError; end
+
+      attr_reader :branch_ref, :ref_list
+
+      def initialize
+        fetch_head_ref
+        fetch_branch_ref
+      end
+
+      def prepare_for_download
+        fetch_unreachable_refs
+        fetch_ancestry_refs
+        fetch_ordered_refs
+      end
+
+      private
+
+      def fetch_head_ref
+        @head_ref = `git rev-parse HEAD`.chomp
+
+        raise GitOperationError, 'Could not find HEAD commit sha' unless $CHILD_STATUS.success?
+      end
+
+      def fetch_branch_ref
+        @merged_parents = []
+        @ignored_refs = []
+
+        unless merged?
+          @branch_ref = @head_ref
+
+          return
+        end
+
+        @ignored_refs << @head_ref
+
+        fetch_merged_parents
+        fetch_merged_branch_ref
+      end
+
+      def merged?
+        system('git', 'rev-parse', 'HEAD^2', out: File::NULL, err: File::NULL)
+      end
+
+      def fetch_merged_parents
+        first_parent = `git rev-parse HEAD^1`.chomp
+        @merged_parents << first_parent if $CHILD_STATUS.success?
+
+        second_parent = `git rev-parse HEAD^2`.chomp
+        @merged_parents << second_parent if $CHILD_STATUS.success?
+
+        raise GitOperationError, 'Could not find merged commit parents' if @merged_parents.length != 2
+      end
+
+      def fetch_merged_branch_ref
+        @origin_head_ref = `git rev-parse origin/HEAD`.chomp
+        @branch_ref = nil
+
+        if @merged_parents.first != @origin_head_ref
+          @branch_ref = @head_ref
+          @ignored_refs = []
+
+          return
+        end
+
+        @branch_ref = @merged_parents.last
+        @ignored_refs = @ignored_refs.to_set | `git rev-list #{@branch_ref}..origin/HEAD`.chomp.split
+
+        raise GitOperationError, 'Could not find ignored refs' unless $CHILD_STATUS.success?
+      end
+
+      def fetch_unreachable_refs
+        command = <<-COMMAND.strip.gsub(/\s+/, ' ')
+          git fsck
+              --no-progress
+              --unreachable
+              --connectivity-only #{@branch_ref}
+            | awk '/commit/ { print $3 }'
+            | head -n 25
+        COMMAND
+
+        @unreachable_refs = `#{command}`.chomp.split
+
+        raise GitOperationError, 'Could not find unreachable refs' unless $CHILD_STATUS.success?
+      end
+
+      def fetch_ancestry_refs
+        @ancestry_refs = `git rev-list --max-count=25 #{@branch_ref}`.chomp.split
+
+        raise GitOperationError, 'Could not find ancestry refs' unless $CHILD_STATUS.success?
+      end
+
+      def fetch_ordered_refs
+        unordered_refs = (@unreachable_refs.to_set | @ancestry_refs) - @ignored_refs
+
+        command = <<-COMMAND.strip.gsub(/\s+/, ' ')
+          git rev-list
+            --topo-order
+            --no-walk=sorted
+            #{unordered_refs.to_a.join(' ')}
+        COMMAND
+
+        @ref_list = `#{command}`.chomp.split
+
+        raise GitOperationError, 'Could not find refs to download cache' unless $CHILD_STATUS.success?
+      end
+    end
+  end
+end

data/lib/rspec_tracer/runner.rb

@@ -208,10 +208,11 @@ module RSpecTracer
       rspec_root = RSpec::Core::RubyProject.root
       rspec_path = RSpec.configuration.default_path
 
-      RSpec.configuration.requires.map do |file_name|
+      RSpec.configuration.requires.each_with_object([]) do |file_name, required_files|
         file_name = "#{file_name}.rb" if File.extname(file_name).empty?
+        file_path = File.join(rspec_root, rspec_path, file_name)
 
-        File.join(rspec_root, rspec_path, file_name)
+        required_files << file_path if File.file?(file_path)
       end
     end
 

data/lib/rspec_tracer/version.rb

@@ -1,5 +1,5 @@
 # frozen_string_literal: true
 
 module RSpecTracer
-  VERSION = '0.1.0'
+  VERSION = '0.5.0'
 end

data/lib/rspec_tracer.rb

@@ -17,6 +17,7 @@ require_relative 'rspec_tracer/coverage_reporter'
 require_relative 'rspec_tracer/defaults'
 require_relative 'rspec_tracer/example'
 require_relative 'rspec_tracer/html_reporter/reporter'
+require_relative 'rspec_tracer/remote_cache/cache'
 require_relative 'rspec_tracer/rspec_reporter'
 require_relative 'rspec_tracer/rspec_runner'
 require_relative 'rspec_tracer/ruby_coverage'

metadata

@@ -1,29 +1,35 @@
 --- !ruby/object:Gem::Specification
 name: rspec-tracer
 version: !ruby/object:Gem::Version
-  version: 0.1.0
+  version: 0.5.0
 platform: ruby
 authors:
 - Abhimanyu Singh
 autorequire:
 bindir: bin
 cert_chain: []
-date: 2021-08-26 00:00:00.000000000 Z
+date: 2021-09-03 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: docile
   requirement: !ruby/object:Gem::Requirement
     requirements:
-    - - "~>"
+    - - ">="
       - !ruby/object:Gem::Version
         version: 1.1.0
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '1.1'
   type: :runtime
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
-    - - "~>"
+    - - ">="
       - !ruby/object:Gem::Version
         version: 1.1.0
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '1.1'
 - !ruby/object:Gem::Dependency
   name: rspec-core
   requirement: !ruby/object:Gem::Requirement
@@ -85,6 +91,8 @@ files:
 - lib/rspec_tracer/html_reporter/views/examples_dependency.erb
 - lib/rspec_tracer/html_reporter/views/files_dependency.erb
 - lib/rspec_tracer/html_reporter/views/layout.erb
+- lib/rspec_tracer/remote_cache/cache.rb
+- lib/rspec_tracer/remote_cache/git.rb
 - lib/rspec_tracer/reporter.rb
 - lib/rspec_tracer/rspec_reporter.rb
 - lib/rspec_tracer/rspec_runner.rb
@@ -97,7 +105,7 @@ licenses:
 - MIT
 metadata:
   homepage_uri: https://github.com/avmnu-sng/rspec-tracer
-  source_code_uri: https://github.com/avmnu-sng/rspec-tracer/tree/v0.1.0
+  source_code_uri: https://github.com/avmnu-sng/rspec-tracer/tree/v0.5.0
   changelog_uri: https://github.com/avmnu-sng/rspec-tracer/blob/main/CHANGELOG.md
   bug_tracker_uri: https://github.com/avmnu-sng/rspec-tracer/issues
 post_install_message: