rspec-tracer 0.1.0

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA256:
+  metadata.gz: fd68858a56e3f240a0710c3dc4d7af601268bc2f4029c3a67f5e93ccd301dc48
+  data.tar.gz: b89cb78a0fef102e99dd60ccb53b45238d0785d912750c13b9546e0e2fcbe6f3
+SHA512:
+  metadata.gz: 6e203ace8d29199999af09cd82418e0160160dae0077b5f4cb56c9c4de7bf40c25c5506e5fbc2d9081af4e7e69c47a80b62ce0a5de94aaee473f9506c62b4d83
+  data.tar.gz: f955b25c90b5fad1a0f0d4dd1c46f6c3b8c7e1c7d9d5a330a2cb4a13ba543d5264cd0fcdf7fcd545f44fa8d5c636dc6194341cb1ea484a22fd37c925b254d05c

data/CHANGELOG.md

@@ -0,0 +1,10 @@
+## [Unreleased]
+
+## [0.1.0] - 2021-08-27
+
+**Initial Release**
+
+### Added
+
+- Ability to run RSpec Tracer with SimpleCov and without SimpleCov
+- Support for HTML reports

data/LICENSE

@@ -0,0 +1,21 @@
+The MIT License (MIT)
+
+Copyright (c) 2021 Abhimanyu Singh
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in
+all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+THE SOFTWARE.

data/README.md

@@ -0,0 +1,248 @@
+![](./readme_files/rspec_tracer.png)
+
+RSpec Tracer is a **specs dependency analysis tool** and a **test skipper for RSpec**.
+It maintains a list of files for each test, enabling itself to skip tests in the
+subsequent runs if none of the dependent files are changed.
+
+It uses [Ruby's built-in coverage library](https://ruby-doc.org/stdlib/libdoc/coverage/rdoc/Coverage.html)
+to keep track of the coverage for each test. For each test executed, the coverage
+diff provides the desired file list. RSpec Tracer takes care of reporting the
+**correct code coverage when skipping tests** by using the cached reports. Also,
+note that it will **never skip any tests which failed or were pending** in the last runs.
+
+Knowing the examples and files dependency gives us a better insight into the codebase,
+and we have **a clear idea of what to test for when making any changes**. With this data,
+we can also analyze the coupling between different components and much more.
+
+**First Run**
+![](./readme_files/first_run.gif)
+
+**Next Run**
+![](./readme_files/next_run.gif)
+
+## Note
+
+**RSpec Tracer is currently available for use in the local development
+environment only.**
+
+## Installation
+
+Add this line to your `Gemfile` and `bundle install`:
+```ruby
+gem 'rspec-tracer', group: :test, require: false
+```
+
+And, add the followings to your `.gitignore`:
+```
+/rspec_tracer_cache/
+/rspec_tracer_coverage/
+/rspec_tracer_report/
+```
+
+### Compatibility
+
+RSpec Tracer requires **Ruby 2.5+** and **rspec-core >= 3.6.0**. If you are using
+SimpleCov, it is recommended to use **simplecov >= 0.12.0**.
+
+## Getting Started
+
+1. **Load and Start RSpec Tracer**
+
+    - **With SimpleCov**
+
+        If you are using `SimpleCov`, load RSpec Tracer right after the SimpleCov load
+        and launch:
+
+        ```ruby
+        require 'simplecov'
+        SimpleCov.start
+
+        # Load RSpec Tracer
+        require 'rspec-tracer'
+        RSpecTracer.start
+        ```
+
+        Currently using RSpec Tracer with SimpleCov has the following two limitations:
+        - SimpleCov **won't be able to provide branch coverage report** even when enabled.
+        - RSpec Tracer **nullifies the `SimpleCov.at_exit`** callback.
+
+    - **Without SimpleCov**
+
+        Load and launch RSpec Tracer at the very top of `spec_helper.rb` (or `rails_helper.rb`,
+        `test/test_helper.rb`). Note that `RSpecTracer.start` must be issued **before loading
+        any of the application code.**
+
+        ```ruby
+        # Load RSpec Tracer
+        require 'rspec-tracer'
+        RSpecTracer.start
+        ```
+
+2. Run the tests with RSpec using `bundle exec rspec`.
+3. After running your tests, open `rspec_tracer_report/index.html` in the
+browser of your choice.
+
+## Environment Variables
+
+To get better control on execution, you can use the following two environment variables:
+
+### RSPEC_TRACER_NO_SKIP
+
+The default value is `false.` If set to `true`, the RSpec Tracer will not skip
+any tests. Note that it will continue to maintain cache files and generate reports.
+
+```ruby
+RSPEC_TRACER_NO_SKIP=true bundle exec rspec
+```
+
+### TEST_SUITE_ID
+
+If you have a large set of tests to run, it is recommended to run them in
+separate groups. This way, RSpec Tracer is not overwhelmed with loading massive
+cached data in the memory. Also, it generate and use cache for specific test suites
+and not merge them.
+
+```ruby
+TEST_SUITE_ID=1 bundle exec rspec spec/models
+TEST_SUITE_ID=2 bundle exec rspec spec/helpers
+```
+
+## Sample Reports
+
+You get the following three reports:
+
+### Examples
+
+These reports provide basic test information:
+
+**First Run**
+
+![](./readme_files/examples_report_first_run.png)
+
+**Next Run**
+
+![](./readme_files/examples_report_next_run.png)
+
+### Examples Dependency
+
+These reports show a list of dependent files for each test.
+
+![](./readme_files/examples_dependency_report.png)
+
+### Files Dependency
+
+These reports provide information on the total number of tests that will run after changing this particular file.
+
+![](./readme_files/files_dependency_report.png)
+
+## Configuring RSpec Tracer
+
+Configuration settings can be applied in three formats, which are completely equivalent:
+
+- The most common way is to configure it directly in your start block:
+
+    ```ruby
+    RSpecTracer.start do
+      config_option 'foo'
+    end
+    ```
+- You can also set all configuration options directly:
+
+    ```ruby
+    RSpecTracer.config_option 'foo'
+    ```
+
+- If you do not want to start tracer immediately after launch or want to add
+additional configuration later on in a concise way, use:
+
+    ```ruby
+    RSpecTracer.configure do
+      config_option 'foo'
+    end
+    ```
+
+## Filters
+
+RSpec Tracer supports two types of filters:
+
+- To exclude selected files from the dependency list of tests:
+
+    ```ruby
+    RSpecTracer.start do
+      add_filter %r{^/helpers/}
+    end
+    ```
+- To exclude selected files from the coverage data. You should only use this
+when not using SimpleCov.
+
+    ```ruby
+    RSpecTracer.start do
+      add_coverage_filter %r{^/tasks/}
+    end
+    ```
+
+By default, a filter is applied that removes all files OUTSIDE of your project's
+root directory - otherwise you'd end up with the source files in the gems you are
+using as tests dependency.
+
+### Defining Custom Filteres
+
+You can currently define a filter using either a String or Regexp (that will then
+be Regexp-matched against each source file's path), a block or by passing in your
+own Filter class.
+
+#### String Filter
+
+```ruby
+RSpecTracer.start do
+  add_filter '/helpers/'
+end
+```
+
+This simple string filter will remove all files that match "/helpers/" in their path.
+
+#### Regex Filter
+
+```ruby
+RSpecTracer.start do
+  add_filter %r{^/helpers/}
+end
+```
+
+This simple regex filter will remove all files that start with /helper/ in their path.
+
+#### Block Filter
+
+```ruby
+RSpecTracer.start do
+  add_filter do |source_file|
+    source_file[:file_path].include?('/helpers/')
+  end
+end
+```
+
+Block filters receive a `Hash` object and expect your block to return either true
+(if the file is to be removed from the result) or false (if the result should be kept).
+In the above example, the filter will remove all files that match "/helpers/" in their path.
+
+#### Array Filter
+
+```ruby
+RSpecTracer.start do
+  add_filter ['/helpers/', %r{^/utils/}]
+end
+```
+
+You can pass in an array containing any of the other filter types.
+
+## Contributing
+
+Read the [contribution guide](https://github.com/avmnu-sng/rspec-tracer/blob/main/.github/CONTRIBUTING.md).
+
+## License
+
+The gem is available as open source under the terms of the [MIT License](https://opensource.org/licenses/MIT).
+
+## Code of Conduct
+
+Everyone interacting in the Rspec Tracer project's codebases, issue trackers, chat rooms and mailing lists is expected to follow the [Code of Conduct](https://github.com/avmnu-sng/rspec-tracer/blob/main/.github/CODE_OF_CONDUCT.md).

data/lib/rspec_tracer/cache.rb

@@ -0,0 +1,109 @@
+# frozen_string_literal: true
+
+module RSpecTracer
+  class Cache
+    attr_reader :all_examples, :failed_examples, :pending_examples, :all_files, :dependency
+
+    def initialize
+      @run_id = last_run_id
+      @cache_dir = File.join(RSpecTracer.cache_path, @run_id) if @run_id
+
+      @cached = false
+
+      @all_examples = {}
+      @failed_examples = Set.new
+      @pending_examples = Set.new
+      @all_files = {}
+      @dependency = Hash.new { |hash, key| hash[key] = Set.new }
+    end
+
+    def load_cache_for_run
+      return if @run_id.nil? || @cached
+
+      load_all_examples_cache
+      load_failed_examples_cache
+      load_pending_examples_cache
+      load_all_files_cache
+      load_dependency_cache
+
+      @cached = true
+
+      puts "RSpec tracer loaded cache from #{@cache_dir}" if @run_id
+    end
+
+    def cached_examples_coverage
+      return @examples_coverage if defined?(@examples_coverage)
+      return @examples_coverage = {} if @run_id.nil?
+
+      load_examples_coverage_cache
+    end
+
+    private
+
+    def last_run_id
+      file_name = File.join(RSpecTracer.cache_path, 'last_run.json')
+
+      return unless File.file?(file_name)
+
+      JSON.parse(File.read(file_name))['run_id']
+    end
+
+    def load_all_examples_cache
+      file_name = File.join(@cache_dir, 'all_examples.json')
+
+      return unless File.file?(file_name)
+
+      @all_examples = JSON.parse(File.read(file_name)).transform_values do |examples|
+        examples.transform_keys(&:to_sym)
+      end
+
+      @all_examples.each_value do |example|
+        example[:execution_result].transform_keys!(&:to_sym)
+
+        example[:run_reason] = nil
+      end
+    end
+
+    def load_failed_examples_cache
+      file_name = File.join(@cache_dir, 'failed_examples.json')
+
+      return unless File.file?(file_name)
+
+      @failed_examples = JSON.parse(File.read(file_name)).to_set
+    end
+
+    def load_pending_examples_cache
+      file_name = File.join(@cache_dir, 'pending_examples.json')
+
+      return unless File.file?(file_name)
+
+      @pending_examples = JSON.parse(File.read(file_name)).to_set
+    end
+
+    def load_all_files_cache
+      file_name = File.join(@cache_dir, 'all_files.json')
+
+      return unless File.file?(file_name)
+
+      @all_files = JSON.parse(File.read(file_name)).transform_values do |files|
+        files.transform_keys(&:to_sym)
+      end
+    end
+
+    def load_dependency_cache
+      file_name = File.join(@cache_dir, 'dependency.json')
+
+      return unless File.file?(file_name)
+
+      @dependency = JSON.parse(File.read(file_name)).transform_values(&:to_set)
+    end
+
+    def load_examples_coverage_cache
+      file_name = File.join(@cache_dir, 'examples_coverage.json')
+
+      return unless File.file?(file_name)
+
+      @examples_coverage = JSON.parse(File.read(file_name))
+    end
+  end
+end

data/lib/rspec_tracer/configuration.rb

@@ -0,0 +1,134 @@
+# frozen_string_literal: true
+
+require_relative 'filter'
+
+module RSpecTracer
+  module Configuration
+    DEFAULT_CACHE_DIR = 'rspec_tracer_cache'
+    DEFAULT_REPORT_DIR = 'rspec_tracer_report'
+    DEFAULT_COVERAGE_DIR = 'rspec_tracer_coverage'
+
+    attr_writer :filters, :coverage_filters
+
+    def root(root = nil)
+      return @root if defined?(@root) && root.nil?
+
+      @cache_path = nil
+      @root = File.expand_path(root || Dir.getwd)
+    end
+
+    def project_name(proj_name = nil)
+      return @project_name if defined?(@project_name) && proj_name.nil?
+
+      @project_name = proj_name if proj_name.is_a?(String)
+      @project_name ||= File.basename(root).capitalize
+    end
+
+    def cache_dir(dir = nil)
+      return @cache_dir if defined?(@cache_dir) && dir.nil?
+
+      @cache_path = nil
+      @cache_dir = dir || DEFAULT_CACHE_DIR
+    end
+
+    def cache_path
+      @cache_path ||= begin
+        cache_path = File.expand_path(cache_dir, root)
+        cache_path = File.join(cache_path, ENV['TEST_SUITE_ID'].to_s)
+
+        FileUtils.mkdir_p(cache_path)
+
+        cache_path
+      end
+    end
+
+    def report_dir(dir = nil)
+      return @report_dir if defined?(@report_dir) && dir.nil?
+
+      @report_path = nil
+      @report_dir = dir || DEFAULT_REPORT_DIR
+    end
+
+    def report_path
+      @report_path || begin
+        report_path = File.expand_path(report_dir, root)
+        report_path = File.join(report_path, ENV['TEST_SUITE_ID'].to_s)
+
+        FileUtils.mkdir_p(report_path)
+
+        report_path
+      end
+    end
+
+    def coverage_dir(dir = nil)
+      return @coverage_dir if defined?(@coverage_dir) && dir.nil?
+
+      @coverage_path = nil
+      @coverage_dir = dir || DEFAULT_COVERAGE_DIR
+    end
+
+    def coverage_path
+      @coverage_path ||= begin
+        coverage_path = File.expand_path(coverage_dir, root)
+        coverage_path = File.join(coverage_path, ENV['TEST_SUITE_ID'].to_s)
+
+        FileUtils.mkdir_p(coverage_path)
+
+        coverage_path
+      end
+    end
+
+    def coverage_track_files(glob)
+      @coverage_track_files = glob
+    end
+
+    def coverage_tracked_files
+      @coverage_track_files if defined?(@coverage_track_files)
+    end
+
+    def add_filter(filter = nil, &block)
+      filters << parse_filter(filter, &block)
+    end
+
+    def filters
+      @filters ||= []
+    end
+
+    def add_coverage_filter(filter = nil, &block)
+      coverage_filters << parse_filter(filter, &block)
+    end
+
+    def coverage_filters
+      @coverage_filters ||= []
+    end
+
+    def configure(&block)
+      Docile.dsl_eval(self, &block)
+    end
+
+    private
+
+    def test_suite_id
+      suite_id = ENV.fetch('TEST_SUITE_ID', '')
+
+      return if suite_id.empty?
+
+      suite_id
+    end
+
+    def at_exit(&block)
+      return Proc.new unless RSpecTracer.running || block
+
+      @at_exit = block if block
+      @at_exit ||= proc { RSpecTracer.at_exit_behavior }
+    end
+
+    def parse_filter(filter = nil, &block)
+      arg = filter || block
+
+      raise ArgumentError, 'Either a filter or a block required' if arg.nil?
+
+      RSpecTracer::Filter.register(arg)
+    end
+  end
+end